Introduction

In many respects, integrative and complementary medicine (ICM)Footnote 1 is a growing scientific field. During the last decades, and especially since the middle of the 1990s, there has been a rapid increase of peer-reviewed ICM publications in a variety of scientific journals (Danell and Danell 2009; Tam et al. 2012). In part, the establishment of specialized ICM journals supports this development, although the expansion is also taking place in general/internal medical journals and various biomedical sub-fields. A great majority of the publications are original research articles, even if the share of reviews is increasing over time (Danell and Danell 2009; Fu et al. 2011) and some of the most cited documents are reviews and survey studies (Tam et al. 2012). The establishment of ICM as a scientific field is also manifested in increasing numbers of scientific forums (such as world congresses), improved opportunities to get research funded (for example, by agencies such as the National Center for Complementary and Alternative Medicine, a branch of the US National Institutes of Health), and the foundation of ICM research centers and research positions at highly reputable universities and hospitals.

Even if it is reasonable to argue that ICM is a growing scientific field, it is also a highly contested area in terms of scientific legitimacy. The ideas and practices in ICM traditions rely on postulates that, at least to some extent, are in conflict with dominant scientific consensus. For example, homeopaths or acupuncturists can use substances, equipment, tools of diagnosis, and/or causal explanations that not are accepted or fully explained by established scientific standards. However, the boundaries between ICM and conventional western medicine—on what is accepted as valid knowledge or not—are under constant negotiation and “translation” by different actors (such as researchers, medical professionals, and politicians) (cf. Callon et al. 1983; Gieryn 1983). There are also important differences between different countries, between different cultures, over time, and concerning what is accepted, desirable, or legal.

From a conventional medical perspective, ICM can and should be tested and evaluated with help from high quality medical procedures, preferably randomized controlled trials (Ernst 2007). However, this ideal is far from unproblematic among ICM researchers and practitioners. While some, such as many chiropractors and parapsychologists, view their methods as scientific, albeit not fully accepted by the scientific community (Hess 1993; Martin 1994), others emphasize and defend alternative forms of knowledge production. Many ICM practitioners are skeptical about the possibility of testing their methods with help from randomized controlled trials, since they value individual adjustments in the treatments and aspects that are hard to measure with help from conventional techniques (Ernst 2007). In some cases, such as acupuncture, it has been suggested that there are different forms of the same tradition; a more technically oriented western acupuncture and a traditional Chinese form (Ulett 1999). Despite this, it is reasonable to expect that positive responses from medical professionals and researchers are crucial for the establishment and legitimacy of ICM, both as a scientific field and in society at large. It is also reasonable to expect that a crucial strategy is to engage in high quality research and publish in well-cited and internationally recognized journals (e.g., Adams 2002; Yoshida 2002), even if this strategy will affect content, experimental procedures, writing styles, and application of central concepts (see Kim 2007).

From previous bibliometric studies on ICM research we know relatively a lot about the general publication pattern (Danell and Danell 2009; Fu et al. 2011; Barnes et al. 1999), different forms of publication biases (Pittler et al. 2000; Sood et al. 2007; Vickers et al. 1998), general content of top-cited articles (Tam et al. 2012), and geographical aspects (Fu et al. 2011; Tam et al. 2012). There are also informative studies on sub-fields, such as acupuncture (Danell and Danell 2011; Fu et al. 2012; Han and Ho 2011) and homeopathy (Chiu and Ho 2005). However, we know much less about ICM’s reception. Is ICM research integrated in conventional medicine or even acknowledged in explicit ICM contexts (other ICM studies, ICM journals, etc.)? In a citation context study, on a few relatively highly cited ICM articles, it was shown that the results migrated to a great variety of journals, both inside and outside the ICM field. However, a closer inspection of the citation contexts revealed that a clear majority of the citing documents were related to ICM in general, or to the specific ICM traditions investigated in the cited articles. The results also indicated that the cited ICM studies, whether or not they were framed positively, negatively, or neutrally, were represented as substantially different from biomedical research (Danell 2012).

The aim of this article is to analyze the reception of ICM research in scientific journals. Is this kind of research acknowledged, in terms of being cited, in explicit ICM contexts (for example, in ICM journals and/or in documents with ICM-related content)? or Is it (also) cited in general or specialized biomedical counterparts? What is the impact of ICM research? Is it possible to identify any shift in focus, concerning the content, from the original ICM research to the documents where it is acknowledged? Are the citing documents about similar topics, problems, disorders, methods, etc. as the cited ones? or Is ICM research integrated in other contexts than originally produced?

Empirical material

Since the aim of this article is to analyze the reception of research the empirical material was divided in two parts: one set of ICM publications; and one set of publications that cited the first set. The science citations index expanded (SCI-E) database on the web of science was used as the source. This database covers high-impact medical journals, includes relevant information about cited and citing documents, and full-text abstracts, in most cases.

The ICM publications are delimited to all citable items in journals classified as “integrative and complementary medicine” according to journals citation reports (JCR) in one specific year. An argument for choosing this journal category is that research published in general or in specialized biomedical journals probably has a higher degree of legitimacy outside the ICM field than research published in explicit ICM journals. Another argument is that it is also problematic to identify individual documents as ICM since they can be classified and indexed in many ways (for example, in relation to specific ICM traditions, methods, and/or diseases, but also with general concepts such as “complementary medicine”, “folk medicine”, or “traditional medicine”).

In order to analyze the reception of research is it necessary to step back in time a few years to get a starting point (actually, to give enough time for researchers to find the publications, to carry on their research, and publish their own works). In this case, I have chosen to delimit the set of ICM documents to one specific year, namely 2007. That year, 12 journals were included in the ICM subject category. This set of documents consisted of 1,329 items (see Table 1). The set of citing documents consisted of all documents that, according to the SCI-E database, cited any of the items in the original set of ICM document (including possible self-citations) during the period from January 2007 to September 2012. This set consisted of 7,573 unique documents (doublets removed).

Table 1 Cited ICM journals and number of document types in each journal
Full size table

To retrieve the data, general and cited reference searches were used, with help from abbreviated journal titles, publication name/cited work, and year published/cited year(s). Subject classification of journals and impact factors were retrieved from JCR.

Methods: citation and co-word analysis

This article combines two main methods, namely citation and co-word analysis. Citation analysis is one of the most common bibliometric methods, which focuses on relationships between cited and citing documents. At a very basic level citations are considered to be some kind of acknowledgement that one document receives from another (e.g., Smith 1981). Researchers, or more precisely authors of scientific texts, are expected to cite works they find relevant and useful. They are also assumed to be well informed (for example, about recent developments in their fields) and honest. From that, we can expect highly cited work to be more useful, or contain more important contributions, than works that only receive few or no citations at all (Wouters 1999). But as several authors have noted, researchers are governed by various expectations and conventions, individual preferences, and limitations (for example, lack of information, personal disputes, and editorial limitations) (MacRoberts and MacRoberts 1996). They also have very different reasons to cite, for example, giving homage or critiquing peers (Garfield 1965). Citing behavior is also quite different in different research fields (Hargens 2000; Wouters 1999). In this case, average citing numbers and impact factors (retrieved from JCR) were used as indicators of the impact of the research in focus, even if these measures give a simplified view of the reception. This paper also analyzes the general bibliometric characteristics of the cited and citing documents and the subject classifications (retrieved from JCR) of the citing journals.

A more detailed analysis of the content was performed with help from co-word analysis. This method was originally developed as an alternative to bibliometric techniques, such as citation and co-citation analysis, in order to identify how documents are related to each other by counting and analyzing co-occurrence of words or phrases. The general idea is that this procedure can be used to detect themes in given areas, and relationships between these themes, which can be used to explore, review, and evaluate research fields, often with help from visualization or mapping techniques (He 1999). A basic assumption is that texts are the main products of science and consequently should be the main units of analysis if we want to understand cognitive and social structures of science (Latour and Woolgar 1979; Callon et al. 1983). Of course, researchers not only read, write, and publish texts; they also build laboratories, perform experiments, analyze different forms of data, draw conclusions, etc. But science, as we now it, is basically manifested, communicated, and negotiated textually. Knowledge is produced by using already existing texts and acting upon them, which, in turn, results in new texts. However, texts not only refer to journal articles or other kinds of scientific publications. The concept can also include such things as notes, research plans, observation protocols, apparatus print-outs, and patents (Callon et al. 1983). Another basic assumption is that researchers produce their texts intentionally and try to convince their audience of their legitimacy. The words in the texts are linked together in phrases, which, in turn, can be linked together in larger networks (of research problems, subjects, methods, interpretations, etc.). If the content changes in an area, for example concerning what topics or research questions are “hot or not”, this can be considered as the combined effect of a large number of individual acts and strategies (Callon et al. 1991).

In co-word analysis, not all words or phrases are considered equally important. At first glance, this may seem contradictory, since words just are symbols and equally “light”. However, depending on contexts, and what words are associated with other words, words and word associations possess varying amounts of power or relevance. Some words, which can be labeled as macro-terms or signal words, are almost impossible to circumvent or ignore, not necessarily because of the frequency with which they occur in texts, but because of how they are used and connected with other words in order to establish meaning. Macro-terms, such as “DNA”, “clinical trial”, or “chemotherapy”, can literally synthesize entire research domains. An important assumption about macro-terms is that they not only are considered signals, but also connectors. They build networks with other macro-terms, research problems, and different kinds of actors. For example, macro-terms connect all the papers that include a specific macro-term, the researchers who have written the papers, the problems that are associated with the term, the methods that are used, and the laboratories where the research is conducted (Callon et al. 1983).

In practice, co-word analysis can be performed on different text corpora uses—full texts, abstracts, titles, index words, etc. (Leydesdorff 1989). There are also different techniques for identifying macro-terms, calculating their relationships, and visualizing the results (e.g., Callon et al. 1983; Callon et al. 1991; Law and Whittaker 1992; Courtial and Law 1989). For this article, I performed the analysis on the text corpora of the abstracts, of both cited and citing documents. Since some document types such as editorial material, letters, and corrections normally don’t have abstracts, this part of the analysis covered a limited share of the initial sets (76 % of the ICM documents, 95 % of the citing documents). In order to identify macro-terms, I used the VOSviewer program.Footnote 2 The VOS-viewer is described as an unified approach to clustering and mapping bibliometric networks, and has been proposed as an alternative to combining clustering and mapping techniques relying on different assumptions (Waltman et al. 2010). This approach is based on a weighted and parameterized variant of the Louvain method for partitioning data into cluster (Blondel et al. 2008), and a variant of the well known Multidimensional scaling technique called visualization of similarities (VOS) which calculates a low dimensional visualization in which the visualized distance between any pair of objects reflects their similarity as accurately as possible (Van Eck and Waltman 2007). The linguistic filters in this program identify so-called noun-phrases, convert plural forms of nouns into singular ones, calculate the relevance of the noun phrases, and cluster and map the phrases/terms. The general idea is that phrases/terms with general meanings (such as “promising result” or “in this paper”) are of less importance than those with specific meaning (for more details on the different steps in the procedure, see Van Eck and Waltman 2011). In this case, I have used the full counting method with a minimum co-occurrence of ten terms in the case of the cited documents, and 20 in the case of the citing documents (since it is a larger body with higher frequencies). This resulted in 409 phrases/terms from the ICM documents and 1,384 from the citing documents.

Findings

General characteristics of the ICM documents

As mentioned above, the initial set of ICM documents consisted of 1,329 items published in 12 ICM journals in 2007. Seventy-three percent of these documents were classified as articles, 6 % as reviews, 12 % as editorials, 4 % as letters, and 5 % as other document types (for example, corrections, news, and biographies) (Table 1). It is worth noticing that the journals, depending on their publication activity, scope, and internal structure, are represented in very different proportion in the set. On the extremes, there are 15 documents published in Acupuncture and Electro Therapeutics compared to 466 in Journal of Ethnopharmacology in the year 2007.

Impact of the ICM documents

On average, the documents had been cited 8.6 times from the date of publication until September 2012. The most-cited document was cited 144 times. About 12 % of the documents were cited 20 times or more. Another 20 % were cited between one and three times. Twenty-two percent had not been cited at all. However, the journals were cited to relatively different extents (Table 2). On the extremes, the documents in Explore: The Journal of Science and Healing were cited 1.3 times on average, while the documents in the Journal of Ethnopharmacology were cited 14.2 times during the same period of time.

Table 2 Average citing of documents in ICM journals and five-year impact factors
Full size table

The average citing numbers of the documents, which were measured diachronous (i.e., cumulatively), were in general much higher than the synchronous impact factors of the journals (Table 2), since it was opened for a larger citation window (Ingwersen et al. 2001; Glanzel and Moed 2002). Of course, this is not a unique feature of ICM journals, but it is worth noticing that several of these journals seem to be much more frequently cited than the general impact factor indicate. Similar discrepancies concerning impact factor and citation per paper were also found by Fu et al. (2011), who analyzed the impact of CAM papers during the period 1980–2009.

General characteristics of the citing documents

Moving on to the general characteristics of the citing documents, 81 % were classified as articles, 14 % as reviews, 2 % as editorials, 1 % as letters, and 2 % as other document types. About 2 % of the citing documents were published in 2007—the same year as the cited documents. Another 13 % were published in 2008, 23 % in 2009, 26 % in 2010, 30 % in 2011, and 5 % in 2012,Footnote 3 an indication that it takes a couple of years to get the research in focus acknowledged by other researchers.

Reception in different sub-fields: the journals

A first measure of the context in which the ICM research in focus was acknowledged was in what journals the citings appeared. Were they related to the ICM research field or any other medical sub-field? All together the documents were cited in 1,900 journals. Fifty-seven percent of the journals only appeared once in the material. Twenty-four percent of the journals appear two or three times. The most frequent journal is Journal of Ethnopharmacology (6 % of the citing documents), followed by Evidence-Based Complementary and Alternative Medicine (5 %), and Journal of Alternative and Complementary Medicine (3 %). Of the top ten citing journals (Table 3), five were included in the ICM subject category according to JCR. Another four journals were related to the field by their focus on natural products, herbs, and medicinal plants. Only the Journal of Agricultural and Food Chemistry was clearly outside the ICM research field.

Table 3 Top ten citing journals, percentage of citing documents, and subject categories
Full size table

In an analysis of the first subject category of the citing journals (Table 4), the dominance of the ICM field among the most frequent journals (Table 3) seemed to decrease. ICM was the most common subject category (with 16 % of the citing documents), but there were also several relatively large subject categories related to general and specialized biomedicine, such as Pharmacology & Pharmacy (14 %), Neurosciences & Neurology (3 %), General & Internal Medicine (3 %), and Oncology (3 %). We could also identify subject categories related to general natural science, such as Biochemistry & Molecular Biology (6 %) and Chemistry (5 %). Another relatively large subject category was Plant Sciences (12 %), a category that included several journals that could be related to the ICM field (such as the Journal of Natural Products and the double-classified Journal of Ethnopharmacology). However, if we include multiple classifications of the journals, almost 25 % of the citing documents belonged to the ICM journal category.

Table 4 Top ten subject categories of citing journals according to first category, and percentage of citing documents
Full size table

Mapping scientific sub-fields: a co-word analysis of abstracts

A first impression of the overview maps (Figs. 1, 2) was that the structure of the phrases/terms is very similar between the cited and the citing documents. In both cases, it was possible to divide the clusters into two distinct parts: one related to pre-clinical research, and one related to clinical research. The pre-clinical part consisted of three clusters (green, yellow, and blue) and the clinical part consisted of one large cluster (red). There was also a small cluster (purple) outside the clinical and pre-clinical ones consisting of a few phrases/terms in both maps. Not only was the structure very similar, but the dominant phrases/terms, which could be considered macro-terms, also overlapped. “Extract” was the dominant term in both green clusters, “cell” in the yellow clusters, and “rat” in the blue ones. In the red clinical clusters, “patient” was the dominant term. In all of these cases, it was reasonable to assume that the macro-terms represented the main focus of the documents and were the objects of investigation.

Fig. 1
figure 1

Overview map of co-word analysis of ICM documents

Full size image
Fig. 2
figure 2

Overview map of co-word analysis of citing documents

Full size image

Looking more closely at the clusters and comparing the visualizations with frequencies and relevance calculations of the phrases/terms, some striking similarities concerning the general content were identified, even if the exact phrases/terms varied between the cited and citing documents.

The green clusters (Figs. 3, 4) were dominated by relatively general phrases/terms, such as “extract”, “acid”, “plant”, “plant extract”, “root”, and “plant species”, and could be labeled as pharmacological or even plant pharmacological. In these clusters, we could also identify several terms connected to laboratory practice (e.g., “assay”, “mic value”, “vitro”, and “concentration”), units of measure (e.g., “mg kg”, “mu l ml”, and “mu m”), substances (e.g., “oil”, “essential oil”, “chloroform”, “hexane”, “alkaloid”, and “acetone”), and specific disorders (e.g., “malaria” and “tuberculosis”). In the case of the citing abstracts (Fig. 4), it was also possible to identify phrases/terms that were related to the ICM research field (such as “folk medicine”, “homeopathic preparation”, and “kampo medicine”), even if they not were particularly salient.

Fig. 3
figure 3

Close-up map of pre-clinical clusters in ICM documents

Full size image
Fig. 4
figure 4

Close-up map of pre-clinical clusters in citing documents

Full size image

The blue clusters were dominated by terms related to animal experiments (Figs. 3, 4). Some general terms such as “rat” and “mouse” were the most frequent, but it was also possible to identify phrases/terms connected to metabolisms (e.g., “protein”, “kidney”, “antioxidant”, “liver”, “blood glucose”, “cholesterol”, “lipid”, and “toxicity”), immune system (e.g., “anti inflammatory effect” and “analgesic effect”), research process (e.g., “expression”, “administration”, “experimental group”, “inhibitory effect”, and “region”), units of measure (e.g., “mg kg”, “g kg”, and “body weight”), and disorders (e.g., “diabetic rat” and, “liver injury”).

The yellow clusters (Figs. 3, 4) could be labeled biological or cell biological ones. The dominant macro-term was “cell”, followed by ones like “protein”, “gene”, “production”, “expression”, and “activation”. In these clusters, we found many terms that were abbreviations for different proteins (e.g., “nf kappa b”, “tnf alpha”, and “vcam”) or related to cells or cell functions (e.g., “macrophage”, “phosphorylation”, “cell line”, “cancer cell”, and “cell viability”).

Compared to the pre-clinical clusters, the clinical ones were more diverse (Figs. 5, 6). The dominant macro-term in both cited and citing abstracts was “patient”. There were also many general phrases/terms related to the ICM/CAM field (e.g., “cam”, “cam use”, “cam user”, “cam therapy”, “cam practitioner”, “alternative medicine”, and “integrative medicine”), surveys or interview studies (e.g., “survey”, “interview”, “respondent”, “focus group”, “questionnaire”, and “experience”), clinical research and/or overviews of clinical research (e.g., “clinical condition”, “clinical trial”, “clinical effect”, “clinical research”, “cohort study”, “controlled clinical trial”, “randomized clinical trial”, “placebo”, “trial”, “database”, and “healthcare”). Especially in the citing abstracts (Fig. 6), it was possible to identify several phrases/terms related to specific ICM traditions or techniques (such as “acupuncture”, “yoga”, “mindfulness”, “quigong”, “massage therapy”, and “manual therapy”.) Among these traditions, acupuncture was represented with help from several terms (such as “acupoint”, “acupuncture group”, “acupuncture point”, “point”, “acupuncture stimulation”, “trigger point”, and “electroacupuncture”, as well as abbreviations of specific acupoints), both in cited and citing abstracts. Some ICM traditions, such as manual therapy and yoga, could be connected to different kinds of physical problems. For example, yoga was connected to “fibromyalgia”, “disability”, and “neck pain”. Manual therapy was connected to “neck pain”, but also to terms related to treatments (like “manipulation” and “mobilization”). Acupuncture was related to terms like “pain”, “anxiety”, and “vomiting”, but also to several terms related to the research process (such as “trial”, “score”, “placebo”, and “measure”).

Fig. 5
figure 5

Close-up map of the clinical cluster in ICM documents

Full size image
Fig. 6
figure 6

Close-up map of the clinical cluster in citing documents

Full size image

Conclusions

Since ICM research is contested and often considered different, deviant, or even unscientific in scientific contexts, the first aim of this article was to analyze the general reception of ICM research in scientific journals. In what contexts was this kind of research acknowledged, in terms of being cited? Was ICM research first of all cited in its own context, in ICM journals and/or in documents with ICM-related content? or Was it (also) cited in general or specialized biomedical counterparts? On these questions, the results, at least in part, pointed in different directions. When analyzing what journals the ICM research was cited in, it was clear that the most frequent journals were related to the ICM field. ICM was also the most common subject category among the citing documents (about 25 % could be related to this specific category). However, the ICM documents were cited in a great variety of journals and a majority of the documents were related to medical subfields other than ICM, such as Pharmacology & Pharmacy, Neurosciences & Neurology, Oncology, and General & Internal Medicine, as well as and general natural science, which indicated that ICM research was highlighted outside a narrowly defined ICM context. This does not necessarily mean that the content of the research has been accepted or confirmed, but it was used and highlighted in other frameworks than originally published. From the analysis, it was also clear that the ICM journals had relatively different impact, in terms of impact factors and average citing numbers, and that some of the ICM journals in focus had much higher average citing numbers than the impact factors indicated. This is particularly evident in the cases of Journal of Ethnopharmacology and Complementary Therapies in Medicine.

Another goal was to analyze whether the focus, in terms of content, changed from cited to citing documents. In other words, were cited and citing documents on similar topics, problems, disorders, methods, etc.? or Was ICM research integrated in contexts other than those originally produced? This part of the analysis was performed with help from co-word analyses of the abstracts, which hopefully gave a more detailed view of the reception of ICM research. The results indicated that both structure and general content of the terms/phrases were strikingly similar in cited and citing abstracts. In both cases, it was possible to divide the phrases/terms into five clusters, of which at least three could be connected to pre-clinical research and one to clinical research. From the dominant phrases/terms, which overlapped in the four large clusters, it was possible to label the pre-clinical clusters Pharmacology, Animal experiments, and Cell biology. The large clinical clusters could be labeled Patient oriented. These terms/phrases could also be considered macro-terms in the material. In the clinical clusters, and especially in the one generated out of the citing documents, it was possible to detect several phrases/terms that were explicitly related to the ICM research field. These phrases/terms were both connected to specific traditions (such as acupuncture, manual therapy, and yoga) and to the use and practice of ICM in general. It was also clear that much of this research was related to surveys and interview studies. However, I would argue that it is more significant that so many of the terms, especially in the pre-clinical clusters, were not articulated in terms of ICM, CAM, or specific ICM/CAM traditions. This could very well be a result of adjustments in writing style or of a focus of the research in order to get research accepted and/or published in biomedical forums (cf. Kim 2007), but it could also be a sign that ICM research was (or was on its way to being) an integrated part of biomedical and natural sciences research, and that there was no need or intention to frame it as ICM. Instead, many of the phrases/terms indicated that both cited ICM documents and citing documents were related to basic research, and that the macro-terms were connected to different aspects of the laboratory practice, objects of investigation (cells, proteins, substances, etc.), units of measure, and specific disorders. It was also striking that there seemed to be a very distinct division of the macro-terms: the pre-clinical clusters were to some extent intertwined and overlapping, but there were very few connections between the clinical and the pre-clinical clusters. So, returning to the initial question, both when analyzing the citing pattern and the content of cited and citing documents, it was clear that ICM research was acknowledged oustide a narrow ICM research context (for example, in sub-fields such as Pharmacology & Pharmacy and Plant Science). However, focusing on the content, there was no dramatic change in focus from cited to citing documents since they seemed to have very similar overall content.