Introduction

One of the greatest challenges faced by parents of children with neurodevelopmental disorders is making treatment decisions. This challenge is compounded by the vast landscape of information available to them: a landscape in which the accuracy and motivation of claims about treatment safety and efficacy is often hard to discern. Parents may seek health information and direct guidance from professionals and other parents, or through media such as hard copy written materials and educational videos, as well as through online resources. The Internet has now become the second most used and preferred source of health information for families after their health care providers (D’Alessandro et al. 2001; Goldman and Macpherson 2006; Tuffrey and Finlay 2002; Wainstein et al. 2006). Most parents turn to the Internet for health information in order to enhance knowledge and understanding of their child’s condition (Semere et al. 2003; Tuffrey and Finlay 2002; Wainstein et al. 2006), establish a sense of parental control over the actions of health professionals, and develop the ability to explain the condition to others, including the affected child (Starke and Möller 2002). While parents using the Internet have been shown to be rather critical of the credibility of online information (Bernhardt and Felter 2004; Khoo et al. 2008; Nettleton et al. 2005) there is evidence that in practice many have enough trust in online health information to be influenced by it when making medical decisions for their children (Semere et al. 2003; Wainstein et al. 2006). Currently, the medium remains largely unregulated and much online health information has been shown to contravene treatment recommendations from the American Academy of Pediatrics (Impicciatore et al. 1997; McClung et al. 1998; Pandolfini et al. 2000; Scullard et al. 2010). Although the positive contribution of online resources cannot be understated—many families have developed a better understanding of their child’s condition and found medical and social support online (Akins et al. 2010; White and Dorman 2001)—physicians and other health care providers are increasingly faced with the task of helping families sift through the diverse range and quality of online health information. Their task is especially difficult when online health information appears to be scientifically validated but is in fact not evidence-based, makes erroneous references to scientific evidence, or is laden with emotional testimonials. While a handful of websites have been developed specifically to assure throughput of high quality evidenced-based information (e.g., Cochrane Library, Healthfinder, MedlinePlus), concerns remain about the common use on many other websites of sophisticated marketing, emotional testimonials, and unproven claims about the safety and efficacy of treatments (American Academy of Pediatrics 2001; McCaffrey et al. 2007; Simpson and Roman 2001). In response, commentators have proposed that peer-reviewed scientific evidence should be the sine qua non of information for families and clinicians as they evaluate the best treatment options for children with disabilities (Levy and Hyman 2005; Zaidman-Zait and Jamieson 2007).

In a previous study we mapped the landscape of treatment information on highly-trafficked websites from major advocacy organizations for neurodevelopmental disorders (Di Pietro et al. 2011). Advocacy websites were examined because they provide a plethora of information about the conditions they advocate for in addition to providing valuable support services for families by hosting on-line networks, charitable events, conferences, funding research, and providing a voice for stakeholders (White and Dorman 2001). We found that these sites generally serve as information warehouses rather than evaluation services, providing messages largely targeted at parents about a wide variety of readily available treatment products and services. Most importantly, we found that information about treatments was overwhelmingly encouraging, regardless of the type of product or service, and regardless of its acceptance by the medical community. The absence of information about the websites’ selection criteria for featured treatment products and services contributed to the impression of uncritical endorsement. These findings gave rise to further questions about the nature and quality of evidence presented. We pursued the answers to these questions here by identifying and quantifying the diverse types of evidence used to support treatment claims. Our study is broadly framed within the context of consumer health information available via the Internet, but grounded in an appreciation of the complexities of evaluating evidence in the context of evolving roles of the ‘e-patient’ and parental expertise, and the particular information needs associated with chronic conditions in children for which treatment is often elusive, unproven, or multifactorial. We conclude with an analysis of how the status quo can be improved, and delineate key issues to be considered in future developments in information provision services by advocacy organisations.

Methods

Following Di Pietro et al. (2011), we conducted an in-depth quantitative and qualitative content analysis of advocacy websites devoted to three common neurodevelopmental disorders—autism spectrum disorder (ASD), cerebral palsy (CP), and fetal alcohol spectrum disorder (FASD). The three neurodevelopmental conditions we consider are the foci of research for a new Network of Centres of Excellence (NCE), NeuroDevNet, a Canada-wide initiative to study children’s brain development from both a basic and clinical perspective. The work reported here is part of the neuroethics component of the initiative.

Content analysis is a widely used research method for analyzing the content of various media (newspapers, television programs, political speeches) including health information on the Internet (Jenssen et al. 2009; Krippendorf 1980; Ostry et al. 2007; Petch 2004; Starman and Gettys 2010; Suggs and McIntyre 2009). Rather than being a single method, approaches to content analysis vary depending on the nature of the content being analyzed and goals of the study (Hsieh and Shannon 2005). Following our initial assessment of website treatment information (Di Pietro et al. 2011), we developed our own coding instrument to evaluate the types of evidence used to support claims about treatment products and services featured across our sample of websites.

Website Selection

We identified a sample of advocacy websites for analysis based on Internet traffic ranking scores obtained from Alexa Internet Inc, Compete Inc, and Quantcast Corporation. Websites that ranked the highest based on the greatest number of average daily visitors and page views over the 3 months between May and July 2010 were chosen to establish a pool of nine sites for analysis—three for each of the three target neurodevelopmental disorders. Four experts from NeuroDevNet Inc reviewed and confirmed the final selection of these websites.

Website Accountability and Readability

To contextualize the analysis of the information provided on the websites, we recorded the presence or absence of: a mission statement for the host organization, a list of website contributors and their credentials, a list of the Board of Directors, website administrator contact information, financial disclosures; disclaimers, banner advertisements and brand name products, and corporate sponsorship. We also evaluated reading difficulty level using the US government standard Flesch-Kincaid Readability Test (McClure 1987) that yields a score between 0 and 100 based on text characteristics such as statistical average word length and sentence length of the work. High scores indicate easy reading; low scores indicate high levels of difficulty.

Content Extraction

Content extraction was designed following an initial assessment of the websites and exploration of the texts. We searched the websites for content relating to both free and commercially available treatment products and services. Passages containing the text strings: “treat*”, “purchas*”, or “buy” were located using the website search engine. In the few cases where a search engine was not a feature of the site, we used the “find” function of the browser and augmented this strategy with a manual search using the sitemap. The boundaries of the website were defined by information present in the text of the website itself, or as a download located on the website server. Treatment information was defined as any statement made concerning an item, activity, or program intended to cure or improve a child’s condition, alleviate symptoms, or cope with difficulties arising from the disorder. Although advocacy websites also provide information through online chat rooms or forums, we did not examine content from these social support services. Single treatments (also referred to below as cases) formed the units of analysis, and were defined at the greatest level of specificity available on the website. General information about diets, for example, could represent a single case, but if specific types of diet were presented such as gluten free, sugar free, or Feingold, these would also be treated as separate cases. All text pertaining to a particular case was extracted for analysis, along with URLs and site-map location listed for each entry. To verify the consistency and reliability of data extraction, two independent coders checked the cases against the master text file, noting and resolving any points of disagreement about case identification, case omissions, redundancy or overlap. We assembled the data for this study within a 2 week period between 7/7/10 and 7/22/10, noting the time and date of the most recent website update.

Coding

We used NVivo 8 qualitative research software from QSR International Pty Ltd. to record the data coding and compute statistics. An initial coding scheme was developed based on the descriptive hypothesis, grounded in a literature search concerning the use of scientific evidence in the online health context and on observations from a preliminary exploration of the selected websites. Resolution of disagreements in the application of the initial coding scheme by the two coders was then used to refine the final version of the coding guide. We coded treatment attributes (i.e., treatment type, conventionality, and accessibility) and types of evidence present. The two coders worked independently and inter-coder reliability was calculated using Krippendorff’s alpha as a conservative measure of agreement. We resolved disagreements through discussion and consensus.

Codes for Treatment Attributes

Products and services featured on the websites were first characterized in terms of their attributes—properties of the treatment itself rather than of its representation on the website. Attribute codes were grouped under treatment type, conventionality, and accessibility, and each case was assigned one code from each group. Codes included the following (please see Di Pietro et al. 2011) for a complete list of assigned treatment attributes and their definitions):

(a) Treatment Type

Diet/supplement, pharmaceutical, medical/surgical, sensory, behavioral, educational, mind–body, body-based therapies, technological, and other (e.g., homeopathy, orthotics).

(b) Conventionality

Conventional (CON)—defined as treatments that have proven effectiveness based on peer-reviewed scientific research and strict drug regulations (American Academy of Pediatrics 2001; Angell and Kassirer 1998).

Complementary and alternative medicine (CAM)—defined as ‘‘a group of diverse medical and health care systems, practices, and products that are not presently considered to be part of conventional medicine’’ (The National Center for Complementary and Alternative Medicine, 2010), and ‘‘strategies that have not met the standards of clinical effectiveness, either through randomized controlled clinical trials or through the consensus of the biomedical community’’ (American Academy of Pediatrics 2001). To distinguish conventional from complementary or alternative treatments, the criteria developed by Hanson et al. (2007) in the ASD context was used and adapted to encompass the range of products and services we consider for CP and FASD. Thus, all treatments except for educational techniques, sensory therapies, and prescription drugs (not off-label) were considered as CAM.

(c) Accessibility

Direct-to-consumer (DTC), over-the-counter (OTC), or in clinical trial.

Codes for Types of Evidence

The central objective of the study was to assess evidence provided for treatment safety or efficacy. In the coding scheme we developed, evidence was broadly defined as any statement, reference, or rhetorical strategy used to demonstrate the truth of an explicit or implied assertion, ranging from full citations to peer-reviewed scientific articles to personal anecdotes. We also included in our evidence analysis codes for unsubstantiated claims about safety or efficacy, which could be interpreted as an absence of evidence in support of treatment claims. Types of evidence were thus grouped under three broad categories: references, other types of evidence, and unsubstantiated claims. Each case could receive multiple codes, but could receive any specific code only once, indicating the presence rather than the frequency of a particular type of evidence for each treatment. The codes are described below:

(a) References

Peer-reviewed: Reference to a peer-reviewed journal paper with full attribution (author, year, title, and journal) in the text, as a footnote, or in a bibliography. These were verified for accuracy (author, year, title, and journal) through Google, Google Scholar or the U.S. National Institutes of Health free digital archive of biomedical and life sciences journal literature, PubMed Central (http://www.ncbi.nlm.nih.gov/pubmed). References included for analysis were further verified for the relevance of their content, with two coders noting whether the reference supported the claims (correct), contradicted or did not corroborate the treatment claim (incorrect), or did not directly address the claim (irrelevant). References to books were not included in our analyses, and references that could not be obtained via any of these sources were counted but excluded from the analysis of relevance.

Non-peer reviewed: Full reference to a research study not published in a peer-reviewed form, including papers under review, mentioned in a conference presentation, or appearing in a non-peer reviewed journal such as a newsletter.

Partial: Incomplete citation. For example, the author and year of a publication are given in parentheses, but the article title and corresponding journal information is not provided.

(b) Other Types of Evidence

Advocacy-generated: Evidence collected by the host group (e.g., embedded surveys or forums on the website of the advocacy group) or based on its own publications.

Anecdotal: Evidence bearing on the experience of a single child, described either by a parent or practitioner.

Expert cited: Evidence drawing upon or referencing an expert or expert knowledge, such as that of a medical doctor, researcher, psychologist, educator or health provider, or an identifiable group of practitioners, a conference, panel, or clinic.

Graph: Illustration of efficacy or safety data of a treatment product or service.

(c) Unsubstantiated Claims

Background information: Statements about the mechanism of action of a treatment and/or the aetiology of a disorder that implies the need for a particular treatment but in the absence of any reference to scientific evidence to support the claim (e.g., ‘X causes autism and the main action of treatment Y is on X’).

Safety/efficacy: Statements about safety or efficacy were coded as unsubstantiated when they were provided without reference to a source (e.g., ‘Treatment X is safe’ or ‘Treatment X produces excellent results’).

Scientific allusion: Suggestion that a scientific study has taken place (e.g., “research has shown”) or use of a statistic that implies data collection (e.g., “50 % of parents report…”), but with no reference to author, date or other source for the claim.

Statistical Analysis

Coding results were exported from NVivo8 to Excel for descriptive analysis, consistent with the small cell counts. Standard error of the mean (SEM) was computed when appropriate.

Results

Cases

A total of 146 treatment products and services (i.e., cases) were identified across the nine websites. Some treatments were featured on multiple websites and for more than one disorder. We excluded a priori from the analysis products and services where scientific evidence for safety and effectiveness was judged to be unnecessary (i.e. wheelchairs) or that were not developed specifically for the treatment of the condition (i.e. recreational toys). Specifically, these cases included equipment for mobility (wheelchairs, walkers), personal hygiene products (i.e. diapers), children’s clothing, and descriptions of recreational services such as sports activities and summer camps, or recreational products such as books, toys, and music for entertainment, leaving 98 cases in the final analysis. The 98 cases were distributed across the three disorders as 60 cases on ASD websites, 37 on CP websites, and 15 on FASD websites.

Website Characteristics and Readability

All nine websites provided mission statements and seven of the nine provided disclaimers stating that website content may be subject to change, is for reference or informational purposes only, does not substitute for medical advice from a medical professional, or a combination of the three. Although all nine websites provided administrator contact information, none provided a list of website contributors with their credentials. Banner advertisements and brand name products were not featured on any of the websites; however corporate sponsor logos were prominent on donor acknowledgement pages and only four out of nine websites provided financial disclosures or a list of Executive Committee Board members.

The average Flesch-Kincaid readability score was: 38 (±4.0) for ASD websites; 46.8 (±1.7) for CP websites, and; 50.7 (±4.0) for FASD websites. The three websites that cited peer reviewed literature in support of treatment claims had scores of 29 (autism.com), 14 (autismsociety.org), and 36 (faslink.org), with an average of 26.3 (±1.2).

Treatment Attributes

(a) Treatment Type

Diets/supplements were the most featured types of treatments across the sample of websites (Table 1). By disorder, body-based therapies (22 %) and technological devices for improving mobility or aiding with communication (16 %) were the two most common types of products and services on the CP sites, while diets/supplements (33 %) and behavioural therapies (23 %) were most common on the ASD sites, and diets/supplements (67 %) and pharmaceuticals (20 %) were most common on FASD websites.

Table 1 Percentage of cases identified under each treatment type and further categorized by disorder, conventionality, and accessibility
Full size table

(b) Conventionality

Thirty-nine out of the 98 cases analyzed were categorized as conventional (CON) treatments. The remaining complementary and alternative (CAM) treatments (n = 59) mostly consisted of diets/supplements, off-label use of pharmaceutical drugs, and mind–body interventions (Table 1).

(c) Accessibility

Eighty-one out of the 98 cases analyzed were readily available OTC products and services, primarily diets/supplements, behavioural therapies, mind–body interventions and body-based therapies. Fifteen cases were for DTC treatments (all of which were pharmaceutical products) and two cases were only accessible via enrolment in clinical trials (stem cell therapy and electromyographic-type sensors).

Types of Evidence

Three categories of evidence codes emerged from the analysis of treatment claims: references, other types of evidence, and unsubstantiated claims. The first two categories consist of the seven codes for the different types of evidence we encountered, split according to the presence or absence of citations. The third category reflects the three different types of unsubstantiated claims—or lack of evidence—encountered.

(a) References

Three of the nine websites (autism.com, faslink.org, and autism-society.org) contained cases with references to peer reviewed publications, representing 13 % (n = 13/98) of all cases. Eleven percent of cases cited non-peer reviewed work, and 6 % provided incomplete citations (Fig. 1).

Fig. 1
figure 1

Percentage of cases that provided references to empirical studies (column 1), that contained other types of evidence (column 2), and that contained claims which were not supported by any type of evidence (column 3). Note that these codes are not exclusive; one case could be presented with multiple types of evidence

Full size image

A total of 109 peer-reviewed papers were referenced in relation to 13 of the 98 cases analysed. Through the institutional journal subscriptions of the University of British Columbia, we were able to obtain full copies of 94 out of the 109 (86 %) papers; 70 of these papers were cited across two autism sites (autism.com and autism-society.org), the remainder were cited on one FASD site (faslink.org). For the remaining 15 papers, we were only able to obtain their abstracts and thus excluded them from our analysis.

Of the 94 papers verified for accuracy, the majority of references to peer-reviewed research articles supported the statements that cited them (81 ± 3 %). The remaining references were either incorrectly cited (12.6 ± 4.7 %) or judged to be irrelevant (6.5 ± 3.5 %) to the citation context. For example, four references were cited to support the following statement: “Children with FASD have damaged intestines, kidneys and livers” (source: www.faslink.org), but three of the four cited studies were based on results from animal studies rather than from children with FASD and the accompanying text failed to discuss the use of animal models. These citations were thus coded as irrelevant to the claim. As an example of an incorrect citation, one autism website claimed that “…DMSA [i.e., succimer] increases copper excretion (Smith et al. 2000)” (source: www.autism.com). According to Smith et al. (2000) however, “multivariate analyses of a subset of elements (Cu, Fe, Mn, Zn)…indicated no significant effect of succimer treatment…”

(b) Other Types of Evidence

The percentage of cases that contained other kinds of rhetorical evidence strategies is shown in Fig. 1, column 2. Most frequent were references to experts (29 % of cases) and anecdotes (25 % of cases), followed by references to data generated by the advocacy organization itself (12 % of cases). Only one case (from a CP website) contained a picture of a graph to demonstrate the effectiveness of the treatment. When considering each disorder separately for “Other Types of Evidence”, cases located on ASD websites contained references to experts (35 % of cases) and advocacy generated data (22 % of cases) most often. For CP websites, expert references (24 % of cases) and anecdotes (22 % of cases) were most frequent. In contrast, we found a particularly high proportion of anecdotes (60 % of cases) on the FASD sites, with the second most frequent code in this category being references to experts (20 % of cases).

(c) Unsubstantiated Claims

Unsubstantiated claims about safety/efficacy (61 % of cases) or background information about the disorder (41 % of cases), and allusions to scientific studies (50 % of cases) dominated website content across all websites (Fig. 1, column 3), exceeding the frequency of full or partial references and other types of evidence described above. This was a consistent trend across disorders.

Types of Evidence as a Function of Type of Treatment

Peer-reviewed evidence was only used for diets/supplements (present for 34 % of cases of this treatment type), pharmaceuticals (present for 13 % of cases of this treatment type), and mind–body interventions (present for 4 % of cases of this treatment type). Notably, however, non peer-reviewed evidence was also frequent for pharmaceutical treatments (present for 53 % of cases of this treatment type). Unsubstantiated claims were found most frequently regardless of treatment type.

Types of Evidence as a Function of Conventionality

Figure 2 illustrates the percentage of conventional (CON) vs. complementary and alternative (CAM) treatments featured across the nine websites that were supported by at least one instance of each type of evidence. The types of references that were more common for CAM cases were peer reviewed evidence (present for 2 % of CON cases vs. 22 % of CAM cases) and partial references (4 % of CON cases vs. 8 % of CAM cases) whereas non-peer review evidence was more common for CON cases (13 % of CON cases vs. 9 % of CAM cases). With regard to “Other Types of Evidence”, advocacy generated (2 % of CON cases vs. 20 % of CAM cases), and anecdotal evidence (15 % of CON cases vs. 32 % of CAM cases) were more frequently used in support of CAM treatment claims while references to experts (33 % of CON vs. 27 % of CAM cases) were more frequent for CON cases. Unsubstantiated background information (30 % of CON cases and 48 % of CAM cases) and allusions to scientific studies (46 % of CON cases and 53 % of CAM cases) were also more common for CAM treatments while CON cases were surrounded more frequently by unsubstantiated claims about safety/efficacy (67 % of CON cases vs. 56 % of CAM cases). Note that unsubstantiated claims were found most frequently regardless of the conventionality of treatments (see Fig. 2).

Fig. 2
figure 2

Percentage of cases, categorized as either conventional (CON) or complementary and alternative medicines (CAM), supported by at least one instance of each form of evidence

Full size image

Types of Evidence as a Function of Treatment Accessibility

Non-peer reviewed evidence was present more frequently for direct-to-consumer (DTC) treatments (present for 28 % of DTC cases vs. 4 % of OTC cases) whilst the percentage of DTC and OTC cases that were supported by peer reviewed (11 % of DTC cases vs. 14 % of OTC cases) and partial references (6 % of DTC cases vs. 7 % of OTC cases) were similar. Although advocacy generated evidence was used more frequently for DTC cases (22 % of DTC cases vs. 11 % of OTC cases), anecdotes (17 % of DTC cases vs. 26 % of OTC cases) and expert references (22 % of DTC cases vs. 29 % of OTC cases) were slightly more common for OTC cases. Furthermore, more DTC than OTC treatments were accompanied by unsubstantiated claims about background information (67 % vs. 37 %) and use of scientific allusions (72 % vs. 46 %) with only a moderate increase in unsubstantiated claims about safety/efficacy (67 % of DTC cases vs. 60 % of OTC cases) observed for DTC cases. Claims about the two featured treatments that are currently in clinical trials—stem cells and electromyographic-type sensors—were evidenced by anecdotal stories in addition to unsubstantiated background information, unsubstantiated safety/efficacy claims, scientific allusions and in the case of electromyographic-type sensors, expert opinion. Again, we note the dominance of unsubstantiated claims, regardless of the accessibility of treatments, in relation to the other evidence categories.

Discussion

Our findings indicate that the majority of treatment claims on advocacy websites are unsubstantiated or unsupported by peer reviewed evidence. Instead, treatment claims are grounded in a variety of rhetorical evidence strategies for making claims appear scientifically credible. These strategies include scientific allusions, references to experts, and non-peer reviewed citations, or personal anecdotes by practitioners and data generated by advocacy groups. This pattern of evidence was largely invariant to the type, conventionality or accessibility of the treatments featured, and across the three disorders explored. The reporting of empirical studies published in peer-reviewed scientific journals was infrequent and largely used in support of claims about complementary and alternative treatments; those which apparently by definition tend to lack sufficient scientific evidence to establish safety and efficacy. In the cases where references to peer-reviewed scientific publications were invoked to back up treatment claims, 20 % of citations failed to support the relevant claim or were irrelevant to the citation context. On informal examination of the incorrect citations, errors appeared to arise from reliance on the abstract in isolation from the full text of the article as well as from the direct misinterpretation of research findings reported in the cited article. This concurs with more general studies on the accuracy of scientific information online and is consistent with an emerging consensus that health information over the Internet is typically not based on scientific peer-reviewed evidence (Impicciatore et al. 1997; Pandolfini et al. 2000; Pereira and Bruera 1998; Silberg et al. 1997). Allen et al. (1999) for example, found that up to 34 % of websites on the theory of evolution in a sample of 59 were inaccurate or misleading relative to currently accepted scientific opinion, and as many as 48 % of websites that purvey scientific information contained no peer-reviewed references.

Peer review is the standard currency of communication between scientists and remains a high priority requirement in most academic disciplines, especially in biomedical research (Rowland 2002). However, only three of the major advocacy websites we reviewed made use of peer reviewed publications as a form of supporting evidence for claims about treatments. Of course, the absence of scientific citations does not necessarily signify that a given claim is incorrect or misleading. Some advocacy organizations may choose not to cite research articles when providing health information on their websites in order to streamline or simplify health messages, or because they do not expect the user to locate and read the cited research studies. Indeed, 52 % of the cited publications in our sample required institutional subscriptions to medical journals or out of pocket payment. Thus, it is unclear whether citations of peer-reviewed papers or links to them ought to serve the same vital function as the currency of communication between advocacy organisations and those they are intended to serve.

Discussion about the utility of providing citations to scientific literature on the Internet must also be framed within debates over the rise of the expert ‘e-patient’, which are often polarised between assertions that more information empowers patients whose increasing expertise should be welcomed by doctors, and arguments that the proliferation of online health information leads to confusion, time-wasting and ultimately to poorly evaluated treatment decisions (Kopelson 2009). In a survey conducted by the Health on the Net Foundation, Boyer et al. (2002) reported that two thirds of patients visit medical sites that contain more complex information because they feel that the information to which they otherwise have access is too basic. In this regard, our sample of advocacy websites may serve this cohort well; the readability scores of the websites were well outside the range of 60-70 that is considered to be ideal for the majority of Internet users to be able to understand online content and for popular newspapers and magazines aimed at a general public audience (Kunz and Osborne 2010). It is encouraging that other public health researchers (Bouche and Migeot 2008; Boyer et al. 2002; Kunz and Osborne 2010; Stevenson et al. 2007) have found that patients will often later return to their doctors to clarify information they have mined independently; i.e., that the Internet is used to supplement rather than replace medical advice. Thus, to accommodate varying levels of knowledge and the needs of readers such as parents who may be highly knowledgeable about a neurodevelopmental disorder, health messages ought to be presented in easy to understand language while, wherever possible, supported by complete and relevant citations to scientific literature with links to those papers or credible websites that can provide more in-depth scientific information for those who desire it.

The Art of Persuasion

The presence of scientific material on websites has been reported to be an important indicator of authority and credibility by Internet users (Eysenbach and Köhler 2002). On advocacy websites, authority is often unclear as content is authored by a confusing and often opaque mixture of professionals with medical credentials, website content managers, and variously sourced personal anecdotes or testimonies (Di Pietro et al. 2011). Moreover, references to peer reviewed publications in support of treatment claims were especially prevalent when there was an absence of clear medical guidelines, such as for autism where the aetiology of the disorder remains uncertain (Wikgren 2001). Indeed, the majority of the citations to scientific literature were made on ASD websites in relation to claims made about CAM treatments (see Fig. 2) that often lack extensive study to determine safety and efficacy (American Academy of Pediatrics 2001). In this context, references to scientific literature may be used to boost credibility and legitimize claims and to persuade readers that an argument is valid (Gilbert 1977). It is also critical to acknowledge that scientific research does not operate in isolation from society, and that study design or interpretation can be shaped in order to influence policy (Hess 2007; Kraemer and Gostin 2009). Hence, significant questions arise about the objects of persuasion themselves, as well as about the ends to which citations and other appeals to expertise are used.

Many scholars have suggested that the lack of regulation and editorial control over online health information has resulted in the selective disclosure of evidence to influence sales of healthcare products and services (Shepperd and Charnock 2002; Sonnenberg 1997; Wyatt 1997), or to increase support for a particular treatment or healthcare policy (Perez 2010). In cases where a treatment is controversial, has unwanted side-effects, or where the reader is unlikely to have background knowledge about competing claims, the selective disclosure of information threatens to undermine the role of online health information in facilitating informed decision-making. Although we did not directly assess the comprehensiveness of citations surrounding each treatment claim, we observed several instances whereby selective reporting of scientific findings was used to advocate for controversial treatments known to have serious side-effects.

For instance, two of the three autism websites we surveyed (autism-society.org and autism.com) encouraged the use of chelation therapy, a CAM treatment deemed ineffective and unsafe by many scholars and pediatricians (Doja and Roberts 2006). Traditionally, chelation therapies have been used to treat heavy metal poisoning through the removal of excess heavy metals such as lead and mercury from the body, but have not been approved for the treatment of autism. Reports of chelation-related childhood deaths (Brown et al. 2006) amidst FDA warnings (Voelker 2010) that OTC chelation products should not be used on children, and the halting of clinical trials in 2008 due to safety concerns and a lack of observed efficacy (Mitka 2008; Stokstad 2008), were not mentioned on either website encouraging its use. A future study analysing how treatment information in our sample of websites compares against published recommendations and accepted standards of care would be useful in determining the comprehensiveness of content within each website as well as to identify content biases. Although we did not assess how treatment information is perceived by parents who visit the autism websites we studied or how this relates to the perceived credibility of the advocacy organisations that produce the websites, it is noteworthy that chelation therapy remains one of the most popular CAM treatments for children with an ASD (Rossignol 2009).

Despite the difficulty of determining the appropriate level of scientific information to present on advocacy websites, in order to prevent potentially harmful treatment recommendations, we nonetheless urge advocacy organizations to collaborate with a panel of medical experts to review and certify websites for quality information and ethical conduct, addressing the misleading information practices we have observed. One organization that offers this service for medical and health websites on a voluntary basis is the swiss-based health information on the net foundation (HON). Alternatively, consumers of health information may also benefit from using free online tools such as the DISCERN instrument (available at: www.discern.org.uk/index.php) which is a brief questionnaire that assesses the quality of written information on treatment choices for a health problem.

Communicating Controversial Science

Many health communication researchers have argued that comprehensiveness of information, markers of which include discussion of negative side effects or constraints on efficacy alongside discussion of positive effects, is an important indicator of website credibility, as is the presentation of competing opinions and contradictory research findings (Dutta-Bergman 2004; Eysenbach et al. 2002). It is also important to recognize, however, that invoking notions of balance in communication of controversial science is not unproblematic. Presenting scientific evidence for and against a particular treatment in order to avoid bias can give the potentially misleading impression that each has equal weight (Clarke 2008; Mooney 2004). Moreover, the framing of comprehensiveness in terms of scientific evidence can also neglect the experience of practitioners and parents, which may legitimately speak to the potential efficacy of treatments not yet represented in controlled studies (Golnik and Ireland 2009) and would arguably be akin to policing or limiting access to information that may in fact be relevant (Lewis 2006; Silberg et al. 1997). Hence, we suggest that treatment recommendations made on advocacy websites be based on principles of evidence-based medicine while integrating clinical expertise, the experience of parents, and the best available evidence for treatments that lack comprehensive study. Definitions of best available evidence must take into consideration the hierarchy of study types in evidence-based medicine, comprised of randomized controlled trials at the top, followed by well-designed but nonrandomized studies including both controlled (case–control) and uncontrolled (cohort) designs, and finally descriptive case reports from physicians and parents (Levy and Hyman 2008). Several recent reviews of the scientific evidence surrounding CAM therapies have assigned grades to treatments reflecting the strength of evidence used to support or refute their use along these lines (Akins et al. 2010; Levy and Hyman 2008; Rossignol 2009). By using this type of information, advocacy groups could assist stakeholders in evaluating treatments by categorizing them according to the best available evidence and clearly indicating the forms of this evidence, including the experience of practitioners and parents.

Strengths and Limitations

This work highlights critical problems and seemingly paradoxical practices in how evidence-based treatment information is disseminated over the Internet by some of the most influential advocacy organizations for three neurodevelopmental disorders. The findings are limited, however, to the singular website voices of the major organizations they represent and, although heavily accessed, the sites represent only a portion of those in existence for the disorders of interest and in a single snapshot of time. Caution should thus be used in extrapolating these findings to advocacy websites of neurodevelopmental disorders in general, and future studies could explicitly assess the broader prevalence of the evidence strategies identified here on health websites. We also selected websites based on Internet traffic ranking scores rather than ranking from search engine results, which is another common approach to such investigations (Eysenbach and Köhler 2002; Eysenbach et al. 2002). The sample is therefore more representative of site popularity rather than search engine optimization strategies. Another caveat to the present study is that the quality of information on the websites is not interpreted within the larger context of information in other media. Studies assessing information in traditional media such as broadcast television, magazines, and newspapers also frequently report a high prevalence of inaccurate or incomplete scientific information (Eysenbach et al. 2002). Our findings cannot directly address whether poor quality health information on the Internet should be understood as a new form of misinformation or a variation of an already endemic phenomenon (Coiera 1998).

Conclusions

While we concur with Tuffrey and Finlay (2002), Wainstein et al. (2006) and others that the physician should remain the primary source of medical information for parents, we assert that physicians, along with parents who are also gaining new forms of expertise from online sources, would benefit from more reliable and transparent online repositories of health information (Bouche and Migeot 2008; Silberg et al. 1997; van Woerkum 2003). The websites of advocacy organizations are well situated to contribute to this service, and thus to help stakeholders evaluate highly complex health information to make appropriate treatment decisions for children with neurodevelopmental disorders. We have identified ways in which advocacy websites could better meet this goal while taking into consideration parental expertise and best available evidence for unproven therapies. Toward this end, advocacy organizations must seize the opportunity to review their approaches to disseminating treatment information, and consider the importance of clearly indicating the levels and definitions of evidence being provided in support of treatment claims. Through further collaborations with experts, significant benefits for children with neurodevelopmental disorders would ensue.