Introduction

Cognitive impairment is a common manifestation of multiple sclerosis (MS), affecting 40–65% of patients irrespective of disease subtype. Cognitive compromise can be present early in the course of the disease and impacts patient’s fulfillment of role in the family, social and work life. Over the last two decades, interventions for cognitive compromise in MS have been published.

This study examines the methodological challenges of randomized controlled trials (RCTs) on cognitive symptoms in MS and discusses specific problems related to rehabilitation trials. Clinical aspects of these interventions are discussed in other articles of this issue.

A search of the following databases was carried out in March 2010: Medline (Pubmed), embase, cinahl, psycINFO, Cochrane Collaboration’s trial register, and Database of Reviews of Effectiveness (DARE). The following five methodological criteria were considered [1]: the target population (eligibility criteria stated; baseline comparability); allocation (random and concealed); masking (subjects and assessors); outcomes (explicit primary outcome stated; sample size calculation reported; adequate follow-up); and analysis (intention to treat; between-group comparisons). Furthermore, description of the intervention in nonpharmacological trials was considered.

Results

A total of 12 RCTs, published between 1996 and 2010, were found. Of these, five were on pharmacological interventions (three symptomatic treatments and two disease-modifying) (Table 1), while seven were on rehabilitation (one on compensatory strategy [7] and six on PC-based retraining [8–12]) (Table 2).

Table 1 Published RCTs assessing the efficacy of pharmacological interventions
Full size table
Table 2 Published RCTs assessing the efficacy of cognitive retraining
Full size table

The target population

Eligibility criteria depended on the type of intervention. For disease-modifying drugs, trials should target cognitively preserved MS patients and follow them for a sufficient time to assess the treatment effects on prevention of cognitive compromise. The two published RCTs on disease-modifying treatment provide exploratory data on RCTs designed for different primary outcomes [2, 3]. In the interferon β 1a trial, patients’ cognitive functioning at baseline was quite heterogeneous [3]. In the glatiramer acetate trial, baseline neuropsychological assessment was within the normal range and remained stable in both arms during the 2-year follow-up. Trials on symptomatic drugs [4–6] and those on rehabilitation [7–13] included mildly to severely compromised patients, mostly on the basis of neuropsychological assessment. In all the RCTs considered, eligibility criteria were specified and, despite the limited size of most of these studies, comparability at baseline was fine in all but one instance [6].

Allocation

Three of the included studies (25%) had a randomization method that was considered inadequate (alternate assignment) [11, 12] or not reported [5, 9, 13]. Concealed allocation was not specified in two pharmacological (40%) [4, 5] and three rehabilitation studies (43%) [8, 12, 13].

Concealment

Four drug trials (80%) and two rehabilitation trials (29%) [10, 11] were double-blind studies [2–4, 6]. It should be noted that having the patient blind is often unfeasible in rehabilitation trials; however, a sham intervention or a low-intensity intervention can be conceived for PC-based retraining, allowing patient’s masking [10, 11].

Outcomes

Only one drug trial (20%) [6] and one rehabilitation trial (14%) [10] specified the primary outcome measure and provided the sample size justification. A practice effect was almost universally reported. The length of follow-up depends on the chosen study population, the main study outcomes, as well as the type and predicted efficacy of the study intervention. It was considered to be adequate in three drug trials (60%) [2, 3, 6] and five rehabilitation trials (71%) [7–11].

Analysis

All the published studies correctly performed between-arm comparisons, but intention to treat analysis was conducted only in two drug trials (40%) [2, 6] and three rehabilitation trials (43%) [8, 10, 12]. Considering the interferon β 1a trial [3], attrition was very high, exceeding 40%.

Description of the intervention in rehabilitation trials

The CONSORT statement on reporting clinical trials recommends that “precise details of the interventions intended for each group and how and when they were actually administered” should always be reported [14]. Adherence to this recommendation is particularly important for RCTs on rehabilitation interventions: full description of the specific rehabilitation technique or theory on which the intervention is based, the setting, intensity, duration of the retraining program, and the required experience and training of the therapist is essential to document the intervention, and make it understandable and replicable. Another reason why rehabilitation interventions should be described in detail is that their potential value can only then be appreciated. It must be clear whether the trial is pragmatic (focussing on measuring benefit of the intervention when introduced into current clinical practice) or primarily exploratory (aiming to measure benefit under ideal conditions or determine which component of the intervention is efficacious). In drug RCTs, the experimental intervention is compared to placebo or an established drug. The comparator in rehabilitation RCTs is much more problematic. An adequate sham intervention can be devised only in selected cases. Waiting list patients are not a good solution, since the patients know they are receiving/not receiving the intervention, and this can result in bias. A more acceptable option is to use a low-intensity intervention as comparator, although this is likely to reduce the study power and require recruitment of more patients. In view of these issues, the CONSORT Group recently developed an extension of the CONSORT statement for nonpharmacological treatments [15]. Furthermore, the Medical Research Council has produced guidance on how to develop and evaluate such interventions (Fig. 1) [16, 17].

Fig. 1
figure 1

Phases in the development of a clinical trial involving a complex intervention (adapted from the UK Medical Research Council framework for the development and evaluation of complex interventions by RCT) [16]

Full size image

Conclusions

The methodological quality of an RCT has been defined as the likelihood of the trial design generating unbiased results that are sufficiently precise and allow application in clinical practice [1]. Evidence for the effectiveness of interventions to help cognitively compromised MS people is scarce. Twelve RCTs [2–13] have been published since 1993 and their quality is overall insufficient. Pitfalls of RCTs on interventions for cognitive compromise in MS include difficulty in choosing the appropriate study population and outcome measure, and a limited study size. It is also noteworthy that the most recent trials are not of high quality. Other problems are specific to rehabilitation studies and arise because: these interventions are multifaceted, complex, and difficult to define and quantify; the training and skills of clinicians administering the intervention can vary; and patient blinding is difficult and often impossible. Furthermore, the resources necessary to conduct rehabilitation trials according to good clinical practice and high quality standards are generally lacking. The recently updated CONSORT statement and MRC guideline should improve clarity, completeness and transparency of reporting trial findings; moreover, they should help researchers in designing and conducting future studies [14–17].