Introduction

Attention-deficit hyperactivity disorder (ADHD), characterized by inattention, hyperactivity and impulsivity, is one of the most prevalent childhood psychiatric disorders [4]. Children with ADHD often show social deficits [44], comorbid ODD or CD [6], or anxiety disorders [31]. Parents of children with ADHD often experience higher levels of parenting stress than parents of normal control children [25]. Results about children’s self-esteem are inconclusive. Children with ADHD are frequently described as having poor self-esteem [42], but inflated self-esteem is also reported [24].

Several studies have shown the effectiveness of methylphenidate for the reduction of ADHD and related symptoms [31]. Research into the additive effects of behavior therapy to methylphenidate has shown mixed results. Klein and Abikoff [27] have demonstrated incremental results for adding behavior therapy to methylphenidate in terms of the reductions of ADHD symptoms, but other studies have shown no such additive benefits [2, 31].

The largest treatment study to date is the MTA study [31]. The MTA study compared four treatment arms. First, medication management (methylphenidate), second, intense multimodal behavior therapy, third, a combination of both treatments and, finally, standard community care, in which almost 70% of the children received stimulant treatment. All conditions showed significant reductions in ADHD and related symptoms after treatment. Especially, the medication management and combined condition showed significant improvements in ADHD and related symptoms compared to standard community care and multimodal behavioral treatment. Surprisingly, the medication management and combined condition did not differ significantly.

Based on these results, the MTA cooperative group [31] concluded: “If one provides carefully monitored medication treatment similar to that used in this study as the first line of treatment, our results suggest that many treated children may not require intense behavioral interventions.” However, several MTA results indicate additional value of behavioral therapy, especially for ADHD-related symptoms. First, alternative outcome analyses, using composite measures, showed significant differences in effect sizes between the medication management and combined condition, in favor of the latter [11]. Second, children in the combined condition received lower doses of medication at post-test than those in the medication management condition [31]. Third, parents preferred both conditions including behavioral treatments above the medication management condition [31]. Also, the combined condition showed more excellent responders than the other conditions [32]. Further, the combined condition yielded the most favorable outcomes on 12 of the 19 dependent measures [31], although these differences were not statistically significant. Finally, for oppositional and internalizing behavior, social skills, and parent–child relations the combined condition was superior to behavioral treatment condition, while the medication management condition was not [31].

Several authors have commented on the design and methods used in the MTA study [e.g., 9, 36] and question the lack of difference between the medication management and combined conditions. One of the comments was that active medication was compared with faded behavior therapy [e.g., 36], although a recent study shows identical results during active behavior therapy [5]. Also, the benefit of the study was questioned because psychosocial treatments were too extensive for use in standard community care [9].

Both results and commentaries of the MTA study urge a need of replication in a real-life outpatient clinical setting and different cultural context. The present study investigates the additional value of short-term intensive multimodal behavior therapy to optimally titrated methylphenidate. Active medication was compared with active behavior therapy plus active medication. Treatment arms were designed to be applicable in standard outpatient clinics. Combined treatment was expected to outperform optimally titrated methylphenidate for ADHD- and related symptoms.

Method

Participants and procedure

Fifty children aged between 8 and 12 years (M = 9.9, SD = 1.2) participated. School psychologists, pediatricians or general practitioners referred children to five different child psychiatric outpatient clinics in the Netherlands for diagnosis and treatment of ADHD. Inclusion criteria for participation were (1) a DSM-IV diagnosis of ADHD established with the parent version of the Diagnostic Interview Schedule for children (DISC-IV) [13, 38] and (2) an estimated full scale IQ of 75 or above based on a short version of the Wechsler Intelligence Scale for Children-Revised (WISC-R) [45]. Exclusion criteria were inadequate mastering of the Dutch language by the child or both parents, and a history of methylphenidate use. Before participation children gave their verbal and parents their written informed consent. Shortly before the beginning of the treatment phase, participants were randomly assigned to one of the two treatment conditions. One week before the beginning of treatment, the pretest was conducted: parents and teachers completed questionnaires and children were assessed at the outpatient clinic. Within a week following the last session of the behavior treatment, approximately 11 weeks after pretest, participants in both treatment arms received the post-test.

Treatment

Medication management

A four-week pseudo randomized multiple blind placebo controlled crossover medication design, as described for the MTA study [15], was used for individual methylphenidate dose titration. In this titration trial 5, 10, and 20 mg of methylphenidate and placebo were administered in a pseudo random order twice daily at breakfast (around 7.30 a.m.) and at lunch (around 12.30 p.m.). All children weighed above 22 kg, thus the highest dose never exceeded 0.9 mg per kg of the body weight. Coding of doses was kept at a hospital pharmacy and in case of immediate side effects the pharmacy could be reached to unblind the coding.

The effects of methylphenidate were monitored daily at school and home by teachers and parents, respectively, using rating scales, measuring symptoms of ADHD, ODD, and impairment [see 15]. Similarly to the MTA, parents and teachers completed the Conners, Loney and Milich (CLAM) [29] scale to assess ADHD symptoms and the McBurnett, Swanson, Kotkin, Agler, Flynn & Pelham (McSKAMP) [41] to assess impairment. Additionally, parents completed the ADHD subscales of the Disruptive Behavior Disorder Rating Scale (DBDRS) [37]. Parents observed behavior of the child during weekends and Wednesday afternoons, teachers during all weekdays except Wednesday afternoon. To assess side effects, the MTA Side Effect Rating Scale [15] was utilized. Parents and teachers received extensive verbal and written information about medication administration and rating of the child’s behavior. Parents met on a weekly basis with a research assistant to assure medication was taken, the behavior was rated as instructed, and to receive medication for the coming week. Teachers received weekly telephone calls to assure the behavior of the child was rated as instructed and medication was taken at lunch.

All children started with a lead-in phase of 4 days to assess side effects, starting with placebo, followed by 5, 10, and finally 20 mg of methylphenidate, twice a day. None of the children showed significant side effects. Then, 4 weeks of medication titration started. After these 4 weeks, five independent clinical raters, using a standardized manual [as described in 16], blindly reviewed mean dose–response graphs. During this evaluation period of 1 week, children were medication free. By consensus each child’s best dose was selected. Again using a standardized manual, the child was classified as responder, placebo-responder or non-responder to methylphenidate. Low symptoms scores and no room for improvement on all doses indicated a placebo-response. A non-responder was defined when there was room for improvement on all four doses and no significant difference between these doses. A child was classified as a responder when there was a significant difference between at least two doses. Following the consensus procedure, a child psychiatrist subscribed the selected best dose (if not placebo) to the child.

Of the 45 titrated children (see ‘‘Results’’ section for drop-out rates), one did not complete titration due to side effects, however, post-test scores were collected. Of the remaining 44 children, 25 (59%) were assigned to an individually optimally titrated dose of methylphenidate, with an average individual dose of 20.8 mg/day (SD = 10.18). The remaining 19 children were classified as placebo-responders. With respect to the high level of placebo-responders there may be an association between responder status and baseline severity of ADHD. This issue will be detailed in the results section. Further, placebo-response may reflect a compression of ratings that is an artifact of rating methodology. Without objective measures a real placebo-response and rating artifacts are difficult to differentiate (Pelham in [15]). Therefore, manualized instructions for psychiatrists included the option of prescribing 5 mg twice daily for placebo-responders, in case of recurring ADHD symptoms during the medication-free week. Using this procedure, eight children were prescribed 5 mg twice a day. During the remaining 5 weeks of treatment subjects were specifically instructed to remain on the assigned dose, however, in case severe symptoms or side effects emerged, the child psychiatrist was consulted and dose adjustments could be made. In seven of the 44 titrated children (15%) the dose was changed from titration to post-test. In three children the dose was lowered, due to emerging side effects. In four children the dose was raised, because of re-emerging ADHD symptoms.

Multimodal behavior therapy

The multimodal behavior therapy integrated family-based and school-based interventions with cognitive-behavior therapy of the child. The multimodal behavior therapy started in the first week of medication titration. Treatment selection was based on empirical efficacy in reducing ADHD or related symptoms and applicability in outpatient settings.

Parent behavior therapy.

The parent behavior therapy consisted of 10 weekly sessions of 90 min group therapy for four or five parent couples, provided by two therapists. The parent training was based on Barkley’s training: “Defiant children: A clinicians manual for parent training” [8]. Components included psycho-education on ADHD, structuring the environment, practicing positive attending skills, giving effective behavioral commands to the child, contingency management skills, and knowledge of parenting techniques such as time-out.

Teacher behavioral training.

The teacher training was based on the teachers training manual by Pelham: “Attention deficit hyperactivity disorder, diagnosis, nature, etiology and treatment” [35]. The teacher training consisted of a two-hour workshop, in which psycho-education on ADHD, structuring the classroom environment, implementing contingency management in the classroom, and a daily report card (DRC) system [e.g., 22] were explained to the teacher. The DRC is a classroom contingency management technique where parents provide rewards based on the teacher’s ratings of the child’s classroom behavior for that day. Teachers received an extensive handout of the training and weekly additional contacts by phone, during which the implementation of behavioral techniques was monitored, the use of the DRC was evaluated, and possible problems were discussed.

Child cognitive-behavior therapy.

The child cognitive-behavior therapy consisted of 10 weekly 75-min group sessions for four or five children, provided by two therapists. The program used was adapted from Kendall and Braswell [26]. Cognitive-behavioral techniques consisted of the children acquiring problem-solving techniques. Relaxation and contingency management techniques were also used. Training comprised modeling by the therapists, role-playing, and guided practice. Academic and interpersonal problems were extensively covered, to ensure generalization across the wide range of problem behaviors. In addition, a token reinforcement system was used during the group sessions.

Multimodal component.

The treatment impact was maximized by simultaneously implementing the 10-week child cognitive-behavioral therapy, parent behavioral therapy, and the teacher training. Using the DRC system further enhanced integration of home and school interventions. Since it has been suggested that cognitive-behavioral therapy is only effective if embedded in a multi-modal approach [35], parents were taught to help the child to apply the problem-solving steps at home. Also on a weekly basis, parents watched video-recordings of the child treatment sessions.

Treatment integrity.

Manualized treatments were used, but individualized specific targets were defined for each child. After each session, therapists completed a treatment integrity checklist, determining whether all manualized topics had been covered. Based on the therapist’s average ratings, 88% of the parent and child training manualized elements were extensively covered. During the 10 weeks of therapy, three sessions were randomly videotaped and reviewed by first author (S. v.d. O), using the checklists, it was revealed that 85% of the manual topics were covered both in the parent and child training.

Therapists were experienced child-psychologists, familiar with behavioral treatment of ADHD. All eight therapists were extensively trained in the protocol by a senior behavior therapist. One of the therapists provided both the parent and the child treatment for a particular subject. Further, during the 10 weeks of multimodal therapy, supervision sessions were conducted biweekly with a senior child behavior therapist (second author P.P.). Treatment fidelity of the medication titration was assured by daily medication compliance forms filled out by teachers and parents (see medication management section) and weekly empty blister counts. After titration, parents and teachers received weekly phone calls to monitor treatment fidelity.

Selection measures

Diagnostic Interview Schedule for Children—Fourth edition (DISC-IV)

The ADHD, ODD, and CD sections of the DISC-IV parent version [13, 38] were administered by a trained clinical child-psychologist. The DISC-IV, a structured diagnostic interview, generates DSM-IV diagnoses. Adequate reliability and validity have been reported for precursors of the DISC-IV [40]. Using the DISC-IV, 31 children met DSM-IV criteria for ADHD combined subtype, 16 for inattentive subtype and three children met criteria for the hyperactive/impulsive subtype. Twenty-three children showed comorbidity with ODD, and of these children two met criteria of an additional CD diagnosis.

Revised Wechsler Intelligence Scale for Children (WISC-R)

Intelligence was assessed with four subtests of the WISC-R: Vocabulary, Arithmetic, Block Design, and Picture Arrangement. Estimation of the IQ as obtained by these four subtests correlates between r = 0.93 and r = 0.95 with Full Scale IQ [18]. Only children with an estimated IQ of 75 or above were included.

Outcome measures

Disruptive Behavior Disorder Rating Scale (DBDRS)

The DBDRS [37] assesses DSM-IV disruptive behavior disorder symptoms. The DBDRS consists of 42 items and contains four subscales: Inattention (9 items), Hyperactivity/Impulsivity (9 items), ODD (8 items), and CD (16 items). Parents and teachers rated the child’s behavior on a 4-point Likert scale ranging from 0 (not at all) to 3 (very much). The Dutch translation has adequate reliability (alpha range = 0.88–0.94) [34]. For this study the Inattention and Hyperactivity/Impulsivity subscale scores were combined into one ADHD score. Higher scores indicate more severe symptoms.

Social Skills Rating Scale (SSRS)

Both the parent and the teacher version of the Social Skills Rating Scales [17] were used to assess social skills. The teacher version (SSRS-T) consists of 30 items, the parent version (SSRS-P) of 38 items. The SSRS-T and SSRS-P are rated on a 3-point Likert scale ranging from 0 (never) to 2 (often). The total score served as the outcome measure. The SSRS-T and SSRS-P have high reliability in a Dutch ADHD sample (alpha 0.87 and 0.88, respectively) [44]. Higher scores indicate better social skills.

Parenting Stress Index (PSI)

A Dutch short version of the PSI [1] was used, containing 25 items tapping “parenting stress.” Items are answered on a 6-point Likert scale, ranging from 1 (do not agree) to 6 (totally agree). This short version has excellent reliability (alpha range = 0.92–0.95) [12]. Higher scores indicate more parenting stress.

Self-Perception Profile for Children (SPPC)

The subscale Global Self Worth (GSW) of the Dutch version of the SPPC [19, 43] was used to assess self-worth. In this self-report measure, the child is first asked to identify which of two behavioral possibilities applies most to him or her, and then asked whether this is only sort of true or really true for him or her. The six items are scored from 1 to 4, with higher scores indicating higher self-worth. The GSW subscale shows adequate reliability in a Dutch sample (alpha = 0.74) [43].

State Trait Anxiety Inventory for Children (STAIC)

Anxiety was assessed with the trait anxiety scale of the Dutch version of the State Trait Anxiety Inventory for Children (STAIC) [39]. The child rates each of the 20 items on a 3-point Likert scale ranging from 1 (almost never) to 2 (often). Reliability is high in a Dutch sample (alpha = 0.80) [7]. Higher scores indicate higher levels of anxiety.

Statistical analyses

The current sample size was sufficient to detect large differences with a power level of 0.79 [10]. This study was designed to detect large group differences, since only large differences are clinically meaningful and important to clinical decision-making. First, analyses of variance tested for group differences at pretreatment. Chi-square tests were used for categorical variables. Also, association between placebo-responder status and severity of baseline ADHD symptoms was assessed. Then, to assess differences between treatment groups in medication dose directly after titration and at post-test, chi-square analyses were conducted. Finally, treatments effects were examined with 2 (group: medication or combined condition) by 2 (time: pre–post-test) repeated measures ANOVAs. Effect sizes (η2) are reported for all analyses. Using Cohen’s [10] guidelines for η2 effect sizes, effect sizes smaller than 0.06 were considered small, effect sizes between 0.06 and 0.14 were considered medium, and effect sizes above 0.14 were considered large.

Results

Of the 50 children included in the study, 23 children were randomized to the medication management and 27 to the combined treatment condition (Fig. 1). One of these 50 children refused to participate after randomization to the medication management condition. Further, one child did not show up at post-test and two parent couples dropped out, because of marital problems and inconvenient time of treatment, respectively. Criterion for treatment attendance in the combined condition was attending at least 75% of all treatment sessions. One child was omitted from the analyses because this criterion was not met. Results are presented for 21 children completing all assessments in the medication management condition and 24 children with complete data assigned to the combined condition.

Fig. 1
figure 1

Flow-chart describing the flow of patients to the two treatment conditions and drop-outs

Full size image

Of these 45 children, 40 (89%) children were from Caucasian parents, 1 child (2%) was from Caribbean parents, and 4 children (9%) from mixed origin, with one of both parents from Asian, Turkish, or Moroccan origin. The primary caretaker completed the rating scales, which was the mother for the majority of the participants. Mean treatment attendance in the combined condition was 88.6%. At pretest teachers rated 95%, and parents 85% of the children at least 1.5 SD above the normal-control mean of the DBDRS. Teacher rated an additional 5% and parents an additional 10% of the children 1 SD above this mean. Further, baseline DBDRS scores of responders and placebo-responders were compared. On the teacher rated DBDRS placebo-responders scored significantly lower than responders (F (1,41) = 3.67, P ≤ 0.05), the parent-rated DBDRS showed no difference between responders and placebo-responders (F (1,41) = 12.20, P ≥ 0.05). All but two of the children had an IQ of 80 or higher.

Baseline equivalence

One-way ANOVAs and chi-square analyses showed no significant differences between conditions in terms of baseline demographic characteristics (Table 1). Further, one-way ANOVAs showed no significant group differences and low effect sizes for parent and teacher rated pretest outcome measures (η2 range = 0.00–0.02).

Table 1 Demographic characteristics of children randomized to the medication management condition (Med) and the combined condition (Med + Beh)
Full size table

Outcome

Chi-Square analyses were conducted to compare the difference between titration results of both treatment groups (χ2 = 1.40, df = 4, P ≥ 0.05). Also, at post-test the treatment group did not differ on dose of methylphenidate (χ2 = 0.76, df = 3, P ≥ 0.05).

Table 2 displays means and standard deviations of both conditions at pre and posttest and results of the ANOVAs.

Table 2 Pre and post-test scores in the medication management (Med) and the combined condition (Med + Beh)
Full size table

The repeated measure ANOVAs showed a consistent pattern. On all measures significant improvements were found from pre to post-test. Large effect sizes were noted with one exception: the SPPC, measuring self worth showed a medium effect size. Further, there was no significant time by group interactions. These analyses show that both groups did not differ in their improvement from pre to post-test. Also, all time by group effect sizes were small (η2 range = 0.00–0.02).

Discussion

This study examined differences between the efficacy of an optimal dose of methylphenidate combined with brief, intensive multimodal behavior therapy and an optimal dose of methylphenidate alone. Active medication was compared with active behavior therapy. In the MTA study the treatments were designed to have maximum impact, and therefore were criticized to be unsuitable for clinical practice [36]. In this study, treatment arms were designed to be applicable in standard outpatient mental health clinics. Finally, all participating children were medication naïve, whereas in most previous treatment outcome studies children were not [28, 31].

In this study, individual titrated medication improved ADHD symptoms and social skills, anxiety, oppositional and conduct behavior, self-esteem, and parenting stress. Large effect sizes were found for all outcome measures, except for child rated self-esteem, which showed a medium effect size. However, no evidence was found for the additive effect of multimodal behavior therapy to stimulant treatment.

Previous studies also report no additive effects of long-term multimodal behavior therapy on the improvement of ADHD symptoms [2, 31], social skills [3, 31], oppositional and conduct disorder symptoms [2, 31], and self-worth of the child [20], nor on anxiety [31]. The MTA study showed that parenting stress, parental cognitions, low self-esteem of the mothers and low parenting efficacy in fathers were associated with worse child treatment outcome [23]. However, no differences between treatment conditions in outcomes (including parenting stress) were observed when the MTA treatment conditions were stratified according to the presence of negative/ineffective parenting practices [21, 46]. This study also showed no difference between the two treatment groups on parenting stress at outcome. However, our study did not assess other parenting variables.

Consistent with Klein et alal. [28], both treatment conditions did not differ in the titrated dose of methylphenidate directly at post-test. However, the MTA study [31] reported a significant lower dose for combined strategies at 14 months. Possibly, this is related to the short duration of our treatment that allows less time for medication adjustments.

In this study the behavior therapy was adapted to and used in a clinical setting. Utilization of an extensive multimodal behavior therapy of almost a year, such as in the MTA study, showed no different results than the 10-week multimodal behavior therapy as used in the present study on ADHD symptoms. In fact, effect sizes for the combined conditions of both studies are highly similar. Both the MTA-study and our study showed very large effect sizes for ADHD symptoms, and comparable large effect sizes for ADHD-related symptoms.

Positive effects of methylphenidate cease as medication is discontinued [2]. Additive effects of behavioral treatment might only become evident after long follow-up periods. However, up to now, findings regarding the long-term effectiveness of behavioral treatment next to methylphenidate are inconclusive. A recent study shows that extensive one-year behavior therapy did not facilitate methylphenidate withdrawal [2]. Further, the MTA 24-month follow-up data did not show significant benefits of the combined treatment over medication treatment alone, although the dose of methylphenidate was lower in the combined condition [33]. All in all, the present study clearly shows that in medication naive children with ADHD, intense multimodal behavior therapy does not enhance the effects of optimal titrated methylphenidate, yielding support for short-term efficacy of optimal titrated methylphenidate.

Limitations

Some limitations should be considered. First, the small number of participants per treatment condition limits the statistical power. However, our a priori power analysis showed that the sample size was large enough to detect clinically significant differences between the two groups. Also, results of this study are consistent with a large-scale trial such as the MTA [31] and consistent with smaller scale studies [e.g., 28] including comparable numbers of participants. However, to detect more subtle differences between treatments, a replication or extension of the present study is needed.

Next, treatment duration was brief. However, findings of the MTA study [31] indicate largest gains of treatments after the first 3 months of treatment, which is comparable to our treatment duration. Further, during treatment and assessments, children, parents, teachers, psychologists, psychiatrists, and researchers were not blind to treatment condition. Especially in the combined treatment condition, parents and teachers were actively involved in treatment delivery, which may have caused bias for this treatment. Also, prior to randomization, parental preference for condition was assessed: 88.9% of the parents preferred the combined treatment, causing possible bias for combined treatment. In spite of these possible biases, there were no significant differences between treatment conditions. Due to simultaneously starting of medication titration and behavioral treatment, an early impact of behavioral intervention may have been possible, resulting in a lower titrated medication dose for the combined treatment. However, directly after titration there were no significant differences in assigned dose between both treatment groups. Further, a more intensive teacher training would have been preferable. However, during our pilot study, teachers could not participate in a more lengthy treatment, due to heavy workloads.

Finally, our study shows higher rates of placebo-responders (41%) than the MTA study (12.5%). Our placebo rates are also somewhat higher than reported in the literature. Previous studies have established that 20–30% of the children show a placebo or non-response to stimulants [16]. It is unclear whether the high placebo rate is reflective of rating artefacts or sample characteristics. In our sample placebo-responders have lower baseline scores than responders, thus less room for improvement for methylphenidate. Therefore, the placebo-response may have been overestimated in our sample.

Clinical implications

This study does not support the expectation that optimally dosed stimulant treated children with ADHD should routinely receive a multimodal behavioral treatment to reduce ADHD and related symptoms. Both treatment conditions show large treatment gains for ADHD- and related symptoms. Nevertheless, for subgroups of ADHD children, for example children with comorbid anxiety disorders additional behavior therapy may still be useful [30]. Possibly in clinical practice and research it is more useful to individualize and tailor treatments to specific patients needs, rather than using a “one size fits all” approach [14].