Introduction

Attention-deficit/hyperactivity disorder is one of the most prevalent psychiatric conditions in childhood with an estimated prevalence around 5%. Children with ADHD also display significant impairments in a number of cognitive functions compared to typically developing controls [1, 2]. However, there is no specific cognitive profile for individuals with ADHD as their cognitive deficits are heterogenic in the type and severity of dysfunction with great variation at the individual level [2]. Sustained attention and executive functions are the most affected areas [1, 2], although only half of the children with ADHD have an actual executive function deficit [3]. Pharmacological treatment is very effective for the core symptoms of ADHD [4], but the impact on cognition, particularly executive functions, is limited [5,6,7] which makes it important to investigate other treatments.

For more than a decade, research has focused on cognitive training as a possible new treatment approach for ADHD [8, 9]. Cognitive training is theoretically based on the concept of neuroplasticity, which implies that the brain can be changed by new experiences. The brains of individuals with ADHD show both structural [10] and functional [11, 12] anomalies, and the target of cognitive training is to strengthen deficient networks and areas by external stimulation in hope that these effects will decrease symptoms and improve functional outcomes. The search for a new intervention has resulted in a range of different approaches to cognitive training being driven by different theoretical frameworks. For instance, studies on working memory training have lead the field [8, 9, 13], followed by attention and some executive function training [14,15,16,17,18]. The focus of most cognitive training approaches has been to achieve improvement both on the directly trained functions such as attention or working memory measured by tests dissimilar to the intervention (near transfer) and, more importantly, to other untrained cognitive functions and symptoms (far transfer).

The field of cognitive training has grown so much over the past years that several meta-analysis and systematic reviews have been conducted on the subject [19,20,21,22,23,24]. Despite different inclusion criteria across the meta-analysis and different understandings and definitions of what cognitive training approaches target, there is a consistent evidence of moderate near-transfer effects on working/short-term memory [20, 21, 23], while there are no significant far-transfer effects on inhibition, attention ratings or academic performance [20, 21, 23]. These results are particularly true for working-memory training. On the other hand, studies investigating attention or executive function training have found no or limited near-transfer effects on the trained functions [17, 21].

In general, there are large discrepancies in teachers’ and parents’ ratings of a child’s behavior, where parents often report a greater severity of symptoms [25]. This discrepancy is also reflected across cognitive training trials that often use parent and teacher ratings as outcome measures for symptoms and executive functions. The meta-analyses show significant effects of cognitive training on both ADHD total and inattention symptoms and rated executive function, with moderate effect sizes on parental ratings and somewhat smaller effects on teacher ratings [19, 20]. Working memory training was found not to generalize to severity of ADHD symptoms, while interventions targeting multiple cognitive functions were shown to have large effects, when rated by parents [20].

Past research is based primarily on programs targeting short-term/working memory [22,23,24]. Very few trials target several cognitive functions. There are few trials targeting two cognitive functions (working memory and inhibitory control) [15, 26] and only two trials targeting three executive functions (working memory, cognitive flexibility and inhibition) [17, 27]. Considering that ADHD is cognitively very heterogeneous and that the interventions used in the past trials were limited to few cognitive functions with modest outcomes, it is important to investigate the effect of cognitive computer programs targeting a range of cognitive functions. This is an underexplored field of cognitive training in ADHD.

In the current randomized controlled trial, we tested ACTIVATE™, that targets a wide range of cognitive functions: sustained attention, response inhibition, cognitive flexibility, working memory, pattern recognition and category formation and use. ACTIVATE™ has been previously tested in a different setting, where it was incorporated as a part of a multifaceted intervention program, the Integrated Brain, Body, and Social (IBBS) intervention for children with ADHD or subthreshold ADHD. In that trial, ACTIVATE™ was combined with physical exercises and a class behavioral intervention three times a week with very modest outcome [18]. That trial was a school-based multifaceted intervention and we were interested to investigate the effect of the cognitive training component of ACTIVATE™ solely in a more intensive home-based setting in children with ADHD only. There have not been previous trials testing cognitive training interventions targeting a wide range of cognitive functions. We hypothesized that computer games targeting a number of cognitive functions might have a better effect on the symptoms and cognition than interventions targeting single or a few cognitive functions (working memory).

Method

Setting and sample

Participants were recruited at the Child and Adolescent Psychiatric Departments Aabenraa (including Augustenborg), Kolding and Odense from January 2013 to October 2015. A detailed protocol for this trial has been published previously [28]. A total of 164 families provided informed consent and were invited to participate in the diagnostic interview, Development and Well-being Assessment (DAWBA) via an online platform [29]. The DSM-IV criteria for ADHD, conduct disorder, autism spectrum disorder, depression and schizophrenia were assessed for this trial. DAWBA was filed out by the parent(s), child if older than 11 years and in the majority of cases also a teacher. If parents failed to complete the DAWBA online within 10 days of invitation, they were contacted and reminded to do so by the principal investigator. Of 164 invited families 122 participated in the DAWBA interview, which was then rated by one of two medical doctors (residents at child and adolescent psychiatry), trained as clinical DAWBA raters. To ensure a high inter-rater reliability, the first ten interviews rated by each of the raters were also rated blindly by a child psychiatrist (S. Dalsgaard), who had extensive clinical experience and was trained as a clinical DAWBA rater by the developer of the instrument, professor Robert Goodman. Overall, the inter-rater-reliability test showed a high composite agreement percentage of 87.5% (95% CI 60.4–97.8%) and an overall Cohen’s κ of 0.75. According to Landis and Koch [30], a value of 0.61–0.80 corresponds to a substantial agreement. Inconsistencies between ratings in these initial interviews were discussed and a consensus about diagnoses was reached.

Participants meeting full or subthreshold criteria for an ADHD diagnosis in DAWBA (n = 86) were invited to a clinical interview by one of three trained psychologists, to confirm the ADHD diagnosis, using the ADHD section of the Kiddie-Schedule for Affective Disorders and Schizophrenia (K-SADS) [31]. To ensure inter-rater reliability for the K-SADS, the first ten cases of each of the three psychologists were videotaped and also rated by an experienced K-SADS rater (N. Bilenberg or A. Bikic). After the parent(s) completed the K-SADS interview, the intellectual level of participants was tested by a trained psychologist, using the Reynolds Intellectual Assessment Scales (RIAS) [32].

Inclusion criteria for participation in the trial were: Fulfilling DSM-IV criteria for ADHD (in DAWBA interview, and verified with K-SADS); age between 6 and 13 years; access to a computer and internet connection and informed consent obtained. Furthermore, the following exclusion criteria were applied: Diagnosis of comorbid conduct disorder, autism spectrum disorders, depression or schizophrenia; medical history of head injury or a verified neurological disorder; intelligence quotient (IQ) < 80; motor or perceptual handicaps which would interfere with computer use; medical condition requiring primary treatment; and no informed consent from custody. Finally, 78 participants were considered eligible for the trial. Eight families decided not to participate for various reasons (lack of time, change of mind, starting medication treatment, and/or family difficulties) hence 70 participants were included in the study. Participants were asked not to change their medication status during the intervention period. However, two participants (one in each group) started medication during the intervention. They were, like all other participants required to not take medication 24 h prior to the cognitive test.

Cognitive outcome measures

All participants were tested with following tests from the Cambridge Neuropsychological Test Automated Battery (CANTAB) [33]: The Motor Screening Task (MOT) screening for visual, movement and comprehension difficulties. Attention tests: Attention switching task (AST) (Total omission and commission Errors) is a test of the participant’s ability to switch attention and to ignore task-irrelevant information. Rapid Visual Information Processing (RVP) (Probability of Hit and Mean Latency) is a test of sustained attention. Executive functions: Spatial working memory (SWM) (Between errors) is a test of ability to retain and manipulate spatial information. Stockings of Cambridge (SOC) (Problem solved in minimum moves) is a spatial planning test. Intra-extra dimensional set shift (IED) (EDS Errors) is a test of rule acquisition, reversal, attentional set formation maintenance, shifting and flexibility of attention. Stop signal task (SST) (Direction errors stop and go and SSRT last half) is task measuring response inhibition. Reaction time: Reaction time (RTI) (5-choice movement time and simple error score inaccurate) provides motor and mental response speeds and movement time.

Behavioral outcome measures

The following questionnaires were used: (1) ADHD-Rating Scale-IV (ADHD-RS) is a symptom rating scale [34]. The Danish version of the ADHD-RS-IV is a translation of the 26-item version, comprising nine items on inattentiveness, nine items on hyperactivity/impulsive behavior and eight questions on oppositional behavior [35]. (2) Behavior rating inventory of executive function (BRIEF) is a 86-item rating scale for parents and teacher assessing executive function behaviors in the school and home environments [36]. BRIEF consist of eight clinical scales (Inhibit, shift, emotional control, initiate, working memory, plan/organize, organization of materials, monitor) and two validity scales (Inconsistency and negativity). The clinical scales form two broader Indexes (Behavioral Regulation and Metacognition) and an overall score, the global executive composite. (3) Weiss functional impairment rating scale-parent report form (WFIRS-P) is 50-item questionnaire where parents are asked to rate their child’s functional impairment over the past month [37]. There are six domain scores (Family, learning and school, life skills, child’s self-concept, social activities and risky activities).

Interventions

Both the intervention and control group received treatment as usual (TAU). TAU may have involved diagnostic and cognitive assessment, psycho-education, pedagogical counseling, and questionnaires for parents and teachers, home and school visits and, for some children, medical treatment. Besides TAU, the intervention group was encouraged to use the computer program ACTIVATE™ (http://denmarkstudy2.c8sciences.com/?language=da) six times a week for 8 weeks. We only used the cognitive computer games part of the ACTIVATE and we did not use the physical exercises, that are offered with the program. We used the first version of ACTIVATE™ at home consisting of three exercises: Catch the Ball, Butterflies and What Comes Next. These games are targeting a broad range of cognitive functions with focus on sustained attention, response inhibition, cognitive flexibility and control, speed of information processing, multiple simultaneous attentions, working memory, category formation and pattern recognition. For a detailed description of the games please see our protocol [28].

Procedures

This was a parallel, two arms, single blind, randomized and controlled trial. Prior to randomization, the parents and a teacher completed the ADHD-RS and BRIEF questionnaires. In addition, the parents completed the WFIRS-P questionnaire. All participating children were assessed with a series of cognitive tests from the CANTAB test battery at four time points: T0 = baseline; T1 = after 8 weeks of intervention; T2 = 12 week follow-up and T3 = 24 week follow-up after ended intervention. Participants were assessed at approximately the same time of the day at each visit and always between 8:30 a.m. and 2 p.m. to avoid time of day impacting cognitive functions. Children receiving pharmacological treatment were asked not to take their medication 24 h prior to the cognitive testing. The parents were reminded to do so by a text message via mobile phone. The eligible 70 participants were then randomized 1:1 with stratification for site and medication status. The Copenhagen Trial Unit, an independent clinical intervention research unit in another city, performed the randomization, described in detail in Bikic et al. [28].

The investigators performing the cognitive test with CANTAB were blind to the child’s allocation at each assessment. After randomization, participants in the intervention group received an individual username and password by e-mail and used these to access the computer game at a secure online web-based platform, designed for this trial. Each log-on, progress on the games and time of playing was registered for all participants and these data were used to measure compliance in the intervention group. In the event of any technical problems, with the intervention, the parents (n = 8) contacted the principal investigator by e-mail or phone, who then contacted IT-support.

Ethics

The trial was conducted in accordance with the Declaration of Helsinki and was approved by the Danish Data Protection Agency (ID.nr. 2008-58-0035) and the Regional Scientific Ethical Committee for Southern Denmark (nr. S20120096). The trial was registered at ClinicalTrials.gov (NCT01752530) and the trial protocol has been published [28].

Sample size

When planning our intervention, we have based our sample size calculation on our primary outcome measure. Assuming CANTAB RVP was normally distributed with standard deviation (SD) 0.22 points. If the true difference in the experimental and control means is 0.13 points, we needed to include 61 experimental participants and 61 control participants to be able to reject the null hypothesis that the population means of the experimental and control groups were equal, with a 90% probability (power). The type I error probability associated with this test of this null hypothesis is 5%. We thus aimed to include 122 participants in total, but were only able to include 70 participants, even after having extended the recruitment period with 1 year. Due to lack of funding, we were not able to further extend the recruitment. Post hoc power calculation showed that, we would be able to identify moderate treatment effects (i.e., standardized mean differences > 0.68) with a power of 80% and a 5% significance level with the smaller sample size of 70. Substituting missing values with the minimum and maximum observed values, best–worst and worst–best sensitivity analyses of primary outcome found beta coefficient ranging from − 0.07 in the worst-case scenario to 0.03 in the best-case scenario. No significant effect could be detected in the sensitivity analysis.

Statistical analyses

We performed intention to treat analysis. All variables with normally distributed residuals were analyzed with a structural equation model (SEM) using a Full Information Maximum Likelihood Estimator to address missing data. We used a robust variance estimator, because some outcomes had moderate violations of the normality assumption. Outcomes were treated as observed variables. Correlations between exogenous variables were estimated. Means and variances were estimated for exogenous variables with missing values. All variables were adjusted for the stratification variables “center” and “pharmaceutical treatment” at baseline. In addition, we have adjusted for baseline scores of all variables in our analysis. As we only recruited one patient from the center in Odense, this patient was assigned to another center (Kolding) by flipping a coin. Based on SEM, we estimated beta values with 95% confidence intervals. Means and standard deviation estimates were based on a Full-Information Maximum Likelihood (FIML) estimator. All analyses were performed and analyzed according to a two-sided significance level of p < 0.05.

For estimation of effect for the primary and secondary outcomes, we rapport the beta coefficient, that is a standardized regression coefficient.

All statistical analyses were performed in STATA 13.1 [38]. The primary outcome in this trial has been defined a priori as the continuous response variable, ‘CANTAB RVP probability of hit’. Secondary outcomes have been defined as the total scores for the ADHD-RS and BRIEF for the parent and teacher version, respectively.

Compliance

There was a great variation in the number of sessions performed in the intervention group (M = 26.2, SD = 15.89, min = 0, max = 48). Compliance was low and only 66.5% of participants performed more than 20 sessions.

Results

Seventy participants were randomized in this trial. Participants ranged in age from 6 to 13 years (M = 9.95, SD = 1.7) and were all Caucasians. A total of 40 (57%) participants used ADHD medication during the intervention, with no significant differences in medication status between the two groups. Four participants dropped out of the trial before completion of the intervention. One participant in the control group did not participate in the T1 assessment, but returned to the two follow-up sessions (T2 and T3). Missing data for this second visit were estimated based on FIML. A flowchart of included participants is shown in Fig. 1. There were no serious or non-serious adverse events reported.

Fig. 1
figure 1

Flow diagram of participant enrollment in the trial

Full size image

Baseline characteristics

The 70 participants allocated to the two groups were comparable on a number of measures at baseline (see Table 1).

Table 1 Diagnostic and demographic characteristics of children at baseline
Full size table

Primary cognitive outcome

Results indicate that the intervention had no effect on our primary outcome measure, the CANTAB RVP Probability of hit compared to the control group: b = − 0.017, CI (− 0.0907 to 0.0560), z = − 0.46, p = 0.643 (see Table 2).

Table 2 Results: Effects on the cognitive outcome measures from CANTAB
Full size table

Secondary outcome measures

The secondary measures were defined a priori as the total score on BRIEF as rated by parents and teachers and the total score on ADHD-RS parent and teacher version (see Table 3). Results indicate that there were no significant effects of training on BRIEF total scores for the parent version b = − 2.12 (− 5.5 to 1.26), z = − 1.23, p = 0.22 or teacher version: b = 3.68 (− 1.11 to 8.48), z = 1.5, p = 0.13. There were no significant differences for the ADHD-RS parent total score b = − 1.02 (− 6.13 to 4.09) z = − 0.39, p = 0.69 or ADHD-RS teacher total score b = 3.11 (− 3.63 to 9.85) z = 0.90, p = 0.37.

Table 3 Results: Effects of the intervention and treatment as usual on the behavioral measures from T0 to T1
Full size table

Standardized regression coefficient for the primary and secondary outcomes

There was no significant effect on the primary outcome, sustained attention (β = − 0.047; CI − 0.247 to 0.153) or the secondary outcomes ADHD-RS-parent [β = − 0.037; CI (− 0.224 to 0.150)]; ADHD-RS-teacher [β = 0.093; CI (− 0.107 to 0.294); BRIEF-parent (β = − 0.119; CI − 0.307 to 0.069)] and BRIEF-teacher [β = 0.136; CI (− 0.048 to 0.322)].

Exploratory measures

Cognitive outcome measures

All secondary and explorative variables were on continuous measurements. SEM analysis indicated a highly significant effect of the intervention on executive functions as measured at T1, by the variable SOC problems solved in minimum moves: b = 1.22 (0.347–2.10), z = 2.74, p = 0.006 with the intervention group outperforming the control group (and also a significant effect on this outcome at T2 and T3, see later). However, the difference between the two groups on the accuracy in planning corresponds to 0.30 of a standard deviation on the outcome indicating a modest effect. There were no significant group differences on any of the other cognitive measures (see Table 3).

Questionnaires

No subscales of the BRIEF were significantly different between groups. There were no significant differences between the intervention and the control group on the ADHD-RS subscales or any other measures (see Table 3).

Follow-up: time T2 and T3

At the 12-week follow-up (T2) data on the cognitive outcome measures was available for 54 participants and at the 24-week follow-up (T3) for 41 participants (for details on drop out see Fig. 1). Results on the follow-up data for the CANTAB cognitive test indicate that the significant difference on SOC Problems solved in minimum moves was maintained over both time points, at T2 (p = 0.035) and at T3 (p = 0.017).

At the 24-week follow-up (T3), there were significant effects on two measures that did not differ significantly at T1: RTI 5-choice movement time (p = 0.008) and AST total commission errors (p = 0.014) and SST SSD 50 last half-approached significance (p = 0.065). Due to a large number of drop-outs (over 50% for the parents and 65% for the teachers) on the behavioral scales returned at T3 (n = 36 parent ratings and n = 18 for teacher ratings) and T4 (n = 34 parent and n = 20 for teacher ratings) we did not calculate results for ADHD-RS, BRIEF and WFIRS at these time points.

Post hoc analysis

Interactions with age

To explore possible interactions with age, we divided participants in two groups: 6–9 years old (n = 43) and 10–13 (n = 27) and compared participants in the intervention group to the controls in each age group (see Supplemental Table S1). For a number of CANTAB measures, there was a significant difference in the older group, but not in the younger group.

Interactions with subtype

An examination of ADHD subtype showed that the gains of intervention were largest for the inattentive subtype (ADHD-I) compared with controls for a number of outcomes.

Discussion

The aim of this trial was to investigate the effect of ACTIVATE™, a computerized intervention targeting multiple cognitive functions, compared to treatment as usual. Our primary hypothesis, that this intervention would have an effect on an objective measure of sustained attention, could not be confirmed. Although we recruited fewer participants than initially anticipated, we should have been able to detect treatment effects above standardized mean difference of 0.68. Our study might, therefore, have been underpowered to detect changes of small to moderate effect sizes.

We found no significant effect on the secondary outcome measures, defined as the total scores on the BRIEF and ADHD-RS questionnaires for parents and teachers, indicating that there was no effect of ACTIVATE™ on parents- and teacher-reported symptoms.

Significant differences between the intervention and control group were seen on one exploratory measure. ACTIVATE™ had an effect on the accuracy in planning (SOC). There was a highly significant difference between groups (p = 0.006) after intervention, indicating that the ability to plan was improved in the intervention group as compared to the control group with a modest effect. Furthermore, the significant difference was maintained at both the 12-week (p = 0.03) and 24-week (p = 0.017) follow-up, thus the effect was still observable 6 months post intervention. This change had a modest effect at best. This result is not explained by near transfer as the ACTIVATE™ games and the planning task SOC do not have very much in common. Nevertheless, this result might be random due to multiple comparisons in our trial. It would be necessary to test this hypothesis as a primary outcome in future studies to determine if there is a real effect of the intervention on planning ability.

There were no near or far-transfer effects on parent or teacher rating scales for ADHD or the parent-rated functional scale after the intervention.

ACTIVATE™ has been tested previously as part of a multifaceted intervention [18] in a randomized, controlled trial with children with ADHD or subthreshold ADHD in a school group setting. Despite the methodological differences between our trials, our results are similar in finding no effect on the majority of cognitive outcome measures, severity of symptoms and executive function behaviors. We did find an exploratory effect on the ability to plan, while Smith et al. [18] did not include such a measure. Overall, these results indicate that ACTIVATE™ is not very useful for children with ADHD in general.

Our results also add to a small number of randomized trials, which have investigated interventions focusing on combined inhibition and short-term memory training [15, 26], and two trials additionally including set-shifting [17, 27]. Our intervention was somewhat similar to the two latter trials of Dovis et al. [17], and the Van der Oord et al. [27] as the games in ACTIVATE™ also focus on working memory, set shifting and impulse inhibition. Dovis et al. [17] found significant changes of working memory and short-term memory, inhibitory performance and interference, which we did not find. It is important to mention that, neither Dovis et al. [17] nor Van der Oord et al. [27] used a measure of planning ability, which we found a significant change on. A difference to our trial is that in the Van der Oord et al’s. [27] study participants completed at least 20 out of 25 sessions and in the Dovis et al. [17] study participants completed 25 sessions with only 3% failing to meet compliance criteria. As we adopted an intent-to-treat design in our analyses, we kept everyone in the intervention group regardless of the number of sessions performed. The compliance in the intervention group was low and only 66.5% of our participants performed 20 or more sessions. Adherence to the intervention varied among participants randomized to the treatment. The low compliance might also have influenced our findings and partly explain the lack of beneficial effects of the intervention. Several previous studies [9, 15, 17, 27] have chosen an intervention period of 5 weeks producing mixed results. We choose a longer intervention period expecting that more training might have a positive impact on the outcomes. Obviously this was not the case and it is worth discussing if compliance might be higher in shorter-term interventions, as it puts less strain on the child and the family. We decided to follow-up on the outcomes 3 and 6 months later to investigate if possible results were maintained. Future studies should examine whether effects of cognitive training depend on compliance. The described results of other studies together with our trial indicate a very limited impact of broader cognitive training on cognition and symptoms in children with ADHD. The results of cognitive training trials for children with ADHD have so far not met the great expectations that have led the field.

A general problem across all cognitive training studies is that so far not a single study has tailored the interventions to the existing cognitive deficits of the trial participants. The common practice has been to include participants with ADHD regardless of the individual cognitive deficits profile at baseline. Individuals with ADHD are thus assumed to have identical needs and expected to benefit from the intervention equally, despite evidence that individuals with ADHD exhibit heterogenic cognitive profiles and symptoms [1,2,3]. If an individual with ADHD exhibits problems with attention, but has normal working memory, it would make little sense to train working memory and expect this to generalize to attention. Indeed, the connection between specific cognitive deficits, their hierarchical order and interaction and the generalization to other cognitive dysfunctions and symptoms have not yet been empirically proven. Most cognitive training approaches focus on the assumption that the largest cognitive deficits presented in individuals with ADHD somehow might be the most central ones for the disorder. Nevertheless, the central cognitive deficit in ADHD has still not been identified and it is still questioned if this kind of core cognitive deficit exists for the whole ADHD population [1]. Considering the heterogeneity of the disorder on the cognitive, neural and symptom level, it would be important to look at the effects of cognitive training on a subgroup level and identify specific groups that might benefit from certain kinds of cognitive interventions. Although we did not have a very large sample, we performed some exploratory post hoc analysis to investigate possible subgroup effects and found that the gains of intervention were largest for the inattentive subtype (ADHD-I) compared with controls. Age of the participants might also play a role. These analyses were purely exploratory, and future research could focus on the question if the effect of cognitive training might be age and ADHD subtype dependent.

Several meta-analyses [19, 20] have indicated a considerable difference in the perceived effect of interventions in parent and teacher ratings. Parents tend typically to report higher scores than teachers. One explanation for this difference might be due to that teachers tend to be blind to the allocation of the child more often than the parents. Thus, the significance seen in parent reports might partially be explained by a placebo effect. To avoid these issues, we have chosen an objectively measured primary outcome, sustained attention (RVP). This is a clinically very relevant outcome, as sustained attention is one of the most affected cognitive functions in most individuals with ADHD [1, 2]. On the other hand, the clinical and ecological validity of both the RVP and other similar cognitive outcome measures like the continuous performance task (CPT), is still undetermined [39]. Even if children score better on these instruments after training, it is unknown how much this will translate into everyday life. Therefore, we have added parent- and teacher-rated questionnaires defined as the symptom rating scale ADHD-RS and the scale of executive functions BRIEF as our secondary outcomes. Both scales are very clinically relevant and are used in the clinic as a part of the assessment as they reflect symptoms and issues with executive functions in everyday life.

As mentioned above, the clinical relevance of the RVP and other sustained attention measures is undetermined. In our power analysis, we assumed to see a change of half a standard deviation, which would equate to a moderate change of standardized mean difference of 0.68. A change of this magnitude would equate to 0.13 points on the RVP. This result would approach the mean seen in typically developing children and this change would have been clinically relevant. At the same time, we were not able to detect smaller changes that might have been clinically relevant as well. Regarding our secondary outcomes, a standardized mean difference of 0.68 would mean, that we could detect a change on the ADHD-RS questionnaire exceeding seven points. This result would equate to an improvement of around 22% compared to the baseline data and would indicate a clinically relevant change. Additionally, we could have detected a change beyond 7% in regard to the initial score for the BRIEF.

Another issue in cognitive training trials in general is the difficulty to perform double-blind trials and to find the right kind of control group. The optimal control group in cognitive training trials would be an active placebo group performing a control intervention that has no impact on cognition. An active placebo group does not only control for the contact with the therapist and the computer, but also allows to blind participants and their parents ensuring a double-blind design. The real challenge is to identify an active placebo-training program without any impact on cognitive functions and we, therefore, chose not to use it and use TAU instead. Some studies, especially those using Cogmed [9, 40], have used the actual intervention on a consistently low level in a non-adaptive fashion as an active control. In cognitive intervention trials, it is challenging enough to engage the participants in the demanding intervention for several weeks, maintain good adherence to the trial, a high motivation and prevent participants from dropping out. If control participants have to engage in an intervention with very low cognitive load for several weeks, this could be perceived as boring and cause attrition. Additionally, the blinding could be broken, because participants and parents probably could figure out which group they are in. In a previous trial [41], we have used the game Tetris as a control condition. In that trial, we found no differences between the groups at the end of the intervention, but there were several different pre-post effects for both the intervention and Tetris group, individually. Importantly, Dovis et al. [17] used a new control condition with good adherence, by the application of game-design elements and game principles in a non-game context: a gamification of the intervention. Interestingly, in that trial, the active control group also showed some significant pre-post changes, indicating that there might be a cognitive effect even in low load interventions, which could have obscured group differences [17] and could explain the differing results between the Dovis et al. [17] and the Van der Oord et al. [27] trials.

Strengths and limitations

Our trial has several strengths. The number of drop-outs during the intervention was small. We performed intent-to-treat analysis, using FIML to account for missing data. Additionally, we performed adequate generation of allocation sequence, adequate allocation concealment and adequate blinding wherever possible. We tested a priori defined primary, secondary and exploratory outcome measures as they were published in our trial protocol [28] and there is no for-profit bias. Still, our trial does have some limitations.

We were not able to blind the participants and their parents or teachers to group allocation. Although we included objective outcome measures, our secondary outcome measures were based on questionnaires rated by parents and teachers, who were not blind to group allocation, which can induce possible placebo effects. Our trial was likely underpowered to detect small to moderate changes. In addition, it is important to mention the low-treatment compliance as the most children only trained the half dose recommended. Low compliance in the intervention group may also partly explain the lack of effect. At the two follow-up time points, the drop out was substantial for the returned questionnaires and we were not able to analyze survey data for these time points. We have performed a high number of analyses why some of our significant results might be at random. Our exploratory analyses were performed on a relatively small number of participants. The teacher ratings were not always provided by the same teacher, which can induce a natural variability in scores and might explain some of the unusual results we found.

To conclude, ACTIVATE™ did not show an effect on any of our primary or secondary outcomes. We found that it may have beneficial effects of the ability to plan, and this effect seemed to be sustained over time. Before dismissing ACTIVATE™ as a possible treatment, it would be important to investigate the effects on specific cognitive functions, particularly planning ability, in future studies. Considering that ADHD is a very heterogenic disorder at the individual level, future studies with larger samples should investigate effects on subgroup levels, as the overall evidence indicates that cognitive training is not very beneficial for children with ADHD in general. The approaches of the future should be driven by the specific needs of individual cognitive profiles.