Introduction

Despite the alarming prevalence of childhood mental disorders (13–20%) [1, 2] and the accompanying personal, economic, and social costs, the focus in mental health has been primarily on treatment rather than prevention [3]. Developing effective prevention strategies for children and adolescents, in an effort to reduce the prevalence and impact of mental health disorders, is therefore crucial. Cognitive-behavioral preventive programs have been found to decrease the number of new cases in adolescents [4] as well as the depressive and anxiety symptoms among children and youth [5]. However, most of these prevention programs have not been designed to be tested in community-wide contexts and to reach a large number of youths and thus efforts are needed to improve the wide-scale implementation of efficient prevention strategies.

Although the vast majority of psychological research on video games has focused on their negative impact, there is a growing body of evidence for their potential mental health benefits [6–10]. Thus, recently a new mental health intervention and prevention strategy has been proposed: the use of online therapeutic games. Games represent a very promising alternative for prevention because they have the advantage of being attractive and immersive for the general population, with 97% of children and adolescents playing them in their leisure time [11]. More so, online video games can be played anywhere at any time. Overall, the use of video games holds great promise for a radically new approach of delivering prevention programs for children and adolescents.

In this context, the REThink game was designed to respond to these concerns, aiming to offer a theory-based prevention tool that can develop psychological resilience in undiagnosed children and adolescents. REThink is a therapeutic videogame accessible online, meant to be used as a standalone intervention to promote emotional resilience in children and adolescents. REThink is based on positive and preventive curricula of Rational Emotive Behavioral Therapy (REBT [12]), specifically on Rational Emotive Behavioral Education, (REBE [13]) principles, which focus on teaching children to change their irrational beliefs, to promote functional emotions, effective problem solving and positive emotions. REThink features a main character, RETMAN, first introduced at Albert Ellis Institute in 1970s [14], and later developed by David [15]. RETMAN is a superhero that helps children to think rational and have functional emotions. RETMAN also has five friends that help him in his missions. The REThink game was designed to help children and adolescents, aged between 10 and 16 years, to learn healthy strategies for coping with dysfunctional negative emotions such as depression, anxiety, and anger.

Although recent research using rigorous design pinpoints REThink as a promising, effective solution for preventing emotional difficulties in youth David et al. [16], studies also highlight the importance of understanding the mechanism of action for such an intervention (David, Mogoșe, and Costescu, in press). No such studies have been published to date and we are yet to understand what mechanisms impact the efficacy of preventive therapeutic games. Psychopathology and dysfunctional emotions are conceptualized by REBT based on the ABC model [14] as being caused by irrational beliefs. Irrational beliefs are a specific type of evaluative cognitions which are rigid, illogical, not consistent with reality, and unhelpful. The major irrational beliefs that have been connected to distress and psychopathology [17] are demandingness, awfulizing, frustration intolerance and global evaluation. In turn, rational beliefs such as preferences, badness, frustration tolerance and unconditional acceptance are considered resilience mechanisms and have been the target of youth prevention programs [18].

As such, our primary aim was to examine the mechanisms of change proposed by Ellis [19] to explain the efficacy of REThink intervention designed to help children and adolescents, aged between 10 and 16 years, to develop emotional resilience. Based on the theoretical foundation on which REThink was developed (REBT), we hypothesize that the intervention’s impact is mediated by changes in rational and irrational beliefs. We were also interested to explore if the effect of the REThink intervention on emotional outcomes was moderated by gender or age.

Methods

Since methodological details of this clinical trial have already been described elsewhere (David et al., under-review), only the most important aspects are presented here.

Participants

A total number of N = 165 healthy children and adolescents, aged between 10 and 16 years, were invited to participate in the present study. Participants were recruited on a voluntary basis from one middle school (‘Ioan Opris’ Gymnasium School, Turda, Cluj, Romania) and one high school (National College ‘Mihai Viteazul’, Turda, Cluj, Romania), located in an urban area. Informed consent to participate in the experiment was obtained from their parents and from the school principal. Out of the randomized subjects (n = 56 in the Waitlist condition; n = 55 in the REBE condition; n = 54 in the REThink condition), 23 (13.94%) did not complete the initial assessment (n = 10–17.86% in the Waitlist condition; n = 7–12.73% in the REBE condition; n = 6–11.11% in the REThink condition). These subjects were thus not introduced in the analysis and were treated as dropouts (see Fig. 1). No other dropouts were recorded. The Chi-squared test that compared the frequency of the dropout in the three groups indicated no statistical differences form and excepted distribution due to chance alone, χ2(2) = 1.14, p = 0.564.

Fig. 1
figure 1

Flow diagram of the progress through the phases of the trial

Full size image

The final sample used for data analysis consists of 142 subjects: 46 in the waitlist condition, 48 in the REBE group and 48 in the REThink group. The mean age of participants was equal to 13.02 (SD = 2.06), 12.75 (SD = 1.95), and 12.93 (SD = 2.20) years old in the REThink group, Rational-Emotive Behavioral Education (REBE) group and waitlist condition, respectively. Sample consisted of 91 girls and 51 boys, with 71.8% of the participants being enrolled in secondary school (grades 5–8) and 28% being enrolled in high school (grades 9 and 10). Table 1 summarizes the demographic characteristics of the participants in each group.

Table 1 Demographic characteristics of the participants included in the statistical analysis by group
Full size table

The randomization of the participants was done in a stratified manner, with the aim to ensure balance of the treatment groups with respect to the children’s grade. Trial participants were subdivided into seven strata (from the fourth to the tenth grade) then permuted block randomization was used for each stratum.

Procedure

Two protocols were elaborated for this study, one for each condition (REThink, REBE), including guidelines for intervention in each condition. One experienced psychologist (certified in CBT/REBT) assisted the REThink intervention and provided intervention in the REBE condition. Participants in the REThink and REBE groups have completed the 7 modules developed for this study. The psychological content of the modules was the same in both groups, the method of delivery being different. In both groups, the application time of a module was approximately 50 min, meetings with students taking place after the classes. If participants from the REThink group did not manage to navigate the module’s levels in 50 min, the experimenter stopped the game and scheduled them to the next module. The delivery time was set at 50 min having as reference the maximum playing time spent in the respective modules by five children who tested the game before implementing the study. Prior to the first module, children and adolescents completed pre-intervention questionnaires. After finalizing module 4, children and adolescents completed the mechanisms’ questionnaires. Finally, after finalizing all modules, during 1 month, children and adolescents completed the post-intervention measures. In the REBE group, pre-intervention, intermediate and post-intervention questionnaires were provided by the psychologist in a group context. In the REThink group, each questionnaire was individually filled in by the participants on the iPad. Questionnaires were displayed before a module (Module 1) or when modules were completed (Module 4 and Module 7). Before leaving the experiment room after every module, in both groups, participants were debriefed and thanked for their participation. The debriefing session was a short one (5–10 min) and aimed to debrief the pupil regarding the game contents and outcomes and thereby focus and reflect the player experience. At this stage, the psychologist asked open-ended questions to children, questions about their experience, what they learned, and about the game. During 1 month, participants in both groups completed all the modules and the assessment phases. During the first 3 weeks, 6 modules (2 modules/week) were applied and the last module, the most complex, was applied in the last week. Thus, the intermediate assessment phase took place 2 weeks after the pre-test and the post-test took place 1 month after the pre-test.

The waitlist condition (WL) was assigned to a waiting list and will receive the REThink intervention after the 6-months follow-up assessment, these analyses being reported in another study. WL served as an untreated comparison group during the study, participants taking part only in the three assessment stages during the trial. The time between the assessment phases was the same as for the intervention groups.

Interventions

The REThink game

REThink is a therapeutic videogame accessible on iOS Apps, meant to be used as a standalone application to promote emotional resilience in children and adolescents. The player’s mission is to help people on Earth to escape from the power of Irrationalizer, an irrational character that is bad for humanity, due to his capacity of cultivating bad minds and instilling irrational thinking and unhealthy emotions.

The main character of the game is RETMAN, a character that promotes rational thinking and guides the player. RETMAN has five friends, five rational characters that help him in his missions: Preferilizer (representing preferences beliefs), Ponderancer (representing non-awfulizing beliefs), Toleraser (representing high frustration tolerance beliefs), Acceptableizer (representing unconditional acceptance beliefs) and Optimizer (representing happiness). Irrationalizer, is thus RETMAN’s enemy, who promotes irrational thinking together with his servants: Necessitizer (representing demandingness beliefs), Awfulizer (representing awfulizing beliefs), Frustralizer (representing low frustration tolerance beliefs) and Discourager (representing global evaluation beliefs). RETMAN and his friends have the mission of helping the player to teach people from Earth to be more rational and happier. Besides these characters, there is the player’s avatar, which the player can choose at the beginning of the game. The game has seven levels. Each level has a trial part at the beginning in which RETMAN is explaining what the player has to accomplish. Each level has various degrees of complexity, which increase as the player progresses in the game. Level 1 aims to help the player to distinguish between 3 types of emotions—positive, negative and neutral states, and between functional and dysfunctional emotions (i.e., anxiety vs. fear). The player must choose the character’s emotion or a neutral state represented on the screen, and at a more complex sub-level the player must specify whether the emotion depicted on screen is a functional or a dysfunctional one. Level 2 aims to familiarize the player with the cognitive processes, such as: irrational beliefs and rational beliefs. The player’s role is to help a tree to grow by scaring the birds which have irrational beliefs that can harm the tree. Level 3 helps the player to identify the relation between their beliefs and their emotional and behavioral reactions. The player has to plant seeds (rational and irrational) and to prevent irrational servants from planting seeds that contain more irrational thoughts. From good seeds (rational thoughts) rise blue flowers (functional emotions), while from bad seeds (irrational thoughts) grow red flowers (dysfunctional emotions). The player’s mission is to cultivate more good plants, because these bring happiness to the planet. Level 4 aims to help the player to acknowledge the process of changing irrational cognitions into rational cognitions. The player has to identify people’s irrational thoughts and to provide them with a specific potion that contains the rational version of their irrational cognition. Level 5 aims to help the player to develop problem-solving skills. The action takes place in a maze, where the player has to discover the code “DECODE”, which stands for the solving problem steps. At the end of the level, the player completes a quiz where he/she earns points if for each letter in the code he attaches the correct problem-solving step (i.e., D = define the problem). Level 6 aims to help the player develop relaxation skills, using the abdominal breathing exercise. The action takes place in the wood and the player has to get closer to irrational servants to find out about their plans to conquer the territory in the next level. To get closer, the player needs to relax and in this way he/she becomes invisible. For assessing his level of relaxation, the player has to wear a belt that detects his heartbeat. The device used is called Polar Heart Rate Sensor (http://www.polar.com), which is a heart rate monitor designed to have a comfortable modality to wear it, around the chest. The Polar Heart Rate Sensor is supplied to every child/adolescent that uses the intervention.

Level 7 is the most complex level of the game and aims to help the player build happiness skills. In the first sub-level, the player has to quickly identify the happy face out of several angry faces. At the second sub-level, the player has to identify from the crowd the people who are happy and hug them. At the final sub-level, the player has to share his happiness with those people who have negative emotions and irrational thoughts.

The REBE group intervention

The REBE group intervention protocol was based on the Passport to success curricula, using class lessons format, based on experiential lessons [13]. The objectives of the seven modules were: (1) to learn different emotional states and behavioral reactions, and differentiating between basic emotions, complex emotions and functional and dysfunctional emotions, (2) to identify the cognitive processes and to differentiate between irrational and rational beliefs, (3) to identify the relation between cognitive processes, emotions and behavioral reactions, (4) to learn to change irrational cognitions into rational ones, (5) to identify and exercise the steps of a problem-solving process, (6) to develop relaxation skills using the abdominal breathing exercise, and (7) to review the skills learned in previous meetings and to develop positive emotions.

Measures

All participants were evaluated at baseline, at the middle of the intervention (only the mechanisms and satisfaction measures), and at the time of termination. Children and adolescents’ primary and secondary outcomes were examined using the following measures:

Primary outcomes

Strengths and Difficulties Questionnaire-Child Version (SDQ [20]) is a 25-item scale that was used to assess emotional symptoms, as a primary outcome. The SDQ comprises five subscales that measure difficulties in several psychological domains: emotional symptoms, conduct problems, hyperactivity/inattention, peer problems, and prosocial behavior. The score for each of the five scales is generated by summing the scores for the five items that make up that scale, thereby generating a scale score ranging from 0 to 10. Higher scores are indicative of more emotional impairment. Each item is scored on a three-point Likert-type scale from 0 (“not true”) to 2 (“certainly true”). The SDQ-child version has demonstrated acceptable psychometric properties [20]. Internal consistency for the current study is α = 0.75 for emotional symptoms subscale.

The Early Adolescent Temperament Questionnaire-Revised (EATQ-R [21]) contains 65 questions in the self-report form and is designed to measure temperamental effortful control, affiliativeness, surgency, and negative affectivity. The questionnaire asks adolescents how true each statement is for them. Response options used a 5-point Likert-type scale, from 1 (“almost always untrue”) to 5 (“almost always true”). We used in the present study only 1 subscale from the original questionnaire: depressive mood (primary outcome). For this subscale, we calculated a total score. A high score indicates that the assessed dimension is high for that individual. The EATQ-R demonstrated acceptable psychometric properties in previous studies [21]. Internal consistency for the current study is α = 0.64 for depressive mood subscale.

Hypothesized mediating variables

The Child and Adolescent Scale of Irrationality (CASI [22]) is a 28-item scale that was used to measure irrational cognitions in children and adolescents. We used in the present study items that reflect irrational cognitions in several domains: demandingness for fairness (DEM-F), low frustration tolerance for work (LFTW), low frustration tolerance of rules (LFT-R), and total score of irrationality level. Children and adolescents were asked to express their agreement/disagreement with the 28 statements on a 5-point Likert-type scale, from 1 (“strong disagreement”) to 5 (“strong agreement”). A high score indicates that the assessed dimension is high for that individual. Previous research has reported adequate psychometric properties for CASI [23]. Internal consistency coefficients for the current study are α = 0.65 for low frustration tolerance for work, α = 0.80 for low frustration tolerance of rules, and α = 0.80 for CASI total score. The demandingness subscale showed lower reliability, α = 0.27 and was excluded from all analyses.

Children’s Automatic Thoughts Scale-Negative/Positive (CATS-N/P [24]) is a 50-item questionnaire that assesses negative and positive automatic thoughts in children and adolescents. The questionnaire comprises five subscales: physical threat, social threat, personal failure, hostility, and positive thoughts. Children and adolescents rate to what extent they had a specific thought over the past week on a 5-point Likert-type scale, from 0 (“not at all”) to 4 (“all the time”). A high score indicates that the assessed dimension is high for that individual. The CATS-N/P has demonstrated good psychometric properties [24]. Internal consistency for the current study, is α = 0.81 for physical threat subscale, α = 0.85 for social threat subscale, α = 0.80 for personal failure subscale, α = 0.81 for hostility subscale, α = 0.85 for positive thoughts subscale, and α = 0.88 for CATS total score.

Hypothesized moderating variables

We explored if the effect of the REThink intervention on emotional outcomes was moderated by gender or age.

Results

Data analysis strategy

To investigate the mechanism of change for emotional problems, we focused on the outcomes that provided evidence for the efficacy of the REThink game. That is to say the mechanism of change analysis was conducted in relation to SDQ emotional symptoms and EATQ-R depressive mood. The analysis of the efficacy of the intervention revealed that SDQ emotional symptoms significantly decreased after the REThink intervention (Cohen’s d = 0.46), while no other significant change in scores from pre- to post-test, on this variable, was observed for the other groups. Also, the REThink group experienced a significant decrease in EATQ-R depressive mood scores (d = 0.84) and had significantly lower scores as compared to the WL condition (d = 0.66). Descriptive statistics on these outcomes are available in Table 2.

Table 2 Descriptive statistics for emotional outcomes and cognitive mechanisms by group
Full size table

We used in our analysis of the mechanisms of change the total scores for the two scales assessing dysfunctional/irrational cognitions, namely CASI and CATS total scores. To analyze the mechanisms of change, we first checked if groups were equivalent in baseline on these variables, using one-way univariate analyses of variance (ANOVAs). Then, we conducted a mixed within–between MANOVA, having the time of measurement (pre-test, intermediate assessment, and post-test) as the within-subjects factor, and the groups of treatment (WL vs. REBE vs. REThink) as the between-subjects factor. We followed significant main effects and the interaction effects with Bonferroni adjusted pairwise comparisons of estimated marginal mean, both for within- and between-subjects effects. We computed \(\eta_{\text{p}}^{2}\) as an indicator of effect size for the main and the interaction effects, while for significant pairwise comparisons we computed Cohen’s d index of effect size, based on observed means and standard deviations (within-subjects ds were adjusted for the correlation between the two measurement times).

Next, for the alleged mechanism that showed evidence of change following the expected direction, we correlated the change scores (from pre-test to post-test) with the change scores computed for the emotional outcomes (SDQ emotional symptoms and EATQ-R depressive mood). Finally, we conducted a mediation analysis to verify if indeed the between-groups differences in outcomes evolution are mediated by the changes in this mechanism.

Finally, we conducted an additional mixed within–between MANOVA using SDQ emotional symptoms and EATQ-R depressive mood as dependent variables, to investigate the effect of gender and age as possible moderators of the effect of the interventions in the study.

Missing values and imputations

Missing values for eight subjects were inputted using common procedures (average of the items of the same scale where a few items were missing, the average of the group where a complete scale was not filled, pre-test scores were used for the intermediate assessment scores that were missing). Separate analyses without these inputted values yielded identical results. Only results with the inputted values are reported here after.

Descriptive statistics and baseline comparisons

Descriptive statistics for all measures and effect sizes for changes from pre-test to post-test in each group are presented in Table 2. We first checked for pre-test differences between the three groups on the measures of the hypothesized mechanisms of change. One-way ANOVAs having the treatment group as the independent variable and the CASI and CATS total scores as dependent variables indicated no differences between groups: Welch (2, 90.87) = 0.54, p = 0.582 for CASI total score, and F(2, 139) = 0.23, p = 0.796 for CATS total score.

Changes in the hypothesized mechanisms

Next, we examined the changes in the hypothesized mechanisms across the three groups of treatment by computing a mixed within–between 3 × 3 MANOVA, having as the within-subjects factor the moment of the assessment (pre-test vs. intermediate vs. post-test), and as the between-subjects factor the group of treatment (Waitlist vs. REBE vs. REThink). Both CASI and CATS total scores were introduced as dependent variables. We used repeated univariate contrasts for the within-subjects effects to check where significant changes emerged. We identified a significant within-subjects main effect (pre-test vs. post-test), Wilk’s lambda = 0.82, F(4, 136) = 9.06, p < 0.001, \(\eta_{\text{p}}^{2}\) = 0.18, but no significant between-subjects main effect, Wilk’s lambda = 0.03, F(4, 276) = 0.95, p = 0.438. The interaction effect was statistically significant, Wilk’s lambda = 0.89, F(8, 272) = 2.10, p = 0.036, \(\eta_{\text{p}}^{2}\) = 0.11.

We followed the significant multivariate effects with univariate analyses of variance (sphericity assumed). We found a significant univariate time effect for CASI total score F(2, 278) = 16.41, p < 0.001, \(\eta_{\text{p}}^{2}\) = 0.11 (see Fig. 2), but not for CATS total score, F(2, 278) = 1.46, p = 0.233. The interaction effect (time × group) for CASI total score was also significant, F(4, 278) = 3.517, p = 0.008, \(\eta_{\text{p}}^{2}\) = 0.05, but not for CATS total score, F(4, 278) = 1.90, p = 0.110. Figure 3 presents the evolution of the three groups on CASI total score.

Fig. 2
figure 2

Estimated marginal means for CASI total score across all groups, at the three assessment moments. Statistical analysis has indicated a significant time effect for this mechanism between pre-test and intermediate assessment

Full size image
Fig. 3
figure 3

Estimated marginal means for CASI total score for each group, at the three assessment moments. Statistical analysis has indicated a significant interaction effect for this mechanism between pre-test and intermediate assessment

Full size image

Repeated measures contrasts for CASI total score indicated that significant changes had occurred in the evolution of the groups on this measure between pre-test and intermediate assessment, for both the time effect, F(1, 139) = 25.61, p < 0.001, \(\eta_{\text{p}}^{2}\) = 0.16, and the interaction effect F(2, 139) = 5.80, p = 0.004, \(\eta_{\text{p}}^{2}\) = 0.07. Visual inspection of the time effect depicted in Fig. 2 indicates that scores have decreased between these time points. To explore the interaction effect, we computed within- and between-groups pairwise comparisons of estimated marginal means at pre-test and intermediate assessment, using Bonferroni correction for type-I error. These tests indicated that the control groups showed no change from pre-test to mid-intervention, p = 1.000, while for the REBE condition this change was marginally not significant, p = 0.075. The REThink group had, however, experienced a significant high magnitude decrease in scores, p < 0.001, d = − 0.80. Between-groups comparisons indicated no significant differences between the three groups at pre-test or intermediate assessment (all ps > 0.05). Figure 2 indicates slight increase in scores in the REThink group from intermediate assessment to post-test, while for REBE the scores continue to decrease slowly. Although we did not compute the contrast for the interaction effect from pre-test to post-test, we explored (1) if the participants in the REThink group have reached a significantly lower level of irrational beliefs at the end of the intervention as compared to the initial assessment, and (2) if the decrease in the REBE condition has led to a significant difference when comparing the same time moments. The difference between pre- and post-test for the REThink group was still significant, p < 0.001, d = − 0.66, while for REBE was still marginal, p = 0.063.

Association between changes in outcomes and changes in hypothesized mechanisms

Next, we computed changes scores from pre- to post-test on the CASI total scores which showed significant decrease in the REThink intervention and investigated its association with the pre- to post-test change scores for the emotional outcomes. Table 3 presents these associations across all three groups. CASI total score changes were significantly associated in the expected direction with both SDQ emotional symptoms, r = 0.30, p = 0.033, and EATQ-R depressive symptoms, r = 0.37, p = 0.011 in the REThink condition.

Table 3 Pearson’s r correlation coefficients for the associations between change scores on CASI total scores and emotional outcomes
Full size table

Mediation analysis

Furthermore, we investigated if the changes observed on the two outcome measures for emotional problems are mediated by the changes in mechanisms, which would be a critical proof for the role of mechanism played by irrational beliefs. To do so, we used the procedure and the PROCESS macro developed by Hayes [25]. We tested two separate mediation models in which the predictor was a binominal variable contrasting the REThink and the WL conditions, the mediator was the pre- to-post changes for CASI total score, and the dependent variable was sequentially the change scores for SDQ emotional symptoms and EATQ-R depressive mood. A mediation effect was considered present if: (1) the predictor (the contrast coding for the REThink vs. WL groups) had a significant association with the mediator (the change scores for the CASI total score); (2) a significant association was also present between the predictor and the outcome; (3) the direct effect for the association between the predictor and the outcome decreased when controlling for the mediator and the 95% bootstrap confidence intervals (CI) with 10,000 samples for the indirect effect did not include the value “0”.

The group contrast (REThink = 1; WL = 0) significantly predicted changes in CSI total scores (mediator), β (standardized coefficient) = 0.27, t(91) = 2.73, p = 0.008. The same contrast predicted the changes in SDQ emotional symptoms, β = 0.29, t(91) = 2.85, p = 0.005. This effect decreased but was still significant, β = 0.21, t(90) = 2.01, p = 0.040, after introducing the mediator in the model, which was also an independent significant predictor, β = 0.28, t(90) = 2.75, p = 0.007, of the SDQ emotion symptoms change scores. The 95% bootstrap CI for the indirect effect [0.09, 0.86] did not include 0. Taken together, these results indicate that a partial mediation was present. Table 4 and Fig. 4 summarize the results for this mediation analysis.

Table 4 Result of the mediation analysis for SDQ emotional symptoms
Full size table
Fig. 4
figure 4

Standardized regression coefficient for the relationship between group contrast (Waitlist vs. REThink) and changes in SDQ emotional symptoms as mediated by changes in CASI total score. The direct effect when controlling for the mediator is presented between brackets

Full size image

For EATQ-R depressive mood, the group contrast was again a significant predictor, β = 0.35, t(91) = 3.56, p < 0.001. The direct effect was also significant, β = 0.24, t(90) = 2.56, p = 0.012 when controlling for the mediator, which also predicted changes scores on EATQ-R depressive mood, β = 0.41, t(90) = 4.36, p < 0.001. Once again, the 95% bootstrap CI for the indirect effect [0.28, 2.25] did not include 0, which points to a partial mediation. Table 5 and Fig. 5 summarize the results for this mediation analysis.

Table 5 Result of the mediation analysis for EATQ-R depressive mood
Full size table
Fig. 5
figure 5

Standardized regression coefficient for the relationship between group contrast (Waitlist vs. REThink) and changes in EATQ-R depressive mood as mediated by changes in CASI total score. The direct effect when controlling for the mediator is presented between brackets

Full size image

Moderation analysis

Finally, we were interested to explore if the effect of the intervention on emotional outcomes was moderated by gender or age. To do so, we ran a second mixed within–between MANOVA, having as the within-subjects factor the moment of the assessment (pre-test vs. post-test). As between-subjects factors we included the group of treatment (Waitlist vs. REBE vs. REThink), the gender of the participants (male vs. female) and their age, as a binominal variable, coding for pre-adolescents and adolescents (ages between 9 and 12 years vs. ages between 13 and 17 years). We also included 2-way interaction effects between group and gender, and group and age. As dependent variables, we introduced the outcomes that showed evidence for the efficacy of the REThink game in the initial analysis, namely SDQ emotional symptoms and EATQ-R depressive mood. Similar to the initial analysis on efficacy, the multivariate results indicated a significant time effect, Pillai’s trace = 0.139, F(2, 132) = 10.67, p < 0.001, \(\eta_{\text{p}}^{2}\) = 0.14, a significant time × group interaction effect, Pillai’s trace = 0.10, F(4, 266) = 3.53, p = 0.008, \(\eta_{\text{p}}^{2}\) = 0.05. All other interaction effects for time × gender, time × age, and the three way interactions between time × group × gender, and time × group × age, were not significant (all ps > 0.05). As for between-subjects analysis, we found only a significant effect for age, Pillai’s trace = 0.05, F(2, 132) = 3.36, p = 0.038, \(\eta_{\text{p}}^{2}\) = 0.05. The effects for group, gender, and the interaction effects for group × gender, and group × age were not significant (all ps > 0.05). These results point to the idea that the efficacy of the intervention did not vary based on the age or gender. However, age had a significant main effect, suggesting that pre-adolescents and adolescents responded differently on the measures assessing emotional outcomes. Univariate between-groups analysis indicated that the effect of age was present on both SDQ emotional symptoms, F(1, 133) = 4.58, p = 0.034, \(\eta_{\text{p}}^{2}\) = 0.03, and EATQ-R depressive symptoms, F(1, 133) = 6.59, p = 0.011, \(\eta_{\text{p}}^{2}\) = 0.05. Figures 6 and 7 present the effects of age on SDQ emotional symptoms and EATQ-R depressive mood, respectively.

Fig. 6
figure 6

Estimated marginal means for SDQ emotional symptoms (across all groups, and across pre-test and post-test) as a function of age group. Statistical analysis has indicated a significant main effect of age on this outcome

Full size image
Fig. 7
figure 7

Estimated marginal means for EATQ-R depressive mood (across all groups, and across pre-test and post-test) as a function of age group. Statistical analysis has indicated a significant main effect of age on this outcome

Full size image

Discussion

The present study investigated the mechanisms of action and moderators of REThink, a therapeutic game designed to help children develop emotional resilience. The results pinpoint changes in irrational beliefs as a mediator for the REThink intervention. Specifically, our findings suggest that a decrease in irrational beliefs contributed to a decrease in depressive mood and negative emotions overall. Irrational beliefs decreased at the end of the intervention, along with a decrease in emotional symptoms and depressive mood at the same time, and they were found to mediate obtained effects. Our results are in part consistent with those found in previous meta-analyses of cognitive-behavioral interventions for children and adolescents [26], and in previous treatment outcome studies indicating that cognitive distortions are mediators of treatment effect [27]. Indeed, according to CBT theory [19, 28, 29], distorted beliefs generate dysfunctional feelings and behaviors. However, our results add more specificity to these findings, in that they indicate the type of beliefs—irrational ones that acted as mechanisms of change for our intervention. At the same time, our results indicate that no such effect was found for negative automatic thoughts.

Our findings are in line with Rational Emotive Behavioral Theory, which predicts that an improvement in treatment is associated with a reduction of irrational beliefs rather than in automatic thoughts. While many forms of CBT target the change in automatic negative thoughts, our interventions was founded on the general REBT principle [12] that interventions should strive to directly target core dysfunctional beliefs, which can save time and, if successful, also lead to changes in automatic thoughts [24]. Although investigating such a path was beyond the scope of our paper, our results confirm changes in children and adolescents’ irrational beliefs as a mechanism of change for this intervention. No significant changes in irrational beliefs were found for the REBE group and this is an unexpected result, given the documented efficacy/effectiveness of this program [18]. However, considering the low changes found in terms of outcomes for this intervention group and a greater satisfaction with the REThink game (David, Cardos, and Matu, under-review), this might suggest that a habituation effect relating to the fact that all students previously received REBE as regular service in the school. Indeed, mental health promotion campaigns based on REBE have been previously implemented by the school psychologist and thus children and adolescents in all groups have already been acquainted to the subjects like those approached by the REBE group. Although the focus of both interventions was on reducing irrational beliefs, the REThink group might have worked better due to using game-based indirect tools for change.

As for moderators, age and gender were examined as possible predictors of response to the treatment. Our results indicate that the game was equally effective for children and adolescents. The obtained results are similar to those reported by previous meta-analyses looking at age effects on cognitive-behavioral interventions which indicated that they are equally effective across age groups [30].

Our study was not without limitations. We used standard assessment points, which means that we might have missed effects that unfolded throughout the study at more nuanced time points. To fully understand this mechanism of change and the dosage of the intervention require to impact it, future studies should include multiple in-treatment assessment phases, to gage the specific time points (and techniques) when a change in irrational/dysfunctional beliefs occurs. Another limitation is the fact that both interventions were delivered in schools and students have been previously exposed to such programs. Future studies will need to control for this variable in their design to be able to clarify mechanisms of change in REBE.

Our study adds relevant knowledge for determining mechanisms of change in empirically supported mental health prevention efforts in children and adolescents. Thus, results obtained contribute to the understanding of the mechanisms that take part in the process of change during CBT-based prevention programs for youth emotional disorders. Our findings suggest that changes in irrational beliefs trained during the online therapeutic game are mediating the change in symptom reduction and therefore supports the theoretical model upon which REThink was founded.