Introduction

Community surveys indicate that 13–24 % of children in Brazil are affected by mental disorders, the highest rates being found in low-income communities [1, 2]. A survey assessing child and adolescent mental disorders in a representative sample of the population of the United States reported prevalence rates of 23.4 % in the last 30 days and 40.3 % in the last 12 months [3]. Despite the fact that mental disorders in childhood are so prevalent and have been associated with long-lasting impairments, there are significant gaps in services for those affected, especially in low- and middle-income countries [4].

In children and adolescents, mental disorders are rarely treated. In the United States, it is estimated that only approximately 14 % of youths with a mental disorder have received pharmacological treatment in the last 12 months [1]. In Brazil, it is estimated that no more than 20 % of individuals with attention deficit hyperactivity disorder (ADHD), including adults, receive first-line medications [5]. In developing countries, the average rate of receiving treatment for a mental disorder is 159/100,000 affected children, whereas it is 664/100,000 affected adults [2]. This discrepancy could be a attributable to various factors, such as a lack of public policies targeting childhood mental disorders, a lack of services, a shortage of trained specialists, limited community knowledge of such disorders, and the stigma associated with mental illness [3, 4, 6].

The school system and teachers are important elements of activities aimed at promoting childhood mental health and treating mental disorders, especially in low- and middle-income countries [4, 7]. In the school environment, there is often a lack of knowledge about mental health and a stigma attached to mental illness, both of which impair the ability of teachers to identify children suffering from mental disorders, as well as to educate and relate to those children [8]. Although school teachers are familiar with normal and abnormal development, they are usually not adequately trained to recognise and manage children with mental disorders or special needs in general [9]. A survey conducted in inner-city elementary schools in the United States evaluated teacher beliefs about mental health and found that a great majority of them reported had received minimal formal education in mental health problems, before or after graduation [8]. A study conducted in South Africa reported a significant lack of knowledge and misconceptions about ADHD among primary school teachers [10]. Greater knowledge about mental disorders has been associated with reduced stigma [2, 11]. Therefore, educating teachers on child mental health is a critical component in the agenda for mental health promotion [12].

There have been few methodologically sound studies designed to test educational programs targeting teacher knowledge of child mental health issues. In a study conducted in England, 80 primary schools were randomised into two groups: one in which the teachers received training in mental health and one in which they did not [13]. The results of that study have yet to be published. In Brazil, we identified only one such study, the primary objective of which was to test an intervention aimed at reducing school dropout rates, based on the concept that mental health disorders constitute the primary cause of school absenteeism and dropout. The study compared two schools with similar dropout rates, offering a package of universal preventive interventions to one school and nothing to the other. After the one-year intervention, the rates of truancy and absenteeism were significantly lower for the school at which the interventions had been implemented [14].

Despite the potentially significant effects of educational programs for teacher training, there are major methodological barriers to testing such programs, as well as to scaling them up and sustaining them over time. Such barriers include the high cost of hiring the necessary trained personnel, the lack of access to all regions of a country (especially in those of continental dimensions), the need for periodic refresher sessions, and the need for continuous contact with experts for consultations. Web-based programs show great potential for overcoming these limitations [15], and initial studies of web-based programs focusing on specific disorders have produced promising results [16, 17].

In this study, we aimed to develop a web-based program to educate primary school teachers on child mental disorders and to test its effectiveness in comparison with the same program delivered via print and video materials only, as well as in comparison with no intervention. The only differences between the web-based and non-web-based programs were the web-based tools used in the former: a discussion forum to interact with consultants and a web conference with a child psychiatrist as a refresher session. These tools were especially designed to promote participatory learning as a means of reinforcing the concepts. We hypothesised that the web-based program would be more effective in increasing knowledge, as well as in reducing misconceptions and stigma, related to child mental health among teachers.

Methods

Design

This was a cluster-randomised, parallel group clinical trial comparing two active interventions and a control (no intervention): receiving web-based interactive education (WBIE); receiving text and video-based education (TVBE); and being on a waiting list (WL, no intervention). Interventions were implemented over a period of 3 weeks.

Participants

The trial was conducted in the city of Campos do Jordão, in the state of São Paulo, Brazil. The São Paulo State Department of Education approved the project. Before beginning the trial, we presented the program to the principals of all public primary schools in the city. After discussions on the benefits of the program, all of the principals consented to allowing their schools to participate in the trial. The final sample was composed of nine public primary schools, all of which were in urban areas and were similar in terms of infrastructure and other characteristics.

Determination of eligibility

To be included in the study, schools were required to have the infrastructure necessary to receive a web-based program and the staff were required to give consent. The nine participating schools were visited before and during the implementation of the program, and all were considered eligible for inclusion in the study.

Randomisation

The number of students and teachers at each school were considered stratification variables. The distribution of these variables was calculated, and the two tertiles were used to define three blocks (of three schools each). We subsequently used simple randomisation (with a random number table) within each block to assign schools to one of the three trial groups.

Interventions

The concept of the educational program is based on similar programs from other countries [18, 19] and was developed in consultation with experts in child and adolescent psychiatry and early education who are members of the Brazilian National Institute of Developmental Psychiatry for Children and Adolescents. The educational content conceptualises mental disorders according to current scientific evidence, in accessible language, and addresses common misconceptions, together with the associated stigmas. The program includes descriptions and examples of the clinical manifestations of mental disorders, as well as general strategies to manage them in the classroom context. The tele-education tools, embedded in the learning management system (Cybertutor), were developed by experts in the Department of Telemedicine at the University of São Paulo School of Medicine.

The educational program is composed of the following elements: a website providing the training schedule and information regarding the challenges of the program, as well as information about the research team and other aspects of the program; a tutorial of the learning management system; and six educational videos that include interviews with child psychiatrists, teachers, and parents of children with mental disorders. The tutorial explains how to use the Strengths and Difficulties Questionnaire. The first two videos address basic and general concepts about child mental disorders, such as aetiology, risk factors, functional impairment, consequences throughout development, and treatment principles. Each of the four other videos addresses clinical presentations and classroom management principles for children with depressive and anxiety symptoms; inattentive and hyperactivity symptoms; conduct problems; and problems related to social interaction. Participating teachers were provided with a booklet detailing the content presented in the videos. The program also features a web forum to be joined by all teachers and members of the research team. At the end of the third week, there was a web conference involving all of the teachers and a child and adolescent psychiatrist. The educational components are depicted in detail in Chart 1. The WBIE intervention included all of the components described above. The TVBE intervention was restricted to the tutorial, the videos, and the booklet.

Chart 1 Components of the web-based educational program
Full size table

The interventions were implemented simultaneously at all of the participating schools. The WBIE and TVBE interventions both consisted of 9 h of training, delivered in one three-hour session per week for 3 weeks. The sessions were scheduled for periods during which teachers are routinely assigned to training activities. Teachers allocated to the WBIE group were able to access the program at any time, via the Internet. Teachers assigned to the TVBE group were also able to access the program at any time, via the booklet and a DVD of the same videos and tutorial, but had no access to the interactive tools (web forum and web conference).

Measures

Before and immediately after the intervention, teachers in all three groups (WBIE, TVBE, and WL) were assessed in terms of socio-demographic characteristics, as well as their knowledge, beliefs, and attitudes related to child mental health. The WBIE group answered a satisfaction questionnaire specifically designed to evaluate web-based educational programs.

The assessment questionnaire comprised 48 questions among different domains: 27 assessing knowledge of child mental health; and 21 assessing beliefs regarding and attitudes towards mental illness in childhood. Knowledge of mental disorders was assessed with dichotomous (true/false) questions, and the responses were treated as binary variables (false = 0; and true = 1), with the objective of quantifying learning in regard to the identification and classroom management of mental disorders in general and to each of the mental disorders addressed in the educational material. Examples of such questions include statements like “Children with anxiety disorders are inattentive, restless, and sad.” and “Students who are eager to leave the classroom should be allowed to do so whenever they want.” Questions about beliefs related to child mental health were based on the questionnaire prepared by the World Psychiatric Association Taskforce on Awareness [20]. Those questions assessed stigmatised opinions regarding mental health, pre-conceived ideas about vulnerability to mental health problems, and other common myths regarding mental health. (e.g. “What is the most likely reason a child is angry all the time, or feels sad or worried, or simply cannot have a relationship with anyone?”). The questions about attitudes towards mental health assessed behaviours adopted by teachers in relation to a student or colleague with probable mental health problems, as well as evaluating teacher interest in the issue (e.g. “In the last month, in which of the following activities related to dealing with mental health problems did you participate?”). The measures are available upon request.

Sample size calculation

The sample size calculation, which was conducted with the statistical power analysis program G*Power 3 [21], considered the following parameters: estimated effect size (δ = 0.14); three intervention groups; power (1 − β error probability) set at 0.8; and two measurements correlated at 0.4. As a result, the sample size required to achieve the pre-established parameters was 123 subjects. Because we estimated a minimum dropout rate of 20 % based on a pilot study, we invited all of the teachers from all nine of the participating schools. Schools were randomly stratified according to the numbers of students and teachers. One important assumption behind this calculation should be considered: although the randomisation was conducted at the school level, we did not assume a multilevel structure. Therefore, we assumed that there was a minimal or no intraclass correlation between schools, values near zero indicate that individuals within clusters (schools) are no more similar to each other for the variable of interest than are individuals across clusters. Because this is a major assumption, we have handled the possibilities of variation among the nine schools with robust standard errors.

Statistical analysis

We conducted complete case (also known as “available case”) and intention-to-treat (ITT) analyses. Complete case analysis included only those teachers whose outcomes are known, and ITT analysis included all teachers from all randomised schools. The baseline distributions for the primary and secondary outcome variables were classified as zero-inflated, normal, or gamma. Baseline significance tests comparing groups of teachers were conducted via Student’s t tests or Mann–Whitney tests for continuous outcomes (depending on the homogeneity of variance and normality of the distribution of the data), and Chi-square tests were used for binary outcomes. Regression models were used to test the effectiveness of each intervention for each outcome. For categorical data, we employed Poisson regression, whereas we employed linear regression for normally distributed, continuous outcome data.

The data were analysed assuming the dependence of observation (teacher reports), because ignoring the clustering would underestimate the standard errors and the corresponding confidence intervals. Consequently, the standard errors were adjusted for the survey design (i.e. taking the clusters into account), resulting in robust standard errors.

To evaluate the effect of the intervention under ITT inference, we performed sequential regression multiple imputation. This approach fills in the missing data on a variable-by-variable basis, matching the imputation model to the distribution pattern of each variable. The imputed values were constrained minimums and maximums of each outcome/predictor. Five imputed datasets were created as recommended by Rubin [22, 23] and pooled to produce the estimates. The multiple imputation assumed the missing completely at random data mechanism because of unexpected teacher personal events (vacation, relocation to another school district, etc.) The variables selected for imputation were baseline values (imputed and used as predictors) and type of school intervention (WBIE, TVBE, and WL; only used as predictors). Multiple regressions followed the same structure used in complete case analyses. Statistical analyses were conducted with STATA, version 11.0 (StataCorp LP, College Station, TX), and imputation was conducted with SPSS Statistics software package, version 19.0 (IBM Corporation, Armonk, NY, USA). The level of significance was set at p < 0.05.

Ethical aspects

The project was approved by the educational administrative authorities of the city and by the Research Ethics Committee of the University of São Paulo School of Medicine Hospital das Clínicas (Protocol no. 1130/08). Personal information was treated confidentially and anonymously. Teachers and school principals agreed to participate, and the participating teachers gave written informed consent form. The trial was registered in the ClinicalTrials.gov database (identifier: NCT01860313).

Results

The study included all public primary schools in the city, randomly allocated to one of the three groups. Of the 75 teachers at the three schools randomised to the WBIE group, 52 took part of the program and completed the assessments, compared with 32 of the 67 teachers at the three schools randomised to the TVBE group. Adherence to the training program was ≥80 % for both groups, with no significant difference between the two. Of the 71 teachers at the three schools randomised to the WL group, 31 completed the assessments. The CONSORT diagram is presented in Fig. 1. As can be seen in Table 1, there were no significant differences among the groups in terms of any socio-demographic characteristic, except sex composition (female: 100, 96.9, and 92.9 % for the WBIE, TVBE, and WL groups, respectively; p = 0.047).

Fig. 1
figure 1

CONSORT flowchart

Full size image
Table 1 Comparison between groups in regard to socio-demographic characteristics (n = 115)
Full size table

Knowledge

We used multiple linear regression to determine the group effects with respect to post-intervention changes in knowledge in eight domains: basic concepts of mental health; classroom management of children with mental disorders; identification of mental disorders in general; and (five domains of) specific knowledge on the identification and management of mental disorders (anxiety, depression, ADHD, conduct disorder, and social problems). We also calculated the total score for all eight domains combined.

Considering the complete case analyses, we found that the WBIE group was superior to the TVBE group in 4 of the 8 domains—classroom management (β = 0.59, p = 0.02); identification of mental health problems (β = 1.50, p < 0.01); and specific knowledge of depression (β = 0.71, p = 0.02) and conduct disorder (β = 0.45, p < 0.01)—as well as on total score (β = 1.44, p = 0.01). Considering the ITT analyses, we found that all of the statistically significant differences cited above were maintained, with exception of the total score (β = 0.76, p = 0.258). In the complete case analysis, the WBIE group was superior to the WL group in 3 of the 8 domains: identification of mental health problems (β = 0.51, p = 0.03); specific knowledge of depression (β = 0.48, p = 0.03); and specific knowledge of conduct problems (β = 0.24, p = 0.042). In the ITT analysis, the WBIE group was superior to the WL group only in the specific knowledge of depression domain (β = 0.47, p = 0.02). In the complete case analysis, the TVBE group was inferior to the WL group in 3 of the 8 domains: classroom management (β = −0.68, p = 0.02); specific knowledge of anxiety (β = −0.16, p = 0.043); and specific knowledge of conduct disorder (β = −0.21, p = 0.046). In the ITT analysis, the TVBE group was inferior to the WL group only in the specific knowledge of conduct disorder domain (β = −0.33, p = 0.043). Tables 2 and 3 show the results of the complete case and ITT analyses, respectively.

Table 2 Effects of the interventions on knowledge, by pair-wise comparison and complete case analysis
Full size table
Table 3 Effects of the interventions on knowledge, by pair-wise comparisons and ITT analysis
Full size table

Beliefs and attitudes

We also used multiple linear regression to determine the group effects with respect to post-intervention changes in beliefs and attitudes. The WBIE group was superior to the TVBE group regarding stigmatised concepts in mental health (β = −0.92, p < 0.001), meaning that after the intervention, members of the WBIE group had fewer stigmatised opinions than did those of the TVBE group. The WBIE group was also superior to the WL group with regard to non-stigmatised concepts in mental health (β = 1.18, p < 0.001). The WL group was superior to the TVBE group regarding stigmatised concepts in mental health (β = 0.98, p = 0.001), meaning that after the intervention, members of the TVBE group had more stigmatised opinions than did those of the WL group. No differences in attitudes were detected in any of the pair-wise comparisons. The results of the ITT analysis were consistent with those of the complete case analysis. Tables 4 and 5 show the results of the complete case and ITT analyses, respectively.

Table 4 The effect of interventions over beliefs and attitudes, by pair-wise comparison and complete case analysis
Full size table
Table 5 The effect of interventions over beliefs and attitudes, by pair-wise comparison and ITT analysis
Full size table

Satisfaction

After the intervention, we assessed satisfaction on the part of the teachers in the WBIE group (n = 52). Of those teachers, over 60 % were satisfied with the training and 70 % considered the information presented in the learning management system important.

Discussion

In this cluster-randomised controlled trial, we tested the effectiveness of a web-based educational program designed to educate primary school teachers on the recognition and classroom management of children with mental health problems, in comparison with that of an otherwise similar text- and video-based educational program and with that of no educational program (the control). Teachers trained with the web-based program showed greater gains in knowledge than did those trained with the text- and video-based program and those receiving no such training. We were surprised to find that the teachers in the control group gained more knowledge than did those trained with the text- and video-based program. In regard to beliefs about mental health, the post-intervention prevalence of stigmatised concepts was lower in the WBIE group than in the TVBE group, as well as being lower in the TVBE group than in the WL group, whereas the post-intervention prevalence of non-stigmatised concepts was higher in the WBIE group than in the WL group. There was no effect on attitudes regarding mental health. Our results are consistent across different analytical strategies (complete case and ITT), although a lesser positive effect was demonstrated by the ITT analysis, as expected.

The present study showed that teachers randomised to a web-based educational program presented an increase in knowledge regarding mental health. It should be borne in mind that both active groups had access to the same educational content. The difference between the two was in the WBIE group access to a discussion forum and to web conferencing. Our results indicate that these components are related to greater improvements in knowledge gain. Nevertheless, the increase in knowledge and the decrease in stigma were not followed by a modification in attitudes in regard to mental health.

The intervention had lesser effects on beliefs and attitudes than on knowledge. This finding has been previously reported [24], underscoring either the difficulty in changing attitudes or the difficulty in measuring modifications. One possible interpretation is that the intervention had no effect on attitudes. Alternatively, it is possible that we were not able to detect the modification, either because of the short interval between assessments (only 3 weeks) or because more ecological measures (of school performance, absenteeism, the school dropout rate, referrals of children and adolescents to mental health services, and changes in school environment) are more well suited to detecting such changes.

Motivation and adherence are important aspects that are likely to influence the effects of such interventions. Teachers from one of the schools randomised to the web-based program showed poor adherence and low motivation to participate because of political conflicts with the municipal department of education, which may have negatively affected the results. To investigate the effect of adherence and motivation as predictors of compliance, a complier average causal effect should be estimated, which is beyond the scope of our study. Indicators of low teacher motivation included a lack of access to the educational program outside school time, low participation (at one WBIE group school), and a low completion rate of post-intervention questionnaires. However, as negative attitudes towards mental health are frequently found in the educational field, it is important that a training program has the ability to overcome these issues and present positive results despite them. In this respect, the attrition rate was high for all groups (especially for the WBIE group)—31, 52, and 56 %, for the WBIE, TVBE, and WL groups, respectively. The high rates of attrition might be attributable to a number of factors not associated with the study (high rates of teacher absenteeism and teacher unavailability to complete the assessments and join the program due to high workloads), as well as to factors associated with the characteristics of the interventions or the theme of the study. In this regard, the authors of other studies testing interventions for school teachers have chosen to include only those who volunteer to participate [24, 25]. The differences between our complete case and ITT analyses suggest that such a strategy would favour the intervention tested here, although the intervention would probably have a more limited effect in terms of real-world circumstances.

Despite these limitations, our study has important strengths. The schools were randomised to the different interventions, and preferences towards a particular program therefore played no role in the results. In addition, the results indicate that web-based interactive tools were consistently more effective than traditional educational tools in increasing knowledge and reducing the prevalence of stigmatised concepts, and the teachers presented an overall positive attitude towards those tools. Our findings have implications for the development of sustained educational programs in child mental health for teachers, especially in low-resource environments. Future studies should assess how gains in knowledge of mental disorders translate to the school environment.

Children with mental disorders are consistently under-identified and under-treated worldwide. Stigma, misconceptions, and ignorance are major barriers to access to mental health services, and the educational system plays a key role in the referral process. Educating teachers to recognise and manage children with mental disorders is an important initiative, and web-based interactive technologies have the potential to be important tools for disseminating sustainable educational programs.