Introduction

Around 50–60% of children with conduct problems at home also show clinically significant problem behaviours in day-care or school settings i.e., displaying pervasive conduct problems [9, 22, 25, 34]. These children are likely to be not only more aggressive but also have problems interacting with parents, peers and teachers. They are at higher risk for developing serious problems later on as compared to children with conduct problems exhibited only in one setting [12, 19]. To reduce the risk for children with pervasive conduct problems, they need to improve their everyday psychosocial functioning in day-care/school settings, in addition to enhance their positive behaviours at home. If such changes do not occur, negative social day-care or school experiences may further contribute to the development of conduct problems [36].

Conduct problems in young children are often treated with parent training (PT) procedures having produced well-documented positive changes in parental skills and children’s behaviour at home [15, 28, 30, 34, 36]. In the present study, positive outcomes at home were found for about two-thirds of the children after treatment and at the 1-year follow-up [20]. Further, child therapy (CT) approaches targeting social competence in the child have been found to add to the effects of PT, in particular regarding social problems with peers [33]. Whether PT produces positive changes in child behaviours in day-care or school settings is a less clear issue having received limited attention in research but has also produced conflicting findings. For example, previous studies have not found improvements in child behaviours in day-care/school after effective PT [4, 7, 13, 29, 33], while others have reported positive generalisation effects [14, 18, 36]. About one-third of the children whose parents have received PT still exhibited peer-relationship problems, academic and social difficulties at school 2–3 years later [36]. Studies have shown that treatments targeting both school risk factors and home risk factors produce more effective changes in children’s behaviour problems at school as at well as home [8, 24, 25].

To evaluate cross-setting generalisation effects Mc Neil [18] emphasised the need to document clinically significant improvements after treatment in the primary targeted setting, i.e., the child’s overall adjustment has to be affected to such a degree that improvements could be observed across settings. A further prerequisite is to document that children referred because of conduct problems at home also show sufficient levels of conduct problems in day-care and school, so that positive changes achieved in the child’s behaviours at home are possible to generalise back to these settings.

The overall purpose of this study was to investigate generalisation effects of treatment with Incredible Year’s PT and PT combined with CT (PT+CT) [36] administered to an outpatient clinic sample of children aged 4–8 years referred because of ODD or CD problems at home. Generalisation effects were assessed across time, settings and other areas potentially related to child aggression. In specific, the following hypotheses were addressed:

  1. (1)

    Does improvement in child aggression levels at home also generalise to reduction of aggression problems in day-care/school settings? What is the influence of treatment condition and does combined intervention with PT+CT (as compared to PT only) further add to positive generalisation effects?

  2. (2)

    Does improvement in child aggression levels at home also generalise to the following areas related to aggression: child attention problems, social competence, social problem-solving strategies, peer-interactions, student–teacher relationships and parents involvement in day-care/school?

  3. (3)

    Are positive generalisation effects obtained after treatment also maintained 1-year later?

Method

Participants

The subjects consisted of 127 children, 4–8 years old, referred for treatment to 2 child psychiatric outpatient clinics because of oppositional or conduct problems as experienced by parents. The study was conducted in two university cities in Norway, Trondheim and Tromsø. Exclusion criteria were children with gross physical impairment, sensory deprivation, intellectual deficit or autism. Three families (2.4%) dropped out during the treatment period and these families were removed from the analyses. All but one family in the study were native Norwegians. Child and family characteristics are presented in Table 1.

Table 1 Child and family characteristics. (Percentages and number of subjects within parenthesis)
Full size table

Procedures

Information about the study was given to referral agencies or professionals such as teachers, physicians, health nurses and child welfare workers throughout the project period. All clinically referred children were first screened by means of the Eyberg Child Behavior Inventory (ECBI) [26] using the 90th percentile as a cut-off score according to Norwegian norms [23]. Children who attained such a cut-off score or higher were subsequently interviewed by one of three trained interviewers using the KIDDIE-SADS (see description below), and those who received a possible or definitive diagnosis of ODD and/or CD were offered to participate in the study. The term “possible diagnosis” refers to those children who scored one criterion less than the 4 required for a formal DSM-IV ODD diagnosis or the 3 items required for a formal DSM-IV CD diagnosis, and had diminished function, a procedure suggested by Angold and Castello [3]. All clinic children included had severe conduct problems at home as rated by their parents on the ECBI.

To assess conduct problems in day-care the aggression subscale on the Preschool Behavior Questionnaire (PBQ) [5] was used, and for school children, the aggression subscale on the Teacher Report Form (TRF) [2] was used. To estimate optimal cut-off points between our clinic group and normative data obtained from a national survey [23], ROC curve analysis was used. The results of these analyses showed that a value of nine corresponding to the 80th percentile was found to be the optimal cut-off point for the PBQ, and a value of six was the optimal cut-off point for the TRF, corresponding to the 88th percentile. About 83% of the children in the present study scored above these clinical cut-off points before treatment.

Treatment

Parent training

A total of 10–12 parents met in groups with 2 therapists at the clinic during 12–14 weeks for a weekly 2-h session and participated in the Basic Incredible Years Parenting Programme. This programme teaches parents the use of positive discipline strategies, effective parenting skills, strategies for coping with stress and ways to strengthen children’s social skills.

Child therapy

A total of 6 children and 2 therapists met weekly in 2-h session during 18 weeks at the Incredible Years Dinosaur School Programme administered at the clinic. The treatment programme addresses interpersonal difficulties in young children with ODD and aims at increasing social skills, conflict resolution skills, playing and cooperation with peers. For further descriptions of the two treatments, see Webster-Stratton and Hammond [33].

In total 47 of the children in the study were randomised to PT treatment and 52 children to combined PT+CT treatment.

Waiting-list group condition (WLC)

A total of 28 families assigned to the WLC had no contact with the clinic or researchers between pre- and postassessments in the treatment groups. For ethical reasons these families in the waiting-list control group were offered treatment after 6 months and were excluded in the 1-year follow-up assessment.

Strategies to promote generalisation in day-care/school setting

All parents in the study were asked to inform the teacher about how they worked with promoting positive behaviours in the child in the PT programme. The teachers were offered to participate in one meeting together with the parents, and one of the therapists to receive information about the treatment programme. About 60% of the teachers participated in such a meeting. Therapists in the CT group sent six letters to the teachers and had 1–2 telephone calls about current issues addressed in CT sessions.

Assessment

Parental report

Eyberg Child Behavior Inventory (ESBI) is a 36-items inventory for parents to assess child conduct problem behaviours among children aged 2–16 years on a 1–7 scale [6] [26]. Total scores range from 36 to 264. In this study, only the total intensity scores were used to indicate frequency of conduct problems. Reliability for internal consistency was 0.82, and test–retest has been reported to be 0.86 [32].

Child Behavior Checklist (CBCL). The problem part of the CBCL consists of 118 items rated by parents on a 0–2 scale, addressing emotional and conduct problems in the child [1]. In this study, the Aggression and Attention subscales were used, in addition to the Internalising syndrome scale. Total scores for the subscales vary between 0 and 40 and 0–22, respectively, and for the syndrome scale from 0 to 62. Test–retest reliability has been found to be high, and inter-parent agreement to range from 0.65 to 0.75 for these subscales [1].

KIDDIE-SADS. This is a semi-structured diagnostic interview designed to assess psychopathology in children and adolescents according to DSM-IV criteria [16]. Here, only the diagnoses most relevant for the 4–8-year age group were included being based on parental reports of current episodes of psychopathology among children. Three trained persons conducted the diagnostic interviews. All interviews were recorded and random checks showed high reliability, in that all Kappa scores were above 0.90.

Child report

The Wally Child Social Problem-Solving Detective Game (WALLY). This measure is designed to assess both qualitative and quantitative dimensions of the child’s problem-solving ability [35]. The child is presented with 12 illustrations of hypothetical social problem situations and then asked to resolve the problems in the picture. The child is encouraged to give as many answers as he/she can for each situation. The answers are scored on the basis of 16 prosocial or 17 negative solutions and a ratio of positive to negative solutions is computed. Inter-rater reliability for coding responses was checked for 20% of the Wally-tests and agreement was above 0.80.

Teacher report

Preschool Behavior Questionnaire (PBQ). The PBQ includes 30 items of conduct problems and are completed by day-care teachers for children aged 4–6 years [5]. In this study, items in the aggression (7 items), hyperactive/distractible (4 items) and internalisation (5 items) subscales were used. Items were scored on a 0–2 scale, and sum scores for the subscales range from 0 to 14, 0 to 8 and 0 to 10, respectively. Internal consistency was 0.80, 0.63 and 0.82 and in the Behar study [5] test–retest reliabilities for these subscales were 0.93, 0.94 and 0.60.

Teacher Report Form (TRF). On the TRF, teachers are asked to rate school children’s academic performance, four general adaptive characteristics, and 112 conduct problems scored on a 0–2 scale [2]. In this study, the aggression and attention subscales and the internalising syndrome scale were used. Sum scores for these scales were from 0 to 50, 0 to 40 and 0 to 70, respectively. Test–retest reliability for the problem scales has been found to be 0.90, and 0.55 for inter-rater agreement [2]. For both the PBQ and the TRF measures, the aggression subscales are compared with data from the Norwegian normative study to estimate optimal cut-off points between a school sample and our clinical group [10].

Social Competence and Behavior Evaluation (SCBE). The SCBE includes 80 items designed for day-care and schoolteachers to assess patterns of social competence, affective expression and adjustment difficulties in the child [17]. In this study, we used the subscales for social competence and peer-interactions (egotistic-prosocial). Scores range from 0 to 5, and sum scores for the subscales range from 0 to 50 and from 0 to 200, respectively. Test–retest reliabilities range from 0.74 to 0.87 [17], and alpha coefficients for internal consistency varied from 0.67 to 0.77 for the different subscales.

Student–Teacher Relationship Scale (STRS). The STRS is a 30-item rating scale (with scores ranging from one to five) designed to assess teacher perceptions of their relationships with a particular child [21]. In this study, total scores for overall relational quality were used and sum scores range from 30 to 150. Internal consistency for this scale showed an alpha level of 0.67.

Involve-Teacher (INVOLVE-T). This measure based on teacher ratings and derived from the Oregon Social Learning Centre (OSLC) questionnaire addresses the degree of parental involvement in day-care/school [32]. In this study, the sum score of parental involvement with teachers in day-care/school (7 items include parent calls to teacher, attendance to school/day-care meetings, questions asked) were used. Scores range from one to five (five indicating high involvement), and an average score was computed. Internal consistency was found to be 0.79.

Statistics

Analyses of differences between group means between pre- and posttreatment were conducted by means of ANCOVAs using pretreatment scores as covariate and treatment condition (PT, PT+CT and WLC) as a between-group factor, followed by Bonferroni post hoc test when overall effects were significant. For analyses of follow-up data, repeated measures of ANOVA were used to examine main interaction effects from posttreatment to the 1-year follow-up. PBQ and TRF scores were combined and transformed into z-scores. Effect sizes (eta square) were estimated using Cohen’s criteria [11] for small (1–5.8%), medium (5.9–13.7%) and large (13.8% and more) effects. An alpha level of P<0.05 indicated a statistically significant result.

Results

Generalisation of aggression problems across day-care/school settings

The results of ANCOVAs showed a significant main effect of treatment condition, F(2,108)=7.20, P<0.01, ES=11.8%, on child aggression problems in day-care/school settings (Means and SDs are presented in Table 2) posttreatment, using pretreatment scores as covariate. Subsequent Bonferroni post hoc analyses showed that children in combined PT+CT treatment significantly reduced their aggression levels in day-care/school settings after treatment as compared to those in the PT (P<0.05) and the WLC (P<0.01) groups. However, the two latter groups did not differ significantly from each other.

Table 2 Means and SDs for child conduct and internalising problems (TRF/PBQ z-scores), social competence, peer-interactions and student–teacher relationships pre- and posttreatment and at the 1-year follow-up
Full size table

Further analyses of changes across posttreatment and follow-up evaluations by means of repeated measures of ANOVA on this measure indicated a significant time by treatment-group interaction, F(1,75)=4.69 P<0.05, ES=6%. Analyses conducted for each treatment group showed a significant (P<0.01) increase in aggression scores for the combined PT+CT group from post to follow-up, however, the scores at follow-up were still better than before treatment. The change for the PT group was nonsignificant from post to follow-up, indicating stable high results for this treatment group across all three times of assessment (Means and SDs in Table 2).

Change of clinical levels of conduct problems in day-care/school

Based on clinical cut-off scores on the PBQ and the TRF aggression subscales, 83% (n=100) of the children exhibited clinical levels of conduct problems in day-care/school setting before treatment, i.e., having pervasive conduct problems. After treatment, 74% (n=84) of the children showed such levels of conduct problems in day-care/school. Of the total sample 44 (39%) of them had pervasive conduct problems (i.e., clinical levels of conduct problems both at home and in day-care/school). At the 1-year follow-up, 83% (n=70) of the children exhibited clinical levels of conduct problems in day-care/school, and 30 of them (35% of the total sample), had pervasive conduct problems. Table 3 presents stability and changes in clinical levels of conduct problems in day-care/school in all subjects across treatment and follow-up evaluations. A total of 17 children (15% of the total sample) changed from clinical to non-clinical level between pre- and posttreatment, whereas 6 children (5%) deteriorated. This difference in direction of change was significant (P<0.05) on the McNemar change test. When each treatment group was examined, only children in the combined PT+CT group changed significantly (P<0.05) more often from clinical to non-clinical level than vice versa. However, the PT group and the WLC groups showed no significant change.

Table 3 Stability and changes in clinical levelsa of conduct problems among all children in day-care/school between pre- and posttreatment assessment and at the 1-year follow-up
Full size table

From post treatment to the 1-year follow up, the direction of change for the whole group of children was non-significant on the McNemar test. However, when changes for each treatment group was studied, significantly (P<0.05) more children in the combined PT+CT group changed from non-clinical level to clinical than the opposite way. For the PT group the McNemar test was non-significant.

Generalisation across aggression-related day-care/school factors

To broaden the analyses of possible generalisation effects to other areas than child aggression, the following day-care/school factors were investigated: attention and internalising problems (teacher report), social problem-solving strategies (child report), social competence (teacher report), peer-interactions (teacher report), student–teacher relationships (teacher report) and parent–teacher involvement (teacher report).

Significant result was found only for social problem-solving strategies. For all other factors investigated, there were found no significant differences in scores across treatment and the 1-year follow-up.

The results of ANCOVAs (see Means, SDs in Table 2) from pre- to posttreatment indicated significant main effects of treatment for social problem-solving strategies, F(2,100)=8.71, P<0.001, ES=14.8%. Post hoc analyses revealed that children in the combined PT+CT group had a significantly (P<0.001) higher increase on social problem-solving strategies than those in the PT (showing a decrease from pre- to posttreatment); however, no difference was found in comparison with the WLC.

Further analyses of follow-up data from posttreatment to the one-year follow-up by means of repeated measures of ANOVA, showed a significant time by treatment-group interaction effect for social problem-solving strategies, F(1,73)=5.36, P<0.05, ES=7%. The PT group showed a significant score increase from posttreatment to the 1-year follow-up (P<0.05). However, because of the relapse in this treatment group from pre- to posttreatment, the scores at 1-year follow-up were not higher than before treatment. The combined PT+CT group did not change significantly from posttreatment to follow-up, indicating the positive treatment effect remained stable for this group.

Discussion

In the present study, generalisation effects from home to day-care/school settings were examined in a clinic sample of children aged 4–8 years treated because of ODD or CD with The Incredible Years PT programme or PT combined with CT. A striking characteristic of the present sample was that 83% of the children exhibited clinical levels of conduct problems both at home and in day-care or school settings before treatment, i.e. exhibiting pervasive conduct problems. In parental ratings of child conduct problems at home, positive outcomes of PT approaches as well as combinations with CT have been found for about two-thirds of treated children in previous studies as well as in the present study [20, 27, 28, 30, 33, 36].

Positive generalisation effects defined as reduced aggression levels in day-care/school after treatment were found for children who had received combined PT+CT intervention in the present study. However, this positive effect was not maintained 1 year later in that children in this treatment group relapsed significantly after treatment and during the 1-year follow-up period. When clinical levels of aggression problems in day-care/school were examined, most children still exhibited such problem levels both after treatment and at the 1-year follow up, thus indicating that most children still were rated high on aggression problems by their teachers in spite of reduced aggression levels in the home after treatment.

When generalisation effects were investigated in regard to a broader spectrum of problems potentially associated with aggression problems in day-care/school settings, it was found that children treated with combined PT+CT also significantly increased social problem-solving strategies from pre- to posttreatment further underlined by a high effect size. This treatment group showed stable levels in social problem-solving strategies one year later, while the PT group increased their positive strategies during this time period. However, because the PT group showed a decrease from pre to posttreatment, they at the 1-year follow-up showed scores at the same level as at the preassessment.

Overall, the combined PT+CT treatment produced more positive generalisation effects than the PT only group, in particular after treatment. This finding is in line with outcomes of similar intervention studies showing that social skills training for the child is a valuable component of treatment of young children with conduct problems [31, 33, 36]. However, in the present study a relapse in the PT+CT group after treatment during the 1-year follow-up period as shown in several outcome measures, indicates the need for developing specific strategies to maintain positive treatment effects also for children who have received combined treatment. The positive findings in significant reduction of aggression problems and enhancement in social problem-solving strategies from pre- to posttreatment among children in the combined PT+CT group may depend on teachers having received letter information regarding the Dinosaur school (CT) during the treatment period. They may have adapted their behaviour towards the child supporting he/she to use the self-control strategies they have learned during treatment. During the follow-up period many children were put into a new classroom setting, and they may have lacked the support needed to continue to use self-control strategies they already have learned. This may explain the relapse found in more measures in the combined PT+CT group at the 1-year follow-up. An interesting tendency, however not significant and difficult to explain, is that the PT only group on several measures showed a more positive development from posttreatment to the 1-year follow-up as compared to the combined PT+CT group. This tendency was found on measures of aggression problems, attention problems, social competence and peer-interactions and may indicate a delayed time-effect for the PT only treatment. The delayed improvement for children in the PT only group may be a result of that parents continue to consistently work with children at home after treatment, and indicate a strength of this intervention used by itself.

Altogether, the positive generalisation effects obtained across time, settings and problem areas in the present study have to be viewed as limited in spite of powerful reductions of children’s oppositional and aggressive behaviours at home [20]. It is therefore, important that conduct problems in day-care/school need to be targeted in specific in these settings to optimise outcomes for young children with pervasive conduct problems. Webster-Stratton et al. [36] also reported that teacher training addressing children’s difficulties in school need to be used as a complement to home-based intervention, a conclusion further supported by findings of the present study.

Although the present study included a highly selected clinic sample of children with oppositional or CD recruited to a controlled treatment trial, this sample exhibited a high proportion of pervasive problems indicating that these children have serious conduct problems with a poor prognosis, in particular if left untreated. In line with suggestions by McNeil [18] the inclusion of a high proportion of children with pervasive conduct problems as carried out in the present study, provides necessary conditions to estimate generalisation effects across settings. The positive generalisation effects obtained in the study are therefore likely to be fairly robust.

However, in spite of these advantages of the present study, a limitation was the small number of children included reducing the statistical power in the analyses. Further, the lack of a WLC group in the follow-up assessment restricted our comparisons to the two active treatment conditions. Although day-care and school problems among children were primarily assessed by means of questionnaires, in addition to the Wally-test, clinical levels of aggression problem scores were based on cut-off points derived from comparisons with national, normative data for children in the same age groups.

Overall, the results of the present study indicate that potential generalisation effects across time and settings are important to study further in clinic groups of children with ODD/CD. Conduct problems in day-care/school children are associated with a broad spectrum of problems in these settings, in particular social problems with peers, but also with more negative student–teacher relationships. Generalisation effects across other important areas, for example, social relations with children outside the home should, therefore be, addressed in evaluations of treatments for children with pervasive conduct problems. The results of the present study further emphasise that conduct problems in day-care/school need to be targeted in specific, and that a high involvement between parents, therapists and teachers is likely to further enhance generalisation effects. This finding is also in line with outcomes reported by Webster-Stratton, Reid and Hammond [36] from their study evaluating the effects of a teacher programme added to PT and CT.

Future research should further address and evaluate strategies to enhance positive generalisation effects to day-care/school settings when PT methods are used to help reduce conduct problems in children, and to further develop strategies to maintain positive generalisation effects obtained after intervention.