Introduction

Conduct problems are the most common of childhood mental health difficulties [1], and recent evidence suggests that this problem is becoming more widespread [2]. Data from the Growing up in Ireland Study indicate that 7–8% of 9-year-olds in Ireland experience abnormal emotional and conduct problems. Such problems involve a range of oppositional or antisocial behaviour such as disobedience, lying fighting and stealing. In some cases, the severity and persistence of the problem is sufficient to warrant a psychiatric diagnosis of ‘conduct disorder’. However, research has shown that the adverse consequences of conduct problems are large and persistent even with sub-threshold levels of conduct problems [3]. Among the negative effects associated with conduct problems are poorer educational attainment [4–12], increased criminal activity [10, 13–16], reduced labour market success [11, 12, 15, 17, 18] and poorer adult mental health [19]. Consequently, early behavioural problems place large costs on individuals, families and society.

Research on the causes of conduct problems has identified a number of risk factors [20]. These include socio-economic factors, such as family poverty and deprived neighbourhoods, parental characteristics, such as low education and mental illness, and family relationships, such as abuse and inconsistent or neglectful parenting. Not all risk factors are of equal importance, and evidence suggests that early family relationships and parenting style are particularly significant forces [21, 22]. In addition, interventions focused on changing parenting style can offer an effective and economically efficient way of identifying and managing the risks associated with childhood conduct problems [23, 24]. Research supports the efficacy of such programmes in reducing the intensity of conduct problems, both in the short term [25–28] and in the longer term [15, 29, 30]. A systematic overview of the economic evidence on the effectiveness of parenting programmes is given by Charles et al. [31]. Importantly, this review highlights the lack of economic evaluations of parenting programmes, from either a cost-effectiveness or a cost-benefit perspective.

This paper contributes to this literature by conducting both an incremental cost-effectiveness analysis of the Incredible Years (IY) parenting programme in Ireland and a long-term cost-benefit analysis.

Methods

Objective and plan

The objective of this study is to examine the cost-effectiveness of the IY parenting programme in reducing childhood conduct problems in Ireland. Cost-effectiveness is assessed by relating the differential cost per treatment and control group to the differential effectiveness of the programme. Uncertainty is examined using probabilistic sensitivity analysis and presented as cost-effectiveness acceptability (CEA) curves. The results from this analysis are combined with data from secondary sources to provide indirect measures of the long-run rate of return to the IY programme.

The intervention

The IY parent, teacher and child training series was developed at the University of Washington over the last 30 years and is specifically designed to prevent and treat emotional and behavioural difficulties in children aged 0–12 years. This paper considers the BASIC Preschool/Early School Years Parent Training component of the programme, which targets children between the ages of 3 and 7 years. Parent competences are developed in areas such as communication, limit setting, problem-solving and anger management. Parents are also encouraged to develop support networks. Two trained facilitators take parents in groups of approximately 12 for one 2-h session each week over a 12- to 14-week period. The programme uses a collaborative approach between group leader and parents including the analysis of video vignettes of family behaviour for discussion.

Study design

The outcome study identified a sample of 149 families to take part in the randomised trial [32]. The trial was carried out in typical community-based services in Ireland located in four urban areas, all of which were designated as socio-economically disadvantaged. Families were recruited to the study using existing community service systems such as Health Board waiting lists and local schools. At baseline, all eligible children had to score over the clinical cut-off on either the Intensity or Problem subscales of the Eyberg Child Behaviour Inventory (ECBI) [33]. The ECBI was also the primary outcome measure of child problem behaviour in our analysis. The ECBI is a parental report of the frequency and intensity of problem behaviour in children and was developed as a screening instrument for the differentiation of normal and conduct problem children [33]. The administration of parental questionnaires is often the first step in the diagnosis of child mental health conditions. The test is used to assess 36 individual problems, including difficulty following orders, difficulty interacting with other children, problems with attention seeking and problems with concentration and attention span. Many of these characteristics have been identified by teachers as important determinants of readiness to learn [34, 35]. The final ECBI test score, which ranges from 36 to 252, is an aggregate of the scores over individual problems. The clinical cut-off for serious behavioural problems with the ECBI is 127.

Sample members were randomly allocated, on a 2:1 basis, to either the parent training intervention group (T) who participated in the programme or a control group (C) who was placed on a waiting list for the intervention (offered after 6-month follow-up). Participants were assessed at baseline in early 2008, before they received the intervention, and again 6 months later by which time all members of the treatment group had participated in the programme. Of the original sample, 12 failed to respond to the follow-up survey and 5 more families failed to provide adequate data on the ECBI and/or service use. We exclude these observations from the analysis. One could impute values for the missing [36]. However, almost all imputation techniques rely on the assumption that the data are ‘Missing at Random’. Under this assumption, the complete case approach adopted here provides consistent estimates, though imputation may lead to efficiency gains. However, the efficiency properties of the imputation approach are based on asymptotic theory, which may not be appropriate given our sample sizes. Excluding the missing data resulted in a valid sample of 132 parents, of which 93 were members of the treatment group and 39 were members of the control group.Footnote 1 Of the 93 treatment group members, 7 were assigned to treatment but did not complete the programme. These families are nevertheless included in our analysis, and as such, our results should be interpreted on the basis of the intention-to-treat principle. To examine the robustness of our findings to this choice, we also report results based on the smaller sample of compliers.

Cost data

For purposes of conducting the cost analysis, an adapted version of the Client Service Receipt Inventory (CSRI) was used to record the frequency with which health, educational and social services were used by parents and their children during the previous six months [38]. The CSRI was administered by means of a face-to-face interview with the main caregiver at two time points: at baseline, before parent/caregivers began the parenting course and again 6 months later. In a previous study, much of the unit cost data for services could be obtained from official annual government publications [39]. The Irish government does not publish such a detailed description of unit costs. For this study, data on unit costs were obtained from a variety of sources and agencies. Table 1 lists the key health services for children in Ireland and the corresponding unit costs. For some categories (e.g. general practitioner (GP) visits), the costs are well-established. For others (e.g. A&E, outpatient and overnight stay in paediatric hospital), costs were obtained from the Casemix/HIPE unit of the Health Service Executive, the organisation charged with running the public health system in Ireland.Footnote 2 For other services (e.g. special needs assistants (SNAs)), official government payscales were used to determine an hourly rate for the service.

Table 1 Demographic statistics by treatment group status
Full size table

In addition to the service costs, we also collected data on the direct recurrent costs per parent of running the programme. These costs were obtained from ‘cost diaries’ completed by each of the group facilitators during each week of the programme. These costs covered the full range of recurrent costs involved in implementing the programme. These included (1) costs of recruiting the parents to the group; for instance, time and mileage for each family visited and telephone call; (2) costs directly related to ongoing group (e.g. session preparation time, home visits to parents and supervision time); and (3) costs incurred through facilities provided, such as the provision of crèche facilities or payment for childcare, taxis, food and catering, and other administrative costs directly related to the programme. Non-recurrent initial training and group set-up costs were not included for purposes of this analysis. A total of 18 cost diaries were completed covering all facilitators.

Results

Demographic characteristics

At the start of our study, each parent completed a Profile Questionnaire that collected detailed information on family background, employment status and income levels. Summary statistics, by treatment status, for the sample used in this study are given in Table 2. As expected, given randomised allocation between groups, there are no significant differences in the demographic characteristics of the families or the pre-intervention ECBI score across groups.

Table 2 Unit costs data for service use
Full size table

Impact of programme on child behaviour

Figure 1 shows the full distribution of pre- and post-intervention ECBI scores for both the treatment and control groups in our sample. The left-hand panel of this figure confirms the similarities in the distribution of test scores prior to the intervention. The right-hand panel, in contrast, shows a significant shift in the distributions post-treatment. In particular, the mean difference in the ECBI test between treatment and control groups post-treatment is 20.33.Footnote 3 This compares with the estimate of 21 reported by McGilloway et al. [32] using a slightly larger sample in which missing data were imputed. The distributions in Fig. 1 show that this mean effect is not driven by outliers and instead reflects a genuine improvement in test scores throughout the sample. Significantly more of the treatment group had obtained ECBI scores below the clinical cut-off following the intervention (60% for the treatment group vs. 35% of the control group).

Fig. 1
figure 1

Density of Eyberg intensity scores at baseline and 6-month follow-up for treatment and control groups

Full size image

Service usage and recurrent costs

The details on service use are given in Table 3. In terms of primary care, the most commonly used services were GPs, nurses and speech therapists.Footnote 4 The remaining primary care services such as social workers, paediatricians and physiotherapists tended to be used by less than 10% of either group. In terms of special resources during schooling, only one-to-one help, which is likely to mean accessing SNAs, was a commonly reported service. A statistical analysis of the baseline data showed no significant difference in service use between the control and treatment groups at the start of our analysis.

Table 3 Proportion using medical services at baseline and follow-up survey
Full size table

Looking at the change in service use over time, we see that while there is evidence of a decline in service use by the control group, the reduction is more pronounced for the treatment group. In particular, there is a substantial decline in the use of many primary care services, as well as less contact with social workers for those who received the treatment. To determine whether these differences were statistically significant, we tested the null hypothesis that service use was unchanged between baseline and follow-up. The P-values for this test are given in columns 6 and 7 of Table 3, for the control group and treatment group, respectively. The results show no significant change in service use for the control group. However, members of the treatment group reported significant reductions in the use of a number of services, including GP visits, use of speech therapists and visits to social workers. This decline in service use in the treatment group is consistent with improved child behaviour and enhances the overall cost-effectiveness of the programme.

Table 4 provides a summary of recurrent costs. For simplicity, we summarise these costs in three categories: direct wage costs covering the costs of the facilitators time, travel costs and other costs and expenses. As expected, direct wage costs constituted the largest component of total cost, accounting for almost 90%. Additional expenses accounted for the majority of the remaining costs, with travel expenses amounting to a minor component of overall costs. Since the average group size observed in this study was 11 parents per group, these data imply an average recurrent cost per parent of €1,463.

Table 4 Recurrent costs of programme provision using leader cost diaries (€)
Full size table

Table 5 combines these data with the service utilisation costs to estimate the incremental costs associated with the programme. The final row indicates that the additional net cost of the programme was approximately €1,759.74. This figure is used to construct an estimate of the overall cost-effectiveness of the programme.

Table 5 Health social care and special education services used by children
Full size table

Cost-effectiveness of IY parenting programme

When the data on outcomes and costs are combined, we estimate an incremental cost-effectiveness ratio of €87 per 1 point reduction in the Eyberg intensity score (1,759.74/20.33),Footnote 5 with a 90% bootstrapped confidence interval of [€43–€158]. This estimate compares to the point estimate of £73 (€83) obtained in the cost-effectiveness study of the IY programme in Wales [39]. Using our estimate, it would cost €9,483 ((235−126)*87) to bring the child with the highest intensity score to below the clinical cut-off point and €2,784 ((158−126)*87) to bring the average child in the study below this limit.

The probabilistic results are summarised in the CEA curve and the cost-effectiveness plane shown in Figs. 2 and 3, respectively. Given uncertainty about the parametric distribution of the ICER, we use a 1,000 replication bootstrap to estimate the CEA. From Fig. 2, we infer that the probability of the IY programme being cost-effective would exceed 90% provided the willingness to pay threshold was at least €137. This probability falls to 80% if the threshold is reduced to €115 and rises to 95% if the threshold increases to €158.

Fig. 2
figure 2

Cost-effectiveness acceptability curve for IY evaluation

Full size image
Fig. 3
figure 3

Cost-effectiveness plane for IY evaluation

Full size image

Cost-benefit implications

To date, there have been very few cost-benefit analyses of early intervention programmes of the type considered in this paper. Aos et al. [42] undertook a series of cost-benefit analyses for a range of early intervention programmes.Footnote 6 Their results for Pre-Kindergarten Education Programmes and Home Visitation Programmes indicate that while some programmes (such as Nurse–Family Partnership for Low-Income Women and Home Visitation Programmes for At-Risk Mothers and Children) achieve significantly higher benefits than costs, others (e.g. Early Head Start, Comprehensive Child Development Programme and Infant Health and Development Programme) were less successful in this respect, yielding net losses of between $16,203 and $49,000 per youth.

While many of the programmes discussed by Aos et al. have features in common with the IY programme, this programme itself was not included on the grounds that the outcomes associated with the programme (i.e. reductions in child conduct disorder) are difficult to value. While we accept that this is difficult, we nevertheless believe that there is merit in trying to use the results from the cost-effectiveness analysis to determine a longer-run rate of return for the IY programme.

Cost-benefit analysis of IY programme

To carry out the cost-benefit analysis of the IY programme, we combine the results from our cost-effectiveness analysis with estimates of the effect of conduct problems on key outcomes in adult life. For pragmatic reasons, we focus on three outcomes: education, crime and unemployment. There is an extensive literature of the effects of conduct problems on each of these outcomes, and in addition, it may be possible to assign monetary values to the associated gains using secondary data sources.

There is a substantial body of research showing that early conduct problems have a negative effect on educational attainment [4–12]. In particular, children with conduct problems were more likely to require remedial help at primary and secondary school, were more likely to have to repeat a grade, performed less well on exams and were less likely to obtain recognised educational qualifications. To capture these effects in our cost-benefit analysis, we assume that the educational cost of children with conduct problems amounted to an additional-hour-a-week contact with a SNA for each of the first 4 years of primary school. Given a school year of 37 weeks and an hourly pay rate of €15.20 for SNA, this translates into an annual savings of €562.40 per year. In our ‘Discussion’ section, we consider a broader measure of this educational effect.

There is also evidence of a significant link between conduct problems and criminal activity [10, 13–16]. The report by the Sainsbury Centre for Mental Health estimates that around 80% of all criminal activity is attributable to people who had conduct problems in childhood and adolescence [14]. Children with conduct problems were more likely to have engaged in criminal activity, more likely to have been arrested and more likely to have spent time in jail. Friedli and Parsonage estimate that the overall lifetime costs of crime was €182,262 per case for those with conduct disorder in the UK and €51,250 per case for those with mild conduct problems [45]. Fergusson et al. [13] estimated that the gap in the probability of imprisonment between those with behavioural problems and those without was 7% points.Footnote 7 Figures for Ireland suggest that the average cost of keeping a prisoner for the year 2007 was €97,700 [46]. This figure gives an approximate measure of the additional cost of prison of the order of €6,839 per person. In our conservative calculations, we assume that the savings from reductions in imprisonment occur only once and at the age of 30, which is the average age of the prison population in Ireland in 2007 [46].

Finally, evidence of an association between conduct problems and poor adult labour market performance is provided in a number of studies [11, 12, 15, 17, 18]. In our initial analysis, we consider only the effect of higher unemployment, though potential wage losses are discussed later. We assume that individuals with conduct problems spend an additional 5 months in unemployment. This is consistent with the range of estimates reported in Moffitt et al. [18] who followed individuals up until age 26. It is estimated that the annual cost of unemployment in Ireland, in terms of welfare payments and losses in taxes, is approximately €15,000 which, when combined with unemployment effect, leads to an expected savings of €6,250. Again, we make the conservative assumption that that this is a once-off saving occurring at the age of 30.

The pre-intervention mean ECBI score for members of our sample was 158. Thus, a 32-point decrease is required to reduce this score below the critical level. When combined with our estimated incremental cost-effectiveness ratio, this means the cost of bringing the average problem child below the clinical cut-off is €2,784. Combing these cost and benefit data yields an internal rate of return for the programme of 11%.Footnote 8

Sensitivity analyses

To examine the sensitivity of our estimated returns, we consider alternative valuations for the crime and unemployment effects, and for our cost estimates. For crime, we consider both a 2% point gap and a 14% point gap in imprisonment rates. These span the range of estimates reported in Koning et al. [10]. With these estimates, the internal rate of return ranges from 9 to 12%. To consider the sensitivity of our results to the valuation of unemployment, we consider savings based on both a 1-month and a 1-year reduction in unemployment. Since our estimated unemployment effect is based on work-histories up until the age of 26, we also consider a more extreme unemployment effect of 3 years to allow for effects in later years. With these valuations, the internal rate of return ranges from 9 to 17%. When varying our cost estimates, we use the upper and lower bounds of our 90% confidence interval reported earlier (€43–€158). Costs of these magnitude lead to returns of 6 and 25%, respectively. Finally, we take a rather extreme assumption and assume that the reduction in crime is the only benefit arising from treatment of conduct disorder.Footnote 9 In this case, the rate of return falls to 4%.

Discussion

The first objective of this study was to examine the cost-effectiveness of the IY parenting programme as a means of tackling conduct problems in young children. In keeping with previous work, we find that the IY programme was cost-effective in reducing conduct problems (with an ICER less than €90). When interpreting these results, it is important to bear in mind the short-run nature of the analysis presented here. In this study, we only followed participants for 6 months after the treatment. While we know of no long-run randomised evaluation of the IY programme, the evidence available does suggest that the initial gains reported for the IY programme may have a significant longer-run component. Hutchings et al. [48] report on a 4-year follow-up evaluation of a structured parenting programme, which is similar in many ways to the IY programme. They found that the substantial improvements in child behaviour observed after the 6-month follow-up were still evident four years after the intervention had stopped. By contrast, the control group received a less intensive programme but exhibited no evidence of long-run gains. Furthermore, Bywater et al. [30] show that significant improvements in primary measures of child behaviour resulting from an evaluation of the IY parenting programme conducted in Wales were maintained in the medium term (18 months after baseline). In addition, Webster-Stratton, Rinaldi and Reid carried out assessments of participants 8–12 years after contact with the programme [49]. Although no control group was available in this case, the results suggest that the treated children showed less severe conduct problems at adolescence (e.g. limited contact with the criminal justice system) than might have been expected given their earlier clinical levels of conduct disorder.

Using similar methodology, McGilloway et al. [50] report results from a 12-month follow-up of the 103 members of the base treatment group in the current trial. Members of the control group were subsequently offered the intervention and therefore could not be included in the longer-run study. In addition, 16 members of the treatment group were lost to the 12-month follow-up. Nevertheless, these data provide some evidence on the extent to which the improvement in behaviour identified at 6 months persists. The results suggest that the positive early effects of the IY programme were maintained at the 12-month follow-up, thereby suggesting sustained benefits for children and their families in the longer run.

To estimate the economic rate of return to the IY programme, we combined our cost data with benefit data taken from a variety of sources. Our estimated returns compare favourably with previous evaluations of policies aimed at tacking disadvantage. The estimated internal rate of return of the Perry Preschool Project, perhaps the most heralded early childhood intervention programme in the United States, was 8% [51], while many of the adult training programmes are reported to have yielded negative returns [52].

The strength of this analysis is heavily dependent on the quality of the data inputs used. Our model is based on multiple data sources: RCT for effectiveness, published data for costing and an array of secondary international sources for the likely effect of conduct problems on adult outcomes. While this is problematic, it is also unavoidable since this is the first paper of its kind that uses Irish data.

While this array of sources is not ideal, the data used are the best available and follow a detailed review of the literature. We focused on three outcomes, education, crime and unemployment, for pragmatic reasons. Data on these outcomes were taken from a range of studies, which avoided relying on one single source. In addition, these outcomes were most easily valued in monetary terms. Other benefits such as improvements in second- and third-level educational attainment and associated increases in productivity and earnings capacity, reductions in substance abuse, decreased teenage pregnancy, benefits to victims of reduced crime and benefits accruing to the parents and/or other siblings as a result of the programme were omitted and are likely to push the estimated returns higher. For example, the OECD report that the net present value to society of completing second-level education in Ireland was approximately €22,000 [53]. This includes the benefits and costs to both the individual and society. Cleary et al. [16] report a difference in second-level graduation rates in Ireland between those with behavioural problems and those with none of approximately 40% points. If we use this broader measure to value the educational return of the parenting programme, the estimated net present value of the programme increases from €2,927 to €9,830.

These caveats suggest that our estimated long-run returns may be conservative. On the other hand, the short-run nature of our evaluation leaves open the question as to whether additional programme costs may be needed in the future in order to maintain the short-run benefits identified in our evaluation. In addition, the difficulties associated with valuing the benefits of reduced behavioural problems, in the absence of a long-term follow-up, need to be recognised when considering our results.

Finally, in conducting our cost-benefit analysis, it was often difficult to distinguish between private benefits/costs, accruing to the family directly, and public benefits/costs that accrue to society at large. For instance, when costing primary care, we use a rate of €45 per GP visit. For most families in Ireland, this is paid out of pocket on a fee-per-service basis and is not reimbursed even when the family have health insurance. In this instance, the cost is borne by the family. However, for families who have a medical card—generally because they fall below the income threshold typically in the order of €300 for families or lone parents—the costs of GP visits are borne by the state and thus reflect a cost to society and not the individual. While primary care costs constituted a relatively small proportion of overall costs in our study, this distinction between private and social gains/cost may be more relevant in other dimensions. When analysing crime, we only consider the savings to the taxpayer from reductions in the prison population, and none of the private benefits that arise from the reduced probability of imprisonment. Further analysis may be useful in distinguishing between private and social benefits/gains though this is likely to prove difficult at present given the format and accessibility of currently available data.

Conclusion

The call for an increased policy focus on early childhood interventions and in particular policies that tackle the role of the family in early childhood development has been growing in recent years. This call is based on the dual premise that ‘skill begets skill’, so that early childhood intervention can result in cumulative gains over the life cycle and that the family environment plays a significant role in the development of early childhood behaviour and health. This paper uses the findings from a randomised controlled trial to examine one such early childhood intervention programme. Our findings suggest that the IY programme offers a cost-effective policy option for reducing behavioural problems and in doing so may generate favourable long-run economic returns.