Introduction

The effectiveness of universal screening to ensure children with autism spectrum disorder (ASD) are diagnosed and access evidence-based intervention earlier has been widely debated (Al-Qabandi et al. 2011; Dawson 2016; Fein 2016; Mandell and Mandy 2015; Pierce et al. 2016; Powell 2016; Robins et al. 2016; Silverstein and Radesky 2016; Veenstra-VanderWeele and McGuire 2016). The American Academy of Pediatrics (AAP) recommends that all children be screened with an ASD-specific tool, such as the Modified Checklist for Autism in Toddlers (M-CHAT) (Robins et al. 2001), at 18 and 24 months, a period when rapid neurodevelopment occurs (Johnson et al. 2007). This is in contrast to a current clinical practise often referred to as “surveillance monitoring”, where clinicians continuously monitor children for signs of abnormalities over the course of development (Filipek et al. 2000; Nachshen et al. 2008; Volkmar et al. 2014). Structured screening early in life could potentially identify initial presentations of ASD, such as atypical social and communication development that occurs as early as 12 months (Ozonoff et al. 2008; Zwaigenbaum et al. 2015). Moreover, ASD could be accurately diagnosed in some children prior to age 2 years (Chawarska et al. 2007; Guthrie et al. 2013) and improvement in adaptive behaviour, social skills and IQ have been reported in children with ASD who received behavioural and/or developmental intervention prior to age 3 years (Zwaigenbaum et al. 2015). Although there is strong theoretical rationale for the AAP recommendation, there is no direct evidence that universal screening can lead to earlier access to treatment or improved ASD outcomes over time (Siu et al. 2016). This, in part, could be due to the complexity to assessing ASD services; some studies reported benefits of each component (i.e. early screening, diagnosis or intervention), but none has connected the entire process (Silverstein and Radesky 2016).

Given the moderate accuracy of most published ASD screening tools and the low prevalence of the condition in the general population (Centers for Disease Control and Prevention 2016; Johnson et al. 2007; Siu et al. 2016; Zwaigenbaum et al. 2015), one consequence of universal screening is that many children without ASD would be unnecessary referred for additional investigations. The wait time for diagnostic assessment is already high in Ontario, Canada (median 26 weeks) (Penner 2016) and the additional children waiting for diagnosis from false positive ASD screening would further delay access for children who require in-depth evaluation. As ASD diagnostic assessment is a lengthy process that can involve multiple clinicians, unnecessary assessment can greatly increase healthcare expenditures and parent productivity losses.

A potentially more efficient strategy could be active screening targeted towards subgroups known to be at heightened risk or to be under-diagnosed for ASD. For example, children with a first-degree family member diagnosed with ASD are considered to be at heightened risk for ASD given high familial recurrence rate (Grønborg et al. 2013; Ozonoff et al. 2011), along with babies of preterm birth or those who have specific genetic conditions (Limperopoulos et al. 2008; Richards et al. 2015). Since they typically require closer monitoring, they might benefit from ASD-specific screening to detect early risk markers that are often unrecognized.

The primary objective of this study was to estimate the incremental cost-effectiveness of universal or high-risk screening to surveillance in ASD from the provincial public payer and societal perspectives. The estimated ages at diagnosis and at treatment initiation for children with ASD in each of the three screening strategies were also compared.

Methods

Model Overview

A discrete event simulation (DES) model was constructed to estimate the costs and consequences of a cohort of children born within 1 year in Ontario, Canada under three screening strategies (Fig. 1). The model was calibrated to reflect the ASD epidemiology and the heterogeneity of ASD clinical manifestation in Ontario. A child underwent the 1st screening session at 18 months and was referred for ASD diagnostic assessment if screened positive. Positive ASD diagnosis was based on the diagnostic accuracy of clinical assessment (Huerta et al. 2012; McPartland et al. 2012) and the child’s latent ASD diagnosis. Inputs for the diagnostic accuracy were the inverse-variance weighted averages of published values (Huerta et al. 2012; McPartland et al. 2012). Children with a negative screen or diagnostic assessment returned to scheduled well-child visits. Those diagnosed with ASD, true and false positive diagnoses, were referred to generic applied behaviour analysis (ABA)-based therapy or to early intensive behavioural intervention (EIBI) based on their adaptive functioning status. Children with a false positive diagnosis remained on the wait list for intervention until their developmental trajectory scores revert to an age-appropriate level. This is assumed to reflect clinical reality as clinicians cannot know a child’s latent ASD diagnosis and are unlikely to remove a diagnosis unless the child shows significant improvement. All children exited the model when they reached age 6 years or were correctly diagnosed and initiated ASD intervention.

Fig. 1
figure 1

Schematic diagram of the discrete event simulation model

Full size image

The DES model was created in MATLAB (MATLAB 2017) to capture the dynamics of developmental changes and corresponding resource use in young children. Model parameters were randomly drawn from distributions estimated from published data or a prospective cohort study (Table 1 and Supplementary Table 1). Wait time for ASD intervention was estimated from current wait times and the number of intervention spots available in Ontario, and the length of the intervention. Wait time for diagnostic assessment was estimated from current wait times in Ontario, and the number of children referred for diagnostic assessment in the model. The time horizon was from birth to age 6 years as children may be referred by sources other than primary screening as they enter school. All costs and outcomes were discounted at 3% annually (Sanders et al. 2016).

Table 1 Parameters used in the discrete event simulation model
Full size table

Participant Characteristics

The target population was children living in Ontario at risk for ASD. Each child was assigned a developmental trajectory and clinical characteristics (Table 1 and Supplementary Table 1) that influenced their pathway of care. For this study, high-risk was defined as having one or more full siblings diagnosed with ASD. The probability of being classified as high-risk was estimated by first assigning the latent ASD diagnosis, then using the joint distribution of recurrence risk of ASD in full siblings and the probability of having an older sibling (Centers for Disease Control and Prevention 2016; Ozonoff et al. 2011; Statistics Canada 2015a). Latent ASD status is a binary variable representing whether a child with underlying ASD would be diagnosed during their life time and it was based on sex-specific ASD prevalence from the Centers of Disease Control and Prevention (CDC) (Centers for Disease Control and Prevention 2016). Observed ASD status is the probability of a child exhibiting symptoms that a clinician would diagnose as ASD at a specific time point. Developmental trajectories were described by two time-varying attributes: attainment of age-specific developmental milestones as defined by the CDC (Centers for Disease Control and Prevention 2015), and presence of severely low adaptive functioning as measured on the Vineland Adaptive Behavior Scales (Sparrow et al. 2005). The observed ASD status for all children and trajectories for high-risk children were estimated from on the Infant Sibling Study (Zwaigenbaum et al. 2012), and trajectories for low-risk children were based on population norms (Hagan et al. 2008; Sparrow et al. 2005). As our preliminary analyses did not identify significant differences between methods of generating the trajectories (e.g. using observed trajectories, using distinct trajectories identified by group-based analysis), we used a probabilistic approach to be consistent with other variables in the model. The present approach did not account for within-subject correlations.

Screening Strategies

The reference strategy was surveillance monitoring for potential signs of developmental delay for all children, hereon referred to as surveillance. The two comparators were universal screening (i.e. screening using the M-CHAT for all children) and high-risk screening (i.e. M-CHAT screening for high risk children and surveillance for low-risk children). The M-CHAT was selected for this model as it is a commonly used clinical tool that has been validated in multiple study populations and performs with moderate accuracy (Siu et al. 2016; Zwaigenbaum et al. 2015). The different screening approaches occurred at the 18- and 24-month well-child visits and all children, regardless of strategy, underwent surveillance monitoring at 36, 48 and 60 months. The frequency of screening followed the well-child visit schedule recommended by the AAP (American Academy of Pediatrics 2015). A child was screened positive by surveillance if they did not attain the age-appropriate developmental milestones or if they had severely low adaptive functioning. A positive screen on the M-CHAT was based on the joint probability of having an observed ASD status and the accuracy of the M-CHAT (Table 1).

Measurement of Effectiveness

The study outcomes were (1) the number of children correctly diagnosed with ASD and (2) the number of children having a correct ASD diagnosis and initiation of ASD intervention at each assessment time point. ASD intervention included government-funded generic ABA-based therapy and EIBI.

Resources Use and Cost

In the US and Canada, ASD interventions are typically available at no cost for eligible children in publicly sponsored state or provincial programs. This study was conducted from the provincial public payer and societal perspectives and Table 1 summarizes the sources used to estimate resource use and cost. Provincial government costs included publically funded health services and societal perspective included productivity loss in caregivers. The costs of clinical visits for screening and diagnostic assessments were based on physician fee schedules (Ministry of Health and Long-Term Care 2015; Régie de l’assurance maladie du Québec 2015) or obtained from professional associations (British Columbia Psychological Association 2014; Ontario Association of Speech-Language Pathologists and Audiologists 2016; Ontario Psychological Association 2015). The distributions for cost items with more than one data source were estimated by assuming the two reported estimates were the upper and lower 95% confidence interval. Otherwise, the distributions were estimated by assuming a 10% change above and below the point estimate. The duration and the cost of ASD diagnostic assessment varied by the type of clinician(s) involved; the probability of each was based on a recent Canadian survey (Penner 2016). The cost of ASD intervention that was implemented within the time horizon of the model was also included. This model assumed that the primary caregiver was female and productivity costs were estimated by valuing the time lost from accompanying their child to clinical visits using a national sex-specific hourly wage (Statistics Canada 2015b). All costs were expressed in 2016 Canadian dollars (median 2016 conversion CAN $1.00 = US $0.77).

Cost-Effectiveness Analysis

The predicted outcomes and costs for the three screening strategies were compared using incremental analysis. In order to highlight the clinical utility of the screening strategies, the outcomes were dichotomized at two time points—ASD diagnosis by 36 months and treatment initiation by 48 months. In turn, the incremental cost-effectiveness ratios (ICERs) were expressed as the incremental cost per additional child correctly diagnosed with ASD by 36 months and the incremental cost per additional child correctly diagnosed and initiated ASD intervention by 48 months.

Influence of key input parameters on the ICERs was quantified using one-way sensitivity analysis, where each parameter changed from a low to a high value (Table 1). Variations in outcomes were also assessed using bootstrap sampling to mimic 1000 study replications; the ICERs from each replication were plotted on a cost-effectiveness plane. The 95% credible intervals for incremental costs and outcomes were determined.

Results

Based on Ontario population estimates, a hypothetical birth cohort of 139,789 children, 2065 (1.5%) of whom had a latent ASD diagnosis, was generated and put through the model. Of the 2,065 children with ASD, the proportion correctly diagnosed within the 6-year time horizon of the model was 26.7% in surveillance, 32.1% in high-risk screening and 77.7% in universal screening. A similar proportion of children with ASD initiated either generic ABA-based therapy or EIBI by age 6 years for the three strategies (Table 2). Although more children were correctly diagnosed and initiated treatment in universal screening compared to the other strategies, the median age at diagnosis was 40.7 months (Interquartile range [IQR] 31.3–49.8) and at treatment initiation was 55.1 months (IQR 45.3–62.9).

Table 2 Results of discrete event simulation model by screening strategy
Full size table

Cost-Effectiveness Analyses

Table 2 summarizes the discounted median cost per child and number of children diagnosed and/or initiated treatment at each time point for the three screening strategies. Children were typically diagnosed at older ages in the surveillance group compared to high-risk screening and the incremental cost per child correctly diagnosed by age 36 months was $39,408.9 from the provincial public payer perspective and $41,651.6 from the societal perspective. Due to the high volume of children referred at 18 and 24 months in universal screening, many of whom did not have ASD, most children did not undergo diagnostic assessment until after age 36 months. In turn, the incremental cost per child diagnosed by age 36 months for universal screening was high at $652,032.3 from the provincial public payer perspective and $757,116.9 from the societal perspective.

Similarly, the total number of children with ASD initiating intervention was at least twofold higher in universal screening compared to the other two strategies, but the majority did not initiate treatment until age 60 months. High-risk screening identified more children with severe forms of ASD compared to the other two strategies and 21.4% of the total cost was attributed to EIBI (10.7% in surveillance and 8.1% in universal screening). The incremental cost per child initiating treatment by 48 months was $79,832.2 for high-risk screening and $218,757.1 for universal screening from the societal perspective.

Results from the bootstrap simulation indicated high variation in incremental cost in the societal perspective (Fig. 2). Due to the high cost of EIBI, the iterations with one additional child completing the full course of intervention within the time period greatly increased the incremental cost. Surveillance was more expensive than high-risk screening in 22.7% of the iterations but universal screening was consistently (99.7%) more expensive. There was minimal variation in incremental effectiveness using either outcome. For diagnosis by 36 months (Fig. 2a), 100% of iterations for high-risk screening and for universal screening were more effective. Similarly, more children initiated treatment by 48 months in 80.6% of the iterations for high-risk screening and in 95.0% of the iteration for universal screening (Fig. 2b).

Fig. 2
figure 2

Results of 1000 bootstrapped samples demonstrating the incremental costs and outcomes from the societal perspective for the high-risk and universal screening for a the number of children correctly diagnosed with ASD by age 36 months and b the number of children initiated treatment by 48 months. Ellipses indicate the 95% credible intervals

Full size image

Sensitivity Analyses

The influence of changes in key parameters on outcomes are described in Fig. 3 and Supplementary Table 2. Reducing wait time for diagnostic assessment (Fig. 3a) shifted the age at diagnosis to earlier time points, such that the proportion of children correctly diagnosed by 24 months was 49.7% for universal screening, 16.4% for high-risks screening and 8.1% for surveillance. The impact of wait time reduction was most evident for universal screening given the high number of children referred for diagnostic assessment at 18 and 24 months. The incremental cost per child correctly diagnosed with ASD by 36 months was $55,492.0 for high-risk screening and $63,647.4 for universal screening from the societal perspective.

Fig. 3
figure 3

The effects of changing the wait times for ASD services. a Reducing and b prolonging wait times for ASD diagnostic assessment and the cumulative proportion of children with ASD correctly diagnosed at each time point. c Eliminating and d prolonging wait times for both generic ABA-based therapy and EIBI and the cumulative proportion of children with ASD correctly diagnosed and initiating intervention at each time point

Full size image

Eliminating wait time for both generic ABA-based therapy and EIBI also led to more children accessing ASD intervention at an earlier time point, some by age 24 months (Fig. 3c). The total number of children with ASD that initiated either form of ASD intervention by age 4 years increased by 78% for surveillance, 67% for high-risk screening and 140% for universal screening. Given the high cost of EIBI, however, this greatly increased total healthcare expenditure. The incremental cost per child initiating ASD intervention by 48 months was $91,181.6 for high-risk screening and $164,129.0 for universal screening from the societal perspective.

Using a more accurate screening tool resulted in more children being diagnosed by age 36 months, with incremental costs of $12,263.7 for high-risk screening and $169,969.1 for universal screening. As the number of children with false positive M-CHAT screens decreased at a higher prevalence and ASD recurrence risk, the ICER for universal screening decreased to $550,040.4 per additional child diagnosed by 36 months. Changes in discount rate had minimal impact on the ICER for either outcome (Supplementary Table 2).

Discussion

The hypothesized effectiveness of universal screening for all children at 18 and 24 months in identifying children with ASD at an earlier age has been largely based on evidence from cross-sectional studies or clinical experience (Dawson 2016; Fein 2016; Pierce et al. 2016; Powell 2016; Robins et al. 2016; Silverstein and Radesky 2016; Veenstra-VanderWeele and McGuire 2016). While universal screening may be able to identify children with ASD at younger ages in isolated research settings (Pierce et al. 2011), they fail to account for healthcare system inefficiencies and the potential consequences when implemented on a population-level. Looking at the problem from a health economics angle, this simulation study demonstrates that universal screening may not be a cost-effective approach to ensure children with ASD are diagnosed and initiate treatment earlier. In Canada, children cannot access publicly-funded ASD intervention until they have a clinical diagnosis of ASD. While more at-risk children were identified and were correctly diagnosed with ASD in universal screening over the 6-year time horizon of the model, the majority were diagnosed after age 36 months and initiated intervention at around age 60 months. Rather, study findings show that reducing or eliminating wait time for generic ABA-based therapy and EIBI led to more children initiating treatment at a younger age compared to more frequent screening.

The sensitivity analysis (Fig. 3c) indicates that the number of children initiated either form of ASD intervention by age 48 months increased by 1.7–2.4 times across all three strategies when wait times for both forms of intervention were eliminated. Comparing surveillance monitoring without wait time to universal screening with current wait time, much more children started treatment prior to age 3 years in surveillance. There are evidence-based interventions tailored to children with ASD under age 3 years (Zwaigenbaum et al. 2015) and some studies suggest greater positive change when children started intervention before age 3 compared to after age 5 years (Harris and Handleman 2000; Kasari et al. 2012). Eliminating wait times for ASD diagnostic assessment not only resulted in more children being diagnosed at a younger age (Fig. 3a), some as early as 24 months, but also more children initiating treatment earlier compared to base case. The number of children initiated treatment by 48 months increased by 1.2–3.1 times across the three strategies without diagnostic wait time. Reducing wait times, however, comes with a price.

While the cost of each screening assessment is minor, the annual cost of EIBI is an estimated $56,000 per child in Ontario and treatment typically spans 2–3 years (Auditor General of Ontario 2013). In turn, each additional child starting EIBI due to reduced wait time would have a significant impact on the overall health budget. Compared to base case, the total cost from the societal perspective without wait time for either types of ASD intervention increased by 40% for surveillance monitoring, 43% for high-risk screening and 98% for universal screening. This cost increase is a conservative estimate as it does not account for the cost associated with creating more intervention spots (e.g. start-up, training). Although the initial cost is high, investing resources to ensure children with ASD initiate treatment earlier may offset costs incurred later in life. The cost of caring for an individual with ASD heightens in adult years due to productivity loss and need for support services (Buescher et al. 2014; Cidav et al. 2013). Some studies (Granpeesheh et al. 2009; Harris and Handleman 2000; Kasari et al. 2010; Perry et al. 2011) reported earlier treatment initiation to be associated with improved IQ and adaptive functioning, which may translate to increased independence later in life. A recent Ontario study estimated lifetime savings of CAD $267,000 and gains of 2.52 disability-free life years per individual if EIBI wait time was eliminated (Piccininni et al. 2017). Other studies (Chasson et al. 2007; Cidav et al. 2017; Jacobson et al. 1998; Peters-Scheffer et al. 2012) have also reported future cost-savings from various ASD interventions due to reduced need for special education and other interventions later in life.

The sensitivity of study results to changes in the accuracy of the screening tool emphasized the importance of selecting an appropriate tool at the population-level, especially when most available tools perform at low-to-moderate accuracy (Johnson et al. 2007; Siu et al. 2016; Zwaigenbaum et al. 2015). Due to the low specificity of the M-CHAT, a large proportion of children referred for ASD diagnostic assessment in universal screening did not have ASD nor did they require additional behavioural assessment or treatment. In turn, the high volume of children with false positive referral greatly prolonged wait time for diagnostic assessment. While the harm of a false positive screen may be low (Fein 2016), the consequences of delayed service access for children with ASD may be severe. The estimated delay could impact children without ASD as well, considering most clinicians who carry out diagnostic assessments also attend to children with other behavioural or psychiatric concerns. While some studies (Chlebowski et al. 2013; Pandey et al. 2008) reported that children screened false positive on the M-CHAT had other non-ASD developmental concerns and would benefit from comprehensive diagnostic assessment, there may be more efficient methods to identify these children than through false positive ASD screening.

It’s worth noting that while the estimated median age at treatment initiation for universal screening was higher than what is considered optimal clinically, it was lower than previously reported for children in Ontario (Auditor General of Ontario 2013). Also, the children who remained undiagnosed by age 6 years (i.e. time horizon of the model), particularly for surveillance monitoring or high-risk screening, could have led to underestimation of age at screening or treatment initiation. The proportion of children with ASD diagnosed at each age is lower than previously reported as this study only estimated children identified through primary care screening (Ouellette-Kuntz et al. 2009, 2014).

Study findings suggest that a more reserved screening approach may be optimal. The incremental cost per additional child diagnosed by age 36 months was $41,651.6 for high-risk screening, but whether this could be considered cost-effective depends on society’s willingness-to-pay to find another child with ASD earlier. This study defined high-risk children as those with one or more full siblings diagnosed with ASD, but an alternative criterion to target subpopulations known to be at heightened risk or those who may be under-diagnosed may be appropriate. For example, children of pre-term birth or with specific genetic conditions are reported to be at higher risk of developing ASD (Limperopoulos et al. 2008; Richards et al. 2015). Structured screening has also been reported to reduce biases in clinical judgement and delayed parental reports of initial developmental concerns that may be associated with parental education, socioeconomic status or ethnicity (Begeer et al. 2009; Guerrero et al. 2011; Guevara et al. 2013). However, there are additional financial, geographic and social barriers (Flores and Lin 2013; Liptak et al. 2006; Mandell et al. 2005) for these marginalized populations that would not be addressed by more frequent ASD screening alone.

Limitations

This study demonstrates the complexity in predicting the impact of introducing new ASD services. As shown in the results from the one-way sensitivity analysis (Supplementary Table 2), changes in one parameter did not result in linear change in the outcomes. Moreover, the model made several assumptions on the pathway to ASD diagnosis and determining developmental trajectories in children. This model accounted for the heterogeneity in the clinical presentation of ASD symptoms by modelling the children’s developmental trajectories using a probabilistic approach and by allowing the trajectories to vary by sex, time and latent risk of ASD. However, the trajectories were described with selected variables and might not capture all clinical symptoms typically monitored over time in surveillance monitoring. Therefore, the number of children screened positive, and in turn costs and outcomes, could be underestimated for surveillance, which could bias the ICERs in either direction.

The time horizon of this model was limited to birth to age 6 years. Children have additional avenues to receive an ASD diagnosis (e.g. through the education system) and this could confound measurement of the costs and effects of the screening strategies. As the goal of screening is earlier identification and treatment initiation, this study used two clinically relevant time points (diagnosis by 36 months and treatment initiation by 48 months) as proxies for future health gain. Since the benefits of early diagnosis and intervention initiation might not be apparent until later in the child’s life, a model with a life time horizon could better capture the consequences of the screening strategies. However, there is no published evidence on the long-term health outcomes of children identified through screening.

Lastly, this model did not consider private ASD interventions due to lack of reliable data on resource use. If a large proportion of families are willing to pay out-of-pocket for ASD treatment, thus bypassing the long wait times for government funded interventions, the effectiveness of all three strategies would be underestimated. In addition, while cost to the provincial public payer would be decreased when parents pay out-of-pocket, the higher costs of private services would result in higher societal costs overall.

Conclusion

Universal screening at 18 and 24 months may not be a cost-effective approach to ensure children with ASD receive appropriate intervention earlier. While more children with ASD were correctly diagnosed in universal screening over the 6-year time horizon of the model, they were typically diagnosed after age 36 months and did not initiate intervention until age 60 months. Moreover, the cost was high at ICERs of $757,116.9 per additional child diagnosed by age 3 years and $218,757.1 per additional child initiated treatment by age 4 years. A more reserved screening approach where only children at heightened risk yielded lower ICERs at $41,651.6 per child diagnosed by age 3 and $79,832.2 per child initiated treatment by age 4. Eliminating wait times for diagnostic assessment or for either form of ASD intervention was more effective in ensuring children accessed treatment earlier, but at a high cost that may be offset by future cost savings. Policy makers should also consider the impact on the overall health budget and on health services access for children without ASD when implementing changes along the ASD clinical pathway.