The average age of identification of autism in children has been shown to vary from 3.1 years (Mandell et al. 2005) to 5.7 years, (Shattuck et al. 2009) and for other developmental delays, reports show (Sand et al. 2005) that only 20–30% of children with delays are identified before entering school. Despite this variation in age of diagnosis, experts agree that the tools exist to identify children at younger ages (Filipek et al. 2000), when interventions can be most effective (Heckman 2006). The primary care pediatric practice setting, or medical home (Sia et al. 2004), is an optimal venue for identifying children with and at risk for developmental delays at the youngest possible age. Medical home, a concept that the American Academy of Pediatrics (AAP) developed to describe optimal care for children with special needs, has been expanded to describe care for all children that is accessible, continuous, comprehensive, coordinated, family-centered and culturally sensitive.

The pediatric medical home provides services to the vast majority of children and can take advantage of frequent and longitudinal relationships with families to monitor development over time and in the context of the family environment. The 2007 Medical Expenditure Panel Survey (Center for Financing, Access and Cost Trends 2007) reported that more than 88% of children ages 5 years old and younger had a usual primary care provider, across all ethnic and racial categories. The majority of these children (75%) received primary care in an office, as opposed to a hospital, setting. The primary care pediatric office visit, then, provides an excellent opportunity to reach a majority of children with early screening for developmental problems.

The current recommendations from the AAP (American Academy of Pediatrics 2006) call for developmental surveillance at all of the 14 recommended well-child visits for children birth through age five (Duncan et al. 2008). The AAP also recommends developmental screening with a standardized tool at the 9, 18 and 30 (or 24) month visits. Developmental surveillance includes eliciting parental concerns, documenting and maintaining a longitudinal developmental history, observing the child, identifying protective and risk factors, and obtaining input from others who interact with the child (e.g., day care providers). Formal screening includes the administration of standardized tools, which often are parent completed. Among the tools recommended by the AAP for use with young children are the Parental Evaluation of Developmental Status (PEDS) (Glascoe 2003) and the Ages and Stages Questionnaire (ASQ) (Squires et al. 1997). Both tools achieve at least moderate sensitivity (PEDS: .74 to .79, ASQ: .70 to .90) and specificity (PEDS:.70 to .80, ASQ: .76 to .91) (American Academy of Pediatrics 2006). Parents or caretakers can complete the PEDS in 2–10 min and the ASQ in 10–15 min, making both tools easy to administer as part of pediatric health care visits.

The AAP (Johnson and Myers 2007) also recommends administration of the Checklist for Autism in Toddlers (CHAT) or Modified Checklist for Autism in Toddlers (M-CHAT) at 18 and 24 month visits (Baron-Cohen et al. 1996; Robins et al. 1999, 2001). This tool is available in English and four other languages. The M-CHAT showed sensitivity above .80 and specificity above .90 in initial studies (Robins et al. 2001) with comparable results in more recent studies (Snow and Lecavalier 2008). The CHAT and M-CHAT screening tools also have been validated for use in pediatric primary care practices (Robins 2008). The M-CHAT can be accompanied by standardized, follow-up parent/caretaker interview questions that help those who score and interpret the screening tool further determine how likely it is that a positive response on one or more items warrants further and more extensive evaluation. Follow up questions are available for each item of the M-CHAT and when used, increase the positive predictive value of the screening process (Kleinman et al. 2008).

A study from a pediatric practice in Oregon (Hix-Small et al. 2007) showed that without formal screening tools, child health providers are highly specific in recognizing children with developmental delays, but not very sensitive. This study had pediatricians note evidence of a developmental delay at the 12 month exam and also had parents complete a formal screening tool, but the pediatricians were not told of the results of the tool. Nearly all (95%) of the children whom the pediatricians identified based on clinical observation were deemed eligible for early intervention services after full evaluation. However, the screening tool identified several additional children who were eligible for services after full evaluation. In total, the pediatricians had missed 67% of eligible children based on clinical observation alone. This finding underscores the potential of formal screening to improve identification of young children with developmental delays.

Studies of the use of formal developmental screening tools in pediatric primary care show varying degrees of implementation. A 2002 survey of 1617 members of the AAP (Sand et al. 2005) found that 23% used formal screening tools in their practices. This percentage rose to 57% when the survey was repeated in 2009 (Radecki et al. 2011). A study of pediatricians in Delaware and Maryland (Dosreis et al. 2006) indicated a high rate of screening for developmental delay with the Denver II (50% of respondents) and a low rate of screening for Autism Spectrum Disorders (ASD) (8%). Sices et al. (2003) found similar results in a national sample of pediatricians, with half of survey respondents stating that they use a formal developmental screening tool during routine well-child visits. In an analysis of the 2007 National Survey of Children’s Health, Bethell et al. (2011) reported that parent interviews showed a variation in developmental screening rates across states from a low of 11.7% of children screened in New York to a high of 47% in North Carolina. Miller et al. (2011) showed attainment of an 80% autism screening rate when pediatricians worked in partnership with autism specialists. Barriers cited by pediatricians for not screening with formal tools include: time, reimbursement to cover purchase of tools, screening and scoring (Honigfeld and McKay 2006; Sices et al. 2003) and lack of information about screening tools (Golnik et al. 2009).

Literature on successful efforts to improve developmental screening in pediatric practice is limited, and has been particularly scarce since the publication of the AAP developmental screening guidelines in 2006. The North Carolina Assuring Better Child Health and Development (ABCD) (Earls and Hay 2006) initiative was able to improve the rate of developmental screening in pediatric practices to more than 70% of well-child visits. ABCD used physician champions, workflow maps, networks of community services, staff training and periodic sharing of process and outcome data to change practice throughout the state. Schonwald et al. (2009) demonstrated the feasibility of implementing developmental screening in two urban pediatric primary care settings. King et al. (2010) reported on the results of an AAP pilot project to implement developmental screening in 17 pediatric practices. With a small amount of financial support to implement screening and a one-day educational training program, practices reported screening rates from 68 to 85% 9 months into the study. These practices were all committed to screening before participation in the study, and did not collect pre-study screening rates. In addition, since the publication of the AAP guidelines, programs and policies have emerged to support the practice of developmental screening in pediatric primary care. Several states (Kaye and May 2009) now reimburse providers for screening with a formal screening tool on the same day as a well-child exam, and an increasing number of educational opportunities are now available to train child health providers in developmental screening (King et al. 2010). Additional literature is needed on successful efforts to improve developmental screening in current pediatric practice settings.

The Child Health and Development Institute of Connecticut (CHDI) has a long history of using academic detailing to educate pediatric healthcare providers about best practices in child health care. In collaboration with the state’s Children’s Trust Fund, CHDI supported dissemination of a presentation on developmental surveillance and screening to 150 child health practices in 2004. Follow up chart audits (McKay 2006) showed that twice as many children were identified as at risk for developmental delay after practices had the presentation, and twice as many children were identified in practices that had the presentation compared with practices that did not. Although this educational effort was successful in promoting earlier identification of children at risk for delay through developmental surveillance, no practices increased their use of formal screening tools.

The present study is a pilot study which evaluated the effectiveness of a CHDI-sponsored academic detailing program, Educating Practices in the Community (EPIC), intended to improve developmental screening in child health practices in Connecticut. Academic detailing involves educational outreach through a personal visit by a trained person to health professionals in their own settings. (O’Brien et al. 2007) Successful academic detailing programs are developed as tailored interventions to overcome barriers to behavior change using simple messages, and are delivered by a respected colleague. As described by Soumerai and Avorn (1990), the key components of academic detailing interventions include: (1) investigating the baseline knowledge and motivations for clinical behavior patterns and potential barriers to behavior change, (2) defining clear educational and behavioral objectives, (3) establishing credibility through a respected organizational identity, (4) referencing authoritative and unbiased sources of information and presenting both sides of controversial issues, (5) stimulating active participation in educational interactions, (6) using concise graphic educational materials, (7) highlighting and repeating the essential messages, and (8) Providing positive reinforcement of improved practices in follow-up visits. Academic detailing is a multi-component process that incorporates many of the promotional techniques used by pharmaceutical company sales representatives. For example, the EPIC program schedules presentations for pediatric providers in their own practices over the lunch hour, provides a free lunch for all attendees, and distributes copies of educational materials to attendees.

Research suggests (Freemantle et al. 2000) that traditional methods of education, such as didactic, lecture-based continuing medical education (CME) sessions, have little to no effect on the behavior of health professionals. Academic detailing has demonstrated effectiveness at promoting behavioral change among health care professionals in a variety of clinical decision-making areas, including blood transfusion practice (Soumerai et al. 1993), antibiotic utilization (Finkelstein et al. 2001) and psychiatric disorders (Soumerai 1998). Specifically to children’s health care, academic detailing has successfully changed practices in the areas of pain management techniques (Schechter et al. 2010) and improved asthma management (Cloutier and Wakefield (2011). More specifically to the issue of screening in pediatric practices, Gaines et al. (2008) found that physicians receiving educational outreach visits about developmental coordination disorder (DCD) significantly improved their knowledge about DCD and their ability to identify children with this condition.

The research reported herein describes results of a pilot study to evaluate two EPIC academic detailing programs, or modules, which address developmental screening in pediatric primary care. One EPIC module focused on general developmental screening, and the other focused on autism screening. CHDI retained the services of physician experts to develop and present the two modules. The modules were developed independently and presented by different individuals.

Methodology

Intervention

The EPIC Autism Spectrum Disorder (ASD) Screening module was presented by a trained pediatric primary care provider in 43 pediatric and family medicine practices from March 2009 through November 2010. The EPIC Developmental Monitoring module was presented by four trained child development specialists in 14 child health sites between January 2009 and August 2010. Attendees at both presentations included providers and office staff members. Both modules highlighted: (1) information about developmental delays, (2) use of formal developmental screening tools recommended by the AAP, (3) billing codes to ensure reimbursement for developmental screening, and (4) community resources for connecting children to evaluation and intervention services. The developmental monitoring presentation suggested use of the Ages & Stages Questionnaire (ASQ) or Parents’ Evaluation of Developmental Status (PEDS) at 9, 18, and 24 (or 30) month visits, and the ASD presentation focused on use of one screening tool, the Modified Checklist for Autism in Toddlers (M-CHAT), at 18 and 24 month visits. This study reports on three types of data: (1) Medicaid claims data showing billing for screens performed, (2) participant feedback data after attending an EPIC presentation, and (3) data from chart audits performed in 10 pediatric practices.

Participants in the EPIC Presentations

Following each EPIC presentation, attendees completed an evaluation of the information presented, as well as information about their role in the practice. 318 attendees completed evaluations, representing a range of office roles: Pediatrician (32%), Nurse (20%), Medical Assistant (16%), Family Physician (9%), Office Manager (5%), Other Office Staff (6%) and Other (11%).

Participants in the Chart Audit Study

A chart audit to identify the impact of EPIC ASD on the rate of ASD screening at the 18 month well-child visit was conducted in five practices that received the EPIC Autism module (“Intervention practices”) and five practices that did not receive the module (“Control practices”). The Institutional Review Board at Connecticut Children’s Medical Center approved the chart audit portion of the study, and data transmitted to the authors did not include patient or practice identifiers.

The audited practices were members of a large primary care network in Connecticut, whose members include 19 pediatric practices across the state. A senior project manager from the network selected the practices and performed the chart audits. Nine practices in the network had received the EPIC ASD presentation at the time of the audit, and were eligible to participate as intervention practices. All nine practices were invited to participate in the audit, and five agreed to participate. The main reason for practices not agreeing to participate was the introduction of a new electronic medical record system that would make it difficult to systematically find the appropriate charts and screening information. In each of the intervention practices, the research manager audited 20 charts for 18 month well- child visits going backward sequentially from before the date of the EPIC presentation, and 20 charts from 18 month well-child visits moving forward sequentially starting at least 3 months after the EPIC presentation. In one intervention practice, due to inadequate sample size, only 16 charts were audited for the post-EPIC period.

From the pool of ten practices in the network that had not received the EPIC ASD presentation at the time of the audit, the research manager selected five control practices that best matched the intervention practices on practice size and agreed to participate in the chart audit. The research manager audited 20 charts from 18 month well-child visits sequentially from August 1, 2009. The sampling method yielded information from 18 month well-child visits that occurred 5 months after the first EPIC ASD presentation. This allowed the analysis to account for possible environmental confounders that may have encouraged ASD screening without receipt of an EPIC ASD presentation. Characteristics of the intervention and control practices are described in Table 1.

Table 1 Practice size, patient load, and patient characteristics of intervention and control practices
Full size table

Measures of Presentation Effectiveness

Participants in the EPIC presentations completed a short evaluation form about the presented information. Participants rated their answer to the following questions: “How likely are you to use the information presented?” on a scale from 1/Definitely to 5/Definitely Not; “Was this training useful?” on a scale from 1/Very useful to 5/Not useful and; “Which of the following might pose barriers to using the information? (A) Lack of time, (B) Not enough information, (C) Information too difficult to use” on a scale from 1/Not a barrier to 5/May be a barrier.

Chart audit Measures

The research manager reviewed each chart for evidence of a completed ASD screening tool at the 18 month well-child visit, and if appropriate, noted which tool was used. The chart audit data also included: result of screening, patient’s type of insurance (commercial or Medicaid), and patient gender.

Data Analyses

The authors prepared descriptive analyses of program evaluation data. The number of charts in which a complete ASD screening tool was located out of the total number of charts audited was considered as the rate of ASD screening in each practice. Separate “pre” and “post” rates were calculated for intervention practices. Intervention practices’ “pre” and “post” rates were compared using a Student’s T test in Microsoft Office Excel 2007. Intervention practice “post” rates were compared to matched control rates using a Student’s T test in Microsoft Office Excel 2007.

Results

The Connecticut Department of Social Services approved Medicaid reimbursement for developmental screening on the same day as a well-child exam in October 2008. The billing data reflect the entire population of children younger than three insured by Medicaid and cannot be disaggregated by provider or practice. The state’s Medicaid program (Connecticut Department of Social Services 2010) provided the research team with 2007–2009 data for billing for developmental screening (including screening for ASD) with Current Procedural Terminology (American Medical Association 2010) code 96110 at well-child exams for children <3 years. Figure 1 shows the number of times that billing code 96110 was used on the same day as a well-child visit in 2007–2009, with an increasing number of screens performed across the 3 year period. In 2009, the percentage of well-child visits that also included billing for a developmental screen was 10–12% for the 9 month visit, 21% for the 18 month visit, and 11–17% for the 24 month visit.

Fig. 1
figure 1

Number of screens billed to Medicaid with well-child visits for children <3 years, 2007–2009

Full size image

In response to evaluation forms distributed following EPIC presentations, 318 participants provided feedback on the EPIC presentation in which they participated. The majority of respondents indicated intent to use the information presented: 94% chose 1 or 2 on a scale from 1/Definitely (intend to use information) to 5/Definitely Not (intend to use information). Participants also indicated that the training was useful: 95% chose 1 or 2 on a scale from 1/(Information is) Very Useful to 5/(Information is) Not Useful. Participants identified time as the most likely barrier to using the information from the presentation: 26% selected 4 or 5 on a scale from 1/Not a Barrier to 5/May Be a Barrier. Only 6% identified “not enough information” and only 4% identified “information too difficult to use” as potential barriers.

Chart audit data augmented presentation feedback data, which indicated participants’ intention to implement screening in their practices. The chart audit demonstrated that rates of using a formal ASD screening tool at the 18 month well-child visit increased significantly in all five of the intervention practices (Table 2). All (100%) of the ASD screening tools that were completed in intervention practices were M-CHATs.

Table 2 Rates of ASD Screening in Intervention Practices Post-EPIC Presentation, compared with Pre-EPIC Presentation and with matched control practices (No EPIC Presentation)
Full size table

Rates of utilization of a formal ASD screening tool at the 18 month well-child visit in four of the five intervention practices were equal (n = 2) or higher (n = 2) than the rates of screening observed in the matched control practices (Table 2). One intervention practice had a screening rate that was lower than the matched control practice. 100% of the ASD screening tools that were completed in control practices were M-CHATs. The average rates of screening for the two groups were 70.8% for the intervention practices and 46% for the control practices.

Of the 95 children screened with the M-CHAT in intervention practices, seven screens showed ASD concerns. In the control practices, four of the 34 screens showed ASD concerns. Overall, 8.53% of the M-CHAT screens yielded positive results.

Discussion

This pilot study demonstrates that academic detailing is a promising strategy for changing practice behavior and engaging pediatricians in developmental screening at the 18 month well-child exam. Our findings are consistent with other studies that have shown the power of in-office presentations to promote change in health practices. It is likely that payment for developmental screening on the same day as a well-child exam and expansion of evaluation services for children with possible ASD also contributed to the results of this pilot study. The Connecticut Department of Social Services approved reimbursement for the Current Procedural Terminology code 96110 prior to the EPIC educational outreach program for ASD. Commercial insurers adopted the same reimbursement policy as the Medicaid program. At the same time that reimbursement became available, the state’s Part C Early Intervention program expanded evaluation and intervention opportunities for children with ASD concerns. The Part C lead agency, Birth to Three, designated ten agencies across the state as Autism program centers.

Another contributing factor to the success of the academic detailing program was the availability of a free, easy-to use, parent-completed screening tool. The M-CHAT can be completed in 5–10 min and scored by office personnel before the physician or nurse practitioner sees the family. It cues the child health provider as to possible red flags, which can easily open the conversation with parents about possible developmental concerns.

EPIC outreach efforts, then, were supported by the availability of a free, easy-to-use screening tool, favorable reimbursement policy, and the availability of full evaluations through and intervention services from the state Birth to Three program. These factors have undoubtedly facilitated pediatricians’ implementation of formal screening for autism spectrum disorders. However, this study demonstrates the added value of the EPIC ASD program, as all intervention sites improved their rate of screening after the EPIC presentation, and two of the five showed rates of screening higher than matched control practices. We conclude that without the EPIC training, practices may not know about the AAP guidelines for screening, available reimbursement for screening and referral resources for children with ASD concerns.

Intervention practice number five showed only modest implementation of ASD screening, although the practice performed no screening before the EPIC ASD presentation. It is possible that this practice was slow to make changes, and that many of the charts sampled for the chart audit study represented visits from before the practice started its screening program. One of the major limitations of the analysis presented is that it does not allow us to identify the factors that contributed to practice change once the educational intervention was completed. Another limitation of the study is the relatively small sample size of five intervention and five control practices, which was dictated by the limited number of trained practices in the primary care physician network that volunteered to participate in the study. We feel that our sample included somewhat diverse pediatric primary care settings in terms of patient population and size, and that our results may be generalized to other pediatric primary care settings. However, in order to better evaluate the impact of the intervention, a larger sample of trained and untrained practices drawn from more diverse pediatric primary care settings would be needed.

Three control group practices showed impressive rates of screening without receiving the EPIC ASD presentation. There are several possible explanations for this. The practices could have participated in the EPIC developmental monitoring presentation and selected the M-CHAT for the 18 month developmental screening tool. Some pediatric practices in Connecticut are participating in a study underway at the University of Connecticut (UConn) to validate a new version of the M-CHAT, and it is possible that practices in the intervention and control groups are also in the UConn study. Participation in the UConn study provides practices with a guaranteed, follow-up phone interview of the parents of patients who score positive on the revised version of the M-CHAT. In addition, children who are found to have remaining concerns after the follow-up phone interview, receive a timely, free full evaluation with referral to early intervention services as needed. Finally, either intervention or control practices could have taken up ASD screening based on other educational programs and guidelines distributed by the AAP. Despite these limitations, our results still show an impressive increase in screening in practices from before the ASD EPIC presentation.

The implications of this study for improving early detection of ASD and other developmental delays are:

  1. 1.

    State Medicaid programs and commercial insurers should reimburse pediatricians for screening that occurs during well-child services and with standardized screening tools. The added reimbursement makes it feasible for practices to purchase, score and provide follow-up counseling to parents about the results of screening.

  2. 2.

    States should ensure that services are in place to provide full evaluations, and when warranted, appropriate intervention services for children who do not pass practice-based screening. Without these services, it is unlikely that pediatricians will implement formal screening programs in their practices.

  3. 3.

    States should support outreach to child health providers to educate them about screening for ASD and other developmental delays as well as referral options for children for whom there are concerns. Academic detailing has been shown to be a promising strategy for doing this, and certainly yields more practice change than traditional methods of medical education. However, more research is needed to determine the factors that facilitate implementation of screening programs in pediatric practices.

  4. 4.

    States and practices should monitor their performance in screening young children for developmental delays and ASD. This commitment to continuous quality improvement can ensure that all children with, or at risk for delays, are identified at the earliest possible age.