Many states across the USA provide early education programs for preschool-age children ages 3–5 years old to foster school readiness and reduce school failure. Unfortunately, 14–30 % of the children who enter these programs exhibit problem behavior (e.g., disruptive behavior; Barbarin 2007) and are without age-appropriate social and emotional skills, both of which impact their ability to benefit from early educational experiences (Feil et al. 2005). As a result, these children are at risk for long-term behavioral and academic problems (O’Conner et al. 2011). Despite the increased focus on providing high quality early childhood education to foster social-emotional learning and ameliorate problem behavior, many educators in early childhood settings struggle to deliver the interactional and instructional supports designed to prevent or ameliorate social-emotional learning deficits and associated problem behaviors. This has led some researchers to call for specific training and supports for early educators focused on social-emotional and behavioral interventions (Hemmeter et al. 2006).

A logical choice for improving the quality of early educators’ interactional and instructional support for young children with chronic problem behavior is for them to implement comprehensive intervention models (i.e., manualized interventions that include a number of practice components, henceforth called models) and discrete practices (i.e., a specific behavior or action, henceforth called practices) that target children in early childhood classrooms who demonstrate problem behavior and collateral social and emotional needs. The early childhood literature includes a few models that meet evidence-based criteria (e.g., Incredible Years; Webster‐Stratton et al. 2008), along with other models that have been identified as needing additional empirical support to be considered evidence-based (e.g., Second Step, Brown et al. 2012). Educators also have implemented discrete practices that have demonstrated positive social, emotional, or behavioral effects, typically through single case experimental designs (e.g., precorrective statement, Smith et al. 2010). Although a number of models and discrete practices exist, it can be a challenge for stakeholders to determine which models or discrete practices are the best fit for use with specific children in a particular early childhood setting (Powell and Dunlap 2009). This is due, in part, to the fact that there is no consensus in the literature about operational definitions for the practices that have been the subject of study. These definitional issues make it challenging to compare and contrast findings across studies and make it difficult for teachers to implement practices from models or discrete practices with integrity in early childhood settings.

Other fields have attempted to introduce systems for defining practices to facilitate communication related to intervention methodology and findings. In the mental health field, researchers have focused on defining practice elements (i.e., discrete principles or skills) that compose the models and practices evaluated within a specific literature (e.g., Chorpita and Daleiden 2009; Garland et al. 2008; Institute of Medicine 2015). The practice element approach, also called “therapeutic strategies” (i.e., goal or general principle that guides psychosocial treatment; Beutler and Baker 1998) and “evidence-based kernels” (i.e., units of behavior influence that cannot be reduced further without losing their function; Embry and Biglan 2008), presents a different perspective on how the evidence base can be defined and employed (Chorpita and Daleiden 2009; Chorpita et al. 2005; Garland et al. 2008). Instead of a focus on implementing practices from a single model, the elements approach argues for the efficiency of focusing on specific practice elements that are made from the practices used to create the models. In recent years, this approach has gained traction, as some assert that creating a common way to define the practices within a body of literature is key for implementation efforts (Fairburn and Patel 2014; Institute of Medicine 2015).

Two methods have been used to analyze the evidence base to identify practice elements in the mental health field. One approach developed by Chorpita and colleagues started with a codebook of predefined practice elements derived through stakeholder input, including practitioners and intervention developers (see Chorpita et al. 2005; Chorpita and Daleiden 2009). The set of practice elements was then compared to models for youth with emotional and behavioral problems, and the presence of those elements was coded. This approach is useful for cataloging the extent to which a set of common practice elements is represented across a broad literature. However, given that the practice elements are largely predetermined, this approach may not capture all practice elements within a given literature (Garland et al. 2008).

A second approach developed by Garland et al. (2008) starts without an explicit set of practice elements and works to extract practice elements from a target literature. In their paper (Garland et al. 2008), they focused on studies evaluating evidence-based programs (i.e., models that meet defined levels of empirical support) for youth aged 4–13 years old displaying disruptive behavior problems. Through successive iterative steps, they extracted 21 practice elements from eight studies. The approach demonstrated by Garland et al. is more likely to identify the full range of practice elements represented in a literature.

A comparison between the two approaches reveals some similarities. The seven most commonly represented practice elements identified by Chorpita et al. (2005) for disruptive behavior problems were mapped on to the list of practice elements produced by Garland et al. (2008). However, Garland et al. produced a more expansive list, likely due to their focus on identifying the full set of practices found in the empirical literature. Of course, the ideal number of practice elements identified may vary by the purpose. In some instances, knowing the entire range may be needed, whereas in other contexts, knowing a smaller set may prove more useful.

We are not aware of efforts to identify practice elements in the early childhood intervention literature for children who demonstrate social-emotional difficulties and problem behavior. An approach like Garland et al. (2008) took has several advantages as a first step in identifying practice elements in this literature. First, as noted earlier, there are no commonly accepted definitions for the social-emotional and behavioral practice elements used in early childhood classrooms that could be used to guide the development of a codebook. Second, we were interested in identifying all of the practice elements evaluated in the early childhood literature for models and practices designed to address social-emotional deficits and problem behaviors. Thus, we employed an approach most similar to the one used by Garland et al. to define practice elements.

We applied a modified version of the Garland et al. (2008) approach to identify practice elements in the early childhood social-emotional and behavioral intervention literature. Our primary goal was to determine if it was possible to extract and define a set of practice elements that represent the models and practices studied in the early childhood literature. We describe the process used to identify studies focused on models and practices that target young children aged 3–5 years old who have social, emotional, and behavioral learning needs and to distill from these studies common practice elements. Importantly, unlike previous efforts to identify practice elements in the mental health field, we did not attempt to differentiate practice elements by level of empirical support or exclude studies that failed to achieve positive effects. This decision was made based on the challenges of attempting to synthesize findings across group experimental and single-case research designs (see Shadish et al. 2015). This would require us to resolve challenges related to a quantitative synthesis of the data produced by the two research designs reviewed for this paper, which is beyond the scope of the current manuscript. Instead, we identified the practice elements found in the models and practices listed as having empirical support on the What Works Clearinghouse (ies.ed.gov/ncee/wwc/); Collaborative for Academic, Social, and Emotional Learning (CASEL; www.casel.org/preschool-and-elementary-edition-casel-guide); and Blueprints for Healthy Youth Development (www.blueprintsprograms.com/) in order to highlight those practice elements that are associated with agreed upon standards of evidence in the field.

Method

Selection of Studies

We conducted a literature search of published articles evaluating models or practices delivered in early childhood classrooms targeting young children with chronic problem behavior. A computer-based search was conducted using four electronic databases: (a) Education Research Information Center (ERIC), (b) PsycINFO, (c) Elton B. Stephens Company (EBSCO), and (d) Web of Knowledge (Social Science Citation Index). The search spanned January 2005 to August 2014; these dates were chosen to correspond with the 2005 publication of the National Research Council report that sought to advance scientific research in education, with a specific focus on evidence-based education (National Research Council 2005). The following search terms were used in each database: (engage* OR “on task” OR “off task” OR complian* OR noncomplian* OR behavior OR behaviors OR aggress* OR disrupt* OR problem* OR challeng* OR regulat* OR affect* OR social* OR emotion*) AND (intervene* OR treatment* OR therapy OR prevent*) AND (child* OR preschool*). Relevant reviews and meta-analyses (e.g., Barton et al. 2014; Carter et al. 2011; McCabe and Altamura 2011) were used to initiate literature trails. The search yielded 10,363 unique hits. We reduced the number of articles to 228 by reviewing the title and abstract and excluding studies that did not focus on models or practices delivered in early childhood classrooms. We obtained the full-text version of the 228 articles and determined that 49 articles met inclusion criteria (see Fig. 1). Studies included in the review are denoted with asterisks in the “References” section.

Fig. 1
figure 1

PRISMA flow diagram depicting literature review results. This figure illustrates the results of the literature review and inclusion process

Full size image

Inclusion and Exclusion Criteria

To be included, a study had to meet five criteria. First, the study had to include a model or practice designed to target one or more of the following child outcomes: teacher-child relationships, social skills (peers), social skills (adults), problem and/or challenging behavior, self-regulation, emotion identification, emotion expression, social problem solving, or engagement (i.e., task-oriented behavior). Second, the model or practice had to be delivered at a school or early childhood program. We did not include interventions tailored to an individual child that would not generalize across children (e.g., function-based interventions; LeGray et al. 2010). Third, the model or practice had to be evaluated using one of the following designs: (a) randomized controlled trial, (b) quasi-experimental group comparison design in which groups were matched or tested for differences, or (c) single-case experimental designs (e.g., ABAB/reversal, multiple baseline, changing criterion, and alternating-treatment designs). Fourth, at least one participant had to be preschool age (i.e., 3–5 years old). Finally, the study had to be published in English.

Coding of the Studies

We first focused on extracting individual practices from each study. We defined practice as a specific statement, behavior, or action of a teacher that manipulates features of the physical, temporal, interactional, or instructional environment in the classroom to promote child adaptation, behavioral competence, or learning. To extract practices, we sought to obtain the intervention manual if one existed. If the manual was not available, we relied on the most thorough description of the model provided in the published literature (e.g., description of the model). If such a source did not exist, we relied on the description provided in the article. In all, we relied upon 32 unique sources (n = 3 manuals and n = 29 articles) to code practices. Each intervention manual and article was double coded; disagreements were resolved via discussion.

Each practice was coded along the three dimensions: (a) Target goal, the domain of functioning targeted by the practice (i.e., teacher-child relationships, social skills (peers), social skills (adults), problem and/or challenging behavior, self-regulation, emotion identification, emotion expression, social problem solving, and/or engagement); (b) Unit, the components, features, or active ingredients that are a major part of a practice designed to promote child adaptation, behavioral competence, or learning. Units (e.g., emotional regulation training, social skills training) are typically composed of multiple practices. This dimension was only coded when appropriate (i.e., both the practice and the unit level were identified); and (c) Practice.

Translating Practices into Practice Elements

We took the extracted practices and grouped together the practices that were the same. We then combined the remaining practices to form practice elements via an iterative process. We defined practice elements as discrete practices, principles, or skills that comprise the models (see Chorpita and Daleiden 2009; Garland et al. 2008). Practices that targeted the same domain of child functioning via the same mechanism were grouped together to form single practice elements. For example, practices such as “behavior specific praise” and “descriptive praise” were grouped under the practice element “praise” (see Supplemental Table 1 in online material for how the praise, emotion regulation, and tangible reinforcement practice elements were created). Once the initial list of practice elements was produced, we grouped them into two categories (Garland et al. 2008): (a) Content, defined as the general principle that guides a practice; and (b) Delivery, defined as the way in which a teacher provides instruction to the child.

Expert Review

We surveyed five experts in early childhood education with experience developing or evaluating interventions for young children with social-emotional learning needs and problem behavior. Four experts received a PhD in education (75 % early childhood special education, 25 % special education) and one received a PhD in clinical/community psychology. All experts were employed in academic university settings (80 % Full Professor, 20 % Research Scientists). Two experts were co-authors of this paper and provided feedback after the practice elements were identified, defined, and extracted. None of the experts were authors of any of the models or practices included in the review. A survey was created by the first three authors, which included the name and definition of the 24 practice elements. Experts were asked to rate each practice element as to its relevance for use as a practice for addressing young children’s social-emotional learning needs and problem behavior in early childhood classrooms. Ratings included (a) “not necessary” to use in early childhood classrooms; (b) “useful”, but not essential to use in early childhood classrooms; or (c) “essential” to use in early childhood classrooms. The experts were asked to comment on each practice element and suggest additional practice elements missing from the list. For three practice elements, one respondent did not provide ratings. The criterion for retention of the elements was a useful or essential rating by at least 75 % of respondents.

Finalization of Practice Elements

The quantitative and qualitative feedbacks from the experts were reviewed, and consensus was reached on the final item list. This work included producing final definitions for each item.

Results

A total of 475 practices were extracted from the manuals (n = 3) and articles (n = 29). Redundant practices were grouped, which produced an initial list of 56 practices. These 56 practices were then transformed into practice elements via an iterative process (see Supplemental Table 2 in online material), which produced a list of 24 practice elements divided into content (emotion regulation, problem solving, promoting behavioral competence, promoting teacher-child relationship, social skills) and delivery (choices, differential reinforcement, error correction, ignoring, instructive feedback, modeling, monitoring, narrating, opportunities to respond, praise, precorrection, premack statements, rehearsal, rules, scaffolding, supportive listening, tangible reinforcement, time-out, visual cues) categories.

We submitted the 24 practice elements to the experts (see Table 1). Fourteen items were rated as essential to use in early childhood classrooms (choices, emotion regulation, error correction, ignoring, instructive feedback, modeling, opportunities to respond, praise, premack statements, problem solving, promoting behavioral competence, promoting teacher-child relationship, scaffolding, social skills). Three items were rated as essential by four of five experts and useful by one of five experts (differential reinforcement, monitoring, rehearsal). One item was rated as essential by three of five experts and useful by two of five experts (time-out). Two items were rated as essential by two of five experts and useful by three of five experts (narrating, tangible reinforcement). Four items were rated as essential by three of four experts and useful by one of four experts (precorrection, supportive listening, rules, visual cues). Given that the experts rated all of the items as useful or essential, we retained all of the practice elements. Three practice elements were modified, however, based upon expert feedback: promoting behavioral competence, precorrection, and rules.

Table 1 Common practice elements, definitions, expert ratings, and frequency counts
Full size table

Table 1 provides definitions and frequency counts of the 24 practice elements. To identify practice elements found in evidence-based models and practices, we searched the (a) What Works Clearinghouse (ies.ed.gov/ncee/wwc/), (b) Blueprints for Healthy Youth Development (www.blueprintsprograms.com/), and (c) CASEL (www.casel.org/preschool-and-elementary-edition-casel-guide). Social skills training was listed on What Works Clearinghouse; Incredible Years (Webster‐Stratton, C et al. 2008), PATHS (Domitrovich et al. 2007), and Tools of the Mind (Barnett et al. 2008) were labeled effective on CASEL; Head Start REDI (Bierman et al. 2008), and Incredible Years were listed as “promising” on Blueprints. Twenty of the 24 practice elements were included in Incredible Years, the only model listed on multiple lists (see Table 1). Lastly, we examined how delivery items grouped in the five content domains (see Supplemental Table 3 in the online materials).

Discussion

The purpose of this paper was to describe the iterative process used to identify common practice elements of models and practices designed to target the social, emotional, and behavioral needs of young children who exhibit problem behavior. Utilizing a process described by Garland et al. (2008), we performed a comprehensive literature review, coded the identified literature, and consulted with experts. We identified a total of 24 practice elements that describe a range of practices used in early childhood classrooms to ameliorate problem behavior and improve social-emotional outcomes. The list of 24 practice elements represents an initial step in an effort to address some of the definitional issues in the field.

It is important to consider how our list compares with previous efforts to define practice elements in the mental health field as well as the implications such a list can have for efforts to implement these practices in early childhood settings. There is some overlap between our list of 24 practice elements and the lists produced by previous efforts to identify practice elements in psychosocial treatments for disruptive behavior problems (Chorpita et al. 2005; Chorpita and Daleiden 2009; Garland et al. 2008). Garland et al. (2008) identified a total of 21 practice elements and little less than half overlapped with our list (see Table 1). Of the 20 most frequent practice elements identified by Chorpita and Daleiden (2009) in psychosocial treatments for disruptive behavior problems, a little less than half of them overlapped with our list (see Table 1). So even though these efforts focused on mental health interventions, there is overlap in the practice elements identified by the three approaches. The overlap seen across these three independent efforts provides some preliminary validity to the overall approach of identifying practice elements.

As the previous efforts to identify practice elements focused on psychosocial treatments, it is not surprising (and perhaps encouraging) that we identified some unique practice elements. Unique practice elements identified in our study may reflect developmental differences (i.e., practice elements targeting outcomes of young children in our study versus older children in Garland et al. 2008 and Chorpita and Daleiden 2009) as well as contextual needs of stakeholders (i.e., teachers delivering practice elements in the classroom versus practitioners delivering interventions in clinical settings). Specific delivery items identified in our review that were not found in either of the previous efforts such as active supervision, precorrection, and rules may reflect the needs of teachers to promote and manage the behavior of young children who exhibit problem behavior within classroom contexts. Unique items such as opportunities to respond may reflect teachers’ attempts to provide learning opportunities to monitor child progress in the classroom as well as to promote child engagement in individual and group learning formats.

Our ability to identify practice elements has research and practical implications. This list could help inform efforts to develop quality indicators (i.e., a measure of the quality of early childhood classrooms) for the delivery of practices targeting social, emotional, and behavioral outcomes in early childhood classrooms. In mental health, some efforts to improve care have focused on identifying quality indicators embedded in a quality of care framework (McGlynn et al. 1988). Quality of care research seeks to improve the outcomes of individuals by studying how structural elements (e.g., contextual elements including attributes of settings, clients, and providers) and processes of care (e.g., activities and behaviors associated with giving and receiving care) influence outcomes. In this framework, identifying quality indicators is an important prerequisite for engaging in quality improvement (Garland et al. 2010; Pincus et al. 2011). This is because quality indicators provide the means to assess, track, and monitor the performance of providers relative to recommended practice in the field (Hussey et al. 2007).

Of course, an important step in identifying quality indicators is determining the practice elements linked to key outcomes, a point that highlights a limitation of our review. Unlike previous efforts to identify practice elements (e.g., Chorpita and Daleiden 2009; Garland et al. 2008), we did not take strength or level of evidence into account when identifying models or discrete practices as part of the literature review. We opted to take this approach because it allowed us to use a larger dataset to begin developing a set of practice elements in the field of early education. That said, 20 of the 24 practice elements were found in an evidence-based model, suggesting that most practice elements have empirical support. An important future direction will be to accumulate more evidence regarding the empirical support for the individual practice elements. In the long term, it will be important to winnow the set of practice elements using level of evidence. However, doing so will require producing effect size estimates across single case experimental designs, quasi-experimental designs, and experimental group designs, which represent a challenge for the field (see Shadish et al. 2015). It will be important to negotiate this hurdle so that work toward synthesizing the literature across the various research designs can be made, which will aid in the process of identifying quality indicators with clear empirical support.

Turning to applications of the practice elements we identified, it may be possible to train teachers to use them as a part of efforts to improve the process quality in classroom settings. We identified 24 practice elements suggesting that there may be a relatively short list of practices in the early childhood literature that pertain to young children’s problem behaviors and social-emotional learning. Most models and practices evidenced substantial overlap, suggesting that training teachers to use a relatively short number of practice elements would cover most of the content found in existing models. Moreover, most delivery items clustered within a few content domains, suggesting that it may be possible to select particular practice elements based on the area of child need being targeted. This helps to illustrate why some have suggested that adopting a practice element framework for implementation efforts can be more cost effective and scalable than relying on purveyor-driven models (e.g., Chorpita and Daleiden 2009).

Our results could also inform the development of treatment integrity tools capable of supporting implementation efforts. Some have suggested that treatment integrity instruments used in implementation research should focus on three components (Bellg et al. 2004; McLeod et al. 2013): treatment adherence (prescribed interventions), treatment differentiation (proscribed interventions), and competence (quality of prescribed interventions). These treatment integrity components encompass the core components involved in the implementation of practice elements as each plays a critical role in fostering positive outcomes in young children (Durlak 2010). However, to our knowledge, there are no treatment integrity instruments in the early childhood education field that assess all three components and have utility for use within an implementation science framework (Sutherland et al. 2013). Most existing instruments are designed to assess treatment adherence; however, in implementation research, it is important to assess multiple treatment integrity components (McLeod et al. 2013; Sutherland et al. 2013). The 24 practice elements identified herein could form the items of an integrity instrument capable of assessing treatment adherence and differentiation as well as allow researchers to characterize the instructional practices delivered in business-as-usual classrooms. Thus, it is possible to use these practice elements as items for an integrity instrument for early childhood classrooms.

Though our approach to identifying practice elements had a number of strengths, a few limitations should be recognized. First, a small number of experts were used to help establish the content validity of the practice elements, and it is possible that a larger number of experts may have suggested the addition of more items. Second, it is possible that there may be some practice elements included in the models that were not written in the manuals or articles. It could be the case that the description of the practices was mostly limited to those that represented the main focus of study and that some practices were not explicitly described.

In summary, by using a practice element framework we were able to identify 24 common practice elements that appear in comprehensive intervention models or that are discrete practices targeting social, emotional, and behavioral needs of young children who exhibit problem behavior. On the one hand, this process has implications for improving educational services to these children through training and implementation efforts. On the other hand, the practice elements framework may be useful for researchers, policy makers, and practitioners as we search for efficient and effective ways to serve children and youth who are most vulnerable.