Introduction

While early care and education (ECE) teachers report that managing challenging behaviors is the area in which they most want training (Yoshikawa and Zigler 2000), only about 20 % of ECE providers have received recent training on supporting children’s social and emotional growth (U.S. Department of Health and Human Services, Administration on Children and Families, Office of Research and Evaluation 2010–2015), suggesting some may face training access barriers. Federal guidance now strongly encourages states to enhance workforce development opportunities in the social–emotional domain. However, there are few studies of professional development opportunities designed to help ECE teachers support children’s social and emotional development (Han 2014) and studies of various training approaches are needed. To address these gaps, this paper describes the implementation and preliminary evaluation of the Reaching Educators and Children (REACH) program, a training and coaching intervention designed to reduce training access barriers and increase the capacity of early childhood teachers to support children’s social and emotional development.

Fostering Social–Emotional Development in the ECE Classroom

Children learn social and emotional skills beginning in infancy and early childhood. Given that approximately 80 % of young children spend much of their day in non-parental child care (Snyder and Dillow 2015), child care providers play a significant role in shaping children’s social and emotional development. In short, social–emotional development involves children’s growing capacities to recognize and manage their emotions, have positive relationships with adults and peers and solve problems effectively (Elias et al. 1997). The Collaborative for Academic, Social, and Emotional Learning (CASEL), has consolidated research across disciplines to identify a set of interrelated skills thought to be markers of good social and emotional competence (Zins et al. 2007). These skills fall into two broad categories of relational/social skills and emotional skills (Denham and Brown 2010). Important social skills include: Relationship skills (e.g. communicating clearly, listening, cooperating, seeking help and negotiating conflict) and Responsible decision making (Making constructive and respectful choices about one’s own behavior, ability to evaluate consequences of actions and consider the well-being of self and others). Important emotional skills include: Self-awareness (e.g. ability to recognize one’s emotions, sense of self-efficacy and self-confidence, etc.), Self-management (e.g. ability to regulate one’s emotions, including controlling impulses, motivating oneself, setting and working toward goals) and Social awareness (e.g. exhibiting empathy and understanding others’ perspectives, respect for others).

Within this broad grouping of skills, there are different developmental tasks that are central to each age level. In early childhood, key tasks of social–emotional development involve learning to engage with others in positive ways and manage emotions while entering into a world of peers (Denham and Brown 2010). As children move through the early childhood period, adult expectations increase so that by preschool and kindergarten children are often expected to have the ability to sit still and focus, get along with others in group play, follow directions and more (Denham and Brown 2010). Children who do not meet these expectations in the preschool context can be challenging for teachers. This is a serious concern in many classrooms, as national data suggest that 10–20 % of preschool children have some type of emotional or behavioral problem (Egger and Angold 2006; Lavigne et al. 1996). Sadly, children without strong social and emotional skills often elicit negative feedback from teachers, are rarely praised by teachers for appropriate behavior and are often the recipient of ineffective and punitive interventions (Campbell et al. 2002). Without intervention, children with social–emotional delays are likely to go on to have problems in later childhood and adolescence (Campbell 1995; Lavigne et al. 1998), including social and academic problems in school, school dropout, and even adult incarceration (Dunlap et al. 2006). High rates of expulsion in early childhood education settings demonstrate the challenge that these behaviors pose for teachers (Gilliam 2008b).

While these behaviors can be challenging for teachers, there are many opportunities to foster social–emotional competence in the early care and education classroom. Doing so requires both a person-centered focus and an environmental approach (Zins et al. 2007). The person-centered focus recognizes that children can be taught social–emotional skills as they can be taught other academic skills. This focus involves social–emotional education to teach children, in a developmentally appropriate way, the key social and emotional skills described above (relationship skills, responsible decision-making, social awareness, self-regulation and self-awareness). For example, in the preschool years, such skills are learned as teachers’ scaffold children’s play, directly teach problem-solving skills, help children find words for their feelings, talk about values, rules and consequences, and teach children coping skills such as learning to calm down (Copple and Bredekamp 2009).

The person-centered focus must be paired with a focus on the environment. The environmental focus recognizes that effective social–emotional learning takes place within supportive environments, and it is insufficient to only emphasize skill development within the child (Zins et al. 2007). Relationships, emotional climate and classroom structure are all critical elements. Social–emotional development is fostered by nurturing relationships with caring adults—both parents and other key caregivers such as child care providers (Phillips and Shonkoff 2000). In fact, Copple and Bredekamp (2009) suggest that the most important way that teachers support children’s social and emotional development is through establishing a personal, nurturing relationship with each child. Through relationships, social–emotional learning begins in infancy and continues throughout development. Beyond relationships, the setting features are also important. Social–emotional learning is supported through developmentally appropriate classroom environments, where teachers engage children in meaningful activities, provide substantial time for child-focused learning and create structure with predictable schedules, routines and rules. Recent research supports the importance of this environmental focus. Research consistently demonstrates the importance of a positive emotional climate and high quality interactions between teachers and children for children’s healthy social and emotional development (Pianta et al. 2009; Raver et al. 2009). Further, the quality of the interactions between teachers and children is more strongly associated with child outcomes than other aspects of child care quality (e.g. teacher education, physical space requirements, teacher–child ratios) and even global measures of quality, such as environmental rating scales (Beller et al. 1996; Burchinal et al. 2011; Mashburn et al. 2008; Mashburn 2008).

Increased Emphasis on Workforce Development

There is evidence to suggest that many teachers could benefit from workforce development opportunities related to social–emotional development. While we know that a high quality environment is important for the development of social–emotional competence, large-scale studies of preschool programs suggest that classrooms with both high quality instruction and positive child–teacher relationships are not in the majority (Howes et al. 2008). Further, high rates of preschool expulsion suggest a need to support teachers, parents and children. Federal guidance now strongly encourages states to enhance workforce development opportunities designed to help ECE providers foster children’s social and emotional development, including investing in training teachers to implement Positive Behavior Intervention Support (PBIS) models as well as increasing access to early childhood mental health consultation (ECMHC), both described below (U.S. Department of Health and Human Services and U.S. Department of Education 2014; U.S. Department of Health and Human Services, Administration on Children and Families, Office of Child Care 2015).

The primary PBIS model for early childhood, known as the Pyramid Model (Fox et al. 2003) is a widely known tiered intervention framework for supporting the social, emotional and behavioral development of young children and is disseminated by The Center on Social and Emotional Foundations of Early Learning (CSEFEL; www.vanderbilt.edu/csfel) and the Technical Assistance Center on Social Emotional Intervention for Young Children (www.challenginbehavior.org). Consistent with theory on social–emotional learning described above, it includes both an environmental focus and a person-centered skill building approach. The framework addresses three tiers, which are supported by a foundation of an effective workforce. Tier 1 (Universal Promotion) addresses practices that support social and emotional development of all children, including nurturing and responsive caregiving relationships and high quality, supportive environments. Tier 2 (Secondary Prevention) includes direct instruction of social and emotional skills particularly for children with delays in their social and emotional skills and who are at risk of developing more serious challenging behaviors. Tier 3 (Tertiary Interventions) addresses individualized interventions for young children with persistent challenging behaviors that are not responsive to Tier 1 and 2 interventions. Tier 3 is not a departure from Tiers 1–2, but makes use of those foundational strategies more intentional, frequent, comprehensive across settings and paired with close monitoring.

Few studies of professional development opportunities designed to help ECE teachers support children’s social and emotional development have been reported, including related to the Pyramid Model (Han 2014). An exception is a recent randomized study examining implementation of the Pyramid Model in public Pre-K classrooms in Florida and Tennessee in which teachers in the intervention group (n = 20; training workshops and 16 weekly coaching sessions) made more progress toward implementation of the Pyramid Model than those in the comparison condition (Hemmeter et al. 2011). The study also reports that children in intervention classrooms were rated higher on social skills and that target children identified as having behavioral challenges had greater reductions in problem behavior than children in control classrooms. However, questions remain related to the type and extent of training and coaching needed to help teachers implement the strategies of the Pyramid Model. Additional research is needed to understand factors involved in facilitating implementation of the model across ECE settings and to evaluate the impact of Tier 1 strategies on child outcomes (Fox et al. 2010).

Early Childhood Mental Health Consultation (ECMHC), the second strategy specified in the 2014 joint policy statement (U.S. Department of Health and Human Services and U.S. Department of Education, 2014), supports children’s social–emotional development by teaming a specially trained mental health professional with an ECE provider in an ongoing problem-solving and capacity-building relationship (Brennan et al. 2008; Cohen and Kaufmann 2005). ECMHC is an effective approach to build teachers’ capacity to implement interventions across the three Tiers of the Pyramid Model, as well as address other barriers to building nurturing relationships within the child care setting (e.g. teachers’ stress or depression, teacher beliefs, program policies, etc.). There is a growing body of evidence that highlights the effectiveness of ECMHC in increasing teacher sensitivity, reducing stress and improving child outcomes (Brennan et al. 2008; Perry, Allen, Brennan and Bradley 2010). As with implementing the Pyramid Model, there are infrastructure issues that must be addressed (e.g. ECMHC trained mental health professionals are not available in all communities). Communities without universal access to ECMHC may need to look outside ECMHC to support foundational features of programs (e.g. enhancing basic understanding of developmentally appropriate practice).

Addressing Gaps in the Continuum of Supports for ECE Providers

Even with growth in these promising practices to build the capacity of caregivers, the 2012 National Survey of Early Care and Education indicates that only about 20 % of ECE providers serving children under five received specific training on facilitating children’s social and emotional growth in the past year (U.S. Department of Health and Human Services, Administration on Children and Families, Office of Research and Evaluation 2010–2015). In the state in which REACH was developed, state level reports indicated that private and rural ECE providers may be a particular target, given that the more comprehensive (30 or more hours) social–emotional professional development opportunities were primarily attended by teachers in the state and federally-funded ECE programs and those in urban areas (McKelvey and Chapin-Critz 2011). New approaches may be needed to introduce social–emotional learning concepts, consistent with the Pyramid model and broader theory, to ECE providers who are less likely to access more in-depth training opportunities.

Purpose of this Paper

To address the need to make foundational training in social–emotional development (consistent with the Pyramid model) more accessible, we developed REACH (Reaching Educators and Children). The purpose of the current paper is to describe the initial implementation of the REACH intervention and the results of a preliminary evaluation of REACH, using a pre–post evaluation design. The evaluation was designed to examine teacher satisfaction with the program, and to determine whether there is evidence of improvements in the classroom environment, use of targeted social–emotional supports and in children’s prosocial and problematic behavior.

Description of the REACH Intervention

REACH Goal

The overall goal for the REACH program is to increase the capacity of ECE staff to support children’s social–emotional development. Our intention was provide an introduction to basic practices within each level of the Pyramid model. REACH is designed to build a foundation in teachers who have minimal knowledge about supports for children’s social and emotional development. Using a ‘we bring the training to you’ approach, we hoped to reduce access barriers of child care center directors and teachers due to geography, lack of release time, no substitute teachers or other practical barriers. These teachers may be found in any setting, but we expected the need would be greatest in private child care (as opposed to state funded pre-K or Head Start). These ‘hard to reach’ providers were our target, thus we call our teacher training project, REACH.

Rationale for Training Design

In designing the REACH curriculum, we incorporated literature on effective professional development techniques. In general, in-service professional development for the ECE field is under-researched. However, there is general agreement among researchers that more positive gains are made when training has: (1) an extended and continuous format with each session building on earlier sessions; (2) a curriculum that is fixed yet is able to be individualized to participants; (3) the opportunity for participants to apply the knowledge; (4) trainer observation and feedback related to classroom implementation; (5) the opportunity for participants to reflect on what they learned and to share accomplishments and challenges (Epstein 1993; Sheridan et al. 2009; Spodek 1996; Zaslow and Martinez-Beck 2005).

As we describe below, REACH was designed and is being implemented to accomplish those recommendations. First, there are two director workshops and 6 teacher workshops lasting 1.5 h each, which are sequenced so that each builds on the last (see Table 1). Throughout the 6 month training program, our Trainer Guide instructs the coach to remind participants of an earlier concept and to turn back and look again at earlier handouts. Further, consistent with the recommendation of Han (2014) related to best practices in professional development designed to promote children’s social competence, the sequence of the workshops is designed to introduce promotive and preventive strategies first (e.g. Tiers 1 and 2 of the Pyramid Model), rather than jumping to ‘quick fixes’ for challenging behavior. Additionally, the second director workshop relates the main messages of each of the teacher workshops to similar concepts and strategies at an adult level (e.g. problem solving for grownups). These are aimed at helping directors build positive adult–adult relationships and improve organizational climate (Bloom et al. 2010). Second, coaches build relationships with each participant during workshops and classroom visits and through staying in touch by email and texts. This allows coaches to individualize their support for teachers throughout the partnership. Third, REACH workshops include interactive components such as group activities in which teachers get to practice what they have learned. They get additional practice during the classroom visit with their coach. Fourth, during classroom visits, skills and concepts are modeled by the trainer and teachers are coached as they try them. Finally, reflection is encouraged through the use of monthly self-assessments, where teachers reflect on the frequency with which they engage in techniques they are learning through the REACH training. They are also asked to reflect about something they would like to change in their approach to the classroom, and they write that down on a daily practice plan which is shared with their coach and director.

Table 1 Description of REACH teacher workshop objectives, skills and toolkit items
Full size table

Professional Development Training of Teachers

The trainings that are a part of REACH include 2 workshops for child care program directors, assistant directors, or other decision makers. The first director workshop focuses on the importance of supporting children’s social–emotional development in the early years, introducing the social–emotional teaching pyramid and providing a detailed overview of REACH. The second director workshop focuses on how the director can nurture staff so that they can nurture children. It includes strategies the director can use to support implementation of REACH, assess organizational climate, and use the REACH strategies at an adult level to enhance staff to staff relationships and teamwork. REACH trainings for teachers include a strand of 6 workshops (see detailed description in Table 1) that address the levels of the social–emotional teaching pyramid. These 1.5 h workshops are designed to be delivered monthly to small groups at a location chosen by the director (typically within the child care center itself). To reduce training access barriers, REACH coaches present the same training session twice on-site (or more as needed) in the same day or subsequent days to allow all staff to attend. In fact, a key component of the REACH training is that the training is designed to be center-wide and inclusive of all staff, regardless of role.

REACH has an extensive list of materials for training. Each participant receives a manual during the first workshop and each month thereafter a set of handouts and parent pages is added. Each training has a companion implementation ‘toolkit’ with various items such children’s books, schedule pocket chart, CDs, puppets, posters and Calendar Cards with daily practice prompts that teachers can use to practice the skills and techniques taught during the workshop (see Table 1).

Classroom Coaching of Teachers

A classroom visit follows each of the six teacher workshops. The purpose of the classroom visit is to support teachers in using the skills and concepts taught in the workshop sessions. The coach interacts with teachers on ideas to try or how to adapt the concept to their situation/age group. Each workshop has a standardized classroom visit form to provide guidance and structure for the visit in order to ensure fidelity to the REACH curriculum.

The coach sets expectations for classroom visits during the initial director workshop, during each training workshop, and again upon arriving in the classroom. The primary focus of the classroom visit is the session just taught; however, if other opportunities arise regarding a previous session objective, the coach may include that as well, pointing out that each strategy builds on the others. During the visit, the Daily Practice Plan (DPP) may be reviewed. The DPP is a document used at the end of each session that gives the participant an opportunity to self-select one or two ideas or strategies to try. The coach may model, guide, or observe for those techniques or others described during previous training sessions. The coaches are trained to use the classroom visit to build positive, nonjudgmental relationships with each teacher, point out strengths, problem solve, and listen to feedback. The coach might also meet with the director about ways to support staff in practicing REACH strategies.

At the end of the REACH intervention, the coach meets with the director to help create an action plan to continue to increase their ability to support children’s social–emotional development over the long-term. The plan includes linkages with other state-supported initiatives such as ECMHC, technical assistance to facilitate participation in the Quality Rating and Improvement System or more in-depth training in social–emotional development. The director also receives a set of REACH training DVDs and Training Guidebook, which includes a brief training DVD for each of the 6 teacher workshop. Directors are encouraged to use the set as part of staff orientation or as mini-refreshers at staff meetings.

REACH Implementation

Training of REACH Coaches

The REACH coach training process was led by one of the REACH program’s two developers. REACH was implemented by four trainers with master’s degrees and experience in either the fields of early care and education or early childhood mental health. Each training is accompanied by a trainers’ guide with specific instructions for the trainer on how to teach each component and conduct each activity. Each trainer was asked to study the REACH trainer guide and trainers were given supplemental reading material to more deeply familiarize themselves with the content. For each training they first observed the REACH developer deliver the training, then they delivered mock trainings and finally delivered a training observed by the REACH developer.

Recruitment

We targeted programs where the majority of staff were non-degreed, had not attended the state’s more comprehensive trainings in social–emotional development and where directors indicated great difficulty accessing training for their staff. Directors interested in participating were asked to complete an on-line form describing past training received by their staff and indicating whether they had found training difficult to access in the past. When center directors reported being able to send their teachers to the more comprehensive social–emotional trainings in the state, the REACH team referred them to those trainings.

Centers were recruited for REACH participation in a variety of ways. In the first year of implementation of REACH, kick-off events were held in communities identified as underserved (no past year or upcoming training in social–emotional development) based on a review of training opportunities in the state’s professional development registry (web-based system identifying all professional development opportunities for ECE providers). All licensed child care centers in those communities were invited to an open house to learn about REACH. REACH referrals also came through child care licensing or local child care resource and referral center (called Child Care Aware) staff.

Description of Participating Centers

This study focused on teachers and children who received REACH training in 2013 and 2014. The teachers and children were housed within 30 child care centers that varied in size, funding source and quality rating. Of the 30 participating centers, 56.7 % were private licensed child care with no public funding, 3.3 % were state-funded pre-kindergarten (Arkansas Better Chance for School Success or ABC), and the remaining 40.0 % provided a mix of private and publicly-funded programs. The majority (17 or 56.7 %) were not participants in the state’s voluntary Quality Rating and Improvement System, even at the ‘entry’ level. More than one-third (36.7 %) served children with disabilities (providing ECE program paired with therapeutic services funded by Medicaid) and 40.0 % (N = 12) served foster children.

Evaluation Questions and Methods

The evaluation involved a one-group, pre–post-test design and the evaluation was approved by the Institutional Review Board at the University where REACH was developed. Observational and survey data collection took place prior to the start of the REACH program and 6 months later, after completion of the last REACH teacher workshop. The observations were designed to allow us to observe changes in the way teachers interacted with children in their care as well as global changes in children’s behavior.

Our approach to data collection varied depending on the role of the staff involved. We gathered classroom observation data on staff functioning as teachers (lead or assistant) in classrooms with children in toddler or preschool classrooms. We did not complete observations in classrooms serving only infants (under 18 months of age) because our observation tools were not appropriate for evaluating infant classes and we did not anticipate having a large enough sample to analyze separately. We observed a maximum of three teachers per classroom (some classrooms had multiple assistant teachers or floaters), focusing on the teachers with the most direct contact with children. We did not observe substitute teachers due to their limited time in any one classroom, nor did we observe staff working non-standard hours such as late afternoon, evening or night shift. Observations were completed for each teacher independently. Programs were encouraged to have all staff participate in REACH workshops, and many programs chose to train even non-classroom staff such as administrative or support staff (e.g. office staff, cook, van driver).

Classroom observations were completed by trained research assistants using the measures described below. Observations were completed on both lead and assistant teachers for a period of one and one-half hours in the classroom. Each observer completed training with the research team before conducting independent observations. The training consisted of operationalizing each item on the measures described below, watching videos of teachers and children interacting in the classroom, and practice scoring videos followed by discussion of areas of agreement and disagreement between the trainee and the ‘gold standard’ observer. Next, observers went into classrooms in pairs and conducted practice observations, and discussed their scoring afterward to identify and resolve any areas of disagreement. This process continued until exact agreement between observer pairs on the score for at least 60 % of the items, and scores were no more than one point apart (indicating disagreement) on 80 % of the items.

Data Collection Tools to Address Each Research Question

Below we list our research questions and describe the teacher survey and classroom observation tools used to address each research question. With the exception of teacher satisfaction and demographic information, all measures were administered pre and post REACH training.

  1. 1.

    Were teachers satisfied with REACH? To address this question, we report teacher attendance as well as teacher satisfaction with the program overall and individual program components. At the post-survey, teachers were asked to rate the helpfulness of each REACH component (e.g. calendar cards, coaching visits) and were asked several questions about their relationship with their coach and the impact of REACH on their classroom.

  2. 2.

    Is there evidence of improvements in the classroom environment, including nurturing and responsive caregiving relationships and classroom structure? To measure teacher–child interactions we used the Arnett Caregiver Interaction Scale (CIS; Arnett 1989). The CIS is an observational rating scale which consists of items that assess the teacher’s sensitivity (e.g. warm, enthusiastic, and caring behavior), and ineffective or negative behaviors including punitive behaviors (e.g. hostility, harshness, and use of threats), detached behaviors (e.g. lack of involvement, disinterest) and permissive behaviors (e.g. lack of appropriate rules or discipline). Each item is rated on a four-point scale. The CIS has been used in many studies of child care programs, including in multi-site studies such as the Cost, Quality and Outcomes Study (Peisner-Feinberg et al. 1999). We computed an overall score, in which items were coded so that a higher score indicates more sensitive, engaged, positive caregiving (Cronbach’s alpha = .92). Related to the structure of the classroom, we developed an observation of Use of Classroom Schedule and Transitions. In preschool classrooms (not toddler, as routines are more individualized) observers were asked to indicate whether the daily schedule included words and pictures, was posted at eye level for the child, and was being followed at the time of the observation. They were also asked to rate the teacher (on a four-point scale) on the extent to which he/she used transition activities to help children move smoothly between activities.

  3. 3.

    Is there evidence of increased use of targeted social–emotional supports? To address this question, we utilized the Staff Teaching Feelings and Problem-Solving Scale from the Preschool Mental Health Climate Scale (PMHCS). We adapted five items from the PMHCS (Gilliam 2008a) to observe the extent to which teachers engaged in teaching feelings and problem-solving in interactions with children. Examples of items include ‘Staff helps children label their own feelings’ and ‘Staff actively promotes children’s use of language to prevent/negotiate conflict.’ Although the PMHCS is typically used to rate the classroom as a whole (i.e. combining the interactions of all teachers into a classroom score), we adapted these items to rate each teacher separately consistent with our approach to the Arnett CIS. We utilized a four point response scale (same as Arnett CIS), with higher scores indicating more consistent use of these techniques by teachers. The internal consistently coefficient (Cronbach’s alpha) for this scale was 0.92.

  4. 4.

    Is there evidence of improvement in child prosocial or problematic behaviors? To measure child prosocial behavior we utilized a subscale from the Preschool Mental Health Climate Scale (PMHCS). We adapted four items from the Child Interactions subscale from the PMHCS (Gilliam 2008a) as a measure of child prosocial behavior for each group of children we observed. Examples of items include ‘Children interact with peers in a way that shows mutual affiliation, concern or affection’ and ‘Children are involved, well behaved, cooperative and attentive.’ Items were scored on a scale of 1 (‘not at all’) to 4 (‘very much’), with higher scores indicating more prosocial behaviors. Because we were observing these items in the context of an observation of a specific teacher, we adapted this scale to rate the children we observed interacting with the target teacher (rather than the classroom as a whole). The internal consistently coefficient (Cronbach’s alpha) for this scale was 0.91. To measure classroom level behavior problems we developed a measure of Child Observed Behavior Problems. Observers were asked to rate the frequency of observed behavior problems in the classroom as a whole occurring across a 45-min span during the classroom observation period. Observers marked each behavior concern they observed in three categories: verbal aggression, physical aggression and other disruptive behavior (e.g. tantrum, prolonged crying). We summed the observed incidents within each category.

Sample Description

REACH trained 317 staff who reported teaching in a toddler or preschool classroom. The primary analysis sample for the evaluation study includes 197 classroom teachers of toddler and preschool age children who were observed prior to the start of REACH training (120 were not observed for reasons described in the methods section). Of those 197 teachers, we have post-training observations on 139. Reasons for attrition are as follows; 39 teachers left employment, 12 changed roles within a center (going to a non-teaching position, or switching to another age group which was not observed), and 7 were not observed due to medical leave or other prolonged absences. When teachers could not be observed post-training due to a leave or changed role, we attempted to obtain the teacher survey and included those in the analysis when available.

Of the 197 teachers initially observed, 45.7 % were toddler teachers and 54.3 % were preschool teachers. The average number of years of overall teaching experience for the teachers was 7.54 (SD = 7.06), and about half (51.1 %) had been employed for more than 5 years. Three-fourths did not have a college degree. Almost a third (29.7 %) reported having a Child Development Associate credential, while 25.4 % had an Associates, Bachelors or Master’s degree. In terms of classroom role, 44.7 % were lead teachers, while 36.0 % were assistant teachers or ‘aides.’ The remaining 19.3 % had other roles at the center (e.g. directors functioning as teachers). About one-fifth of teachers (17.9 and 20.6 %) had attended the two most commonly provided social–emotional focused trainings in the state.

Analysis

We examined changes from pre-test to post-test using paired t tests for continuous variables and the McNemar change test for dichotomous variables. Effect sizes are presented using Cohen’s d (Cohen 1992), where .20, .50, and .70 represent small, moderate, and large effects.

Results

Evidence of Participation and Teacher Satisfaction

Of the 197 toddler and preschool teachers that were the focus of the evaluation study, participation rates are shown in Table 2. To highlight the level of participation among teachers that remained employed for the duration of the REACH program, we show the stably employed group separately. As seen in Table 2, 85.9 % of those who remained employed received at least 5 of 6 sessions. Among those who remained employed for the duration of the intervention, the average number of sessions attended was 5.46 (SD = 1.07). Table 3 shows teacher satisfaction with the REACH program and with individual components, and perceived impact of REACH. More than 90 % of teachers agreed that they would recommend REACH to other teachers, made changes in their classroom because of REACH and felt they had a good relationship with the REACH Coach. REACH toolkit items were the most positively rated program component.

Table 2 Teacher attendance at REACH trainings
Full size table
Table 3 Teacher perception of REACH effectivenessa and satisfaction with components
Full size table

Evidence of Improvements in the Classroom Environment: Nurturing Relationships and Classroom Structure

Results from observations of teacher–child interactions demonstrate multiple positive effects. Observations using the CIS show significant increases in teachers’ positive interactions with children from the pre-test to the post-test [t(137) = −3.69, p < .001, d = .38]. This increase represents a small to moderate effect.

Results also demonstrate that REACH trained teachers better structured their classrooms in ways that support social–emotional development. Teachers increased their use of transition activities to help children move between parts of the day [t(137) = −4.41, p < .001, d = .45]. This represents a moderate effect. Additionally, in preschool classrooms we observed teachers’ use of a classroom schedule being used in recommended ways (i.e. using pictures and words, posted near circle time so that it can be taught, posted at the child’s eye level). Teachers significantly increased their use of the schedule in each these ways. The proportion of preschool classrooms with a schedule that is posted increased from 69.6 to 89.1 % (p = .02), and those posted at children’s eye level increased from 30.4 to 76.1 % (p < .001). Classrooms with schedules that included both words and pictures increased from 30.4 to 78.3 % (p < .001), and those posted near the circle time area increased from 26.1 to 69.6 % (p < .001). Finally, the proportion of classrooms in which the schedule was being followed at the time of the observation increased from 52.6 to 73.7 % (p < .01).

Evidence of Increases in Direct Teaching of Social–Emotional Skills

Results from observations using the PMHCS suggest a significant increase in teaching feelings and problem solving, including activities such as the teacher labeling children’s feelings and helping children use language to solve problems [t(137) = −6.27, p < .001]. This was a large effect (d = .72).

Evidence of Improvement in Children’s Behavior

In terms of observations of children’s behavior and interactions with one another, results suggest small to moderate significant increases in children’s prosocial behaviors as measured by items of the PMHCS [t(137) = −2.93, p = .004, d = .37]. Note that the PMHCS was completed on each teacher and the children with whom the teacher was interacting at the time of the observation. In terms of behavior problems at the whole classroom level (all children in the classroom), observed incidents of verbal, physical and other behavior disruptions, only incidents of verbal aggression were significantly reduced [t(72) = 1.95, p < .05, d = .32] and the effect size represents a small effect.

Discussion

We evaluated a professional development approach to share foundational knowledge about methods for supporting children’s social and emotional development, consistent with the social–emotional teaching pyramid model (Fox et al. 2003) and broader theory. In this study, the programs served were predominantly privately operated programs that were not part of the state quality rating system and the majority of teachers were non-degreed and had not attended more comprehensive training in social–emotional development.

There is evidence that teachers were satisfied with the REACH program. Of teachers who remained employed during the 6 months of the REACH partnership, more than 85 % of completed at least 5 of the 6 trainings. Results of teacher surveys suggest that teachers found most REACH components (coaching visits, toolkits, calendar cards, etc.) to be useful. The ‘toolkits’ were particularly valued by teachers, with more than 95 % agreeing they found them to be helpful. Reports from REACH trainer/coaches suggest that many programs were operating under significant financial strain that limited the ability of the teachers to purchase needed materials for their classroom and that the provision of these materials ensured that teachers interested in implementing new practices would have some of the tools to do so.

While our pre–post evaluation design does not allow us to speak definitively about the impact of the program on teachers and children, our results are promising and suggest that additional study of the REACH program is warranted. Evaluation results suggest that from the baseline observation to the post-REACH observation, teachers increased their use of practices in the first two tiers of the Pyramid Model. Related to Tier 1—Nurturing and Responsive Relationship, results from observation using the Arnett CIS suggest improvements. Teachers were observed to increase their use of sensitive, nurturing caregiving (such as speaking to children in warm tones, getting down on their level, paying attention to the children, listening attentively to the children) and a decrease in more negative or detached behaviors (such as speaking with irritation or hostility or not seeming interested in what the children are doing). The effect size suggests a small to moderate increase.

Related to Tier 1—High Quality Supportive Environment, results from classroom observations also suggest teachers made improvements from the pre-REACH observation to the post-REACH observation. Specifically, teachers increased their use of a daily schedule that is posted so that children can see and using pictures and words. While our observation approach did not allow us to determine whether it was a well-balanced schedule (with an appropriate mix of large and small group activities, indoor and outdoor play, etc.), we were able to document an increase in the extent to which the planned schedule was actually being followed. We were also able to document an increase in use of structured transitions to help children through changes between activities.

Some of the largest gains for teachers were related to Tier 2—Targeted Social and Emotional Supports. Results from observations using a subscale of the Preschool Mental Health Climate Scale suggest that teachers on average made moderate to large gains in their use of teaching feelings and problem solving, by doing things such as labeling children’s feelings and helping them solve problems using their words.

The time frame of our evaluation did not allow us to evaluate the extent to which teachers adopted practices in Tier 3—Intensive Individualized Interventions, as this concept was introduced in the final training and our post-REACH evaluations took place shortly after. This training was designed to introduce teachers to these concepts, and we would anticipate additional support and consultation would be needed for successful implementation. However, there is some evidence of improvements in children’s prosocial behaviors and decreases in children’s problems behaviors, specifically verbal aggression, from the pre-REACH observation to the post-observation. These changes were generally small, and physical aggression was not reduced. It may take more practice on the part of teachers, or more advanced implementation of the strategies, to see greater improvements in children.

A key limitation of this study is our lack of a control group. While our research design limits our ability to attribute these changes to the REACH program, there are reasons to be optimistic that REACH was at least partially responsible for the improvements. First, the average teacher was employed for more than 7 years at the start of REACH so it seems somewhat unlikely that they would significantly change their approach absent intervention. Second, there is some evidence that teachers’ classroom practices stay generally unchanged over a period of months in the absence of intervention. For example, in the previously cited study of a randomized study of the implementation of the Pyramid Model, without intervention, use of Pyramid Model practices stayed fairly flat over four observations (Hemmeter et al. 2011). Similarly, classroom observation data collected for the purposes of establishing the reliability of the Classroom Assessment Scoring System (CLASS) suggest that the practices of preschool teachers remain generally unchanged across a school year, with no significant changes in CLASS measures of positive and negative climate, teacher sensitivity and behavior management (Pianta et al. 2008). An additional limitation was our inability to ensure observers were blind to whether they were conducting a pre or post-observation. The presence of REACH materials (e.g. feelings posters, transition songs, visual schedules) in the center at the post-observation generally allowed the Research Assistant to know what kind of observation was being conducted.

We should note that teacher turnover was a significant barrier, as almost 20 % of teachers left employment before the end of the REACH program. While we armed directors with videos and training materials to use with future new hires, we do not know the extent to which they will use those resources.

Future research should examine REACH impacts using a stronger evaluation design. Given the problems associated with teacher turnover, longer-term follow-up would also be helpful to know whether REACH has lasting impacts. It would also be helpful to know how various teacher supports can work together; for example, future study could examine how REACH and ECMHC supports combined (or in sequence) may result in greater benefits for children. While additional study is needed, our findings suggest that the REACH program may be a promising approach to supporting teachers who face training access barriers and who have little prior exposure to social–emotional training content.