Introduction

Youth who exhibit behavior problems and social-emotional deficits in elementary school are at risk for a host of academic and behavioral problems across the life course (Kellam et al., 2008) whose impact and costs extend to the broader communities in which these individuals live (Office of National Drug Control Policy, 2004). The past two decades have brought clear progress and a stronger empirical base to the field of school-based prevention, with programs developed to support teacher instruction of social-emotional and self-management skills to students. Reviews and meta-analyses of school-based prevention programs that address substance abuse (Gottfredson & Wilson, 2003), violence and anti-social behavior (Hahn, et al., 2007; Wilson & Lipsey, 2007), mental health (Greenberg, Domitrovich, & Bumbarger, 2001), and positive development (Catalano, Berglund, Ryan, Lonczak, & Hawkins, 2002) have shown that universal prevention programs can reduce problem behaviors and build protective factors that reduce further risk. Many of these school-based universal preventive interventions include the explicit teaching of social and emotional skills as a way to encourage self-regulation and positive peer relations among students (Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011). These interventions typically include practices that teachers are encouraged to use every day to foster the social-emotional skills in students and to create a positive classroom climate. Given that higher quality implementation is associated with better student outcomes (Derzon, Sale, Springer, & Brounstein, 2005; Durlak & DuPre, 2011), the need for widespread and effective training in preventive interventions is strong.

As part of a broader “science to service” mandate and demand for accountability, many federal and state funding agencies require communities to use evidence-based preventive interventions (EBPIs) that have been evaluated in randomized controlled trials. Typical professional development for teachers in EBPIs consists of one-time in-person training workshops (Darling-Hammond, Wei, Andree, Richardson & Orphanos, 2009). In-person trainings for teachers are resource-intensive, requiring significant teacher time and cost for participation, often within the confines of a predetermined workshop schedule. Unfortunately, the teaching profession experiences a high rate of attrition such that 40–50 % of teachers leave the workforce within 5 years (Ingersoll & Smith, 2003). The sheer logistics of providing in-person training, materials, and support to a large and transitory teaching staff is frequently well beyond the resources of most school systems; therefore, it is essential to develop flexible and less expensive modes of professional development that result in outcomes comparable to or better than to those achieved by in-person trainings.

With the continued and rapid development of technology, online learning formats have been explored as potential alternatives to in-person trainings. From a theoretical perspective, online formats may optimize learning due to instructional practices that incorporate a variety of formats, including text, graphics, audio, video, interactive exercises, and instructional communication (Arbaugh, 2005; Sitzmann, Kraiger, Stewart, & Wisher, 2006). Evidence for the effectiveness of online learning for adult learners is accumulating. Findings from two of the first meta-analyses on the effectiveness of online learning practices suggested that learning outcomes for online education and in-person education were not significantly different (Bernard et al., 2004; Cavanaugh 2001). Another meta-analysis of job-related courses even found advantages in knowledge acquisition for online over in-person learning (Sitzmann et al., 2006), although not all studies controlled for amount of instruction. In a more recent meta-analysis that included 45 studies of various adult learners (e.g., graduate students, professionals taking job-related courses), online learning was comparable to in-person instruction on learning outcomes (U.S. Department of Education, 2010). Additionally, classes that blended online and in-person methods produced stronger learning outcomes than in-person approaches (USDOE, 2010).

Research thus far suggests that online trainings can be as effective as in-person trainings; but to our knowledge there are no studies examining an online training introducing teachers to an EBPI. However, a nascent body of literature explores the use of online support strategies to promote high quality implementation for teachers who have already received training in particular EBPIs. For example, Bishop, Giles, and Bryant (2005) developed a training and support website for teachers implementing the All Stars prevention program that included the use of a personalized teaching calendar, just-in-time instructional tips sent via email, links to session feedback forms, and discussion forums. In a qualitative study of nine teachers randomly assigned to receive the web-based support, the authors reported evidence of the feasibility and acceptability of this form of professional development. MyTeachingPartner (MTP; Pianta, Mashburn, Downer, Hamre, & Justice, 2008) is another web-based professional development program for teachers that is often used in the context of evidence-based curricula such as the MTP Curriculum for Language and Literacy Development (Justice, Chow, Capellini, Flanigan, & Colton, 2003), Banking Time (Pianta & Hamre, 2001), and the Promoting Alternative THinking Strategies (PATHS) curriculum (Kusche & Greenberg, 1995). MTP relies on videotaped observations of teachers and uses a web-based platform involving videotape review, teacher consultation and guided reflection, feedback, and goal-setting to facilitate growth in each teaching domain. MTP has a large body of research supporting its effectiveness (Hamre et al., 2012; Mikami, Gregory, Allen, Pianta, & Lun, 2011; Pianta et al., 2008).

The All Stars website (Bishop et al., 2005) and MTP (Pianta et al., 2008) provide web-based support to enhance knowledge and provide performance feedback about an EBPI for which teachers received in-person training. As such, they are considered resources to enhance implementation because they offer information and activities that supplement the training teachers have received from another source. Therefore, the results of studies involving the All Stars website and MTP do not directly generalize to questions involving the feasibility of an online “replacement” training relative to an in-person training. Replacement trainings typically offer identical content to in-person trainings. Studies examining online trainings intended as replacements for in-person trainings offer a test of the comparability of the training methods. If a replacement online training shows comparable outcomes to in-person training, then for reasons of access and cost it might be an attractive option for disseminating preventive interventions. Studies of replacement trainings are essential, given the resource constraints facing the nation’s schools.

We can learn about the feasibility of replacement trainings from the growing body of literature involving online trainings in evidence-based treatments geared toward mental health therapists (e.g., Sholomskas, Syracuse-Stewart, Rounsaville, Ball, Nuro, & Carroll, 2005; Weingardt, Villafranca, & Levin, 2006). For example, one study of training modalities for community mental health therapists in dialectical behavior therapy (DBT) found comparable ratings across online and in-person training conditions of therapist satisfaction, feasibility, self-efficacy to deliver the intervention, self-reported skill use, and observer-reported adherence and competence (Dimeff et al., 2009). Moreover, the online training was associated with greater gains in knowledge of DBT relative to in-person training. Another study of online versus in-person training in cognitive behavioral therapy (CBT) found no differences between training conditions on knowledge and skill gains, although therapists who received in-person training reported higher satisfaction than those who had received online training (Beidas, Edmunds, Marcus, & Kendall, 2012). Therefore, it appears that within the field of mental health services, online trainings can achieve comparability with the in-person trainings they are designed to replace.

The Present Study

This study sought to examine the perceived feasibility and pattern of implementation following an online training for urban elementary school teachers delivering an integrated intervention encompassing two evidence-based preventive interventions (EBPIs): the Promoting Alternative THinking Strategies (PATHS curriculum; Greenberg & Kusche, 2006; Kusche & Greenberg, 1995) curriculum and the PAX Good Behavior Game (PAX GBG; Embry, Staatemeier, Richardson, Lauger, & Mitich, 2003). Information regarding feasibility, intervention attitudes, and patterns of implementation were collected for teachers who participated in an online training and compared to the data of teachers who attended an in-person training on the same EBPIs.

The specific research questions were as follows: (a) How feasible is the online training to complete in terms of access to and navigation of online materials? (b) How positive are attitudes about the interventions of teachers who complete the online training relative to teachers who receive the same training content via in-person training? (c) How does the implementation of PATHS and the PAX GBG by teachers who receive online training compare to the implementation by teachers who receive in-person training? Based on theory (e.g., Arbaugh, 2005; Sitzmann et al., 2006) and previous research (e.g., Dimeff et al., 2009; Sitzmann et al.,2006; USDOE, 2010) it was hypothesized that teacher ratings would indicate that the online training was not significantly different to the in-person training in terms of feasibility and intervention attitudes. Additionally, it was hypothesized that teachers who participated in the online training would achieve high levels of implementation of PATHS and the PAX GBG that were not significantly different from those achieved by teachers who attended the in-person training. This hypothesis is, in part, based on meta-analytic reviews of online training (e.g., Bernard et al., 2004; Cavanaugh 2001; USDOE, 2010), as well as the provision of coaching following both training conditions. Coaching has been shown to be important to the implementation of a program following the initial training (e.g., Fixsen, Naoom, Blasé, Friendman, & Wallace, 2005; Joyce & Showers, 2002).

Method

Participants

Teachers

Participants in the online training included 45 teachers in three urban elementary schools. The majority of teachers were female (90.9 %). The proportions of teachers who taught kindergarten (18.2 %), 1st (18.2 %), 2nd (18.2 %), 3rd (18.2 %), 4th (13.6 %), and 5th grades (13.6 %) were approximately equal due to the intentional enrollment of all members of the teaching staff within each elementary school. Teacher age, ethnicity, and educational background were not assessed in this study. Teachers participated in the current study during the 2011–2012 school year based on their employment at one of three schools that had been randomly assigned to the control group for a randomized controlled trial (RCT) involving PATHS and the PAX GBG during the previous school year (2010–2011). As a condition of their school’s randomization to the control group, teachers were offered the opportunity to receive training in the interventions, which formed the basis for the current study of online training materials.

Of note, the principal of one school receiving the online training requested that all classroom teachers and staff (e.g., resource teachers, paraprofessionals) participate in the online training related to the PAX GBG. Therefore, 32 additional school staff reported on the feasibility of the online materials and their attitudes about the PAX GBG. Due to the anonymous nature of the data collection, we were unable to separate out those responses; thus, results related to feasibility and PAX GBG attitudes data reflect the opinions of teachers and staff together.

In order to provide a benchmark for feasibility and implementation, data obtained from teachers participating in the online training were compared with those obtained from 65 teachers across three elementary schools who received in-person training during 2010–2011 school year, prior to the development of the online training. Schools were matched on the basis of school characteristics and randomized to either an intervention or control condition in the RCT involving PATHS and the PAX GBG. It is important to note that although the RCT involved a large number of schools and therefore permitted school-level analyses, teacher-level analyses were conducted in the current study due to the small number of schools per condition.

Similar to teachers who participated in the online training, teachers who received in-person training were overwhelmingly female (84.6 %). Their distribution across grades was similar due to the enrollment of entire elementary schools: kindergarten (12.3 %), 1st (21.5 %), 2nd (20.0 %), 3rd (15.4 %), 4th (15.4 %), and 5th grades (15.4 %).

Interventions

Teachers in this study delivered the PATHS curriculum (Kusche & Greenberg, 1995; Greenberg & Kusche, 2006) and the PAX GBG (Embry et al., 2003).

Promoting Alternative THinking Strategies (PATHS®)

PATHS is a universal prevention curriculum that is designed to foster the social-emotional development of students in grades K-5 (Greenberg & Kusche, 2006; Kusche & Greenberg, 1995). It includes a developmentally appropriate series of lessons and activities that provide direct instruction and practice opportunities to help students develop social-emotional skills. Teachers are encouraged to use several daily practices and a set of teaching strategies throughout the day to encourage the generalization of the skills that are taught by students and to create a supportive context for their use. PATHS has been shown in large-scale RCTs to have a beneficial effect on off-task, aggressive, and disruptive behaviors by improving prosocial cognitions and socially competent behaviors (e.g., Conduct Problems Prevention Research Group, 1999; 2010; Greenberg, Kusche, Cook, & Quamma, 1995; Kam, Greenberg, & Kusche, 2004; Riggs, Greenberg, Kusche, & Pentz, 2006). Studies of PATHS conducted with regular and special education children have yielded significant reductions in both internalizing and externalizing behavior 1 year following the intervention (Kam et al., 2004; Riggs et al., 2006).

PAX Good Behavior Game

Originally developed by Barrish and colleagues (Barrish, Saunders, & Wolf, 1969), the GBG involves teacher use of social learning principles within a team-based, game like context to reduce aggressive/disruptive and off-task behavior and, consequently, facilitate academic instruction. The PAX GBG represents Embry and colleagues’ (2003) efforts to improve the effectiveness of the original GBG (Embry et al., 2003). Like the original GBG, the PAX GBG is a group-based token economy, where the groups or “teams” are reinforced for their collective success in inhibiting inappropriate behavior. The additional “PAX” elements introduced by Embry et al. (2003) primarily consist of verbal and visual cues that teachers and classmates use to promote attentive and prosocial student behaviors and a positive classroom environment. The GBG has a long history of successful implementation and positive academic, behavioral, and substance use outcomes in urban public schools (e.g., Bradshaw, Zmuda, Kellam, & Ialongo, 2009; Ialongo, Poduska, Werthamer, & Kellam, 2001; Ialongo, Werthamer, Kellam, Brown, Wang, & Lin, 1999). Significant long-term effects of the GBG have been apparent through middle school (Kellam, Ling, Merisca, Brown, & Ialongo, 1998; Kellam, Rebok, Ialongo, & Mayer, 1994) and into young adulthood (ages 19–21) (Kellam et al., 2008; Petras, Kellam, Brown, Muthén, Ialongo, & Poduska, 2008).

PATHS to PAX

Although the trainings consisted of separate modules for PATHS and the PAX GBG, the two interventions were seamlessly integrated in practice into a single intervention referred to as “PATHS to PAX.” The integration was facilitated by components that were unique to each program in form, but whose functions were theoretically similar. Moreover, common language and an explicit sequencing for introducing and using components of PATHS and PAX GBG supported consistent implementation across training conditions (details regarding the integration of PATHS and the PAX GBG can be found in Domitrovich, Bradshaw, Greenberg, Embry, Poduska, & Ialongo, 2010).

Trainings

Online Training (OLT)

Existing standardized PATHS and PAX GBG training materials were used to inform the creation of an online course that included PowerPoint slides, audio recordings, and video demonstrations of key program elements to deliver the core content of the training. The training was divided into 12 modules and included an overview of the theoretical basis of each program, a review of the core elements of the PAX GBG and content domains of the PATHS lessons, and a discussion of high quality implementation with video examples of actual teachers delivering the program in urban classrooms. Each module was narrated by a coach from the Johns Hopkins Center for Prevention and Early Intervention or a certified PATHS trainer. The online training contained the same content as the in-person training but did not include the group discussions or lesson planning that was included as part of the in-person training approach. At the end of each module, teachers were required to complete a knowledge survey and answer 80 % of the questions correct before the system allowed them to advance to the next module.

In order to avoid any technical difficulties with accessing the training online, training sessions were conducted at each teacher’s school during a district-wide professional development day approximately 1 week before the school year began. At least two members of the research team were on hand to provide a brief orientation to the online system, address any technical issues that arose, and answer questions. Teachers were provided with breakfast and lunch and were free to take breaks as needed. The online training took between 5 and 8 h for teachers to complete, including these breaks. Teachers had open access to the online course from that point forward.

In-Person Training (IPT)

The in-person training was conducted over the course of two separate days. Teachers attended a one-day (i.e., 8-h) workshop for the PAX GBG which consisted of didactics, discussion, demonstration, and video review led by Dr. Dennis Embry, the program developer, with support from research staff who served as coaches to support teacher use of the interventions. Training in PATHS also consisted of a one-day (i.e., 8-h) workshop led by certified trainers and co-facilitated by the coaches. This workshop also included didactics, discussion, demonstration, and video review.

Measures

Feasibility

Immediately following the training, teachers in the online training condition completed a web-based survey of four items related to training feasibility: (a) The technology for this online training was easy to use, (b) I was able to see and hear the videos clearly, (c) It was easy to move between the presentation and the videos, and (d) This online course was as effective as an in-person professional development workshop. Additionally, teachers rated their overall comfort with online technology in general as well as the extensiveness of their prior experience with online trainings. Participants responded using a four-point Likert scale (1 = strongly disagree to 4 = strongly agree).

Intervention Attitudes

All teachers, regardless of training condition, responded to a survey about their attitudes toward PATHS (four questions) and the PAX GBG (four questions) immediately following their training. Questions regarding each program were similarly worded and intended to elicit teacher perspectives about: (a) the potential usefulness of the intervention, (b) student need for the intervention, (c) teacher confidence to implement the intervention, and (d) likelihood of teacher use of the intervention. Teachers responded using a four-point Likert scale (1 = not at all to 4 = a great deal).

Implementation Quality

For teachers in both training conditions, independent observers completed the PATHS Implementation Rubric (Domitrovich, Greenberg, Schaffer, Darney, Rouiller, & Ialongo, 2006) and the PAX Good Behavior Game Implementation Rubric (Schaffer, Darney, Rouiller, Embry, & Ialongo, 2006) at three time points throughout the academic year to assess each teacher’s implementation quality. These rubrics took place approximately 6 weeks (round 1), 14 weeks (round 2), and 22 weeks (round 3) after the initial training.

During the rubric observations, teachers were asked to deliver a PATHS lesson for which implementation quality across four dimensions was rated using a five-point scale. These four dimensions consisted of thoroughness of teaching PATHS concepts, level of disruption during lesson, pacing of lesson, and teacher affect and energy during lesson. Higher scores reflect better quality implementation and teachers whose mean rating across all four dimensions was approximately 3.0 or higher were generally regarded as implementing with sufficient quality.

Additionally, teachers also were asked to “play” a 5–10-min PAX GBG game during which implementation quality across seven dimensions was rated on a five-point scale. These seven dimensions reflect core components of the game such as preparing students for the game, accurately recording student behavior, and responding neutrally to misbehavior. Higher scores reflect better quality implementation and teachers whose mean rating across all seven dimensions was approximately 3.0 or higher were generally perceived to be implementing with sufficient quality. Interrater reliability for the implementation rubrics was high (α = .93).

PATHS Dosage

Teachers provided information to coaches on a weekly basis regarding which lessons they delivered to their students. The total number of lessons in the PATHS curriculum was approximately 40, but varied slightly across grades. Therefore, to provide a comparable metric across grades, the proportion of lessons delivered across the entire school year was calculated for each teacher.

PAX GBG Dosage

Teachers also recorded the number of PAX GBG games played each week and the duration of each game using a “scoreboard” designed for this purpose. These data were summed across the 31-week implementation period between training and the end of the school year and yielded two variables: total number of games played and total number of minutes played.

Procedures

Following training in their respective conditions, all teachers received in-person coaching (IPC) in their classrooms from their assigned coach to assist with implementing PATHS and the PAX GBG. Coaching was standardized in terms of procedures, frequency, duration, and intensity across the online and in-person training conditions and included material and classroom preparation, modeling, observation, and technical assistance. Across a period of 6 weeks following the training, coaches conducted an average of 5.48 (SD = 2.07) coaching sessions that totaled 1.99 (SD = .75) hours (Becker, Bradshaw, Domitrovich, & Ialongo, 2013). At the end of 6 weeks, coaches accompanied members of the research team who conducted round 1 of the independent observations of each teacher’s program delivery using the implementation rubrics (see description in Measure section). Teachers received written and verbal feedback based on these ratings and other data regarding their implementation. Following the round 1 rubric and for the remainder of the school year (approximately 25 weeks), coaches provided an average total of 7.81 (SD = 3.55) hours of coaching during which they tailored their coaching strategies to each teacher’s needs (e.g., coaches introduced more advanced components of the program to high implementing teachers, whereas coaches continued to demonstrate core components or help with the delivery of the program for low implementing teachers; see Becker, Darney, Domitrovich, Keperling, & Ialongo, 2013 for a more detailed description of coaching procedures). Rubric observations were also conducted approximately 14 and 22 weeks following the initial training. Teachers did not receive monetary compensation for their participation in the study, although they occasionally received small incentives (e.g., classroom supplies, lunch bag) for their efforts toward using PATHS and the PAX GBG.

Results

Feasibility

Teacher perceptions of the feasibility of the online training format were high, as indicated by the percentage of teachers who endorsed “agree” or “strongly agree” to the following feasibility items. Many teachers reported prior experience with online trainings (75.5 %) and most indicated comfort with online technology in general (86.4 %). The majority of teachers reported that the technology was easy to use (84.1 %), that they were easily able to see and hear the videos (66.6 %), and that they were able to easily move between the presentation and videos (91.1 %). The majority of the teachers also reported that the online course was as effective as an in-person professional development workshop (77.7 %).

Intervention Attitudes

PATHS

Teacher attitudes regarding PATHS were compared across training conditions and analyses yielded no significant differences between the attitudes of teachers who received the OLT + IPC and those of teachers who participated in the IPT + IPC (Table 1). Teachers in both groups reported that PATHS appeared useful and relevant to their students’ needs. Both groups also reported feeling confident that they could implement PATHS and that they were likely to use the program in their classrooms.

Table 1 Teacher reports of intervention attitudes, by training condition and intervention
Full size table

PAX GBG

Teacher/staff attitudes related to the PAX GBG were also compared across training conditions (Table 1). Additional staff ratings regarding the PAX GBG were included from one school in which the principal requested all school staff participate in the OLT related to the PAX GBG. Participants in both training conditions reported that the PAX GBG appeared useful and that they were likely to use the program in their classrooms. However, participants in the IPT + IPC group perceived the PAX GBG as more relevant to the needs of their students and expressed more confidence at the end of the training that they could implement PAX GBG compared to participants in the OLT + IPC condition.

Intervention Implementation

PATHS

Mean implementation quality ratings for teachers at each of three rounds of rubric observations are presented for teachers by training condition in Table 2. Across observation rounds, the quality of implementation of the PATHS curriculum for teachers who participated in the OLT + IPC was not significantly different from that of teachers who completed the IPT + IPC. With regard to the percentage of PATHS lessons completed by teachers over the course of the year in each training condition, both groups implemented over 80 % of the lessons (OLT + IPC: M = .84, SD = .26; IPT + IPC: M = .88, SD = .06; t (51.9) = .81, p = .42).

Table 2 Implementation quality as indicated by rubric observations, by training condition and intervention
Full size table

PAX GBG

Similar to the results of the PATHS analyses, quality of teacher implementation of the PAX GBG was comparably high for teachers in both training conditions (Table 2), although Round 2 ratings were significantly higher for IPT + IPC teachers than for OLT + IPC teachers. Additionally, the total number of minutes teachers in both conditions played the PAX GBG across the school year was not significantly different (OLT + IPC: M = 2,191.64, SD = 1,489.40; IPT + IPC: M = 1,818.81, SD = 1,478.34; t (101) = −1.26, p = .21). Of note, teachers in the OLT + IPC condition played the PAX GBG significantly more games with their students compared to teachers who received the IPT + IPC (online: M = 218.61, SD = 92.06, in-person: M = 175.97, SD = 98.34; t (101) = −2.24, p = .03).

Discussion

This study appears to be the first to examine the feasibility of an online training for teachers delivering universal preventive interventions in their classrooms, and to compare teacher implementation following online versus in-person training plus coaching. It was hypothesized that teacher ratings of the feasibility of the online training would be high and that teacher attitudes toward PATHS and the PAX GBG would be comparably high across training conditions. Additionally, it was hypothesized that teachers in both conditions would achieve similar levels of implementation of PATHS and the PAX GBG. Each of these hypotheses and related findings is discussed in turn.

Feasibility

The results from this study indicate that the online training program of PATHS and PAX GBG was feasible to implement with a sample of urban elementary school teachers. Most teachers indicated the perceived effectiveness of the online training was similar to the effectiveness of in-person workshops they have attended in the past. Moreover, most teachers reported they were able to navigate the online training materials easily. Although feedback from teachers indicated that the quality of the auditory and visual aspects of the videos embedded within the online training could be improved, it appears that teachers felt quite comfortable and satisfied with the online training format in general. This is an important finding, given that one concern about the widespread use of online trainings is that difficulty with technology and lack of technical support could interfere with the effectiveness of the training (Buchanan, Sainter, & Saunders, 2013). Aside from a brief verbal overview of how to navigate the online course, the research team that was present fielded relatively few technical issues specific to the online training. Many of the technical issues that arose involved computer or headset malfunctions that were unrelated to the online training. As technology continues to develop and becomes ubiquitous in the teaching profession and daily life, it is expected that the number of teachers reporting comfort with online trainings will continue to grow. New generations of teachers also will likely perceive online trainings as feasible and desirable.

The online training was also designed with concerns related to learner engagement in mind. Although the online format did not provide opportunities for learners to ask questions or engage in lengthy discussions about key elements, as was provided in the in-person training, the course offered quizzes and reflection exercises where knowledge could be assessed. These knowledge assessments covered questions and concerns that have been raised by teachers during prior in-person trainings, as well as during the course of program implementation. Built-in course evaluations also provided an opportunity for learners to document their questions, which were then passed along to the research team and coaches.

Related to feasibility, but not assessed empirically, is the feasibility of a school or school system to train teachers using an online format. The online training was feasible in three urban elementary schools in which teachers were invited to participate. Teachers all had access to computers at the same time as well as access to the internet. Moreover, school administrators were agreeable to having their teachers participate in the online training during the course of 1 day dedicated to professional development. Therefore, online trainings may be feasible in other schools, even those with significant resource constraints such as those in the present study.

Given that the training was carried out with study team members present, it is likely that the feasibility of an online training could be even higher. For example, more flexibility could be introduced by allowing teachers to take the training at the time and location of their own choice. Of note, the online training required approximately 8 hours to complete, whereas the in-person training took place across two 8-hour days. In the future, teachers viewing the online training on their own could view the training in its entirety or pace and chunk the training according to their own preferences. Added flexibility might enhance learning because teachers’ attention tended to dwindle toward the end of lengthy training days that covered copious amounts of material. At the same time, it is possible that the presence of the research team promoted attention to the material and that increased flexibility in the administration of the online training could reduce its effectiveness as a teaching tool if teachers perfunctorily scan or even skip training content. Future evaluations of the online training format in a more naturalistic setting are necessary to evaluate the effectiveness of the online training format, as are measures to assess effectiveness, such as knowledge tests or skill demonstrations.

Intervention Attitudes

Overall, teachers who participated in the OLT reported positive intervention attitudes comparable to those reported by teachers who received IPT. Participants in both training conditions perceived PATHS and the PAX GBG as highly useful and relevant to their students’ needs. This suggests that information about the value of the interventions was conveyed as clearly through the OLT as it was during the IPT. Teachers in both training conditions reported that they were likely to use PATHS and felt confident in their abilities to deliver the curriculum. In other words, teachers in the OLT felt as prepared to begin implementing PATHS as the teachers in the IPT.

Two significant differences emerged with regard to attitudes toward the PAX GBG. School staff in the OLT condition felt less confident about using the PAX GBG than teachers who attended the IPT, although their mean confidence level was still high. Additionally, school staff in the OLT condition reported that the PAX GBG was less relevant to their students’ needs than teachers in the IPT condition. The PAX GBG involves a set of game procedures that is new to most classroom teachers. The IPT offered opportunities for teachers to ask questions about the game implementation that may have enhanced their confidence and understanding about how to use the game with students relative to school staff in the OLT condition. It is important to note that, by design, ongoing coaching provided the opportunity for teachers and other school staff in both training conditions to engage in in-depth discussion and problem solving about both interventions. As seminal work (e.g., Fixsen et al., 2005; Joyce & Showers, 2002) has demonstrated, workshop trainings are not likely to result in solid implementation of a program in the workplace environment. It was our experience during the RCT that teachers who attended the IPT benefitted from interactions with a coach soon after the training because there was so much information conveyed during the workshop. Therefore, we designed the OLT to be somewhat less dense than the IPT, knowing that coaching support around PATHS and the PAX GBG would be more relevant to teachers when they were back in the classroom.

Another explanation for the difference in attitudes between the two training conditions concerns the involvement of 32 additional staff members (e.g., paraprofessionals, cafeteria staff) who took the online PAX GBG training at the request of school principals who aimed to facilitate school-wide adoption of the GBG principles and procedures. It is likely that school staff members who were not classroom teachers felt less confident and had lower intentions because their school roles do not involves managing the behavior of groups of students; thus, their data may have reduced the group mean for the online training condition.

Implementation

Although high ratings of feasibility and attitudes for an online training are desirable, an online training must be judged based on the level of implementation achieved by teachers. Teachers who participated in the OLT + IPC achieved levels of high quality for their implementation across time of both PATHS and PAX GBG that was not significantly different from that achieved by teachers who received IPT + IPC. Moreover, teachers in the OLT + IPC condition delivered PATHS and PAX GBG across the school year at a high frequency similar to that of teachers in the IPT + IPC condition. These results converge with studies of online training in psychosocial interventions for mental health therapists and add to a growing body of literature supporting the feasibility of online training and it comparability in terms of interventionist implementation relative to in-person training (e.g., Dimeff et al., 2009). These results are promising but should be considered in the study specific context of teachers in both conditions receiving weekly in-person coaching. Although untested in this study, it is a plausible hypothesis that teachers who receive coaching without any prior training could achieve implementation levels that are not significantly different than those who receive either IPT or OLT plus coaching.

Limitations

Whereas the feasibility, intervention attitudes, and implementation outcomes related to the online training and in-person coaching model in this study are promising, it is important to note that these results are from a specific sample of elementary school teachers in an under-resourced urban school district. It is possible that online training opportunities may be more challenging in rural areas with less consistent computer or internet access.

For the purpose of this study, participants completed the online training program together, with the support of the research team. On the one hand, it is possible that the presence of the research team might have enhanced the feasibility of the online training for a few teachers who would have otherwise had difficulty navigating the online materials on their own. Additionally, the presence of the research team and the designated time for the online training may have enhanced the attention which teachers paid to the materials. It is unknown whether teachers who could view the modules on their own time would pay close attention to the material. On the other hand, it is possible that the presence of the research team as well as the designated time and location to take the online course reduced perceptions of feasibility and flexibility for some teachers. Future research should investigate more naturalistic access and usage of the online training program.

Implementation outcomes in the study were assessed after teachers in both training conditions received coaching; therefore, the effects of the training alone on implementation cannot be determined. However, coaching was comparable across training conditions and the training is not intended as a replacement for coaching. Rather, the important point is that the OLT provided initial exposure to new interventions that was comparable to the exposure provided by IPT. At the same time, it is possible that teachers who receive coaching without any training could have achieved adequate implementation.

Additionally, it is important to note that the randomization described in the study occurred at the level of the school within the larger context of an RCT. Due to low number of schools who participated in the DL training and the comparison group in the current study (i.e., three schools per condition), school-level analyses were not ideal. Therefore, analyses were conducted at the level of the individual teachers, which may result in inflated type 1 errors (Murray, Varnell, & Blitstein, 2004).

A final limitation is that additional variables that might have influenced study outcomes may not have been measured. For example, because they had been control group teachers within the RCT, it is possible that the teachers in the OLT + IPC condition had positive expectations for the training and intervention. It is also possible that those teachers had heard positive things about the intervention from colleagues at other schools. Positive expectations might have colored their attitudes and promoted their implementation. It is also possible that positive teacher perceptions of their school leadership and climate could promote positive attitudes about the training and interventions as well as motivate teachers to use the interventions regularly, thereby overcoming any shortcomings of the trainings themselves. Related to this, teachers’ perceptions of their own self-efficacy or emotional exhaustion may have exerted an influence over their attitudes and implementation regardless of the training condition. Teacher attitudes toward online versus in-person training were not systematically measured across conditions prior to the study. It is possible that teachers were satisfied with whichever training they received, but would have expressed a preference for a different format had been asked ahead of time. Another possible variable that could explain the group differences in the current study is the coach-teacher working alliance. It has been demonstrated that teacher alliance with a coach is related to implementation of the GBG (Wehby, Maggin, Partin, & Robertson, 2012). Given the significant presence of the coach in the schools, it is quite possible that the results in large part are accounted for by coaching.

Conclusion

There is a great need for teacher professional development on behavior management and social and emotional learning (Boe, Shin, & Cook, 2007; Reinke, Stormont, Herman, Puri, & Goel, 2011). However, the barriers of in-person training models impede schools and districts from developing and maintaining a workforce proficient in EBPIs. The findings from this study provide evidence of high feasibility, intervention attitudes, and implementation following an online training. As a replacement for in-person trainings, an online training format has the potential to address many of the barriers to broad dissemination. For example, an online format promotes access to schools and school systems that may not have the funds or flexibility to permit all teachers to attend an in-person training. Additionally, an online training allows new teachers to easily participate in training regardless of when they join the staff. New teachers would be able to access the initial online training materials even before the school year begins and be prepared to participate in in-person coaching making these interventions more sustainable for schools and districts. Online materials also make it possible for teachers to tailor the instruction to their own learning needs, going through material at their own pace and revisiting training materials as needed. This format of professional development is especially useful for teachers, who often have less flexible and/or limited time for professional development opportunities, especially for behavioral or socio-emotional professional development programs.

Future research should investigate the effects of online training programs in EBPIs on student outcomes to determine the relative effectiveness of online and in-person training. Additionally, a better understanding of the parameters of online trainings (e.g., topics, activities, length) that will facilitate teacher knowledge and skill would move the field toward designing effective trainings (USDOE, 2010). Furthermore, identifying effective ways to disseminate online trainings is integral to building school and district capacity to promote better academic, social, and behavioral outcomes for all students.