Introduction

Violent and aggressive behavior in childhood remains a key risk factor for later juvenile and adult violence and other social and academic maladaptive outcomes (Dodge et al. 2006; Nelson and Dishion 2004). Reducing such behaviors should have lasting effects for both the individual victims and perpetrators. Schools are a setting in which much interpersonal aggression among children occurs (Dinkes et al. 2009) and the only setting with almost universal access to children. Schools are places of education—changing the way children experience and think about social problems and providing modeling and practice in this new way of thinking is the cornerstone of many social and character development (SACD) programs. Programs designed to prevent conflict and aggression.

Recent reviews (Hahn et al. 2007) and meta-analyses (Durlak et al. 2011; Wilson and Lipsey 2007) indicate that such programs have modest, but important, effects on aggression and other problem behaviors. Durlak et al. (2011) report an effect size of .22 for conduct problems (including aggression and delinquency), .57 for social and emotional skills (e.g., interpersonal problem solving, goal setting), .24 for positive social behavior (e.g., social skills), and .27 for academic performance. Here, effect sizes (ES) were moderated by program implementation quality. Wilson and Lipsey (2007) note a similar effect size (.21) for aggressive/disruptive behavior. They also note larger effects for better implemented programs and for those involving students at higher risk for aggressive behavior.

The present study examines aggressive outcomes in students within a cluster-randomized trial of the Promoting Alternative Thinking Strategies (PATHS; Greenberg et al. 2002) curriculum. PATHS is a comprehensive school-based SACD program for promoting emotional and social competencies and reducing aggression and behavior problems in young children. The evaluation tracked the development of two 3rd grade cohorts over 3 years in 14 schools across three districts. Within each district, schools were randomly assigned to one of two groups: the PATHS treatment group or a control group in which traditional educational practice continued. Children’s progress and change in school climate was assessed through child assessments, parent surveys, teacher surveys, and school observations. Specific research question to be addressed include whether participation in PATHS significantly alters the developmental trajectory of a number of aggressive outcomes in 3rd–5th grade children.

The present study adds to the existing literature by presenting evaluation results that follow 3rd grade students until the end of 5th grade, examining change in aggression and aggression-related developmental trajectories (as opposed to point-in-time or cumulative [summed over time] scores). PATHS, as a preventive intervention, aims to decrease risk factors and increase protective factors with the goal of altering children’s trajectories toward positive outcomes and away from negative outcomes. Further, this is the first independent randomized replication of PATHS to be published. The lack of independent replication has received increased attention in the prevention arena (Gandhi et al. 2007; Holder 2010). Concerns such as conflict of interest and a potential for bias in reporting results have been raised. A recent meta-analysis of comprehensive school reform found that developer-led studies yielded considerably stronger effects than did studies performed by independent others (Borman et al. 2003). While much of the PATHS efficacy trials involve the primary developers (Conduct Problems Prevention Research Group [CPPRG] 1999, 2002, 2010; Greenberg et al. 2002)—the current cluster randomized trial independently adds to the efficacy of the PATHS intervention.

Promoting Alternative Thinking Strategies (PATHS) Theoretical Framework

PATHS is a comprehensive program for promoting emotional and social competencies through the development and strengthening of skills in emotional literacy, positive peer relations, and social problem solving. The PATHS program places primary importance on the developmental integration of affect (and emotion language), behavior, and cognitive understanding (Greenberg and Kusche 2006; Greenberg et al. 2002). Implicit in this model is the idea that during the maturational process, emotional development precedes most forms of cognition. As a result, in early life, affective development is an important precursor of other modes of thinking and later needs to be integrated with cognitive and linguistic functions. A critical developmental achievement occurs when the child begins to regulate internal affective states through verbal self-regulation.

The following four assumptions guide the PATHS prevention strategy: (1) Children’s ability to understand and discuss emotions is related to both communicative development and the ability to inhibit behavior and show self-control; (2) Children’s ability to manage, understand, and discuss emotions operate under developmental constraints and is also affected by socialization practices; (3) Children’s ability to understand their own and others’ emotions is a central component to effective social problem solving; and (4) The school environment is a fundamental ecology that can be a central focus of change. PATHS provides specific ideas and materials for integration of concepts through other academic areas and teachers report that this is an often overlooked, yet critical, area for school-based prevention programming (see Greenberg and Kusche 2006 or Greenberg et al. 2002 for more detail regarding the curriculum).

A number of randomized clinical trials with a number of differing samples of young children, including deaf/hearing impaired students, regular education students, and special education students, have demonstrated the short-term effectiveness of the PATHS curriculum. Specifically, three randomized clinical trials have been conducted spanning 1 year of PATHS training with relevant data collected at pre, post, and follow-up (Greenberg et al. 2002). In all three trials, PATHS was found to increase children’s ability to recognize and understand emotions, understand social problems, develop effective alternative solutions, and decrease the percentage of aggressive/violent solutions. Moreover, teachers reported increases in program children’s self-control, emotional understanding, ability to tolerate frustration, and to use effective conflict resolution strategies. At 1-year follow-up, children in both regular education and special needs classrooms reported reduced depressive symptoms and reduced conduct problems. In a 2-year post-intervention follow-up of special education students (through the 5th grade year), PATHS slowed the rate of growth of teacher-reported internalizing and externalizing behaviors (Kam et al. 2004).

PATHS is also the universal preventive component of the Fast Track project (CPPRG 1999). Involving 54 schools from four high risk communities which were randomized to intervention or control condition, the Fast Track program extends from first through tenth grade following cohorts of students deemed initially to be “at-risk.” Fast Track couples universal prevention services (i.e., PATHS) with more intensive individualized intervention services for those at highest risk. The efficacy of PATHS has been tested in both end of 1st and 3rd grade (CPPRG 1999, 2010). To date, findings are modest to moderate in a positive direction, showing reductions in aggression and improvements in competencies as rated by teachers and students.

Social Information Processing, Aggression, and PATHS

The PATHS curriculum’s central focus on identifying and understanding emotions and effective social problem solving resonates closely with a social information processing (SIP) theory of aggression (Dodge et al. 2006; Huesmann 1988). SIP models of aggressive behavior were developed to describe at a proximal level how cognitive and emotional processes lead a child to engage in aggressive behavior. Here, attribution of intent and social problem solving abilities (generating alternative response options, evaluating responses, selecting the preferred option, and then carrying out the selected response) are fundamental and both are targets for change in the PATHS intervention.

Latent social knowledge structures are also an important component in children’s aggressive behavior. Whereas on-line information processing patterns are conceptualized as proximal antecedents of aggression, knowledge structures are believed to function as a cross-situational, distal storehouse of information that guides behavior by imposing limits on the processing activities. One type of latent knowledge structure is an individual’s beliefs about the legitimacy of aggression. Children who have beliefs that aggression is normative and appropriate are rated by peers, teachers, and self as higher in aggression relative to those who believe aggression is an inappropriate or unacceptable behavior in social situations (Huesmann and Guerra 1997).

The Present Study

PATHS is designed to be delivered by teachers with support from project staff, to be taught on a regular basis throughout the school year, and to provide daily activities for promoting the acquisition and generalization of new knowledge and skills. PATHS consists of six volumes of lessons, pictures, photographs, posters and other materials for feelings identification divided into three major units (Readiness and self-control, Feelings and relationships, and Interpersonal problem-solving). The PATHS units include five conceptual domains: (1) Self-control; (2) Emotional understanding; (3) Positive self-esteem; (4) Healthy relationships; and (5) Interpersonal problem-solving skills. PATHS is a comprehensive, yet flexible, program that allows implementation of the 131 lessons over the elementary school years. Throughout its implementation, homework assignments, parent letters, and other informational packets are sent home periodically, facilitating parental involvement and encouraging generalization to the home environment.

The present study examines aggressive outcomes in students within a cluster-randomized trial of the PATHS curriculum, one of seven school-wide programs evaluated within the Social and Character Development (SACD) Research Program, a 4-year multi-site randomized field trial of SACD programs. The evaluation tracked the development of two 3rd grade cohorts of students over 3 years in 96 schools across six states. Within each site, schools were randomly assigned to one of two groups: a treatment group in which a SACD program was implemented (PATHS in the current site), and a control group in which traditional educational practice continued. The current study is focused on aggressive outcomes in students and students’ social information processing abilities. Aggressive outcomes included teacher and self-ratings of aggression, teacher reported conduct disorder, teacher rated acting out behavior problems, self-reported rate of minor delinquent behavior, and victimization at school. Social information processing variables included normative beliefs about aggression, aggressive social problem solving, hostile attribution bias, and aggressive interpersonal negotiation strategies. Other student outcomes were collected as part of the study but not presented here (see Social and Character Development Research Consortium 2010 for further details on the overall study as well as more detail on other variables collected).

Methods

PATHS was one of seven programs selected by the Institute of Education Sciences and the Centers for Disease Control and Prevention to participate in a national evaluation of SACD. Each site implemented a different program and an overall evaluation of all seven programs was conducted. The seven participating sites shared sampling strategies, consent processes, a battery of core measures, and a random assignment procedure in which schools were systematically matched and assigned to treatment within pairs.

School Selection and Randomization

In the planning year of the study, 10 elementary schools were identified, pairwise matched, and randomly assigned to PATHS intervention or control condition (five schools to each). The initial pool of schools approached for participation in the randomized study was identified based on long-standing past relationships with the Primary Project (Cowen et al. 1996), an indicated prevention program serving students identified as at-risk for school adjustment problems. The goal was the identification of candidate schools that identified social and character development as a need, understood and agreed to the consequences of randomization at the school level, and if selected to participate in the study, would implement the PATHS curriculum with fidelity. Of the 10 schools, four were from a northeastern urban school district, four from a northeastern suburban school district, and two from a midwestern suburban school district. In year two of the study, an additional four schools were added to increase power, all four from the northeastern urban school district.

Prior to randomization, a pairwise matching procedure, stratified by cohort and district, was employed to ensure demographic similarity of intervention and control schools. Specifically, nine school level demographic variables (student enrollment, percent minority, percent free lunch, percent reduced lunch, percent Limited English Proficient, student stability, percent meeting or exceeding state level requirements in English/Language Arts, percent meeting or exceeding state level requirements in Math, student–teacher ratio) were used to compute a multivariate distance from each school to every other participating school in that district. Schools were paired based on this multivariate distance and within each matched pair, randomization was done with the flip of a coin. At each site, participating students were followed from the start of the 3rd grade through the spring of the 5th grade. Students who left the school during this period were not followed into their new school (unless transferring to another study school); students entering the schools during this period were approached to participate in the study and followed if consented positively (Brown et al. 2008; Vuchinich et al. 2012). No schools attrited from the study.

Participants

Students from 14 schools across three school districts participated, resulting in a final sample of 779 students. Cohort 1 (10 schools; n = 588; 76 %) students were from all three school districts; cohort 2 (four schools; n = 191; 24 %) students were recruited in the second year of the study from the participating urban district. Seven schools were experimental (n = 422, 54 %); seven were control (n = 357, 46 %). Figure 1 presents CONSORT information. Individual positive consent for participation in the research study (i.e., all students were exposed to the PATHS curriculum in the intervention schools regardless of consent status) was 58 %.

Fig. 1
figure 1

CONSORT participation recruitment and retention

Full size image

In terms of student demographics, 443 (57 %) of the students attended urban schools (n = 8) and 336 (43 %) attended suburban schools (n = 6). Three hundred thirty-seven (337) students were male (43 %), 442 were female (57 %); 389 of the students were White/Caucasian (51 %), 291 were African-American (38 %), and 79 were “Other” (10 %). One hundred and thirty students were Hispanic (17 %). Two hundred forty (33 %) of the students were from single parent homes.Footnote 1 While the modal response for family income was at the upper extreme of the scale (highest asked was $70,000 or more; n = 140 [21 %]), a number of families were poor. Two hundred sixty families (39 %) reported incomes of less than $20,000/year (12 % reported incomes of less than $5,000/year). Additionally, 140 families (20 %) reported incomes between $20,000 and $39,999 per year; and 100 families (15 %) reported incomes between $40,000 and $59,999 per year (see footnote 1). Three hundred (43 %) families met the federal government defined income to household size poverty ratio (using 2004 formula) (see footnote 1). Three hundred sixty of the head of households reported working full-time (51 %); 140 (20 %) part-time; and 210 were not working (29 %) (see footnote 1). For the highest level of education in the household, “some college” was the mode (n = 230; 38 %). However, 70 of the families (11 %) did not have a high school graduate or have a GED in the household; 120 families had a high school diploma (19 %); and 210 of the families had a college graduate in the household (33 %) (see footnote 1).

Teacher Training and Consultation

Intervention teachers in each grade attended a 2-day training workshop and received consultation and observation from project staff every other week. Over 90 % of teachers attended these trainings; for those unable to attend, small group or individual training was provided. Teachers received continuing education credit or were paid for their attendance. PATHS lessons are targeted to be taught approximately two or three times per week, with lessons lasting 20–30 min, from mid-September until early June. Project educational consultants (ECs) provided bi-weekly consultation from October until June that were intended to enhance the quality of implementation through modeling, coaching, and provision of ongoing feedback regarding program delivery. ECs also provided general feedback on classroom and behavior management. ECs were experienced school personnel hired by the project. They spent an average of 1–1.5 h every other week in each classroom, observing, demonstrating, or team teaching the PATHS lessons. They also met individually or in small groups with teachers on a regular basis.

Intervention Dosage and Quality of Implementation

To assess dosage, teachers reported weekly to their assigned ECs on the lessons they had presented. Over the course of the study, teachers reported teaching an average of 34.8 lessons per year (SD = 14.5, range = 7 – 62). Classroom fidelity was assessed through monthly ratings of quality of implementation made by the ECs on the basis of their direct observation of teacher instruction. Following the system adopted by CPPRG (1999, 2010), four 4-point Likert-scale ratings were assessed monthly. The four ratings were (a) quality of teaching the PATHS concepts, (b) modeling and generalization of PATHS concepts throughout the school day, (c) quality of student compliance during the PATHS lesson (each scored as 1 = has considerable difficulty, 2 = has some difficulty, 3 = does pretty well, and 4 = highly skilled), and (d) openness to consultation (scored as 1 = low, 2 = some, 3 = above average, and 4 = high). These measures were highly consistent across the year (teaching α = .87; modeling α = .90; compliance α = .90; consultation α = .88) and a yearly average score was computed for each teacher. At the school level, the curriculum was implemented well over the years. That is, six of the seven implementation schools averaged greater than three in the quality of teaching the PATHS concepts, modeling and generalization of PATHS concepts, and openness to consultation. Additionally, five of the seven schools averaged greater than three in the quality of student compliance during PATHS lessons. One school, however, averaged below three on all implementation indices and this data coincides with anecdotal reports from the principal and EC. Overall, six of the seven schools did well with implementation; one school did a marginal job of implementing the curriculum.

Measures

Teacher ratings and child self-report are the sources of data for the aggression outcomes. Table 1 presents a timeline for measurement completion by cohort over the longitudinal study.

Table 1 Measurement timeline by cohort
Full size table

Teacher Completed Measures

Teachers completed Teacher Report on Students (TRS) and Teacher-Child Rating Scales (TCRS) for all consented students. For cohort 1 students, TRS were completed in both the fall and spring of each of the first 2 years and in the spring of the third year (5th grade year). TCRS’s were completed in both the fall and spring of all 3 years for cohort 1 students. For cohort 2 students, TRS were completed in the fall and spring of year 1 and the spring of year 2 (4th grade year). TCRS’s were completed in the fall and spring of year 1 and 2 and in the spring of year 3. Teachers completed the questionnaires independently and completed surveys were returned to a central point in the school for collection by the research team. The TRS survey took approximately 15 min per student to complete; the TCRS took approximately 5 min per student to complete. The TRS assesses student responsibility, social competence, altruism, aggression, conduct problems, academic competence and motivation, and quantity and quality of parent involvement in education. The TCRS assesses student acting out behavior problems, shy-anxious behavior problems, learning problems, frustration tolerance, assertive social skills, task orientation, and peer social skills. Teacher completed aggression, conduct problems, and acting out behavior problems were used as dependent variables in the current study.

Teacher Rated Aggression

The Behavior Assessment Scale for Children-2 (BASC-2) Aggression Subscale, Teacher Version (Reynolds and Kamphaus 2004) was administered as part of the TRS and is a 14-item assessment to which teachers respond on a four-point scale about how often (1 = Never to 4 = Almost Always) a student engages in verbal or physical aggression (e.g., bullies others, calls other children names) in the preceding 30 days. Item responses are summed to produce a total score with a possible range from 14–56. Baseline internal consistency for the current study was .94 (range across all measurement waves = .93–.95).

Teacher Rated Conduct Problems

The BASC-2 Conduct Problems Subscale, Teacher Version (Reynolds and Kamphaus 2004) was administered as part of the TRS and is a 10-item assessment to which teachers respond on a four-point scale about how often (1 = Never to 4 = Almost Always) a student engages in socially deviant or disruptive behavior in school (e.g., skips class, cheats, steals) in the preceding 30 days. Item responses are summed to produce a total score with a possible range from 10–40. Baseline internal consistency was .72 (range across waves = .72–.79).

Teacher Rated Acting Out Behavior Problems

The Acting Out subscale of the TCRS (Hightower et al. 1986) is a 4-item subscale assessing the level of acting out/disruptive behavior problems exhibited by the student in the classroom (e.g., overly aggressive to peers [fights], disruptive in class). Each item is rated on a five-point scale ranging from 1 (Strongly Disagree) to 5 (Strongly Agree). Item responses are summed to produce a total acting out score with a possible range of 4–20. Baseline internal consistency for the current sample was .89 (range across measurement waves = .86–.90).

Child Completed Measures

The Child Report (CR) was a group administered self-report questionnaire completed by all consented students in the classroom setting. For cohort 1 students, the CR was administered in both the fall and spring of years 1 and 2 and in the spring of year 3 (5th grade year). For cohort 2 students, the CR was administered in the fall and spring of year 1 and spring of year 2 (4th grade year). The instrument was delivered to groups of 15–20 students per classroom. A proctor read the directions and each question aloud as students followed along in the survey booklets during a 50-min classroom session. The CR assesses normative beliefs about aggression, self-efficacy for peer interaction in conflict and non-conflict situations, empathy, altruism, aggression, minor delinquency, school engagement, school connectedness, feelings of school safety, and victimization at school.

Additionally, students completed the What Would I Do? self-report assessment of social problem solving skills and hostile attribution biases in the winter of each year of the study (both cohort 1 and cohort 2 students completed the instrument in each of the 3 years). This measure is a self-report adaptation of the Home Interview originally developed by Dodge (1986) and the Social Problem Solving Measure developed by Lochman and Dodge (1994, see also Aber et al. 2003). The instrument assesses four features of children’s social-information processing: competent social problem solving strategies, aggressive social problem solving strategies, children’s hostile attribution biases, and aggressive interpersonal negotiation strategies. A proctor read the directions and each question aloud while students followed along in the survey booklet.

Self-reported aggression, minor delinquency, victimization at school, normative beliefs about aggression, aggressive social problem solving, hostile attribution bias, and aggressive interpersonal negotiation strategies were used as dependent variables in the current study.

Self-reported Aggression

Adapted from The Aggression Scale (Orpinas and Frankowski 2001), self-reported aggression was administered as part of the CR with 6 items assessing how often (0 = Never to 3 = Many Times) the student engaged in verbal or physical aggression (e.g., teases others, shoves others) in the preceding 2 week period. Scores are computed by summing item responses (possible range 0–18). At baseline, internal consistency was .81 (range across measurement waves = .81–.91).Footnote 2

Self-reported Minor Delinquent Acts

The Frequency of Delinquent Behavior Survey (Dahlberg et al. 2005) was modified for the current evaluation and administered as part of the CR. Students responded to six items on a four point scale about how often (0 = Never to 3 = Many Times) he or she engaged in minor delinquent acts in school (e.g., taking something that belongs to others, skipping class) in the preceding 2 week period. Scores are computed by summing item responses (possible range 0–18). At baseline, internal consistency was .73 (range across waves = .68–.80).

Victimization at School

Adapted from The Victimization Scale (Orpinas 2009), self-reported victimization at school was assessed as part of the CR with 6 items to which students respond on a 4-point scale about how often (0 = Never to 3 = Many Times) he or she was victimized at school (e.g., teased; pushed, shoved, or hit; left out on purpose) in the preceding 2 weeks. Scores are computed by summing item responses (possible range 0–18). At baseline, internal consistency was .87 (range across measurement waves = .87–.91).

Normative Beliefs About Aggression

The eight-item General Beliefs subscale from the Normative Beliefs About Aggression scale (Huesmann and Guerra 1997) was used as part of the CR. Students respond on a four-point scale (1 = Really Wrong to 4 = Perfectly OK) indicating their beliefs about the acceptability of verbal and physical aggression, either proactive or in retaliation to others’ behaviors (e.g., It is usually OK to push or shove other people around if you’re mad; If you’re angry, it is okay to say mean things to other people). Scores are computed by summing item responses, with higher scores indicating more beliefs that aggression is a normative and accepted problem solving strategy (possible range 8–32). At baseline, internal consistency was .83 (range across measurement waves = .83–.92).

Aggressive Social Problem Solving

Eight hypothetical vignettes (and their accompanying illustrations), based on Lochman and Dodge’s (1994) Social Problem Solving Measure, were included in the What Would I Do? self-report survey and were used to assess children’s aggressive social problem solving. Each vignette describes a social problem requiring some initiative on the part of the child. The Aggressive Social Problem Solving score is created by recoding item responses as either 1 (aggressive; say “You’d better let me play”) or 0 (nonaggressive; just leave) and then averaging across the eight items (α = .81 at baseline; range across measurement waves = .77–.83). Higher scores indicate more aggressive social problem solving.

Hostile Attribution Bias and Aggressive Interpersonal Negotiation Strategies

An additional six vignettes, based on the Dodge (1986) Home Interview were included on the What Would I Do? survey to assess hostile attribution bias and aggressive interpersonal negotiation strategies. In these vignettes, children are asked to imagine themselves as the recipient of a provocation involving a peer, the cause of which was both visually and verbally ambiguous. Children are then asked about the cause of the provocation, and they select one of four possible causal attributions. The Hostile Attribution Bias score is created by recoding item responses as either 1 (hostile; the child got you muddy because they are mean) or 0 (benign; the child was fooling around and pushed too hard by accident) and then averaging across the six items (α = .76 at baseline; range across waves = .76–.79).

Following assessment of their attributions of intent, children are asked what they would do next in each of the six scenarios, and they select from among four possible response strategies. The Aggressive Interpersonal Negotiation Strategies score is created by recoding item responses as either 1 (aggressive; push the child in the mud) or 0 (nonaggressive; clean up your sneakers) and then averaging across items (α = .90 at baseline; range across measurement waves = .90–.93). Higher scorers are indicative of greater hostile attribution bias and tendencies to react aggressively, respectively.

Results

Table 2 presents basic descriptive information by condition for each variable at each measurement point. For main analyses, three-level growth models were utilized with the HLM (v. 6.08) statistical software program (Raudenbush et al. 2004). An alpha probability level of .10 was set as the threshold for determining statistically significant effects. This was chosen for a number of reasons. First, while power was adequate in the national evaluation, the power to detect significant effects at the site level was diminished (n = 14 schools) and increasing alpha inflates power.Footnote 3 Second, the difficulty in detecting interaction effects in field settings has been well documented (McClelland and Judd 1993) and, for the growth curve analyses presented, the slope by intervention interaction is the parameter of main interest. Finally, the increased risk associated with detecting beneficial and/or detrimental effects when none actually exist seems to outweigh the often mechanized nature of “significant” and “nonsignificant” results associated with an alpha level of .05 (Cohen 1994).

Table 2 Sample size, means, and standard deviations of study variables
Full size table

Unconditional growth models were initially examined to assess the developmental trajectories of these aggressive outcomes over the 3rd–5th grades. Results reveal significant linear and/or curvilinear change in many of the aggressive outcomes. Variance components analyses suggest that significant school-level variance exists for most outcomes, though limited school level variability exists for self-reported aggression and minor delinquent acts—limiting ability to detect significant intervention effects in these outcomes. Because curvilinear change was noted for a number of the outcomes, three-level non-linear (linear + quadratic) growth models were utilized with time at level 1, individual at level 2, and school-level variables at level 3 (treatment status and six dummy variables representing school matched-pair status). To aid in interpretation, school level predictor variables were grand mean centered. Time was coded so that the intercept corresponds to baseline status (time = 0) and subsequent time points were coded linearly as assessments were either collected annually or twice annually, with roughly equal spacing throughout. For outcomes with only three data points (self-reported aggressive social problem solving, hostile attribution bias, and aggressive interpersonal negotiation strategies), only linear effects could be examined. The models tested allow for random intercepts conditioned on covariates and random slopes. In all instances, particular interest lies in the slope by intervention interaction estimates, addressing the question “Are PATHS students’ behaviors changing at a different rate than are the control students?”

Attrition/New Enterer Analyses

Selective flow into and out of the study sample can bias results. In schools, students transfer out and new students enter into schools continually throughout the year. Two-level binomial models including the aggression outcome variables (teacher and self-report), conduct disorder, acting out behavior problems, and minor delinquency were used to predict attrition as well as new enterer status. At the school level, six match dummy variables and condition were used as level 2 predictors. For the 201 students who attrited out of the study, condition was not associated with attrition status (b = .141, t = .760, p = .476; 25.8 % of control students [92/357] attrited versus 25.8 % of experimental students [109/422]). At the school level, match indicator variables demonstrate a higher level of attrition in the urban school districts when compared with the rates of suburban schools. No individual aggression level predictor was significantly associated with attrition. Further, none of the aggression outcome variables significantly interacted with condition to predict attrition status. Similar results were noted for new enterers. For the 172 students who entered after the start of the study, condition was not associated with new enterer status (b = −.024, t = −.055, p = .958; 20.7 % of control students [74/357] were new enterers versus 23.2 % of experimental students [98/422]). No school level predictor or individual aggression predictor was significantly associated with new enterer status and none of the aggression by condition interaction terms were significant. These results indicate no differential attrition or new entering between students in the intervention and control groups.

Teacher-Rated Aggressive Outcomes

Table 3 presents degrees of freedom, unstandardized coefficients, 90 % confidence intervals, and significance results from the teacher-rated growth curve models tested. For presentation clarity, match indicator covariate effects are omitted from the table. Effect size (ES), based on the work of Feingold (2009), are also presented. Here, post baseline means at each time were derived from the model and used for effect size calculations (mean difference divided by pooled baseline standard deviation). Following Feingold (2009), intercept intervention effects are constrained to zero when computing post baseline ES.

Table 3 Degrees of freedom, unstandardized parameter estimates, 90 % confidence intervals, and significance levels for growth curves predicting teacher-rated aggressive outcomes. Intervention effect sizes are also presented
Full size table

In terms of teacher-rated outcome variables, PATHS was related to a reduction in conduct problems over time and a trend toward reducing aggression and acting out behavior problems over time. All effects were curvilinear, with PATHS associated with a deceleration in conduct problems (b = −.111, t = −3.022, p = .023) and a trend towards deceleration for aggression (b = −.137, t = −1.797, p = .122) and acting out behavior problems (b = −.058, t = −1.664, p = .147). Figure 2 graphically presents these findings. As noted in the figure and supported by examination of the ES, the effects for aggression and acting out behavior problems become more pronounced in the older grade late deceleration. For conduct problems, the ES suggest early iatrogenic effects through the 4th grade followed by rapid positive effects in the older grades.

Fig. 2
figure 2

Graphic representation of teacher-rated aggressive outcomes

Full size image

Child-Rated Aggressive Outcomes

Table 4 presents degrees of freedom, unstandardized coefficients, 90 % confidence intervals, and significance results from the self-report growth curve models tested. As noted, there were no significant linear or curvilinear intervention effects associated with self-reported aggression, delinquency, or victimization at school. There were, however, significant effects associated with each of the social information processing variables. That is, there was a linear trend towards significance for students’ normative beliefs about aggression (b = −.464, t = −1.779, p = .126). Additionally, there were significant linear effects for aggressive social problem solving (b = −.023, t = −2.606, p = .040), hostile attribution bias (b = −.041, t = −2.811, p = .031), and aggressive interpersonal negotiation strategies (b = −.038, t = −2.767, p = .033). Figure 3 graphically presents these findings.

Table 4 Degrees of freedom, unstandardized parameter estimates, 90 % confidence intervals, and significance levels for growth curves predicting self-reported aggressive outcomes. Intervention effect sizes are also presented
Full size table
Fig. 3
figure 3

Graphic representation of significant student social information processing outcomes

Full size image

Sensitivity Analyses

To examine robustness of the findings, three complementary approaches to sensitivity were taken. First, analyses were re-run including student level demographic variables of gender, race, ethnicity, family income, parent employment, highest level of household education, and single parent status as well as variables found to differ at baseline. Here, 10 multiply imputed datasets were used to account for missing baseline demographic and other individual level study data (Graham 2009). At the school level, significant imbalance in  % of students with limited English proficiency existed (PATHS mean = 10.8, SD = 7.3; Control mean = 4.2, SD = 3.0; t = 2.23, p = .046). At the student level, a two-level mixed model was used to examine for baseline differences with students nested within schools. While there were no experimental versus control group differences in demographics, there were statistical differences in parent rated intergenerational closure (b = −.661, t = −3.110, p = .021), teacher-rated parent-teacher partnership (b = −.709, t = −2.275, p = .063), and in self-reported self-efficacy for peer interaction in conflict situations (b = −1.207, t = −2.139, p = .076). There were also marginal differences (a conservative p value of .20 was used to assess baseline similarities) in teacher rated altruistic behavior (b = −.681, t = −1.912, p = .104), academic competence/motivation (b = −1.132, t = −1.504, p = .183), acting out behavior problems (b = .703, t = 1.665, p = .147, and learning problems (b = .987, t = 1.580, p = .165). Out of 69 measures collected, there were 7 significant/marginal baseline differences, less than might be expected by chance alone. In all instances, baseline differences favored the control group.

In terms of sensitivity, however, the same pattern of results was noted. Not surprisingly, intercept values (start points) were most changed. In terms of trajectories, the largest absolute parameter change was .023, involving the condition by time effect for child self-reported aggression (b = −.218 with individual level covariates versus b = −.195 without individual level covariates). In no instance, however, did substantive findings change. Effect size favored omitting covariates with the largest difference in effect size being .019 for 5th grade hostile attribution bias. A second approach to sensitivity used only students having at least half of the longitudinal data (n’s range from 547 to 580 depending on outcome) and a third approach dropped the PATHS school with marginal fidelity ratings and its matched control from analyses. Despite the loss in power, both of these latter two approaches confirmed the original findings (sensitivity results available from first author) and ES were larger, in absolute sense, with the marginal fidelity matched pair removed.

Discussion

The study results indicate that PATHS is having beneficial effects on student aggressive outcomes at the end of fifth grade. Further, effects were noted by both teacher and self-report and thus show cross-reporter validity. Teachers noted less aggressive behavior, less conduct problems, and less acting out problems. Although statistical effects in these areas were marginal (save conduct problems), end of study ES were moderate (−.20 and −.24 for aggression and acting out behavior problems, respectively, and −.15 for conduct problems). Such findings are in line with recent reports that universal interventions are likely to have mild to moderate effects across an entire population (Cuijpers 2003; Hahn et al. 2007). The smaller end of study effect size for teacher rated conduct problems is less surprising when one examines the shape of change (Fig. 2). Here, it is noted that relatively minor changes occur across the 3rd and 4th grade years with PATHS students decelerating and control students accelerating through 5th grade. This pattern of ES suggest that interventions need to take hold in schools before effects can reasonably be expected (Beets et al. 2009).

Statistically more robust findings were noted in the social information processing variables of aggressive social problem solving, hostile attribution bias, and aggressive interpersonal negotiation strategies. Each is thought to be an important proximal predictor of children’s aggressive behaviors and is a focus of change in the PATHS intervention (CPPRG 2002). That is, PATHS specifically targets social problem solving as an important proximal construct that relates to children’s long-term social and academic adjustment. PATHS lessons are targeted towards interpersonal cognitive problem solving (Greenberg et al. 2002), with a focus on recognizing and regulating emotions, expressing emotions appropriately, understanding contextual cues (facial, social), intentionality (accident, on-purpose), friendship, and teasing. Problem solving class meetings typically focus around appropriate interpersonal problem solving techniques using the PATHS model. That is, students will write down problems and the teacher will pick the most pressing to be addressed (oftentimes related to aggression). Students will then address the problem as a class. In many ways, these problem solving class meetings mirror social information processes at the individual level. That it, students will first identify the problem, generate alternative solutions, determine the best strategy for solving, implement that strategy, and reevaluate after sufficient time has passed.

While changes in the more proximal skills of social problem-solving and hostile attribution biases were significant, changes in students’ normative beliefs about aggression were marginal, though the end of study effect size approached the magnitude of the other significant findings in the study. Such script and database knowledge is likely more embedded within the individual by the 3rd grade and changes in other social information processing abilities may need to occur before change in normative beliefs is noted. This premise is certainly an area for future research.

Reports of aggression and other conduct problems revealed significant findings as noted by teachers only. Three related issues may explain these findings. First, it may be that these findings are contextually limited and that aggression was reduced at school, but not outside of school. Second, the limited variability at the school level of the multilevel models for self-reports of aggression and conduct disorder severely limits our ability to detect intervention effects in these outcomes. Third, an ongoing issue in the use of multi-informant constructs concerns whose view provides the most useful information (Achenbach et al. 1987; Renk 2005). While teachers have a relatively neutral and unique role in observing student behavior in context (ability to observe a large number of children in an important context, that is, school where interaction with peers of a similar age is ongoing), there is no gold standard. Similar to a number of other findings, correlations across raters for these constructs were modest, at best (end of study correlations across teacher and self-ratings ranged from .19 to .42).

While teachers noted less aggression in PATHS students over time, the PATHS students did not report lower victimization over time. As the PATHS intervention reduces aggression and enhances competencies and improves school climate (CPPRG 1999), we expected to note a similar reduction in victimization. Although such effects may take further time to notice, findings at the end of 5th grade indicate no change on this outcome.

Importantly, the current findings independently add to the efficacy of the PATHS curriculum. The need for independent replication has received increased attention in the prevention arena (Gandhi et al. 2007; Holder 2010) as there have been concerns about such issues as conflict of interest and a potential for bias in reporting results. Much of the PATHS efficacy trials involve the primary developers of the curriculum (CPPRG 1999, 2002, 2010; Greenberg et al. 2002)—the current cluster randomized trial independently adds to the efficacy of the PATHS intervention. Further, the ES found in the current trial are quite similar to those found in other trials (CPPRG 1999, 2010).

Limitations

Several limitations must be noted. First, starting the study with 3rd grade students meant that students only received a portion of the PATHS curriculum, although this is the first published study of PATHS that examines grade level effects beginning in 3rd grade. These children were not exposed to the full kindergarten through 2nd grade components of PATHS. Perhaps effects would be even stronger had these students been exposed to the complete curriculum. Moreover, the findings do not take into account differential implementation of SACD content in both intervention and control schools. Similar to SACD researchers in other states (Fraser et al. 2009), all public New York State schools are mandated to implement some form of SACD education. Thus, the current study represents a test of systematic training and implementation of the PATHS curriculum versus ad hoc adoption of SACD programs/lessons in control schools.

Additionally, an intent-to-treat (ITT) design was utilized, whereby new students to the study schools were invited to participate and included in the current data analyses. Similarly, students who left the study were also included (though additional data was not collected once they left the participating school). Because we wished to estimate program effects at the school level rather than the classroom level, and implementation occurs at the classroom level and will likely vary by year, such analyses are complicated and do not easily fit a school-focused multi-level model with an ITT design. As each year schools have children who stay, children who leave, and children who enter, ITT effects include a substantial number of children with limited treatment exposure (Brown et al. 2008; Vuchinich et al. 2012). Further, exposure to the curriculum is also conditioned on teacher implementation and while variability in implementation occurred, the conservative analyses here consider effects independent of implementation quality (though the sensitivity analysis omitting the school matched pair with suboptimal fidelity demonstrated stronger ES), student exposure, and SACD programming in control schools.

Another limitation concerns the fact that raters in the current study were not blind to condition. Teachers and students clearly were not blind to condition—the effect being that social desirability for program effectiveness may bias reporters to give more favorable assessments of student behavior. However, if this were occurring, one might expect results to be more robust, in both magnitude and breadth, and to be similar at each time period. The incorporation of more objective data (e.g., blinded observers rating child behavior in neutral settings, school record academic data) may help alleviate this concern.

Finally, it must be noted that the results presented differ from those presented in the national SACD evaluation report (Social and Character Development Research Consortium 2010). We suggest at least four reasons influencing these contrasts: (1) the current study uses more “site specific” measures which were chosen to align more closely with the content of the PATHS intervention, (2) the differing use of covariates in analyses, (3) the examination of curvilinear change in the current study, and (4) the different approach to using cohort data in analyses. It is hoped that the current results help contextualize findings across differing measurement strategies and analytic approaches.

Conclusion

While it is difficult to alter the early life experiences of children and the social-environmental conditions that shape these experiences, it does appear possible to broaden children’s social knowledge and change the way they process social information, lessening aggression and other maladaptive behaviors. With a diverse sample of schools and children, this independent replication indicates that the PATHS school-based prevention program can strengthen social-emotional skills and produce significant changes in a number of constructs related to students’ aggressive behaviors. Whether such change is long-lasting and translates to other spheres of the student’s life is an area for future research.