Introduction

What constitutes a ‘good’ high school? Traditional metrics include factors such as attendance, test scores, and graduation rates. More recently, there has been increased interest in the broader climate of schools. School climate includes a variety of factors such as physical and psychological safety, teaching practices, and student–teacher relationships. Research shows that school climate influences the academic and social well-being of students, teachers, and other members of the school community (see Wang & Degol, 2016 for a review). For instance, positive school climate is associated with better student academic outcomes such as engagement (Fatou & Kubiszewski, 2018), achievement (Bear et al., 2011; Thapa et al., 2013; Zullig et al., 2010), and fewer behavioral problems such as bullying (Bandyopadhyay et al., 2009; Wang & Dishion, 2012). Given its influence on school members’ daily experiences, there is growing interest among researchers and practitioners in investigating evidence-based practices for cultivating a positive school climate.

In the current study, we aimed to contribute new insights into how to improve school climate. In the following sections, we review literature on existing definitions of school climate and methods for fostering a positive school climate. We then present our conceptualization of school climate and discuss how RULER—an evidence-based approach to social and emotional learning (SEL) designed to cultivate emotional intelligence skills in leaders, teachers, students, and parents—could enhance school climate. Finally, we present a longitudinal study of 37 Mexican high schools that adopted RULER.

Conceptualizing school climate

In the broadest terms, school climate refers to what a school is like, including the school’s norms, values, goals, resources, teaching practices, interpersonal relationships, and organizational structures (Cohen et al., 2009; Wang & Degol, 2016). The National School Climate Council (2007) defines positive school climate in the following terms:

A sustainable, positive school climate fosters youth development and learning necessary for a productive, contributive, and satisfying life in a democratic society. This climate includes norms, values, and expectations that support people feeling socially, emotionally and physically safe. People are engaged and respected. Students, families and educators work together to develop, live, and contribute to a shared school vision. Educators model and nurture an attitude that emphasizes the benefits of, and satisfaction from, learning. Each person contributes to the operations of the school as well as the care of the physical environment. (p. 4)

While there is general agreement that school climate is multidimensional, the dimensions themselves are still under debate. One of the most prominent models was proposed by the U.S. Department of Education, which describes school climate as having three main areas: safety, engagement, and environment. Research indicates that these dimensions are relevant across samples from the U.S. and Mexico (Shukla et al., 2019). This model overlaps with earlier conceptions of school climate that encompass students’ perceptions of clear rules at the school that are enforced fairly, positive relationships between and among students and adults at the school, effective and engaging teaching practices, and school support for students’ healthy social and emotional development (Janosz, Georges and Parent 1998). Other researchers have identified factors related to these broader dimensions. For example, one facet of school safety is discipline or the actions that a school takes to regulate behavior (Osher et al., 2010). Similarly, a factor that is related to both safety and relationships is respect for diversity, which manifests through actions taken to ensure that people of all backgrounds, identities, and abilities feel like they belong (Bellmore et al., 2012). Other research shows that having a sense of connection to one’s school is associated with fewer behavioral problems among students (Loukas et al., 2006). Furthermore, students’ evaluations of their school’s environment often include the physical elements of the school (e.g., cleanliness, organization, and having adequate space and materials) and student voice, which is the extent to which a school values and responds to students’ concerns (Gillen et al., 2011). Together, these qualities represent the overall school climate.

Given the multidimensionality of school climate, there are numerous ways in which a school can have a negative climate—one that detracts from students’ academic success and psychological well-being. For example, a negative school climate could arise when students feel unsafe, when teaching practices lead to disengagement, when students and teachers do not relate well to each other, when students are treated unfairly, or when the building itself is in disrepair. Negative school climate is related to several undesirable outcomes, including lower student achievement, decreased graduation rates, increased rates of bullying and violence, lower self-esteem, and higher rates of depression and suicide (Collins & Parson, 2010; Jiang et al., 2010; Orpinas & Raczynski, 2016; Thapa et al., 2013). These problems are particularly severe for minority students. For instance, negative school climate has been related to greater absenteeism and decreased psychological well-being among LGBTQA + students (Birkett et al., 2009) and with anxiety, feelings of alienation, and lower academic engagement among youth with disabilities (Milsom, 2006). Thus, while a positive school climate enhances students’ psychological well-being and academic performance, a negative school climate impedes it.

School climate can be examined at two levels. On one hand, school climate is a group-level variable; it describes the conditions of a school, which is a shared context, and therefore is an entity within itself. For this reason, school climate assessment and intervention often treats the school as the unit of analysis (e.g., Cornell et al., 2016). An alternative approach is to evaluate school climate based on individuals who interact with the school (e.g., Gage et al., 2016). This method is advantageous because it enables researchers to account for specific aspects of school climate—such as student relationships and perceived safety—that depend on both the school and the individual. Accordingly, some researchers define the unit of analysis as the interaction among members of the school community and the school as a whole. These two perspectives are reflected in measures of school climate and the strategies developed to improve it.

Improving school climate

There are numerous avenues for improving school climate, with some interventions targeting specific factors. For example, bullying prevention programs aim to improve students’ sense of physical and emotional safety by implementing practices such as sharing information about how to respond to bullying at school (Chalamandaris & Piette, 2015). Others programs seek to reduce discrimination and promote respect for diversity through lectures, sharing stories, and encouraging positive interactions among diverse individuals (Abouda et al., 2012; Thornicroft et al., 2016). Another intervention was designed to improve the quality of relationships among middle-school teachers and administrators by encouraging them to identify their school’s needs and take steps to address those needs (Rhodes et al., 2009). These diverse approaches reflect the multidimensional nature of school climate.

In addition to interventions that target specific aspects of school climate, there also are more-holistic approaches. For example, researchers in India developed a whole-school health intervention designed to improve multiple aspects of school climate (Shinde et al., 2018). The intervention included six components: (1) 16 h of classroom teaching about the process of healthy development, including relationships, gender and sexuality, prevention of sexually-transmitted infections, and substance use; (2) four assemblies per month with skit presentations, role play, and discussions related to classroom teaching topics; (3) boxes for students to contribute anonymous concerns, complaints, or suggestions, which were then addressed through discussions; (4) a monthly magazine that published content related to classroom teaching topics; (5) monthly competitions such as debates and essay writing related to the themes of the intervention; and (6) dissemination of zero-tolerance policies regarding bullying and substance use. A randomized control trial with 74 secondary schools in India indicated that schools who implementing this intervention exhibited increases in supportive student–teacher relationships, student belonging, student participation in school activities and decision-making, and students’ and teachers’ commitments to academic values.

Another promising direction for improving school climate is to address social and emotional learning (SEL). In recent decades, there has been a growing impetus towards SEL, which focuses on cultivating social and emotional skills that enable students and the adults who are teaching and leading schools to thrive—both personally and professionally—in contexts beyond school settings (e.g., Bradshaw et al., 2009; Durlak et al., 2011). SEL is an umbrella term that includes a range of competencies rooted in the skills of emotional intelligence (e.g., emotion regulation), empathy, social responsibility, and goal setting. The Collaborative for Academic, Social, and Emotional Learning (CASEL) identifies five main SEL competencies: (1) self-awareness, which is an understanding of one’s strengths and weaknesses, confidence in oneself, and a growth mindset regarding one’s ability to improve; (2) social awareness, which is the ability to understand other people’s perspectives and empathize with others, including those from backgrounds different from one’s own; (3) relationship skills, which include understanding how to listen to others effectively, communicate one’s needs, cooperate, negotiate conflict, and exchange support; (4) responsible decision-making, which is the ability to choose behaviors that are constructive in terms of safety, ethical standards, and social norms; and (5) self-management, which refers to the self-discipline and organizational skills needed to effectively manage one’s stress, resist unhealthy impulses, and motivate oneself to pursue goals (CASEL 2017).

A recent meta-analysis indicates that SEL interventions effectively foster students’ social and emotional skills and also lead to improvements in academic success and student behavior (Taylor et al., 2017). Given that many of these outcomes are encompassed by or related to school climate, it is possible that SEL interventions also could be used to enhance school climate. For example, because SEL directly targets social skills, it could be expected to improve relationships within a school. Indeed, SEL interventions have been shown to enhance student–teacher relationships (Taylor et al., 2017) and classroom climate (Rivers et al., 2013). Another aim of SEL is to improve students’ confidence, ability to communicate, and perseverance in the pursuit of goals that matter to them. As a result, it is plausible that SEL programs could enhance students’ beliefs that they have a voice in the school. Additionally, prior studies indicate that SEL interventions improve teaching quality, perhaps because social and emotional skills influence student–teacher relationships (Hagelskamp et al., 2013). One established evidence-based approach to improving SEL in schools is RULER.

Improving school climate with RULER

RULER is a systemic, whole-school approach to enhancing the social and emotional skills of students, teachers and staff, and school leaders from preschool through high school. RULER is grounded in multiple theories. First, its focus on improving emotion skills is based on the theory of emotional intelligence, which holds that a critical aspect of functioning is the extent to which individuals are able to understand their emotions, understand other people’s emotions, and use emotions as information to guide behavior (Salovey & Mayer, 1990). RULER adopts the assumption that emotion skills are malleable and can be learned and improved (Brackett & Katulak, 2006). The target of RULER—which is the interactions among students and educators in the school context—is guided by ecological systems theory, which views developmental change as driven by interactions among individuals and their contexts (Bronfenbrenner and Morris 2006). Theories also guide the steps of RULER. For example, the first step of RULER involves training a small group of educators based on the diffusion of innovation theory, which states that a small group of early adopters can help to spread new ideas across a population (Rogers, 2003). The next step of RULER is to spend a year training all educators at a school. This approach is based on social-cognitive learning theory, which holds that children learn through observation (Bandura 1977). Indeed, a critical tenet of RULER is that the values, attitudes, knowledge, and skills of educators are essential for improving the emotional intelligence of students (see Brackett et al., 2019 for a full theory of change).

The acronym RULER represents the five skills that comprise emotional intelligence: recognizing emotions in oneself and others, understanding the causes and consequences of emotions, labeling emotions with a nuanced vocabulary, expressing emotions in accordance with cultural norms and social contexts, and regulating emotions with helpful strategies. These skills are taught through the integration of key tools into staff professional development, school practices, policies, classrooms, and lessons. The tools are (1) the Charter, an agreement about how adults and children want to feel in school and behaviors to which everyone commits in order to help everyone experience these desired feelings more often, (2) the Mood Meter, a tool for building self- and social-awareness and emotion regulation, including a nuanced ‘feelings’ vocabulary, (3) the Meta-Moment, a four-step process for handling unwanted emotions to encourage people to observe changes in their thinking and physiology, pause and imagine their best self, and then strategize and respond and (4) the Blueprint, an approach to resolving conflict through empathy and perspective-taking by following a set of guiding questions (for more details, see Brackett et al., 2019).

RULER is a multi-phase intervention. First, a group of teachers and school leaders—referred to as the ‘implementation team’—attend a two-day RULER Institute. The goal of the institute is to support a core team in (1) developing their own emotional intelligence and (2) creating an implementation plan to share the RULER Tools, practices, curriculum, and strategies with their colleagues. The first year of implementation focuses on adults at the school becoming familiar with RULER through personal and professional use of the approach, guided by the implementation team. Schools are supported by the RULER Online learning platform and remote meetings with RULER coaches. In the following years, teachers embed RULER tools and curriculum into their instructional practices, which involves presenting RULER lessons that explicitly teach students how to recognize, understand, label, express, and regulate emotions, as well as integrating RULER Tools and principles into routines, school practices, and policies. At the high-school level, RULER offers 20 45-min lessons at each grade level. Lessons are based on themes that are salient during adolescence, including identity development, building healthy habits (e.g., eating, sleeping, exercising), stress management, handling peer pressure, and setting and achieving goals. The lessons are aligned with the five CASEL competencies (CASEL 2017), as well as the Illinois SEL standards (Illinois State Board of Education, 2010). In a clustered, randomized control trial of 62 fifth and sixth-grade students, RULER led to improvements in social and emotional skills and academic outcomes among students (Brackett et al., 2012), classroom climate (Rivers et al., 2013), emotion perception, understanding, regulation, and work satisfaction among Spanish teachers (Castillo-Gualda et al., 2017).

There are compelling reasons to believe that RULER leads to improvements in school climate. First, when school leaders make the decision to devote resources to RULER, it conveys a commitment to fostering the social and emotional well-being of students, teachers, and other adults at the school. Thus, simply using the approach could increase perceptions that one’s school is a place where people value social and emotional well-being. Second, given that emotions influence learning (Taylor et al., 2017), using strategies to improve emotional experiences during class could enhance teaching practices and student achievement. Third, RULER largely focuses on expressing emotions and understanding others’ emotions in the service of relationships. For example, the Charter tool explicitly focuses on building a positive emotional climate. Indeed, RULER has been shown to increase emotional support within the classroom (Hagelskamp et al., 2013) and social connectedness among students (Rivers et al., 2013). In line with these findings, RULER is expected to strengthen social connections throughout the school community, including relationships among students and teachers. Cultivating interpersonal relationships could also lower rates of threatening behaviors (Chalamandaris & Piette, 2015), thereby improving students’ sense of physical and emotional safety. Along similar lines, RULER emphasizes the skill of empathizing with other people, including those who might have different perspectives. As a result, participating in RULER could build students’ respect for diversity and their skills and competencies to advocate for themselves. For example, RULER is associated with increased student autonomy and leadership (Rivers et al., 2013). Therefore, implementing RULER could also enhance student voice. In summary, there are strong theoretical and programmatic reasons to believe that implementing RULER could lead to improvements in school climate in high schools. This hypothesis was the focus of our study.

Current study

The purpose of this study was to investigate the effects of RULER for improving school climate in Mexican high schools. To gain different perspectives on these changes, we surveyed both students and educators about their perceptions of school climate. We hypothesized that school climate would improve significantly when RULER was adopted. As a second measure of school-wide changes in individuals’ experiences, we were also interested in students’ satisfaction with their school. Given that school climate is strongly related to school satisfaction (Loukas et al., 2006; Zullig et al., 2010), we hypothesized that students’ school satisfaction would also increase when RULER is adopted.

A second aim of this study was to build upon the literature regarding factors that are associated with changes in school climate. Previous research on RULER indicates that fidelity of implementation, including teacher training, the number of lessons taught, and the quality of implementation, are key factors in predicting changes in student outcomes (Reyes et al., 2012). That is, RULER was most efficacious when teachers were rated by evaluators on the research team as moderate- or high-quality implementers (Reyes et al., 2012). Therefore, we hypothesized that higher implementation fidelity—operationalized as teacher-rated school support for RULER, teachers’ intentions to implement RULER, students’ attitudes towards RULER, and student ratings on the quality of RULER lessons (e.g., whether they seemed interesting/boring, natural/forced)—would be associated with increases in school climate.

Method

This study was a one-group pretest–posttest longitudinal experiment.

Procedure

Teachers serving on implementation teams from 37 secondary schools across Mexico were trained in RULER during June, 2016 at one of two RULER Institutes held in Monterrey and Guadalajara. Immediately following the training, implementation teams completed a survey about their intentions to use RULER and their beliefs that it is valued by their school. Next, in accordance with the RULER rollout recommendations, schools used the 2016–2017 academic year for personal and professional staff development. Student implementation then occurred during the 2017–2018 academic year. All students and teachers were invited to complete surveys at three other times: before implementation teams were trained (T1; Spring 2016); one year later when all teachers were had received training from their school’s implementation teams, but before RULER was brought to students (T2; Spring 2017); and another year later, after RULER was implemented with students (T3; Spring 2018). Surveys included the measures described below and other questions about emotions, mindsets, and creativity that formed part of a larger investigation. All surveys were translated into Spanish by a member of the research team. They were administered anonymously through Qualtrics.com.

Participants

Schools

All high schools included in the study were part of Prepa Tec, a branch of high schools affiliated with the Instituto de Tecnológico de Monterrey, which is a university that offers high school and undergraduate education. All schools (Prepas) are private and are equivalent to U.S. grades 10–12. There are approximately 27,000 students and 3,500 educators across all 37 sites. Schools vary in size, with an average of 767 students (SD = 584) and 99 educators (SD = 61) per school.

Students and educators

Students and educators at each location were invited to participate. Demographic information is displayed in Table 1 (students) and Table 2 (educators).

Table 1 Student demographics
Full size table
Table 2 Teacher demographics (n = 111)
Full size table

Measures

School climate

The Emotional Intelligence School Climate Inventory (EISCI) was created for this study to measure students’ and teachers’ perceptions of their school’s climate. The measure originally included 61 items representing 10 dimensions of school climate: teaching quality, negative climate, student relationships, adult relationships, student–adult relationships, feel of the school, discipline practices, support for SEL, student voice, and respect for diversity. Participants were asked to indicate their agreement with each statement using a scale ranging from 1 (Strongly Disagree) to 100 (Strongly Agree). The wording of some items varied slightly to reflect whether the respondent was a student or teacher (e.g., “The students of my school are nice to me” for students and “The students of my school are nice to each other” for teachers). Other items were identical for students and teachers (e.g., “My school supports diversity”). Because this measure was developed for this study, we conducted analyses reported below to evaluate its psychometric properties.

School satisfaction

School satisfaction was measured with the Schools subscale of the Multidimensional Students’ Life Satisfaction Scale (Huebner, 1991). This includes eight items such as “I feel comfortable at school” and “School is interesting”. Each item was rated from 1 (Strongly Disagree) to 5 (Strongly Agree). The full measure has demonstrated good internal consistency (α = 0.70–0.90), test–retest reliability over 2–4 weeks (r = 0.70–0.90), and convergent validity with self-, parent-, and teacher-rated well-being and social desirability (Huebner 1998).

Positive and negative affect

A modified version of the Positive and Negative Affect Schedule (PANAS; Watson et al., 1988) was used to assess positive and negative affect. Participants were instructed to report the extent to which they were currently experiencing each of 28 affective experiences from 1 (Very Little) to 100 (Very Much). This included emotions from the original version of the PANAS (e.g., inspired, nervous) and feelings that a recent study identified as common for adolescents to experience during school (e.g., stressed, connected; Moeller et al., 2020).

Implementation fidelity

Implementation team reports

After being trained in RULER in the Summer of 2016, implementation teams completed a survey that assessed the extent to which they believe that their school values each of the RULER Tools (Charter, Mood Meter, Meta-Moment, and Blueprint). Items included “I think the leader of my school sees the importance of using the Mood Meter” and “I have enough resources to start implementing the Meta-Moment at my school”. The implementation team was also asked to report their intentions to use each RULER Tool, with items such as “I agree to use the Charter at my school” and “I have different ideas about how to use the Meta-Moment throughout the year”. Items were rated on a 4-point Likert scale with options ranging from 1 (Strongly Disagree) to 4 (Strongly Agree).

Student reports

In the Time 3 survey (Spring 2018, after RULER was implemented), students were asked to report their perspectives on the utility of RULER. First, they reported their agreement with three items: “How useful are the RULER tools for… (a) your personal well-being, (b) your relations with classmates, and (c) acquiring more knowledge”. These items were rated on a four-point scale from 1 (Not at all Useful) to 4 (Very Useful). Students were also asked to report whether RULER lessons felt natural, forced, boring, distracting, childish, interesting, or useful. For these items, participants were instructed to check off each descriptor that applied.

Data analyses

Composite scores were created based on means with pairwise deletion. We used the Cronbach alpha coefficient to evaluate the internal consistency of each measure. Given that perceptions of school climate reflect both the individual (student, educator) and their school (Cornell et al., 2016; Gage et al., 2016), we considered using multilevel modeling to evaluate changes in school climate. However, because there were only 11 schools that had student responses across all three time points, using multilevel modeling with such a small number of clusters could lead to biased estimates of variance components (McNeish & Stapleton, 2016). Therefore, we used an aggregated approach to assess student-rated changes in school climate at the individual level and school level. We also analyzed teacher-rated changes in school climate at the school level. To conduct the school-level analyses, we aggregated scores across individuals within each school, which aligns with approaches for studying school climate used in previous studies (e.g., Cornell et al., 2016). Repeated-measures ANOVA were used to evaluate school-level changes in school climate and school satisfaction across times 1, 2, and 3. The Greenhouse–Geisser correction was used when the assumption of sphericity was violated (Maulchy’s test p < 0.05). We used paired t tests to evaluate changes in individual-level, student-rated changes in school climate between times 2 and 3. Because school climate includes 10 dimensions, a Bonferroni correction was applied to the t test to help to minimize the risk of Type I error. Pearson bivariate two-tailed correlations were used to evaluate the relationship between measures of implementation (e.g., teacher confidence) and changes in school climate.

Results

Preliminary analyses

There were three waves of surveys administered to students and teachers (T1–T3). The T1 sample included 11,291 students and 1,262 teachers; T2 included 5,771 students and 462 teachers; and T3 included 18,941 students and 1,588 teachers. Because all students and teachers completed T1 surveys anonymously, analyses including T1 were conducted at the school level, treating each of the 37 schools as a unique case, with mean scores across all teachers or students entered for each variable. In addition, because students and teachers created unique codes at T2 and T3, we matched those cases across time points. Cases that could not be matched were excluded from analyses. The resulting database included 1,662 students and 61 teachers who responded at both T2 and T3 that could be analyzed at the individual level. This number was considerably smaller than the original T1 samples because of decreased participation during T2, as well as the loss of each graduating class of students each year. Given the small teacher sample, we did not analyze changes in teacher ratings at the individual level.

The assumptions relevant for the current analyses—normal distributions of scores and homogeneity of variance—were both satisfied. See Table 3 for information on the distributions of baseline scores.

Table 3 Distributions of baseline scores across main variables
Full size table

Psychometric characteristics of the EISCI

A principal components factor analysis was used to evaluate whether each item accounted for adequate variance within each dimension of school climate, as rated by students and teachers. For each factor, items accounted for 42–65% of the variance. In considering factor loadings, we used a relatively conservative cutoff of 0.55 (Comrey & Lee, 1992; Matsunaga, 2010). When two items were found to have factor loadings below 0.55 in the teacher dataset, we removed them from both the student and teacher versions to keep the versions parallel. The resulting measure included 59 items, with 5 to 7 items measuring each of the 10 dimensions. All subscales of school climate had adequate internal consistency and were related to school satisfaction, positive emotions, and negative emotions in theoretically-consistent ways (see Table 4).

Table 4 Psychometric properties of the Emotional Intelligence School Climate Inventory (student version)
Full size table

Implementation fidelity

At T3, students were asked to report which RULER Tools were used in their school during that school year. Of the 974 students who responded, 51% reported that their school used one tool, 14% reported two, 13% reported three tools, and 4% reported all four tools. The remaining 18% of students reported that none of the tools were used. The most-common tool used was the Mood Meter (60%), followed by the Charter (31%) and Meta-moment (31%); the Blueprint was the least common (14%). Implementation teams’ attitudes about using RULER are reported in Table 5.

Table 5 Teacher initial attitudes towards using RULER (Summer, 2016)
Full size table

Main analyses

Changes in school climate

Results of paired t tests across all students in all schools indicated that there were significant improvements in 8 out of 10 domains of school climate from T2 to T3 (Cohen’s d = 13.–0.25). The dimensions with the largest effect sizes were support for SEL, t(994) = 5.18, p < 0.001, d = 0.20, and student voice, t(994) = 6.55, p < 0.001, d = 0.25. The two dimensions with nonsignificant changes were the look and feel of the school, t(995) = 2.21, p = 0.11, d = 0.09, and negative climate, t(994) = 1.58, p = 0.03 (0.30 with the Bonferroni correction), d = 0.06. See Table 6 for information about changes in each dimension.

Table 6 Student-rated changes in school climate–Student level
Full size table

Based on student reports at the school level, there were significant T1–T3 and T2–T3 increases for teaching quality, student relationships, student–adult relationships, and respect for diversity. There were also significant T1–T3 increases in the feel of school, discipline, and student voice. There were significant increases across all time points (T1–T2, T2–T3, and T1–T3) for adult relationships and support for SEL (all p values < 0.05). There were no significant changes in negative climate across time points (see Table 7).

Table 7 Student-rated school climate changes–School level
Full size table

Based on teacher reports at the school level, there were significant T1–T3 increases in teaching quality, student voice, and respect for diversity. There were also significant T1–T2 and T1–T3 increases in student relationships, discipline, and support for SEL. Meanwhile, negative climate decreased significantly between T1 and T2, but then increased significantly between T2 and T3 (all p values < 0.05; see Table 8).

Table 8 Teacher-rated changes in school climate – School level
Full size table

Changes in school satisfaction

In terms of means across schools, there was a significant increase in students’ school satisfaction from T1 (M = 4.77, SD = 0.19) to T2 (M = 4.88, SD = 32), t(29) = 8.98, p < 0.05, and this change had a medium effect size, d = 0.42. There were no significant changes in school satisfaction from T2 to T3 (M = 4.86, SD = 0.95), t(29) = 0.12, p = 0.17, d = 0.03.

Role of implementation fidelity

Implementation team members’ intentions to use the Charter at T1 were positively correlated with changes in student-rated climate, including negative climate (r = 0.39, p = 0.038), student relationships (r = 0.38, p = 0.046), discipline (r = 0.68, p = 0.011), and respect for diversity (r = 0.46, p = 0.013). Team members’ ratings that their school values the Meta-Moment at T1 were positively correlated with T2–T3 changes in school climate feel of school (r = 0.45, p = 0.027) and negatively correlated with changes in student voice (r = −0.44, p = 0.031). Student ratings of RULER as useful at T3 were positively correlated with T2–T3 changes in negative climate (r = 0.44, p = 0.007). There were no other significant relationships between measures of implementation fidelity and school climate.

Discussion

The purpose of this study was to identify the effects of RULER on school climate within high schools in Mexico. Findings from 37 schools indicate that, on average, students and teachers reported improvements in 8 of the 10 dimensions of school climate when schools adopted RULER. There were especially large increases in perceived support for SEL, which makes sense given that RULER is an SEL-based approach. These findings largely support the first hypothesis and align with previous findings that implementing RULER in elementary and middle schools led to improvements in classroom emotional support, student connectedness, and student autonomy (Hagelskamp et al., 2013; Rivers et al., 2013). This is the first study to demonstrate the effects of RULER on school climate in a high-school setting.

Although the effects of RULER on school climate were mostly in the expected directions, there were two exceptions: perceptions of negative climate and the feel of school did not significantly improve. Given that the items used to assess negative climate (e.g., “The teachers at my TEC Prepa fight or argue a lot”) were similar to items used to assess other dimensions (e.g., “The adults of my TEC Prepa get along with others”—assessing Adult Relationships), and that there were significant improvements in other dimensions, it is surprising that negative climate did not also improve across time. This could reflect that it is easier to promote positive elements of school climate, such as teachers taking an interest in students, than to reduce or eliminate elements of negative school climate, such as teachers acting unkindly towards students. Similarly, in considering why the feel of school did not improve significantly, it is useful to note that this dimension was assessed using questions about the school’s outward appearance (e.g., “My school is clean”; “I like the way my school looks inside”) and its reputation in the community (e.g., “I am proud of my school”). These aspects of the school might be less likely to shift than other aspects because they depend, at least in part, on structural factors such as a school’s wealth. Thus, some aspects of school climate might not improve significantly based on RULER, at least in the short-term.

Another key finding from this study was that changes in student-rated and teacher-rated school climate were significant across T1 to T2 as well as T2 to T3. These findings from T1 to T2 are particularly remarkable in that they occurred during the adult professional development year before students had received any direct social emotional learning in their classrooms. This finding reinforces the idea that training adults is critical for facilitating change among students. Yet, the exact mechanisms of change are unknown. One possibility is that receiving RULER training enabled adults to more effectively respond to students’ emotional experiences. For example, educators could have been more capable of recognizing when students were feeling overwhelmed and modified their instruction to give students the opportunity to recover, which then improved students’ emotional experiences. Another plausible explanation is that the training improved adults’ ability to understand and regulate their own emotions, which then led them to behave in ways that were more conducive to their goals of teaching effectively and cultivating positive relationships with students. For example, learning how to regulate anger might have led teachers to respond to students with greater understanding. The literature provides evidence to support both of these explanations; studies show that both teachers’ social and emotional skills and their emotional experiences tend to influence students’ academic outcomes and psychological well-being (see Sutton and Wheatley 2003 for a review). Given that the adult training alone led to significant changes in perception of school climate, learning more about the mechanisms of these changes is critical for extending this research.

A second goal of this study was to examine whether intervention implementation was related to the effects of RULER on school climate. Our analyses revealed several aspects of implementation that were associated with the effects, including teachers’ intentions to use the Charter, the implementation team’s ratings that the school values the Meta-Moment, and student ratings of RULER as useful. The finding that the Charter most strongly related to improvements in school climate is unsurprising given that this tool most directly targets school climate. That is, it asks members of a group to discuss how they want to feel within a given context, identify practices that they could employ to help everyone feel that way, and commit to implementing those practices. Thus, the Charter gives students a platform for identifying and voicing their concerns about their school. By facilitating the development of an action plan, the Charter also provides concrete directions for students, teachers, and other members of the school to help to improve their school. In conclusion, of all aspects of RULER, the Charter could be the most essential for improving school climate.

The current findings should be interpreted in light of the study’s limitations. The most-critical limitation is that our study did not include a control group. As a result, alternative explanations of the findings, such as maturation and historical effects, cannot be ruled out. Another limitation is that, because of the lack of identifying codes at T1, surveys could not be matched across all three time points. It is possible that participants who remained in the sample across time points differed systematically from others; for instance, they might have been more stable than participants who were not at the same school across time. A related issue was that, because of the longitudinal nature of the study and the fact that approximately a third of secondary-school students change each year (as older students graduate and new students arrive), the composition of the student sample changed across each time point. Consequently, it is possible that changes in school climate reflect differences in individual students rather than school climate. Nonetheless, given that there were similar patterns in school climate changes across all participating schools, it is reasonable to infer that these changes were in response to the intervention rather than biases in the sample.

Findings from this study point towards several directions for future research. One important focus is the effects of RULER in other contexts. The current study was conducted with high schools in Mexico, which could differ from those in other countries. For example, there are documented differences in social norms related to emotional expression (Hareli et al., 2015) and the nature of student–teacher relationships across countries (Chiu et al., 2016). These norms could lead Mexican students to be more or less receptive to RULER than students in other cultures. In addition to testing how RULER influences school climate in other settings, it would also be useful to consider additional contextual factors, such as the school’s region and community setting, that could moderate these effects.

Our research extends previous work on the impact of RULER on school climate high to schools. Findings suggest that, by using RULER, high schools can improve students’ and teachers’ perceptions of school relationships, teaching quality, emotional and physical safety, student voice, and sense of community. Moreover, such effects begin to occur during adult professional development, before RULER is presented directly to students. Such findings speak to the power of systemic approaches to SEL, such as RULER, that target not only student skills but also educators’ skills and personal well-being, and go beyond classroom lessons by providing broader tools for transforming school pedagogy, policies, procedures, and practices.