1 Introduction

One of the most serious educational problems continues to be the underachievement of Hispanic students in urban schools because they have significantly lower achievement and higher dropout rates than white students (Gándara 2017). Instead of focusing on the achievement gap between Hispanic and white students, however, it may be more meaningful if we examine determinants of Hispanics’ success. From this perspective, we need to look at the instructional and environmental characteristics of schools that are effective for Hispanic students because it may enable us to identify “alterable” variables that may distinguish effective from ineffective schools (Waxman et al. 2007). It has been found, for example, that many Hispanic students attend schools that are in “crisis” or “ineffective” because of the school violence, vandalism, inadequate equipment and facilities, and inadequately prepared teachers (Gándara 2017). There have been a few studies that have examined effective schools (i.e., schools that promote high academic achievement) for Hispanic students (García 1988; Brooks and Kavanaugh 1999; Reyes et al. 1999; Jesse et al. 2004), but these studies have not specifically investigated aspects of classroom instruction and learning environment (e.g., school climate, teacher support, student engagement) that may help us distinguish effective and ineffective urban schools. With the numbers of Hispanic students in the USA growing at unprecedented rates, there is a great need on how to best serve this important population of students (Gándara 2015).

If we are going to reform schools through effective schools research, we must address the issue of improving the quality of classroom instruction for Hispanic students. There is a growing belief that the best way to improve urban schools is to provide them with better teachers and classroom instruction (Waxman et al. 2007). Unfortunately, the quality of instruction provided to Hispanic and other minority students in urban schools has been described as a “pedagogy of poverty,” where teachers typically teach to the whole class at the same time and control all of the classroom discussion and decision making (Waxman et al. 1995; Baker 1999; Haberman 2010). Haberman (2010) argues that the over-reliance on this teacher-directed instructional style leads to student compliance and passive resentment as well as teacher burnout. Furthermore, he criticizes this instructional orientation because teachers are generally held accountable for “making” students learn, while students usually assume a passive role with low engagement in tasks or activities that are generally not authentic (i.e., not connected to students’ out-of-school experiences or real-life situations).

In addition to the poor quality of classroom instruction found in many urban schools, students may also be placed at risk by the quality of their school environment (Thapa et al. 2013). In fact, the school climate or environment has been argued to be one of the central problems of urban schools (Slaughter-Defoe and Carlson 1996; Baker 1999). The school environment is the broader context or climate of the school that either facilitates or constrains classroom instruction and student learning. Several educators, for example, have found that there are many features of schools and classrooms that are alienating to students and consequently drive students out of school rather than keep them in (Rodriguez 2008). The increasing reliance on standardized, high-stakes testing has been documented as a key cause of increasing numbers of Hispanic students who drop out of school (Valenzuela 1999). On the other hand, fostering or maintaining an effective classroom learning environment has been suggested as a means of enabling Hispanic students and other students at risk of failure to be successful in school (Padrón et al. 2014). Research has shown that building stronger and more meaningful teacher–student relationships can encourage positive feelings about school and make Hispanic students more likely to attend (Stanton-Salazar 2001). In summary, there is evidence that both teaching practices and the school and classroom environment of urban schools can either be beneficial or detrimental to the academic performance of Hispanic students. The present study addresses the problems of Hispanic students in urban schools by drawing upon two distinct and emerging research paradigms: (a) classroom learning environments and (b) effective schools. The following two sections briefly describe the existing knowledge base from these two perspectives.

1.1 Classroom learning environments

The socio-psychological environment or classroom learning environment as it is more recently referred to has been extensively researched in the past several decades. From a theoretical perspective, classroom learning environment research emphasizes the student-mediating or student cognition paradigm which maintains that how students perceive and react to their learning tasks and classroom instruction may be more important in terms of influencing student outcomes than the observed quality of teaching behaviors (Fraser 1986, 1991; Koth et al. 2008; Wittrock 1986). This paradigm assumes that: (a) the classroom environment experienced by the student may be quite different from the observed or intended instruction and (b) teaching and learning can be improved by examining the ways that classroom instruction and the learning environment are viewed or interpreted by the students themselves since students ultimately respond to what they perceive is important (Nelson and Christ 2016; Rodriguez 2008). Students are considered to be the experts of their own views and experiences of school and their perceptions of the learning environment are also essential for understanding the opportunities for learning that are provided to each student in class (Fraser 1991; Rodriguez 2008).

Students’ perceptions of their instructional and classroom learning environments have been found to explain a significant amount of variance for both students’ cognitive and affective outcomes (Fraser 1989, 1991; Koth et al. 2008; Rodriguez 2008). Generally, the results of these studies and reviews of research have found that the variables such as cohesiveness, task orientation, rule clarity, student satisfaction, and teacher support are positively related to students’ gain in academic achievement. Other reviews have found that classroom environment measures can be effectively used as criterion or outcome variables in a wide range of research (Fraser 1986, 1991). Research has also found that students’ perceptions in primary grades are valid (Aldridge and Galos 2018) and that they moderately agree with observers ratings (Scherzinger and Wettsteien 2019; den Brok et al. 2006). Another strength of this research is that there are many international studies (e.g., Australia, Singapore, Turkey, Canada, Israel, Taiwan) that have used similar learning environment measures (Fraser 2012; Lim and Fraser 2018). These studies have developed and validated instruments and have often focused on the associations between classroom environment and student outcomes (Fraser 2014).

Although the classroom learning environment has been studied extensively in recent years, little is still known about the ways that Hispanic students perceive specific aspects of their learning environment. This is an important area that needs further investigation, especially since several studies have found that Hispanic students have different perceptions of their instructional and classroom learning environment than students from other racial groups such as Black and White students (Waxman 1989; Waxman and Eash 1983; Yosso 2005).

Measures of learning environment have been included in a few school effectiveness studies, but very little is still known about the relations between effective schools and classroom learning environment. Some studies have found that effective schools have more favorable learning environments and more positive school climate than ineffective schools (Edmonds 1986; Fraser 1991; Levine 1991, 1992). School climate, however, is often measured at the school level or teacher level in these studies. Few studies have actually used learning environment indicators in their research that are generally measured at a student-level, making it a more proximal measure of what actually occurs within classrooms.

1.2 School effectiveness research

Prior school effectiveness research has been criticized for a number of methodological, technical, theoretical, and conceptual reasons (Reynolds et al. 2014). One of the specific criticisms aimed at researchers in this field is that they have not investigated classroom processes as extensively as they should, especially since there is some evidence that suggests that instructional and classroom processes account for differences between schools (Teddlie et al. 1989; Teddlie and Stringfield 1993). Consequently, one of the current research thrusts in this area consists of examining those classroom processes or instructional effects within schools that influence student outcomes (Reyes et al. 2012; Reynolds et al. 2014).

Another current emphasis in school effectiveness research is specifically focusing on schools that have been successful in improving student learning outcomes over time and investigating why they have been successful. This perspective is similar to the educational resilience construct (Condly 2006; Waxman et al. 2004) because it focuses on school successes rather than on failure and predictors of failure. Despite being located in economically poor neighborhoods and communities, some schools do exceptionally well, and it is important to know why these schools succeed while other similar schools from equally stressful environments do not (Price and Waxman 2005; Waxman et al. 2007). This approach is important because it focuses on the predictors of academic success rather than on academic failure. This focus may also help us design more effective educational interventions because it enables us to specifically identify alterable factors (e.g., classroom instruction, teacher support) that distinguish resilient and nonresilient schools. The research thrust in this area is to extend previous studies that merely identified and categorized effective schools and to shift to studies that focus on identifying potential individual and school processes that lead to and foster success such as classroom instruction and the classroom learning environment. In other words, the construct of “educational resilience” is not viewed as a fixed attribute of some schools, but rather as alterable processes or mechanisms that can be developed and fostered for all schools.

A third current emphasis in school effectiveness research is examining the context or location of schools. Contextual influences such as the school setting or location of the school influences both teachers and students (Teddlie et al. 1989a, b; Wang and Eccles 2013). Consequently, research needs to be conducted in specific types of school settings so that the findings can be generalized to other similar settings. One particular setting where more research needs to be conducted is in urban schools. Addressing the problems of urban schools is one of our critical educational issues in the USA because the highest percentage of students at risk of failure are found in these schools and the worst social and economic conditions are also found in urban neighborhoods (Balfanz et al. 2007). Nowhere are the social implications of increasing numbers of disadvantaged families more prevalent than in the large, urban school districts in the USA where the deleterious conditions of underachievement, student and teacher alienation, and high dropout rates exist (Waxman et al. 1992, 2007). Other indicators like the high levels of crime, unemployment, drug abuse, broken families, teen pregnancy, juvenile delinquency, density of liquor stores, and high rates of poverty, clearly describe the critical status of students who are currently living in our nation’s urban neighborhoods. Additionally, urban schools face higher-than-average teacher turnover rates that further exacerbates problems results in lower student achievement (Guin 2004; Allensworth et al. 2009; Whipp and Geronime 2017). Consequently, those students attending urban schools represent the most imperiled group of our increasing numbers of students at risk of failure (Allensworth et al. 2009). Unfortunately, there are very few learning environment studies that have specifically focused on urban cities.

1.3 Purpose of the study

The present study addresses some of the previous problems of school effectiveness research by specifically examining the classroom instruction and learning environment in effective, average, and ineffective schools that serve predominantly Hispanic students. Some studies have conducted systematic classroom observations of instruction in classrooms that serve predominantly Hispanic students (Padrón et al. 2015) and a few have examined the classroom learning environments for predominantly Hispanic students (Rivera and Waxman 2011). These studies, however, have not incorporated both methods in their research and focused on classroom processes in effective, average, and ineffective urban elementary schools that serve predominantly Hispanic students. Unlike other “outlier studies” that have typically compared “extremely good schools” with “extremely weak schools” (Levine 1992), the present study also includes “average” schools to reflect more realistic differences that are often found in urban schools serving predominantly minority students. Consequently, the purpose of the present study is to examine whether or not there are differences in the classroom processes and classroom learning environment of effective and ineffective urban schools for Hispanic students. More specifically, this study investigates whether or not there are significant differences between effective, average, and ineffective urban schools on (a) students’ classroom behavior, and (b) students’ perceptions of their classroom learning environment.

2 Methods

2.1 Criteria for identification of schools

This study was conducted in a large urban school district located in a major metropolitan city in the south-central region of the United States. Twelve elementary schools were selected from the entire population of elementary schools in the district that had predominantly (i.e., > 80%) Hispanic students from economically disadvantaged families (i.e., > 80% free or reduced lunch from the Federal Assistance Program). Four schools were randomly selected from the population of “effective” schools in the school district, four from the population of “average” schools, and four from the population of “ineffective” schools.

The schools in the present study were classified as “effective,” “average,” or “ineffective” based on the state rating system for rewarding schools and improving performance. All schools in the state were evaluated according to their scores on the state-wide assessment of academic skills. Based upon the state-wide criteria, schools were placed into the following categories: (a) exemplary, (b) recognized, (c) acceptable, (d) low acceptable, and (e) clearly unacceptable. Their progress on test scores was also sorted into the categories of (a) exemplary progress, (b) recognized progress, (c) acceptable progress, or (d) no progress. In the present study, “exemplary schools” that have made “exemplary progress” were considered “effective schools.” Schools that had been classified as “low acceptable schools” and having made “no progress” were considered “ineffective” schools because there were no clearly unacceptable schools with no progress in this district. Schools classified as “acceptable schools” with “acceptable progress” were considered “average” schools.

The criteria for identifying effective, average, and ineffective schools in the present study has both some strengths and limitations in that it is based on the state-wide assessment of academic skills. One of the strengths is that these state-wide standardized tests have been found to be reliable and valid. In addition, they measure state-adopted curriculum objectives or important educational outcomes that have been emphasized in the classroom. Furthermore, unlike the use of standardized, norm-referenced achievement tests that are generally based on low-level basic skills and compare a student’s score to how others students score on the test, these criterion-referenced state-wide assessments measure many higher-level outcomes (e.g., problem solving and interpretive explanations) and focus on the expected academic performance of students. Finally, another strength of the state-wide classification system is that it uses multiple achievement criteria (i.e., reading, mathematics, and writing) rather than just one achievement outcome. One of the limitations of the criteria used in this study is that it solely based on cognitive outcomes and does not include other important educational outcomes in the social–emotional learning domain.

2.2 Participants

A total of 947 fourth- and fifth-grade students were surveyed and 573 students in grades three, four, and five were observed in their reading and mathematics classrooms. Table 1 summarizes some of the demographic comparisons between the three types of schools. About 88% of the students in the effective schools were Hispanic, and over 94% of them were eligible for compensatory education programs such as special tutoring or counseling sessions. Nearly 83% of all the students in the school qualified for free or reduced lunch, and about 65% of the students were classified by the State as at risk of failure. Over 48% of them were categorized as Limited English Proficiency (LEP) students. About 52% of third-grade students in the effective schools passed all three sections (i.e., reading, mathematics, and writing) of the state-wide assessment test.

Table 1 Demographic comparison between effective, average, and ineffective schools
Full size table

Approximately 96% of the students in the average schools were Hispanic, and about 82% of them were eligible for compensatory education programs. Nearly 85% of the students qualified for free or reduced lunch, and about 72% of the students were classified as at risk of failure. Over 57% of them were categorized as LEP students. About 41% of third-grade students in the average schools passed all three sections of the state-wide assessment test.

About 85% of the students in ineffective schools were Hispanic, and about 85% of them were eligible for compensatory education programs. Nearly 90% of the students qualified for free or reduced lunch and about 71% of the students were classified as being “at risk.” About 54% of ineffective school students were categorized as LEP students. Only about 32% of third-grade students in these schools passed all three parts of the state-wide assessment test.

The student profiles from the three types of schools are very similar, with the exception of the overall passing rate on the third-grade state-wide achievement tests, where the third-grade students from the effective and the average schools had higher passing rates than students in the ineffective schools. A higher percentage of students in the effective schools were eligible for compensatory education programs than in the average and ineffective schools. The effective schools also had a slightly lower percentage of students placed at risk and LEP than the other two types of schools. None of these differences appear to be educationally significant.

The teacher demographics in these schools were quite similar too. In the effective schools, about 25% of the teachers were Hispanic, 49% were White and 26% were African-American. About 24% of the teachers had advanced degrees, and they averaged eight years of teaching experience. Nearly 6% of the teachers were on temporary permits or emergency certification (e.g., alternatively certified). In the average schools, nearly 41% of the teachers were Hispanic, 46% were White and 13% were African-American. About 29% of the teachers had advanced degrees, and they averaged nearly nine years of teaching experience. Only 2% of the teachers were on temporary permits or emergency certification. In the ineffective schools, about 26% of the teachers were Hispanic, 34% were White and nearly 40% were African-American. About 29% of the teachers had advanced degrees, and they averaged 10 years of teaching experience. Nearly 7% of the teachers were on temporary permits or emergency certification.

The average schools have higher percentage of Hispanic teachers and lower percentage of African-American teachers than effective and ineffective schools. Average schools also have fewer teachers with temporary permit than the other two types of schools. Teachers in the average and ineffective schools had slightly longer teaching experience and a higher percentage with advanced degrees than those in the effective schools. Teacher attendance rates were similar across all types of schools. The average schools are relatively larger in student enrollment and smaller in student mobility as compared with other schools. The mobility rates for the effective and ineffective schools were identical, but quite high (40%). All the effective, average, and ineffective schools in this study were located in low socioeconomic communities, and students were assigned to schools according to neighborhood housing patterns.

Approximately eight classrooms from each school were randomly chosen to be observed during their reading or mathematics class. Approximately four students from each class were randomly chosen to be observed. Due to several pragmatic concerns (e.g., teacher absences, testing schedules, availability of classes by grade), the actual number of students observed differed somewhat between the “effective,” “average,” and “ineffective” schools. A total of 573 third, fourth, and fifth-grade students were observed in these classes. About 48% of the students were male and 52% were female. About 6 weeks before the end of the school year, all students in fourth- and fifth-grade classrooms completed a survey that examined their perceptions of their classroom learning environment. The instrument was not administered in third grade because of validity concerns with younger students completing the survey. A total number of 947 students completed these surveys.

2.3 Instruments

The standardized observational instrument used in the present study was the Classroom Observation Schedule (COS) (Waxman and Padrón 2004). Two standardized student survey instruments were also used in the study: (a) the Classroom Environment Scale (Fraser 1982, 1986), and (b) the Instructional Learning Environment Questionnaire (ILEQ) (Knight and Waxman 1990).

The Classroom Observation Schedule (COS) (Waxman and Padrón 2004) was used for collecting student classroom process data. It is a systematic observation schedule designed to document observed student behaviors in the context of ongoing classroom instructional learning processes. Individual students are observed with reference to (a) their interactions with teachers or other students, (b) the setting in which the observed behavior occurs (i.e., whole class, small group, or individualized, (c) whether their school work is teacher assigned or student selected, (d) the type of activity that the student is working on (e.g., working on written assignments, watching or listening, interacting/talking, or reading), and (e) whether the student is on- or off-task. A complete list of all the specific behaviors is included in Table 3.

This instrument is grounded in direct observation and focuses on student learning behaviors rather than the teacher teaching. This observation instrument has been found to be reliable and valid in previous studies with the same grade levels (Waxman et al. 2009), and in the present study the inter-rater reliability (Cohen’s Kappa) was found to be excellent (r = 0 .98).

The Classroom Environment Scale (CES) is a questionnaire that has been widely used in a variety of educational settings, including elementary grades to measure students’ perceptions of their relationships with students and teachers as well as the organizational structure of the classroom. The content and concurrent validities of the CES have been established through correlational studies and classroom observation (Fisher and Fraser 1983; Fraser 1982, 1991). Adequate internal consistency reliability coefficients were also obtained in previous studies (Fisher and Fraser 1983; Fraser 1982). A brief description of the scales and a sample item from each follows:

  • Involvement—the extent to which students participate actively and attentively in their mathematics/reading class discussions and activities (e.g., In my mathematics/reading class, I really pay attention to what the teacher is saying).

  • Affiliation—the extent to which students know, help, and are friendly toward each other in their mathematics/reading class (e.g., I know other students in my mathematics/reading class really well).

  • Teacher Support—the mathematics/reading teacher helps students and takes a personal interest in them (e.g., My mathematics/reading teacher takes a personal interest in me).

  • Task Orientation—indicates the extent to which the mathematics/reading class is businesslike with an emphasis on completing classwork (e.g., Getting a certain amount of classwork done is very important in my mathematics/reading class).

  • Order and Organization—the extent to which the mathematics/reading class is under control with orderly behavior (e.g., My mathematics/reading class is well-organized).

  • Rule Clarity—the extent to which rules are clearly stated in their mathematics/reading class and the students are aware of the consequences of breaking rules (e.g., In my mathematics/reading class, there is a clear set of rules to follow).

Two scales from the Instructional Learning Environment Questionnaire (Knight and Waxman 1990, 1991), Student Satisfaction and Student Aspirations, were also used in the present study. As used in the present study, this instrument measures students’ perceptions of their (a) enjoyment in school and (b) aspirations for continuing their education. A brief description of the scales and a sample item from each follows:

  • Satisfaction—the extent of students’ enjoyment of their mathematics/reading class and work in mathematics/reading (e.g., I enjoy the schoolwork in my mathematics/reading class).

  • Student Aspirations—students’ intentions to attend and succeed in high school and continue their education beyond high school (e.g., I plan to finish high school).

The instrument has been previously been used in elementary grades and has been found to be reliable (i.e., both internal consistency and test–retest reliability) and valid (i.e., both construct and predictive validity) (Knight and Waxman 1990, 1991).

In the present study, all the items from the two surveys were modified to a “personal form” of the instrument which elicits an individual student’s responses to his/her role in the class rather than a student’s perception of the class as a whole (Fraser 1991). In addition, all the items were designed to be content specific (i.e., reading or mathematics), rather than generic so that students did not have to infer their responses across all subjects and classes. “Short forms” (Fraser 1982) of all the instruments were used in this study, which typically included three or four items per scale.

In order to ensure adequate reliability and validity of the scales used in this study, internal consistency (Cronbach’s alpha) reliability and discriminant validity (correlations between scales) were conducted for each of the modified scales. The mean alpha coefficient of these scales for the present sample using the student as the unit of analysis was .74, and the mean alpha coefficient of these scales for the present sample using the class as the unit of analysis was .83. The mean correlation between all possible scales using the student as the unit of analysis was .17, and the mean correlation between all possible scales using the class as the unit of analysis was .35. These results indicate that the survey instrument has adequate discriminant validity. Univariate analyses of variance (ANOVAs) were used to determine classroom effect on students’ learning environment. The results indicate that class membership had significant influence on all eight learning environment scales with task orientation and order and organization accounting for the greatest differences among classes. The values of eta 2 range from .06 to .24, with an average of .15, suggesting that about 15% of the variance in students’ learning environment is attributed to class membership.

2.4 Procedures

Trained research staff observed each class for approximately 45 minutes near the middle of the school year. These observers were not aware the type of school (i.e., effective, average, or ineffective) that they were observing. An average of four students in each class were randomly selected for observation. During the observation period, each student was observed using the COS for ten 30-second intervals using a round-robin strategy across the four students. A total of 573 third, fourth, and fifth-grade Hispanic students were observed in these classes.

About 6 weeks before the end of the school year, all fourth- and fifth-grade students completed the two survey instruments that were administered concurrently by the research staff. A total number of 947 students completed these surveys. Students were told that the survey was not a test and their responses on the survey would not be seen by their teachers or any school personnel. Students completed the questionnaires during their regularly assigned reading or mathematics period. Students were given approximately 30 minutes to complete the questionnaires. The questionnaires were read by the research staff who also explained any words on the survey that students had trouble understanding.

Multivariate analysis of variance (MANOVA) was used to determine whether there were significant differences on the learning environment scales and student behaviors by the type of school (i.e., effective, average, and ineffective). Follow-up univariate analysis of variance (ANOVA) tests were used when significant multivariate differences were found. When the univariate F tests were significant, Duncan post hoc multiple comparison tests were used to examine where the differences were.

3 Results

3.1 Classroom learning environment results

Table 2 reports the means and standard deviations for all the student survey scales. A mean score close to three for each of the scales indicates that students perceived that the particular scale or variable was very prevalent (i.e., strongly agreed with all the items on the scale), while a mean value close to one indicates the students perceived that the particular variable was not prevalent or never occurred in the classroom (i.e., strongly disagreed with all the items on the scale). The midpoint or median score for all the scales was 2.0.

Table 2 Learning environments differences by type of school
Full size table

The descriptive results indicate that all students had above-average perceptions of their classroom environment. The mean values for all of the scales are above 2.0, and most are close to a 2.5 average. The scales with the highest means for students in effective, average, and ineffective schools were Student Aspirations, Rule Clarity, and Affiliation. The scales with the lowest mean values for students in effective schools were order and organization, teacher support, and involvement. The scales with the lowest mean values for students in ineffective schools were order and organization, task orientation, and teacher support. The scales with the lowest mean values for students in average schools were order and organization, involvement, and teacher support. The standard deviations indicated that there was adequate variance on all the scales and that there were no scales that had a large number of extreme scores.

The MANOVA results revealed overall significant (F (16, 1874) = 16.92; p = .0001) multivariate effects attributable to the differences between the three types of schools on the eight learning environment scales. As a follow-up procedure, univariate ANOVAs were used to determine if the overall difference between schools existed for each scale. Table 2 shows the F values and the means and standard deviations for each scale by school type. The results revealed that there were significant differences between effective, average, and ineffective schools on the all scales except rule clarity. Follow-up ANOVA and post hoc multiple comparison results for the ILEQ scales reveal that students in effective schools had significantly greater satisfaction and aspirations than students in average and ineffective schools. Follow-up ANOVA and post hoc multiple comparison results for the CES scales reveal that students from the effective schools reported significantly higher involvement, teacher support, task orientation, and order and organization than students from the average and ineffective schools. Students from the effective schools also reported significantly higher Affiliation than students from the ineffective schools. Students from the average schools reported significantly greater Teacher Support and Task Orientation than students from the ineffective schools. There were no significant differences between students from average and ineffective schools in involvement, affiliation, and order and organization. There was no significant difference between effective and average schools on the affiliation scale.

3.2 Classroom observation results

Table 3 reports the overall findings from the student observations. In effective schools, the predominant setting or context observed was whole-class instruction (68%), followed by individualized or independent work (19%), and small-group instruction (12%). In these settings, students were observed working independently about 32% of the time. They interacted with their teacher about 56% of the time and with students about 11% of the time. Nearly all the activities (99%) they were observed doing were teacher-assigned activities. The most prevalent activity that students were observed doing was watching or listening (62%). The next most prevalent activities were working on written assignments (22%) and interacting (20%). Students were observed being on task about 97% of the time. It should be pointed out that the standard deviations are quite large for all of the observed student behaviors, suggesting there is a great variance among individual student behaviors from these effective schools.

Table 3 Classroom instruction differences by school type
Full size table

In the average schools, the predominant setting or context that was observed was whole-class instruction (78%), followed by individualized or independent work (13%), and small-group instruction (9%). In these settings, students were observed not interacting and working independently about 45% of the time. They interacted with their teacher about 42% of the time and with students about 13% of the time. Nearly all of the activities (99%) they were observed doing were teacher-assigned activities. The most prevalent activity that students were observed doing was watching or listening (52%). The next most prevalent activities were working on written assignments (15%) and interacting (8%). Students were observed being on task about 91% of the time. It should be pointed out that the standard deviations are also quite large for all of the observed student behaviors, suggesting there is a great variance among individual student behaviors from the average schools.

In the ineffective schools, the predominant setting or context that was observed was whole-class instruction (67%), followed by individualized or independent work (19%), and small-group instruction (14%). In these settings, students were observed not interacting and working independently about 37% of the time. They interacted with their teacher about 46% of the time and with students about 17% of the time. Most of the activities (94%) they were observed doing were teacher-assigned activities. The most prevalent activity that students were observed doing was watching or listening (61%). The next most prevalent activities were interacting (16%) and working on written assignments (16%). Students were observed being on task about 88% of the time. It should be pointed out that the standard deviations are also quite large for all of the observed student behaviors, suggesting there is a great variance among individual student behaviors from these ineffective schools.

The multivariate analysis of variance (MANOVA) results revealed a significant multivariate effect for type of school (i.e., effective, average, and ineffective) on the Interaction (F (4, 1138) = 5.76, p < .0001), type of activities (F (14, 1128) = 13.84, p < .001), Setting (F (4, 1138) = 3.67, p < .01), and manner (F (2, 570) = 21.58, p < .0001) sections of the COS. Follow-up univariate tests revealed that there were significant differences between effective, average, and ineffective schools on most of the variables. Table 3 displays the ANOVA results. One of the key findings is that students from the effective schools were observed significantly more frequently Interacting with Their Teachers than students from average and ineffective schools. They were more frequently placed in small groups or individual setting and less frequently in whole-class setting than students from average schools. They were more on task and less distracted or disruptive than students from average and ineffective schools.

Effective school and average students were observed engaging more in teacher-assigned activities and less student-selected activities than students from ineffective schools. As for specific types of activity, students in effective schools were observed spending more time: (a) working on written assignments, (b) reading, and (c) presenting or acting than students from average and ineffective schools. Students from the average schools were more frequently observed in whole-class settings than students from the effective and ineffective schools. These average school students were also found less frequently (a) interacting or talking, (b) watching or listening, (c) getting or returning materials, and (d) working with manipulative materials or equipment than students from effective and ineffective schools.

4 Discussion

The results of the present study indicate that there are several classroom instruction and learning environment differences between these effective, average, and ineffective urban schools serving predominantly Hispanic students. Students from the effective schools perceived more positive learning environments than students from the average and ineffective schools. In particular, students in the effective Hispanic schools generally have higher Satisfaction and Student Aspirations than students from the average and ineffective schools. They also perceived their classrooms as having significantly more task orientation, order and organization, teacher support, affiliation, and having greater involvement than students from the average and ineffective schools. On the other hand, there were no differences found on students’ perceptions of rule clarity. The high means for this variable suggest that students perceived that all teachers in these schools were consistently emphasizing the rules of the class. In other words, this variable did not help discriminate the effective and ineffective schools.

The results from the observational data indicate that students from effective Hispanic schools interacted with their teachers significantly more than students from average or ineffective schools. Another key finding was that students from effective Hispanic schools were observed being on task nearly 10% more than students from the ineffective schools. This finding is similar to other research that found that students in effective schools are engaged more than students from ineffective schools (Waxman and Huang 1997; Waxman et al. 1997). Engaged time is a critical instructional variable that has been found to be significantly related to students’ academic achievement (Stanton-Salazar 2001), and the differences found in present study serving predominantly Hispanic students is very critical. Students in the average schools were seldom observed working with manipulative materials or equipment, presenting/acting, and tutoring peers. They are, however, more on task and perceiving greater Teacher Support and Task Orientation than students in the ineffective schools.

The results of the present study generally support prior school effectiveness studies that have found that effective schools have a stronger academic focus and more orderly climate than ineffective schools (Cohen 2013; Jesse et al. 2004; Price and Waxman 2005). The findings from the present study, for example, reveal that students from effective schools have higher aspirations, and perceive more task orientation and rule clarity than students from ineffective schools. Students in effective schools are also working significantly more on written assignments than students from ineffective schools. Furthermore, students from effective schools were found interacting with their teacher significantly more than students from the average and ineffective schools.

The results of this study lead to several implications for policymakers, and educators. The study’s focus on USA schools in urban settings serving predominantly Hispanic students is an important contribution to the field. Second, the inclusion of both student learning environment measures and classroom observation to examine effective schools is another major contribution. Third, the substantive findings are important because it suggests that schools can have a positive impact on students’ academic achievement and aspirations for future education. The schools in the present study were quite similar and were all located in an urban metropolitan area with high poverty. The teacher characteristics across the three types of schools were quite similar too. Our findings, however, revealed very distinct patterns for the three types of schools. In the effective schools, teachers were interacting more with their students and students were much more engaged (i.e., on task) with their school work. For the most part, the classroom environment in the effective schools was significantly higher than the average and ineffective schools. On the other hand, in the ineffective schools, students’ have lower perceptions of their classroom learning environment, they are less engaged, and they are more distracted. While the correlational nature of this study does not allow us to make inferences about why these schools are effective, average, or ineffective, our findings suggest that the classroom environment and student classroom behaviors may be influential variables to explore in future studies. These findings are similar to previous school effectiveness studies (Reynolds et al. 2014) and highlight the need to address changes at the school level.

4.1 Implications for future study

Although research on classroom observation and classroom learning environments has made significant progress over the past several decades, there are still several areas that need further investigation. In order to capture all the processes and nuances that occur in classrooms, triangulation procedures are needed to collect data from multiple perspectives (Waxman and Chen 2006). Collecting multiple measures or indicators of classroom processes may help alleviate some of the concerns and criticisms of observational and learning environment research and provide us with a more comprehensive picture of what goes on in classrooms. Teacher self-report data, teacher, administrator, and student interview data, and more qualitative, ethnographic data (e.g., participant observation) could all be used to help supplement the classroom observation and learning environment data. Ethnographic observations could be used to explore the quality of teacher–student interactions and student engagement in the classroom. Qualitative studies could also be done to specifically examine what classroom factors foster Hispanic students’ positive attitudes toward schooling and their own learning.

Systematic classroom observations of specific teaching behaviors such as instructional pacing, teacher questioning, and presentation of information would also be useful to examine other instructional differences between effective and ineffective schools. Other observational instruments that focus on the content of lessons and additional aspects of teacher–student interactions would also be useful. More research is also needed to examine why there was such large standard deviations on the student classroom behaviors. It may be related to teacher effects such as quality of classroom instruction or other classroom environment measures.

Future studies should try to administer the long forms of these survey instruments rather than the “short forms.” The added number of items per scale on the long forms would probably increase the reliability of the scales. Future research could also specifically examine the reliability and validity of the state-wide assessment instrument for Hispanic students. Similarly, future studies may want to examine whether teachers in effective schools “teach more to the test” than teachers in ineffective schools.

Future studies should address multilevel issues by using advanced statistical models like hierarchical linear modeling (HLM) that allow researchers to investigate hypotheses about the effects of within- and between-school or class factors (e.g., teacher effects) on students’ classroom behavior and perceptions of their learning environments. HLM allows researchers to identify and separate individual effects from group effects, after statistically controlling for other explanatory variables. A larger sample would also allow us to examine potential differences by content area and grade level.

While the findings from the present study have several important implications, further correlational, longitudinal, and especially experimental research is needed to verify these results. Correlational studies are needed to see what variables are related to Hispanic students’ cognitive and affective outcomes. Experimental studies are needed to examine professional development programs or school reform approaches that help teachers improve their classroom instruction and learning environment as well as investigating its impact on the academic performance of Hispanic students. Additional research could also examine issues such as the leadership qualities of principals in effective schools for Hispanic students. Some studies, for example, could examine whether the characteristics of minority principals that have been found to be effective in predominantly minority schools such as: (a) commitment toward educating, (b) compassion and understanding of their students and communities, and (c) confidence in the ability of students to learn are equally important for principals of predominantly Hispanic schools. These and other issues still need to be examined so that we can continue to understand and improve the educational experiences for Hispanic students in urban schools.