Introduction

Researchers have long maintained that American high school students are not fully engaged in classroom learning (Newmann, 1992; Shernoff, Knauth, & Makris, 2000; Yair, 2000). Recent international studies of mathematics and science classrooms report that a majority of classrooms do not hold students’ attention. When video-taped American 8th grade mathematics classroom teachers’ instruction was analyzed by mathematics pedagogy and content knowledge experts, a majority of sampled classrooms were rated as “very unlikely” to engage their students in mathematics learning (Hiebert et al., 2000). This is a serious problem because student engagement has consequences for students’ academic achievement and other outcomes (Finn & Cox, 1992).

Many researchers have studied engagement constructing it as students’ general attitudes toward schooling (Finn & Voelkl, 1993; Lee & Smith, 1993; Marks, 2000), but have not treated it as a dynamic phenomenon that occurs inside the classroom, bounded by time and space. In most studies, predictors of engagement such as pedagogical variables are derived from one-time surveys. Thus, most studies lack units of analysis capable of capturing temporal changes in engagement level in accord with changes in instructional context in classrooms. It is therefore difficult to know what specific pedagogical strategies and instructional content promote student interest, engagement and achievement.

An additional and equally important question is how students from different social groups respond to classroom instruction. Specifically, although teachers in the context of standards based pedagogical practices are encouraged to engage students in hands-on problem-solving classroom activities, it is not clear if these strategies are beneficial equally to all groups of students. For example, it may be that some students prefer modes of learning that emphasize group collaboration rather than individual work. While the issues of underserved race and ethnicity groups has been well-studied in the content of academic achievement (Hedges & Nowell, 1999) and tracking (Oakes, 1985; Page, 1991; Rosenbaum, 1976), few quantitative analyses have examined the implications of race/ethnicity in the context of instructional processes during classroom lessons. Thus, we undertook our study to remedy these concerns by conducting an intensive examination of classroom processes and student engagement during the course of daily observations with students attending 8 high schools and enrolled in a total of 32 mathematics and science classrooms across 4 geographically disparate sites with a week spent in each of the participating classrooms.

Student engagement as flow

Engaged students pay close attention to ongoing classroom activities, are interested in the content of classroom lessons, and may also experience heightened states of awareness, confidence, and performance. Csikszentmihalyi (1990) treated engagement as the experience of “flow.” The optimal experience of flow occurs when an individual’s ability and the skill level required by classroom activities match, so that a task provides a reasonable challenge. Flow theory has influenced studies of adolescents (Csikszentmihalyi & Larson, 1989) and has been applied to the study of adolescents’ educational achievement, participation in sports, and deviant behavior.

To capture the nature of engagement, a variety of observational methods have been used. Turner and her associates (Turner et al., 1998) observed classrooms, coded incidents of teacher discourse indicative of social support and explored its relationship to student engagement. Similarly, the Third International Mathematics and Science Study (TIMSS) relied on video cameras in seven countries’ 8th grade math and science classrooms (Hiebert et al., 2003). Classroom video clips were evaluated to determine both the nature of teachers’ pedagogical strategies and the likelihood of these strategies and instructional activities to engage students. This approach, however, lacked the capacity to capture both the embeddedness of teaching activities in ongoing classroom events and the relationship of these activities to engagement as reported by students.

Others have applied event history analysis to model the duration of students’ attention during class based on what researchers gleaned from videotapes of classroom activity (Felmlee & Eder, 1983; Felmlee, Eder, & Tsui, 1985; Imai, Anderson, Wilkinson, & Yi 1992). Such an approach to coding can be unreliable if, as Yair, (2000) stressed, students may simply appear to be engaged. On the other hand, Nystrand and his colleagues studied the occurrence of particular classroom events such as student–teacher interactions. The interactive, dialogic discourse of teachers, advocated as the most effective in communicating classroom lessons to students (NCTM, 2000), was critical in determining the type of engagement that students experienced (Gamoran & Nystrand, 1992; Nystrand & Gamoran, 1991; Nystrand, Wu, Gamaron, Zeiser, & Long, 2001). However, the need still exists for evidence to support the assumption that such instructional contexts will consistently engage all students.

Finally, Csikszentmihalyi and his associates used the Experience Sampling Method to focus on adolescents’ cognitive and affective responses to everyday life events (Csikszentmihalyi & Schneider, 2000; Shernoff, 2001; Yair, 2000). Adolescents were provided with survey booklets, as well as watches that were preprogrammed to beep at randomly occurring intervals both during and outside school hours. Upon being signaled, student participants were asked to record what they were doing and how they were feeling. Using school classroom segments of these data, Shernoff et al. (2000) found that individual work is strongly associated with the experience of flow, characteristic of a high level of engagement. Yair (2000), in contrast, determined that students’ engagement in class was more pronounced when teachers employed hands-on problem-solving strategies as opposed to a lecture format in presenting material. In our study we combined both classroom observational methods and the Experience Sampling Method (ESM). Classroom observational data in the form of fieldnotes allowed the collection of detailed information about the social organization of classroom activities. ESM, although somewhat intrusive, permitted the assessment of the affective responses of students and also allowed the rigorous measurement of engagement using Rasch Model analysis (Wright & Masters, 1982; Wright & Stone, 1979). As a result, we improved upon the methods of other ESM researchers. We depart from both Shernoff’s use of ordinal dependent variables and Yair’s use of the dichotomous variable (Yes/No) describing students’ perception of their engagement in the classroom based on potentially unreliable open-ended questions.

Classroom social organization and student engagement

Sociological research on the social organization of the classroom extends back to Bidwell and Kasarda (1980) and demonstrates how processes of schooling are linked to important student outcomes. Classroom instructional activities linked to the social organization of instruction, influence students’ commitment to learning. We argue that teachers’ classroom activities constitute social organizational variables that influence both attitudes and behaviors of students and the extent to which classroom instruction engages students. Bidwell and Kasarda (1980) questioned earlier research findings that showed schools made little to no difference in determining students’ achievement outcomes (Coleman et al., 1966). At the time that Bidwell and Kasarda undertook their research, resources were measured at the school level and researchers were unlikely to take into account how resources were distributed within schools and how individual students were affected in this process. For Bidwell and Kasarda, patterns of classroom learning activities, including lecturing or group work, mediated the use of classroom resources such as instructional materials that in turn affected student outcomes. For example, when lecture and recitation are the dominant mode of instruction, a student’s public performance in class establishes a hierarchy of more to less academically proficient students. This dynamic can have a strong, negative impact on low-performing students who may be discouraged from active engagement in classroom learning.

Other researchers subsequently demonstrated the power of social organizational dynamics in affecting student outcomes. For example, Barr and Dreeben (1983) found that during the course of reading instruction in elementary school classrooms, students were sorted into ability groups, resulting in differentiated student assessment outcomes that varied according to group membership. Bossert’s participant observation in elementary school classrooms (1979) demonstrated that classroom activities chosen by teachers, such as group work or individual student seatwork influenced classroom processes and strongly affected the visibility of differential student achievement levels. More recently, MacFarland (2001) examined the ways peer social networks provided socially supported deviance that in turn encouraged student resistance in classrooms. He concludes that student-led work groups may compromise classroom order because students have an opportunity to collectively resist the established (teacher-determined) rules. Bidwell and Kasarda refer to such classroom organizational arrangements as “schooling processes” and urge that studies in the tradition of school effects research examine them.

In the context of this tradition, we explore how classroom activities affect student engagement and student perceptions and conversations during class. The latter can be considered intervening factors given that teacher activities, such as lecture, group work, and seatwork, affect student engagement. Students’ perceptions during classroom activities can predict engagement. First, student engagement is influenced by task difficulty, broadly defined, inherent in instructional topics. As studies of flow demonstrate, task challenge must be appropriate and not overwhelming for students to feel competent (Nakamura and Csikszentmihalyi, 2002; Shernoff, Csikszentmihalyi, Schneider, & Shernoff, 2003). Previous classrooms studies argue that both a lack of challenge and redundancy in the information transmitted are characteristic of U.S. classrooms (Hiebert et al., 2003; Stigler & Hierbert, 1999). Thus, we argue that student perceptions of classroom task challenge affect their engagement. Second, how relevant classroom instruction is to students’ lives is also a factor, as indicated in previous research (Ainley, Hidi, & Berndorf, 2002). Research conducted by Ainley et al demonstrates that clear instructional goals tied to student lives enhances engagement. Third, researchers have found that collaboration can heighten students’ engagement. When collaborating in small groups with peers, students can express themselves, explore possible outcomes through hands-on experience, and view peers as consultants.

We see the factors we have just described as intervening variables affected by classroom activities and in turn affecting student engagement. Of the classroom activities considered here, group work may provide more favorable outcomes. The challenge level inherent in classroom tasks is difficult for teachers to adjust to the needs of each student during, for example, classroom lectures, as students may vary in levels of ability. Group work and to some extent individual work, on the other hand, may allow individuals to adjust the pace of work. Through hands-on activities, students may find meaningful, new information on their own, and enhance the novelty of instructional content. Furthermore students may see some activities as more relevant than others since problem-solving activities and collaboration may simulate tasks they encounter in everyday life. In contrast, teacher lecture as well as individual student seatwork may not be as effective as group work in making instructional content relevant. Students are more likely to experience boredom and drowsiness during teachers’ lectures, as well as during individual work. Group work, in contrast, requires activity and may keep students alert and focused. Thus, we expect students to hold a variety of perceptions towards ongoing classroom activities depending upon the nature of these activities provided by the teachers.

We also consider students’ classroom conversation as an intervening factor. When students interact with their peers about the academic lesson at hand, they are also likely to be more focused. Talking with friends about class may make class work more fun. If, however, talking with peers is chitchat about matters unrelated to classroom lessons, student academic engagement is likely to decrease. Both social conversation with others and conversation about instruction are likely to take place during student group work. Seatwork, though it may temporarily keep students quiet, may result in idle chatting when students finish their assigned tasks. Lecturing may suppress both talking about class and chatting as teachers would expect students to remain silent. Taken all together, we predict that classroom activities affect engagement through their impact on student social interaction in the classroom.

Classroom social organization and effects of race and ethnicity

We also investigated how students of different racial and ethnic groups respond to classroom activities. Educational research literature contains a number of relevant findings. As an example, in a study of student achievement in mathematics, when teachers used constructivist approaches, mathematics achievement differences among social groups defined by SES levels were reduced (Wong & Lee, 1998). In his ESM study, Yair (2000) found that Latino students had generally lower levels of engagement, and that their engagement fell to lower levels still when students perceived the quality of instruction to be poor. Conversely, Latino students’ engagement levels increased when instruction was challenging, relevant, and academically demanding.

Here we ask why some racial and ethnic minority students are less inclined to perform well academically in class, and why their engagement levels may vary with classroom activity. We attempt an answer relying on various literature sources. On the one hand, sociolinguistic studies claim that minority students are sometimes alienated in classrooms because of important differences between patterns of home and school interaction (Mehan, 1984, 1992). On the other hand, a social organizational approach would predict that, depending on what teachers do to organize classroom instruction, we could expect differences in classroom social climates, including a sense of collaboration in the classroom. To elaborate, we start by reviewing research on sociocultural differences.

Some consider that schools and the schooling process are hostile to historically underserved students because most mainstream, traditionally organized schools and classrooms do not acknowledge or take into account the cultural orientations of these students. Bourdieu (1984) introduced the concept of habitus to explain the cultural dispositions of individuals belonging to different social groups. One’s habitus can be viewed as one’s cultural niche. Habitus profoundly affects values, beliefs, and how group members’ understand the world. Because the school’s curriculum and teaching practices typically embrace middle class values, lower SES students with a habitus or world view that does not overlap or coincide with the norm manifest in schools and classroom may subsequently be alienated from the dominant messages they receive in school. Likewise, Bernstein’s theory of social class, language code and social control suggests that classroom instruction typically utilizes syntax associated with middle class discourse, syntax that children from low SES households may find alienating and as a consequence they may experience less engagement in class work and less academic success (Bernstein, 1977).

Case studies exploring sociolinguistic differences between household and school classroom socialization norms and patterns provide support for this line of reasoning (Mehan, 1984), while also providing notions about how race/ethnicity and classroom activities may interact. Studies of Native Americans have demonstrated that patterns of learning that are familiar to Native American children at home—group oriented and involving collaboration rather than competition—are rarely practiced in traditionally organized classrooms. Such patterns may not be compatible with typical classroom instructional practices, especially in traditional classrooms in which white, middle-class teachers engage in pedagogical practices most familiar to white middle-class children (Philips, 1972). Other case studies have reported that Latino students are alienated in classrooms because their approach to learning typically involves distinctive aspects of the cultural heritage of Latinos (Borman & Baber, 1998; Moll, 1992). Like native American students studied by Philips, a collaborative environment, as opposed to a competitive one, seems to characterize the ethos of Mexican American families, according to some case studies (Genova et al., 1981; Laosa, 1973; Wortham & Contreras, 2002). The claim that home pedagogical culture varies by subgroups is supported by studies using large data sets. Mexican American parents, for example, were found to place less emphasis on independence in their approach to child rearing (Evans & Anderson, 1973).

Because there is a lack of research on this question, it is unclear how white, black, and Asian students might react to specific classroom activities. White students, partly because they are the majority group, are rarely studied in light of their uniqueness. Studies of black students in school have tended to focus on general attitudinal issues, such as stigma associated with “acting white” (Fordham & Ogbu, 1986), and teacher stereotypes (Downey & Pribesh, 2004). Some have reasoned that urban black students prefer active lessons (Boykin, 1978), yet the validity of this argument has not been tested. Asian students’ preferences are also difficult to predict. Some literature on Asian area studies indicate that learning, particularly in the Confucian cultural tradition, is not so much about interaction as about the individual reading of scripts.

Given cultural variations in child-rearing and socialization, how does a research perspective emphasizing social organizational approaches help explain patterns of variation in and among different racial and ethnic groups in the classroom? As argued earlier, group work promotes collaboration among students, and as a result may provide a more supportive learning context for those whose approach to acquiring knowledge is nurtured by cooperative settings. In contrast, individualized forms of instruction, such as seatwork, may curtail collaborative and engaged learning. If we dovetail social organizational factors with variables derived from sociolinguistics research, we can expect that students who vary in racial and ethnic group membership may be affected differently by particular patterns of classroom organization and instructional activity. For minority students who are more successful learning in cooperative small groups, patterns of classroom organization that encourage group work are likely to be more engaging.

Research questions and hypotheses

We seek answers to two broad questions related to both student engagement generally, and to the differential reactions of students by race and ethnicity during ongoing classroom lessons. First, what are the effects of classroom activities on student engagement and how much of these effects is mediated by students’ perception of lesson content or students’ interaction/collaboration with others in classroom? As we reasoned earlier, we expect group work to be more engaging than either lecture or seatwork. We also expect group work to produce positive perceptions about lesson content and more productive social interactions/collaborations among peers and with teachers.

Student perceptions of instructional content include the level of challenge, defined as both the level of difficulty and lack of familiarity with the content of classroom lessons. Relevance of instruction to students’ everyday lives, plans to attend college, future job aspirations, and forthcoming tests are intervening factors. Finally we take into account student subjective reports of their enjoyment, disinterest or confusion. Social, collaborative factors include student perceptions of collaboration and competition in the classroom. Classroom conversation among students is also of interest, since students may or may not be encouraged to interact with their peers and/or the teacher depending upon the nature of the activity at hand. In summary, we hypothesize that classroom activities influence the way students feel about content and interact with others in the classroom, and that this in turn influences their classroom engagement.

Second, we ask how students of different racial and ethnic groups respond to patterns of classroom instruction, asking: How do racial and ethnic group members vary in their perceptions of classroom instruction and how do their perceptions affect their engagement? We expect that groups vary according to their assessment of conditions that best promote their engaged learning. Keeping in mind demographic and historical conditions governing patterns of assimilation or accommodation to mainstream U.S. norms and values, we explore the possibility that Latino students are likely to prefer classroom learning featuring frequent use of small groups organized to undertake problem-solving activities and that black students may favor conditions supporting their active engagement. Because there is as yet insufficient related research, we hesitate to offer preliminary questions concerning how students from other racial and ethnic groups might be affected.

Data

Background: Urban Systemic Initiative

This study of student engagement is part of a large-scale evaluation of the impact of the National Science Foundation’s Urban Systemic Initiative (USI)1 in four cities—Chicago, El Paso, Memphis and Miami. During the course of the 3-year project our research team examined 10–14 participating schools in each of the sites, aiming to provide an understanding of students’ learning processes and patterns of classroom instruction. To avoid the limitations of outcome measures such as student achievement as measured by high stakes state tests, we undertook this study of student engagement using ESM data, student focus group data, and observational data to determine the nature of students’ responses to classroom experiences as well as to determine more generally which experiences constituted favorable conditions for learning in classroom settings.

Schools, Teachers and Students

Our sample is one of convenience and thus not a representative sample of students from each of the urban sites included in this project. From each of four sites, we selected two high schools. From the pool of five teachers in each school who had allowed us to closely observe their classrooms during the tenure of our larger research program over 4 years, we requested one mathematics and one science teacher willing to participate each day for 1 week during two of their mathematics or science classes. We aimed to recruit 10 students from each of the 32 classes taught by 16 teachers. As will be described later, we followed the same students for 5 days, attending their classes and collecting 10 observation points from each student.

This strategy would have produced 3200 observations from 320 students. However, we were not able in all cases to recruit sufficient numbers of students because of difficulties in obtaining parental consent. Securing a number of observations from each participant was also problematic in some cases. Some schools had fire drills and preparations for homecoming week and the like resulting in the cancellation or interruption of instructional time. One school had a holiday during our research trip. As a result, the average number of observations obtained from each individual was 6.8 and the number of students was 345, a little more than originally planned. All together we obtained 2360 observations across all participants. The final analytical sample was reduced to 1936 cases, as a result of missing cases and other factors. We will discuss this in a later section.

Methods

Experience Sampling Method (ESM), Classroom Observation, and Focus Group Interviews

We employed the Experience Sampling Method (Larson & Csikszentmihalyi, 1983) that documents affective and cognitive responses of study participants at specific times during ongoing activities. From each participating mathematics and science classroom, 10 students were selected as subjects for the study. These students were asked to complete a short questionnaire, upon receiving a signal through small vibrating beepers similar to those used to signal waiting restaurant customers. The survey took about a minute for most students to complete and included questions about their engagement in class, as well as about what they were doing and how they were feeling (See Appendix).

During a 45- or 50-min period, we sent out ESM signals every 10 min. The first and third signals went to one half of the participating students and the second and fourth signals to the other half. Thus, in a typical class each subject received 2 signals 20  min apart during each class period over the course of 1 week, resulting in 10 observation points. Some schools or school districts employed 90-min classes block scheduled classes. During these 90-min classes, the two groups of participating students received four signals. Because block scheduled classes meet every other day, most of these students provided either 8 or 12 observation points.

To minimize the effects of the instrument, students were told that the survey was not a test, and, as mentioned earlier, they were given at least 20 min between ESM signals. The signaling schedule was altered daily, so that the timing was not predictable. We noticed in some limited number of cases that participants spent longer than 1 min completing survey questions. Psychometrical analyses, based on Rasch modeling, suggest that quality of measures, defined by misfit statistics, stayed the same throughout the week. Most students said, when interviewed, that the method was not intrusive.

We also conducted classroom observations, with two researchers taking notes about what was occurring in the classroom. Based on the field notes, the types of classroom activities occurring at the time of the signal were noted (e.g., group work, discussion, lecture, and downtime). To increase our understanding about participating students, we conducted focus group interviews with 10 randomly selected students from the ESM participants at each school after our data collection ended on the final day of our visits. We guided students’ discussions based on a protocol focusing on their general experiences of mathematics and science classrooms,2 and also allowed students to talk freely when appropriate.

Design of Analysis

The data structure was hierarchical such that the signal-level repeated measures were nested within participants, who were in turn nested within courses (32 classes taught by 16 teachers). To take into account the resulting correlated errors, we used hierarchical linear model (Raudenbush & Bryk, 2002). Of various submodels of hierarchical linear model, we replicated a random intercept model, using SAS Institute’s PROC MIXED. Consider first the general representation of models used in our exploration of social organizational effects and intervening variables.

$$Y_{ijk} =\beta _0 + {B1^{\prime}\ast {COV}+B2^{\prime} \ast {ACT} +B3^{\prime} \ast {\rm{INTERVENE}}+u_k +r_{jk} +} e_{ijk} $$
(1)

The postscript i stands for ESM signal-level (7 repeated measures per student on average), j stands for students (10 individuals within each course), and k stands for courses (5 days worth of 32 courses taught by 16 teachers). Vector matrices COV, ACT, and INTERVENE contain data for, respectively, covariates, classroom activities, and intervening variables (student perception and classroom conversation). B1, B2, and B3 matrices are vectors containing corresponding coefficients. Errors at level 1 (repeated measure), level 2 (students), and level 3 (courses), represented, respectively, by e, r, and u are assumed to vary randomly, each with a constant variance. For our question regarding the effects of classroom activities on engagement, we first build a baseline model that controls for basic covariates of students, class, and time. To assess the impact of classroom activities on engagement, we then add the activity variables to the baseline model, while controlling for the technology of instruction (e.g., the use of the blackboard or overhead projector). Next, to evaluate the role of intervening variables, we add the variables characterizing student perceptions and patterns of communication to the model. If the entering of these variables lead to reduction of activity effects, we can consider them to be the factors that mediate the relation between the activities and engagement level.

For our second question regarding race–class activity interaction, consider the following random intercept model.

$$ \begin{array}{lll} \hbox{Model} 1:Y_{ijk}&=&\beta _0 +\beta _1 \ast \hbox{ACT}_{ijk} +\beta _2 \ast \hbox{RACE}_{jk} +\beta _3 \ast \hbox{ACT}_{ijk} \ast \hbox{RACE}_{jk} \ldots+u_k +r_{jk} +e_{ijk}\\ \hbox{Model} 2:Y_{ijk}&=&\beta _0 +\beta _1 \ast \hbox{ACT}_{ijk} +\beta _2 \ast \hbox{RACE}_{jk} +\beta _3 \ast \hbox{ACT}_{ijk} \ast \hbox{RACE}_{jk} +\beta _4 \ast \hbox{COOPERATE}_k +\beta _5 \ast \hbox{RACE}_{jk} \ast \hbox{COOPERATE}_k \ldots+u_k +r_{jk} +e_{ijk}\\ \hbox{Model} 3:Y_{ijk}&=&\beta _0 +\beta _1 \ast \hbox{ACT}_{ijk} +\beta _2 \ast \hbox{RACE}_{jk} +\beta _3 \ast \hbox{ACT}_{ijk} \ast \hbox{RACE}_{jk} +\beta _4 \ast \hbox{CONVERSATION}_{ijk} +\beta _5 \ast \hbox{RACE}_{jk} \ast \hbox{CONVERSATION}_{ijk} \ldots+u_k +r_{jk} +e_{ijk} \end{array} $$
(2)

Imagine, for simplicity, that RACE and ACT correspond to any one of our racial/ethnic groups (Black, Latino, White, or Asian) or classroom activities (Lecture, Group work, Individual work, Downtime). To test whether the race/ethnicity effect depends on the activity that is occurring at the time of an ESM signal, we use Model 1 to test the interaction term based on race/ethnicity and teacher activity variables. The second model would test the hypothesis that the race/ethnicity–activity interaction effect occurs as a result of racial/ethnic groups preferring particular social climate of the classroom, i.e., level of cooperation (COOPERATE). This would receive support if (a) cooperative classroom climate interacts with race/ethnicity to affect engagement level and (b) the race/ethnicity–activity interaction effect is reduced in size as a result. The third model adds an interaction term based on students’ classroom conversation and race/ethnicity. Using the same logic as in Model 2, the third model test whether classroom conversation (CONVERSATION) explains why some race/ethnicity groups prefer certain activities. Throughout these models, we control for basic set of covariates, which are to be discussed below.

Variables

Dependent Variable: Level of Student Engagement

Based on a pilot study, we selected 8 survey items that captured the level of student engagement—using response categories of “Strongly disagree,” “Disagree, “Agree,” and “Strongly Agree”: students were asked to indicate one of the following: (a) I was paying attention, (b) I did not feel like listening, (c) My motivation level was high, (d) I was bored, (e) I was enjoying class, (f) I was focused more on class than anything else, (g) I wished the class would end soon, and (h) I was completely into class. The order of survey items correspond to item difficulty ranging from easy to difficult. This ordering was replicated by the data for this study.

We used Rasch modeling to allow us to express engagement measures in logit in stead of ordinal scales, while also controlling for the differences of item difficulty among the survey items (Wright & Masters, 1982; Wright & Stone, 1979). To express this formally, the following equation states that a student’s response pattern is regulated by three factors, engagement level, survey item difficulty, and “step functions” associated with changes in response categories.

$$\log (\pi _{ijk} /1-\pi _{ijk} )=\beta _i -\delta _j +\tau _k $$
(3)

Postscript i is an individual at a time of an ESM signal, j is a survey item, and k is a response category. π is a probability that a person of engagement level β chooses a response category k (e.g., Strongly agree) over a response category k−1 (e.g., Agree), when answering a survey item with item difficulty δ. A parameter, τ adjusts for the difficulty increase from one response category k−1 to a next category k.

We used Winsteps software (Linacre, 2006) to find values for person measure β and item difficulty measure δ that maximize the match between the model’s prediction and the observed effect (the Maximum Likelihood Method). Based on a pilot study, the eight items satisfied the Rasch Model diagnostic statistics consisting of misfit statistics, reliability, and a wide and balanced range of item difficulty. Based on the data collected, we found that person and item reliability statistics, both Rasch diagnostics statistics, were, respectively 86% and 99%, and thus sufficiently high.3 Thus, we concluded that the estimated β can be used as a linear measure for student engagement.

One problem, however, surfaced as we diagnosed our measures derived from the ESM data collection. Misfit statistics is a measure that indicates how deviant the scoring patterns of respondents were juxtaposed to the pattern expected by the model (Wright & Masters, 1982). There are outfit and infit statistics that are sensitive to different dimensions of the diagnostic information. The outfit statistic is more sensitive to the deviation of students, who provide unexpected responses to either difficult or easy items. The outfit statistic is also easier to interpret as it clearly points to cases of odd and outlier responsess. In practice, more than 99% of the cases respondents who were identified as misfitting in outfit were also misfitting in infit statistics verifying our diagnostic decision.

We found that 8% of the derived scores had rather high outfit levels such that standardized outfit (t-score) exceeded the threshold of 2. The extreme cases occurred when responses corresponded to one and only one category, such as “Strongly Agree.” Less severe cases included situations where respondents disagreed with items (e.g., “Were you paying attention?”, and indicated disagreement with other items (e.g., “I was completely into class). These responses likely occurred when respondents did not have sufficient time to complete the survey items; thus, we decided to omit these cases from our multivariate analysis.

We confirmed that the correlation between the level of engagement and the outfit statistic is essentially zero. To safeguard the influence of this diagnostic decision, we compared the results of our regression analysis with and without the deleted observations. The difference in coefficient sizes in all the models are trivial and no statistical test conducted throughout these models changed results.

Independent Variables

Classroom Activities Based on the observational notes of the researchers, we dummy-coded four types of classroom activities as they occurred in between the ESM signals. Whole class instruction referred to a time period when the teacher lectured in front of the entire class. Individual work was used to describe time periods when students were working independently on problem sets at their seats. This included doing problem sets from textbooks or handouts, as well as completing homework assignments at the end of the class. Group work describes a period when students worked on problems together with their peers. This included doing experiments in science classes, or carrying out problem-solving together in mathematics classes. Periods of down-time (named as nothing), noted typically at the beginning and end of classes, referred to the times when teachers and students were not engaged in academic work. These dummy variables are not mutually exclusive categories, as they could co-occur in the same signal periods.

Intervening Variables for Teacher Activity Effects

Characteristics of Instruction and Affective Response Reported by Students

We expected classroom activities to affect student perceptions of instruction. First, we tried to capture the difficulty of the class using two survey items—whether content was very easy, easy, about right, difficult, very difficult or “cannot tell,” or whether the content was something a student either already knew or did not know. The latter should inform the prevalence of redundancy, believed to contribute to tediousness of US. classrooms, especially a the high school level (Stigler & Hiebert, 1999). Second, when signaled, students reported the relevance of what was being taught. Relevance of class to students’ everyday lives, going to college, future jobs, and future tests were reported as dichotomous categories (yes or no). Finally, other items indicate whether students were feeling cooperative, competitive, having fun, sleepy, or confused.

Five Student Conversation Types

Students’ conversation during class was considered to moderate how activities affected students’ engagement level. Students were asked whether they were talking with anyone, either the teacher or a classmate, at the time of each signal. If they responded in the affirmative, they were asked whether the talk pertained to the lesson at hand or not. Based on student responses, five categories were created: (a) not talking, (b) talking with the teacher about class, (c) talking with the teacher about things unrelated to class, (d) talking with peers about class, and (e) talking with peers about things unrelated to class.

Interaction Terms to Examine Race/Ethnicity Effects

Level of Cooperation in Classroom

The measure of cooperation is a social climate variable designed to interact with race/ethnicity variables. Students indicated if they felt “cooperative” at the time they were signaled. Because our intention is to capture the social climate of the classroom specific to each meeting of a class, responses were aggregated at each class/meeting level. As a repeated measure of level control, we also used the original raw response. In this way we could separate the influence of students’ estimate of degree of cooperation from that of social climate.

Classroom Conversation with Peers

To examine how much of the teacher activity–race/ethnicity interaction effect is a function of students’ classroom conversation, we also created an interaction term based on race/ethnicity and classroom conversation. Of the two, the latter variable indicates three states, (a) not in conversation, (b) academic conversation with classmates, and (c) social conversation with classmates.

Covariates

Basic Characteristics of Time, Students, Class, and Sites

Time

ESM Signal ID indicates the timing of ESM signals. In Chicago and Memphis, where class periods were 50 min in length, ESM signal ID ranged from one to four. In El Paso and some schools in Miami and Memphis, class periods were 90 min in length, thus signal ID ranged from one to eight. These variables were treated as categorical, rather than continuous. Our classroom observations suggest that the engagement level dips towards the end of the class. Days of the week were also coded as categorical variables. We expect Friday to be a low engagement day, as students may be looking ahead to the weekend.

Student Characteristics and Sites

The student characteristics of greatest interest to this research included gender, race (African American, Latino, Asian, and White), and grade (9–12). Table 1 summarizes these demographic characteristics by city. Note that distribution of race is highly correlated with cities. As many as 64% of our Chicago participants, and all of our Memphis participants, were black. El Paso and Miami students were predominantly Latino, largely attributable to the close proximity of Mexico and Cuba and the Caribbean nations, respectively. El Paso houses a large military base as well, providing a mix of other racial groups. In Miami, Latinos were primarily from Cuba or Central America and many of the black students were from Haiti. In Chicago and El Paso, there were more girls than boys present in observed classrooms. Our field observations suggested that on the first day of our research, especially during morning classes, more boys arrived late than girls. Thus, gender effect, if we observe any, could reflect this selection effect. Finally, cities are used as categorical variables in our models.

Table 1. Demographic Characteristics of Students
Full size table
Characteristics of Observed Classes

In addition to mathematics/science distinctions, we have subcategories for algebra, geometry, precalculus, and calculus, as well as biology, chemistry, and physics. Honors classes, as well as courses such as calculus and physics, were distinguished as higher-level courses. Depending on the number of students present in the classroom, categories for classroom size included small, medium, and large. Seating arrangements within classrooms included individual seating, roundtable seating, or laboratory seating. Laboratory seating was utilized in science classes where students were seated at lab tables, although students continued to face teachers and work individually. Whether the seating position was decided by the teacher or students it was coded categorically. We expect these classroom features to influence the level of distraction students experienced during class.

Technology of Instruction

To remove a possible source of bias in estimating classroom activity effects, we controlled for classroom technology, as the latter co-occurs with specific types of instruction. The use of blackboards or overhead projectors as demonstration tools during lecturing was coded categorically with the expectation that these devices might affect students’ attention. The use of computers and hand calculators rarely occurred during our visits and were excluded.

Limitation of Data

We lack student level achievement data. As a result, we cannot evaluate the degree to which individual achievement affects engagement, as well as the extent to which effects found with other covariates, such as race/ethnicity, classroom characteristics, and teacher activities, are the function of student achievement levels. However, we did include two variables that may correct this deficiency. At the classroom level, a dummy variable indicates higher level courses. Students’ sense of how easy or difficult an activity is may also serve as a proxy of their level of competence and confidence. Although the researcher took notes on what teachers did in between the ESM signals and created a time-varying characterization of instructional quality, we did not attain reasonable interrater reliability.4 As a result, we cannot separate the effect of teacher activity from quality of teaching associated with the activity. As a proxy, however, we control for the level of confusion students experienced during class.

Results

Descriptive Statistics

Table 2 presents the distribution of classroom activities, and the percentage of yes responses to survey items. The first column shows the whole sample, while the other four correspond to lecture, group work, seat work, and testing. We first describe the activities and then show how they are related to what students reported regarding their perception and conversation types.

Table 2. Descriptive Statistics (Percentage) for Student Variables and Classroom Activity
Full size table

The predominant activity was lecturing (41%), while seatwork was reported by 34% of the sample. Group work occupied only 13%, while testing was observed with only 3% of the sample. The finding that American classrooms employ mostly lecture and individual work is consistent with the findings of other researchers, as well as another national-level observations (Hiebert et al., 2003; Shernoff et al., 2003).

As for students’ perceptions about instructional topic, students reported they were exposed to new content only 37% of the class time. Comparing the values across teacher activities, we see the favorable impact of group work. During group work, students felt that they were exposed to new sources of knowledge 54% of the time, which was 10% more than during teacher lectures. As for seatwork, an activity very common in American classrooms as well as in our sample, students uniformly felt they already knew the content.

As for the level of difficulty that students reported, 30% of the whole sample reported that the level was “about right” and roughly 30% felt it was either “very easy” or “easy.” Difficult or very difficult instruction was indicated only 10% of the time. To allow a convenient activity comparison, we prepared a summary measure of a difficulty index by assuming that the categories form a linear measure ranging from 1 (very easy) to 5 (very difficult). The results showed that class activities did not change the perceived difficulty level of the content. Thus, group work did not reduce the level of content difficulty any more or less than more traditional classroom participation structures.

With respect to relevance, students found instructional content almost always relevant with regard to future tests, with students unaware of its relevance only 8% of the time. Instructional content was seen as relevant for going to college 67% of the time. In contrast, students found the instruction relevant to everyday life 35% of the time, but relevant to a future job only 23% of the time. Thus, during class time, students consistently thought about their upcoming tests, as well as going to college more immediately related to classroom instruction than to everyday life or future jobs. Comparison by classroom activity types shows that during group work students felt that the content was relevant to their everyday life 52% of the time as opposed to about 35% during lecturing or seatwork.

As for their subjective feelings during classes, roughly one fourth of the time students reported having fun (21%) and feeling cooperative (22%), but they were also sleepy (20%) and confused (17%) during much of this time. Competitiveness was rarely felt (5%). Comparing students’ feelings across classroom activities, we see how activities are related to student affect. Group work provided exceptionally favorable experiences. Students reported having more fun (35% as opposed to lecturing’s 19%) and feeling a lot less sleepy (13% as opposed to lecturing 25% or testing 29%). Students were likely to feel more competitive (11% as opposed to lecturing’s 5%) or cooperative (34% as opposed to lecturing’s 26%). The level of confusion, however, did not vary much, except during testing.

Finally, students were silent about half of the time, talking about class-related information with the teacher 6% of the time and with their peers 20% of the time. Chatting with peers or engaging in social conversation occurred 14% of the time and with teachers a negligible 1% of the time. It was encouraging to find that, for a quarter of the time, students were engaged in class-related issues. Group work is substantially different than other activities. Group work promoted exchanges among peers about class-related issues (54%), while surprisingly discouraging social conversation among peers (9%). In fact social conversation was most prominent during lecturing (14%) and seatwork (14%), the two most frequently observed activities in classrooms in this sample.

In summary, our sample showed that what was being taught tended to be old material, redundant, and rarely difficult. However, when students were engaged in conversation, it was likely to be more academic than social. Students talked about class-related matters during a quarter of the sampled time. Comparison by classroom activity suggests that group work is highly related to students’ heightened sense of relevance for their everyday life, competition, cooperation, and student academic conversation with classmates. Students were much less sleepy during group work. These outcomes are consistent with our expectation that the latter factors play the role of intervening factors in our model of student engagement.

Results of Student Engagement Model

Table 3 shows the results for three models whose outcome variable is student engagement level.5 Model 1 is an unconditional model, Model 2 builds a baseline model, and Model 3 adds classroom activity variables. For ease of interpretation, we z-scored the dependent variable and the column on the left reports percentages of all predictors. Model 1 estimated that variance at three levels, .41 at within-individual, .52 at between individuals, and .11 at teacher-course levels. This indicates that engagement level, as measured, is a phenomenon occurring mostly between individuals (51%), as well as within individuals (39%), leaving little room for teachers/courses to vary (10%). This justifies our interests particularly for the time-varying factors of ongoing classroom events, such as pedagogy and the changing social structural arrangements in classrooms.

Table 3. Random Coefficient Model of Student Engagement Level
Full size table

In Model 2 we controlled for basic covariates, which reduced between-class variance to zero. With Miami being a reference category (a score of 0), Chicago had the lowest score (−.21), while El Paso had the highest score (.22). Grade effects and gender effects were not statistically significant. Latinos were as much as .43 SD less engaged in class than African American students. While whites and Asians were also less engaged in class than African Americans, the differences among them were not statistically significant. Differences among less and more advanced courses were negligible. The lowest levels of engagement were recorded on Mondays, with engagement levels increasing monotonically to reach statistically significant .10 SD higher on Fridays. Engagement levels were highest during the first 10 min of class and remained more or less at the same level until the fifth beep, about the time an ordinary 45 or 50-min class ended. Beyond that length of time, engagement level tends to dip significantly, which resonates with our observation in so-called block scheduled classrooms that are typically 90 min long. Many teachers in such classes stopped instruction within the first hour, allowing students to do assignments (usually homework) at their seats and socialize as they finished their tasks.

Other covariates concern classroom size and seating arrangements. First, larger classes generally were associated with higher engagement levels, yet these differences were not statistically significant. Second, individual seating, the most typical seating arrangement, comprised 74% of the observations, and was .33 SD higher in engagement level than roundtable seating. Individual seating may prevent students from becoming distracted by their peers, while roundtable seating may increase the chance of being distracted by other students. During focus group interviews, many students expressed the concern that peers distracted them from completing their work, especially when working on seatwork tasks such as worksheets. Finally, when seats were chosen by students rather than by their teachers, student engagement level was as large as .33 SD lower. Based on our observations, students who chose their own seats tended to sit with their peers enhancing their opportunity to interact freely with each other.

In Model 3 we address our first research question by evaluating the extent to which teacher activities affect students’ engagement levels, controlling for the use of technology use that may co-occur with certain activities. Group work had about a .20 SD higher level of engagement compared to lecturing, seatwork, and testing. During downtime (when no instructional activities were underway), the engagement level was about .38 SD lower than that for group work and about .20 SD lower than that for other types of activity.6 Some moderate reduction in coefficient sizes occurred to time effects expressed as the effects for ESM signal IDs. This is likely explained by the fact that the use of particular pedagogical strategies depended on the amount of time available and used for the activity in question. Lecture was almost always employed at the beginning of a class, while seatwork almost always came toward the end.

Testing Intervening Factors of Classroom Activity Effects

Table 4 begins with Model 3 that we already reported above. The goal of Models 4, 5, and 6 is to see (a) how student perception of content and student conversation may affect engagement level and (b) whether they mediate the relation between classroom activities and engagement levels. As discussed earlier, the latter will be tested by monitoring the reduction in classroom activity effects, as we add variables of our interests.

Table 4. Intervening Factors of Classroom Activity Effects on Student Engagement
Full size table

In Model 4 we added students’ understanding of content. With reference to the difficulty level of content, the easier the content the more engaged students are. Previous knowledge of the content covered in class made no substantial difference, though students’ engagement was substantially lower when they could not decipher if they were familiar with the content of the lesson at hand or not. The teacher activity effects remained unaffected by these variables, except that the downtime now is well accounted for and is now not significantly different from lecturing.

In Model 5, we considered students’ perception about class content, as well as their affective reactions. To discuss main effects first, engagement levels were higher when students found that the content of the lesson was related to their everyday lives (.17 SD) and to tests (.16 SD). The coefficients were negligibly small for relevance to going to college and their future jobs. This suggests that students are more focused and engaged with class work that is relevant to their present concerns than to their future. Students were also highly engaged when they felt cooperative (.36 SD), competitive (.25 SD), not sleepy (−.41 SD), and not confused (.19 SD). Fun, surprisingly, was not found significant. As a result of a series of variables, the effect of group work was greatly reduced from .21 to .14, indicating some of these variables may be mediating the relation between engagement and teacher activities. After discussing the next model, we investigate which specific predictors were responsible for this reduction.

In Model 6 we added students’ classroom conversation, which slightly reduced group effect from .14 to .11, as well as the effects of the class content related to going to college and test. Engagement level, at the time of academic conversation with teachers, was .23 SD higher than during moments of silence and .45 SD higher than the moments the moments of social conversation with classmates. If the content-related conversation occurred with classmates, the engagement level was also higher, but only by .10 SD compared to those times when conversation did not occur at all and by .38 SD compared to times when students chatted with classmates.7

Given the results of Models 4, 5, and 6, we further examined which of the significant predictors played the role of intervening variables. In Table 5, we compared the reduction rate of group work advantage against lecture, as the significant factors are added to the baseline model (Model 4). In the baseline model (Model 4), engagement level for group work was .26, while the lecturing was .04. The difference in mean engagement levels here fluctuated as a variable is added one at a time (Model 4a–4k). Sleepiness and conversations (Model 4a and 4b) accounted for 20% of group work advantage over lecture; thus, we concluded that these were the primary intervening factors. The rest of the factors did not explain away the group work effect.

Table 5. Diminishment of Group Work Effect by Addition of an Intervening Variable
Full size table

In summary, we concluded that group work, among all of the various class configurations that we observed, was related to higher levels of student engagement. We also found that Latino students had the lowest engagement score overall, with an almost .37 SD difference from black students. Somewhat surprising in terms of effect sizes were factors related to seating. The use of roundtable arrangements for student groups in the classroom, though rare in our sample (13%), had an engagement level .40 SD lower than the conventional individual seating arrangement. In addition, when seats were chosen by teachers and not by students, engagement levels were almost .30 SD higher. Our last three models showed that many of our intervening variables had main effects; however, students’ reported drowsiness and conversation effects are the two major influences of group work on student engagement.

The Dynamics of Race and Ethnicity

Our analysis showed that Latino students were the least engaged in the classes that we have studied. Figure 1 is the reassessment of race/ethnicity effects, which is derived by a model that controlled for a set of covariates that were statistically significant and large in effect sizes.8 To interpret race/ethnicity averages, recall that the engagement scale is standardized with a mean of zero and standard deviation of one. Latino student differences as compared to black students was significant while comparisons with white students were marginally significant. A closer examination of this issue is warranted in order to understand the contextual factors contributing to this phenomenon. Testing this involves the use of interaction effects between race/ethnicity and instructional factors. The second model adds the collaborative measure of classroom climate, as well as classroom conversation, to the modeling of race effects and assesses whether it mediates the race and activity interaction effect. The testing of these is based on the models that control for a set of covariates that were found significant in earlier Models (See footnote 8).

Fig. 1.
figure 1

Comparison of student engagement level by race/ethnicity groups.

Full size image

To construct a baseline model, we first tested city and race/ethnicity interaction terms to the model we used in our earlier analysis. We detected that there is some city and race/ethnicity interaction effect for Latinos (statistics not reported here). While there is some fluctuation for all race/ethnicity groups, statistically significant differences of .64 SD in engagement level was detected between Latinos in El Paso (.075) and those in Chicago (−.57). Latinos in El Paso were for the most part first and second generation descendants of Mexican immigrants and constituted the overwhelming majority of students in the classrooms we visited (.69%), while those in Chicago were more heterogeneous and a distinct minority (13%). To reflect this heterogeneity, we decided to code Latinos as three groups, Latinos of Chicago, El Paso, and Miami. Memphis did not have any Latino students in the sample.

Table 6 shows the results of three models to test a set of interaction terms involving race, pedagogy, classroom collaboration, and conversation. In theory we have six categories of race/ethnicity, four categories of teacher activities, and interaction terms based on the two variables. We report on variables relevant to our second research question and omit results for down time category. Controlling for the same set of covariates used in our earlier investigation, Model 7 tested the interaction effect between racial and ethnic groups and classroom activities.

Table 6. Race and Teacher Activity Interaction Effects (Number of Cases = 1936)
Full size table

As interpretation of interaction terms from Table 6 requires summation of coefficients across relevant rows, we present Figure 2 and Table 7 to summarize the results. Based on the estimation of Model 7, Figure 2 shows how the mean engagement level of each group varies. To interpret race/city averages in Figure 2, recall that the engagement scale is standardized with a mean of zero and standard deviation of one. Using the information from the same model, Table 7 reports the results of statistical tests that compared the engagement means by classroom activities within each race/ethnicity group. As predicted by our set of expectations, Latino students of all three cities were engaged more during group work than during lecture and individual work. The results for Hispanics in Miami and Chicago were statistically significant. Group work engaged them more than two all other activities in Miami. El Paso classrooms, however, favored lecture over individual work. This may reflect the fact that El Paso students had an extended period of individual work when students were able to socialize in a way unrelated to content taught.

Fig. 2.
figure 2

Engagement level by race/ethnicity and classroom activities.

Full size image
Table 7. Comparison of Mean Differences in Student Engagement Level (Model 7)
Full size table

Asians served as a contrast to Latino students. Asian students did experience an increase in engagement for group work. Although not statistically significant, Asians are the only group that favors individual work over group work. Individual work is significantly more engaging for Asians than lecture, a finding unique to this group. Similar to Lations, White students respond well to group work and negatively to seatwork. However, their responses during teachers’ lectures were not as strongly negative as was the case for Latinos. In fact they are the only group to favor lecture to individual work. Finally, Black students seemed less affected by changes in classroom activities and were generally highly engaged in all circumstances.

Now we continue from Model 8 in Table 6. We aimed to explain why Latinos appear to be well engaged during group work while Asian students appear to prefer individual seatwork in the classroom. Suspecting that these results signal associations with racial and ethnic group preferences for particular social structures for learning, we added measures of cooperation both as a repeated measure and a classroom level construct. A strong and statistically significant effect was found with the interaction term for being Asian and class-mean cooperation measure. This indicates that Asians in highly collaborative classrooms were not well engaged. We also see a reduction in Asian student–teacher activity interaction effects, such that the difference between individual work and lecture became very narrow, losing its original statistical significance. Thus, for Asians, classroom orientation seems to explain why seatwork is engaging. Latinos’ group work interaction effect was not explained away, as none of the three cities’ Latinos’ effects are affected by the addition of the cooperation measures.

With Model 8 we examine if the reason for differential reactions to class activities may be related to the use of language. To the extent that language minorities are not fully comfortable with English, it may explain why they like or dislike particular activities. When engaging in collaborative activities, Latino students, for example, may be able to use Spanish, which is an extra resource that may complement their learning. We thus modeled the race/ethnicity coefficients by adding interaction terms between race and talk behavior variables, indicating three states, no conversation, and academic and social conversation with classmates.

The results are clear and statistically significant for Latino students in Miami and Chicago. Latino students in these two cities were significantly more engaged while in academic conversation than in other types of conversation. This also reduced the race/ethnicity and group interaction for them, indicating that academic conversation explained some of the group work effect. For Latinos in Miami, the race–group work interaction effect went from significant .41 to non-significant .33. For Latinos in Chicago, the coefficient went from .47 to .08, though both values were not significant. Yet, the initial marginally significant difference between group work and lecture in this race/ethnicity group (reported in Table 7) was completely explained away. However, this reduction did not occur with Latinos in El Paso.

Thus, we replicated Yair’s finding (2000) that effects of race and ethnicity on engagement may be contingent on instructional contexts. Perhaps the most notable contrast in our findings was between Latino and Asian students. Latino students’ engagement levels were generally lower; however, students of Latino heritage, though the El Paso effect was not significant, were more engaged when assigned to group work. Latinos in Chicago and Miami were not well engaged during classroom lectures. The findings support a hypothesis that group work provides them with academic conversation that motivates them particularly. Asians, in contrast, were most engaged during seatwork, which perhaps was the most individualized, self-paced form of instruction. We found that, for Asian students, classroom engagement was stronger when the classroom was not cooperative, perhaps explaining why Asian students preferred seatwork.

Discussion and conclusion

We have shown the utility of a social organizational approach to understanding student engagement in classroom instruction. Our descriptive analysis foreshadowed some findings related to our first research question. The evidence suggests that class activities may influence the ways in which teachers and students respond to the classroom content and how they interact among themselves. During group work, students spoke more among themselves about academic issues and reported positive feelings about the class. Interestingly, during group work, students felt that they were learning something new, while the perceived difficulty of the tasks was not compromised.

In a more direct and explicit testing of our first research question, the results of our multivariate models showed that students’ perception of class and classroom conversation had main effects on engagement levels. In particular, classroom conversation, as well as the level of drowsiness were both found salient intervening variables that mediate the relationship between activities and engagement level. Each of these factors accounted for 20% of differences in engagement levels between group work and lecture. Lecture seems to encourage social conversation and group work suppresses it, while social conversation has a negative effect on engagement; hence, as McFarland (2001) underscored in his study of student resistance in classroom, the importance of social networks that foster such conversation should not be ignored.

We found other factors that are similarly social in origin. There were positive benefits for students who are given no seating choice. This arrangement influences the amount and levels of distraction experienced by students. In classrooms where the seating arrangement is not constrained by the teacher, we saw friends, or sometimes dyads of boys and girls, sitting close to each other, a situation that can easily prompt chatting, as well as the last minute sharing of homework answers. All these findings seem to indicate that classroom organization, defined by classroom conversation with classmates, is distracting. Though the effect was positive, academic conversation did not add much to engagement level, when compared to the times students are silent. One recurrent theme in focus group interviews in almost all of the schools was students’ reservations that, while friends are valuable sources of information, they can also be sources of distraction when friends are not “doers.”

In pursuit of our second research question, we found that classroom conversation works positively, as academic conversation helped remedy the alienation of Latinos students in Miami and Chicago. The findings support the general idea that students of different race/ethnicity groups have preferences for different types of classroom activities, while also providing interesting speculative data about each race/ethnicity group. An interesting contrast is between Latinos and Asian students. On the one hand, Latinos students were most engaged during group work and, at least in Miami and Chicago, this seems related to the finding that their engagement level is high and even higher than that of other racial/ethnic groups, when engaging in academic conversation with classmates. On the other hand, Asian students were most engaged during individual work, a result that seems related to the finding that their engagement is lower in a cooperative environment. This result resonates well with the view that children of voluntary immigrants are motivated to bring honor to the family by demonstrating their achievement as an individual (Schneider & Lee, 1990).9

Some findings remain difficult to explain. White students in general and Latino students in El Paso strongly preferred lecture to individual seatwork. Black students, who were generally higher in engagement levels, were little affected by differences in types of classroom activities, replicating Yair’s finding (2001) that Latino student engagement was sensitive to the quality of instruction, defined by levels of challenge, relevance, skills, and academic demand, while black students were least affected by these. In our sample, most black students were taught by black teachers. We believe that black teachers generally had a unique, interactive rapport with students during instruction. The teachers and students relied upon “call and response,” during which teachers ask questions and students replied without being formally asked to do so.

Taken together, the findings addressing our research questions show the importance of considering the social environment characterized by classroom conversation. Classroom conversation with classmates including the study of classroom social networks in which student conversation occurs is an area that is underdeveloped in studies of classroom processes. A perspective on the enacted curriculum, for example, utilized by dominant school reform research such as in TIMSS (Third International Mathematics and Science Study), treats teacher instruction as scripts of teaching and thus what counts is what is taught and how it is taught. Given the findings of this research, as well as those by McFarland (2001), such a research paradigm may be better served if it treats the classroom as a more complicated social organization punctuated by classroom activities, and by students’ responses.

One important theoretical contribution is derived from how the classroom can become a locus of social reproduction. While sociology of education research has identified various sources of educational stratification, including school sectors, tracks, and ability groups, it has not yet fully explained the classroom as a site for constructing students’ differential experiences. Although primarily qualitative in its methodological orientation, the sociolinguistics literature suggests that some racial and ethnic groups may have pedagogical preferences. Further, the gap between home and school pedagogical orientations may induce further alienation from classroom learning processes. In combination with the sociolinguistic perspective, the social organization approach provides a flexible and adaptive theory of student engagement. As opposed to deterministic notions that the school is an institution that inherently alienates underserved students, the social organization model allows classrooms to vary in the degree to which they are hostile or friendly to students of different racial and ethnic groups. Because a social organizational approach compelled us to look at the classroom context for student learning, we departed from the research question that has dogged social scientists for decades—that is, how the classroom can be an agent of social reproduction. Following the results of this study, we can now frame a perspective on social reproduction that underscores the likelihood that teachers’ actions in classrooms have the capacity to create a stratification system based on how teachers organize classroom lessons.

The findings from our pursuit of our two guiding research questions indicate that urban high school classrooms in the U.S. may be a locus for the social reproduction of existing inequalities for some racial/ethnic groups, but that classroom can also provide opportunities to learn. U.S. classrooms, according to this study, and other related studies, allocate large amount of time for lecturing and seatwork, leaving a fraction of time for group work. This conventional time allocation for classroom activities may be the mechanisms through which alienation is maintained for some minority groups, such as Latinos. However, distinctly American phenomenon of having prolonged seatwork time may provide children of Asian immigrants’ opportunities to be engaged in class and possibly to excel in school.

Nonetheless, we are far from making decisive conclusions about pedagogical strategies that most benefit students from specific racial and ethnic groups. On the one hand, our findings must be replicated using more comprehensive data that includes not only urban public schools but also schools from other strata of American society. In addition, our broad understanding of race and ethnicity is highly limited because our data were organized broadly as white, black, Latino, and Asian categories—an approach that tends to obscure the composition of these groups as sets of diverse and overlapping, each containing individuals with highly varied levels of linguistic, ideological, and attitudinal acculturation with the values and characteristics of mainstream U.S. society. On the other hand, an explanation of the gaps between home and school pedagogies will require a direct examination of pedagogy at home, allowing us to understand the forces at work that alienate certain subgroups of students from classroom learning.

Notes

  1. 1.

    The USI program, funded by the National Science Foundation in 1993 supported math and science teaching and learning reforms in 21 cities over a 5-year period, providing 15 million dollars to each USI/city site over the duration of funding.

  2. 2.

    Our protocol consisted of questions asking students about their favorite class, their friends in classes, the use of mathematics and science at home or at work, and their future plans.

  3. 3.

    We checked the problem of differential item functioning by gender by running the separate models for boys and girls. We compared the item’s difficulty measures to see if they vary by gender. No substantial difference was found and we concluded that boys and girls responded to our survey items in the same way.

  4. 4.

    We tried to code every 10-min block of instruction based on a classroom observation checklist anchored to the five academic process standards: communication, problem-solving, representation, reasoning, and connections NCTM (1991). Professional Standards for School Mathematics. Reston, VA: Authors. We found that 10 min of class time provided too little information to allow this coding. Furthermore, many classroom moments, particularly seatwork or downtime when nothing was occurring, would have returned missing values for this coding. Thus, the quality of teaching variables would not have been a good fit for our modeling design.

  5. 5.

    Percentages right next to the variable names refer to the descriptive statistics of the predictors. The outcome measure is standardized with a mean of 0 and SD of 1. The coefficient, therefore, can be interpreted as a Z-score difference of that category and the baseline category.

  6. 6.

    We observed that individual work in particular creates a rather diverse set of experiences for students because they can socialize when they are done with seatwork tasks. Also, lecturing might have provided very different experiences, depending on the skill of the teachers. Our model lacks appropriate controls to address these nuances.

  7. 7.

    One way of reading this is that teachers gain only .10 SD of student engagement by encouraging them to talk about class rather than sit quietly; however, teachers lose as much as .28 SD if the talk turns to unrelated content. This may explain the relative lack of group work in our sample. Many teachers may feel reluctant to encourage verbal communication among students. If students are easily distracted, teachers may reckon that there is more to lose than gain by encouraging talk among students.

  8. 8.

    Such covariates includes city, grade, date, timing of ESM signals, outlier teacher, seat type, seating choice, difficulty level of instructional content, newness of information, classroom conversation, and various perception indicators, including levels of cooperation, competition, sleepiness, confusion, and relevance of class to everyday life, college, and future jobs.

  9. 9.

    This pointed was raised by Professor Kenneth Addison as he was interviewed about the findings of this paper by an Internet-based magazine, Catalyst Chicago. Temkin, J. (2004). “Student engagement varies by race and type of instruction.” in Catalyst Chicago.