Introduction

Research on implementation of school-based health promotion and prevention curricula makes it clear that fidelity and quality in delivery impacts students’ acquisition of the lesson content (Felder and Spurlin 2005; Resnicow et al. 1998) and outcomes (for review, see Durlak and DuPre 2008). Even “evidence-based” drug prevention programs (i.e., those proven efficacious in other settings), which are now estimated to be delivered in 47 % of the nation’s public middle schools (Ringwalt et al. 2012), may have little or no effect if implemented poorly (Derzon et al. 2005). Despite this knowledge, the research community is just beginning to report ways in which implementers stray from the program design (e.g., Ringwalt et al. 2004a, 2004b). Thus, to maximize the effectiveness of evidence-based programs, scholars and practitioners need a comprehensive understanding of how curricula are being implemented.

The current study seeks to address this issue by describing teachers’ implementation of keepin’ it REAL, an evidence-based, 7th grade substance use prevention curriculum. The scope of this study is limited to enable a detailed and nuanced description of classroom-based implementation. Specifically, this study investigates how implementers deliver the curriculum (teacher behaviors) and how recipients react to the implementation (student behaviors) in an effort to identify and describe variation in classroom-based implementation. Informing this study are several lines of research including education, education psychology, prevention science, and community psychology. These different fields of study each inform knowledge of delivery methods, teaching styles, and consistency of delivery.

Delivery Methods

Research on delivery methods shows that promoting interaction is key to stimulating positive program outcomes. This finding applies to both behavioral outcomes, as shown in a meta-analysis of prevention programs (Tobler et al. 2000), as well as academic learning outcomes, as shown by the review of process–product research on teaching (see Brophy 1986). For example, student-centered management—a classroom management style marked by strong teacher–student relationships and shared control of the class by teachers and students (Freiberg and Lamb 2009)—tends to encourage and facilitate discussions (Ennett et al. 2011; Freiberg and Lamb 2009). In addition, encouraging positive student involvement (e.g., high autonomy supportive behavior, Reeve and Jang 2006) and managing the classroom improves students’ behavioral outcomes (e.g., decreased marijuana use, improved ability to resist peer pressure, improved social competency) (Giles et al. 2008; Hansen et al. 1991; Harachi et al. 1999). Some studies that incorporate student behaviors into the measurement of program delivery also find that student centered delivery is associated with increasingly negative alcohol expectancies (Sloboda et al. 2009) and reductions in problem behavior (Hirschstein et al. 2007). These studies demonstrate that more interactive delivery techniques as well as active student participation accrue measurable changes in student’s risk and resiliency levels.

Although different theories place emphasis on different aspects of the classroom, it is clear that both teachers and students contribute to an interactive learning environment. Examining the classroom through the lens of self-determination theory, Reeve (2012) reviews research that explores how self-reported student engagement mediates effects of motivation on student achievement. Fraser (1998) reviews research on student perceptions of learning environments, including studies that show positive associations between student outcomes and perceptions of cohesive, satisfying, and goal-directed learning environments; conversely, classes high in disorganization and friction were negatively associated with desirable student outcomes. These reviews point to the important role students play in learning and suggest that when students perceive a well-managed classroom they will be more involved. In particular, recent findings reviewed by Reeve (2012) suggest that rather than only consider motivation, which cannot be seen, it may be more important to focus on overt behavior since motivation’s effects are mediated by student engagement.

Emphasizing the teachers’ role, Pianta et al. (2012) posit that interactive delivery hinges on teacher–student relationships in the form of emotional supports, classroom organization, and instructional supports. Building on a strong foundation of classroom observational research (e.g., Brophy 1986), each domain of relationship is further elucidated through dimensions of the classroom environment (e.g., climate, productivity, concept development), which are measured in observational studies by various behavioral indicators (see Pianta and Hamre 2009). Labeled the CLASS system, it has been described as a “theoretically driven” and an “empirically supported” framework (Pianta and Hamre 2009). Although the CLASS system acknowledges the role of the student, each domain of the system (emotional support, classroom organization, and instructional support) depends, primarily, on teachers’ behaviors in the classroom.

The theories and studies reviewed here are complementary—recognizing the role of teachers and students but with different emphases. These studies point out that interactive delivery is a product of both teacher and student behaviors. This literature suggests a focus on teacher behaviors such as classroom management, positive encouragement versus sarcasm and negativity, and level of control versus autonomy granting (Freiberg and Lamb 2009; Pianta and Hamre 2009; Reeve and Jang 2006) and student behaviors such as engagement (Reeve 2012). In addition, these lines of research show that classroom environments can be reliably observed (e.g., student engagement; teacher behaviors). To investigate how teacher and student behaviors are integrated into a climate of learning for prevention education, this study seeks to accomplish the following:

AIM 1: Describe the range of teachers’ classroom management and interactive delivery behaviors during the implementation of a prevention curriculum, and

AIM 2: Describe the range of student behaviors exhibited during the implementation of a prevention curriculum.

Teaching Styles

In addition to looking at interactive delivery, education researchers have proposed that teachers tend to exhibit different teaching styles, which are characteristic ways of interacting with students. These styles transcend classes and classrooms; they characterize the teacher’s generalized response tendency. Some researchers have conceptualized teaching styles as similar to Baumrind’s (1973) taxonomy of parenting styles delineating authoritarian, permissive and authoritative teachers (e.g., Paulson et al. 1998; Walker et al. 2006; Wentzel 2002). Authoritarians are those with moderate to high levels of control but low levels of responsiveness, permissives are those with low control and low responsiveness, and authoritatives are those with moderate to high levels of control and high levels of responsiveness. Empirical research shows that authoritative teachers buffer negative influences of deviant peer effects in class (Hughes 2002) and also positively impact students’ engagement in healthy behaviors (Edmunds et al. 2008). Studies support the notion that authoritative teachers create an “optimal context for student engagement and learning” (Walker 2008, p. 237). Studies on teacher styles, then, suggest that instructors tend to deliver curricula within a particular style and that these styles matter for student outcomes. Given the evidence that teachers interact with their students in patterned ways, this study also seeks to accomplish the following:

AIM 3: Describe patterns of interaction between teacher and student behaviors.

Consistency of Delivery

Regardless of how a program is implemented, an important consideration for delivery of prevention curricula is the level of consistency of teacher implementation practices. Previous research, for example, has shown that even when teaching the same program, teachers within the same school do not implement with the same degree of fidelity (Beets et al. 2008; Rohrbach et al. 1993). This finding raises a question about whether patterns of teacher–student interaction differ when teachers deliver the same curriculum to multiple classes of students. Some evidence tracking implementation fidelity suggests that teachers do adapt over time, usually with a decline in the amount of material covered (e.g., McCormick et al. 1995). In a recent study evaluating program fidelity over 3 years, Ringwalt et al. (2009) also found that implementers altered delivery from year to year with a tendency to regress toward a mean level of fidelity: Teachers with greater levels of adaptation in the first year, adapted less over time and teachers who adapted little at first, adapted more over time. These researchers conclude that developing mastery over a new curriculum is an idiosyncratic and personal process (Ringwalt et al. 2009). Although we found relatively few studies that assess the consistency of delivery, those reviewed here all consider fidelity from year to year rather than from class to class. Little is known about teacher behavior when implementing curricula with multiple classes of students during a single school year, which also may be an important consideration. For example, do teachers exhibit a consistent delivery style regardless of the class or do they customize their teaching to each new group of students during a single implementation trial? To address this issue, the fourth aim of this paper is to:

AIM 4: Describe how patterns of teacher–student interaction differ and remain consistent when teachers deliver the same curriculum to multiple classes of students.

Methods

This implementation study is part of a larger evaluation of the keepin’ it REAL (kiR) middle school substance use prevention curriculum. An overview of the process involved in developing the curriculum can be found in Colby et al. (in press), and Miller et al. (2000). The original curriculum proved effective in reducing substance use in a randomized clinical trial (Hecht et al. 2006) and, as a result, came to be listed as an evidence-based intervention on the National Registry of Evidence-based Programs and Practices. After adoption by D.A.R.E. America, kiR is now believed to be the most widely disseminated curriculum of its kind, reaching over 250,000 U.S. middle school students each year as well as those in 23 other countries. Given its widespread dissemination, it is particularly important to understand how kiR is implemented.

Sample of Schools

Teachers in the sample used in this study taught in 39 rural schools across Ohio and Pennsylvania. Schools were randomly assigned to one of three conditions: a control condition where teachers continued their routine prevention efforts (n = 14), a rural condition where teachers delivered a version of kiR curriculum adapted for rural students (n = 14), and a classic condition where teachers delivered the original version of the kiR curriculum (n = 11). The 25 treatment schools ranged in size from 194 to 1,087 students (M = 552, SD = 272). The kiR curriculum was implemented with 7th grade students in elementary (n = 4), middle (n = 7), and high schools (n = 14). The mean number of 7th grade students was 99 (SD = 59) with a range from 27 to 226 7th graders (school data available from NCES Common Core of Data, 2006–2007 school year: http://nces.ed.gov/ccd).

Teachers and Training

Teachers in the intervention (i.e., rural and classic) conditions (n = 32) received a detailed curriculum manual and kiR training during a standard one-day workshop conducted by the site liaison and project staff. The training had six components: (a) an overview of research on youth drug use; (b) the curriculum model; (c) evidence of the effectiveness of the kiR program; (d) design of the kiR curriculum; (e) instruction on and practice in using the lessons; and (f) guidance in research-related activities including provision of digital video recording equipment and practice in its use.

Observational Recordings (Video Data)

Teachers were asked to video record each of the 10, 40–45 min lessons for each class they taught. Teachers were instructed to set up the camcorder at the back of the room so the video would capture their movements and were provided a $10 incentive each time they: (a) returned the video in a self-addressed, stamped envelope, and (b) completed an online questionnaire about lesson delivery. There were a total of 78 classes resulting 780 possible videos. Some videos, however, were never returned (n = 82), even after following-up with teachers. Others were returned, but had no video or audio data (n = 10). Both these types of videos were considered missing. Still other videos were returned but were missing audio data (n = 59). A major contributor to the missing data was a single teacher who was assigned 50 videos and only returned 15, none of which had audio data. In total, 688 videos were received from teachers, a response rate of 88 %, with 624 or 80 % of these containing both audio and video data. All videos were uploaded into Nvivo 8, a qualitative data management software program.

Selection of Videos

With such an extensive corpus of video data and only limited resources available for coding, videos were randomly selected for coding. The first or last lessons delivered were not selected because they were potentially atypical. Also, four of the ten lessons contained a video teaching component. Thus, we decided to code two lessons that included teaching videos and two that did not for each class. We then generated patterns of numbers for all possible permutations for selecting eligible lessons (e.g., 2, 3, 4, 5; 2, 3, 4, 6; 2, 3, 4, 7, etc.). This resulted in 36 patterns of numbers which were assigned to individual classes so that, for example, teacher A, class 1 was assigned pattern 1 where lessons 2, 3, 4, and 5 were coded whereas teacher A, class 2 was assigned pattern 6 where lessons 5, 6, 7, and 8 were coded.

Patterns were then randomly assigned to classes making sure that each pattern was used an equal number of times (or as close to equal as possible). When videos assigned for coding were missing or had no sound, we selectively reassigned whole patterns of numbers to classes. This maintained the integrity of the data already coded while maximizing the amount of data generated per class. That is, reassigning patterns allowed coding of as many complete audio + video recordings as possible per class. The random assignment procedure provided a representative sample that would allow extrapolation of coded videos to the entire video dataset.

Each of the 36 patterns of four videos was assigned twice and four different patterns were used three times, assigning a total of 76 patterns and a total of 304 videos for coding. Patterns were not assigned for two classes where the teacher did not return any observational recordings. Using a balanced assignment procedure (i.e., using each pattern an equal number of times) allowed coding almost equal numbers of each lesson across the entire dataset while simultaneously allowing coding of four observational recordings of each class—two a video component and two without. This balanced approach to video selection maximized the kinds of questions addressed about implementation of keepin’ it REAL. For the current study, the sample is restricted to include only those recordings that contained both audio and video data (n = 276 videos for 31 teachers and 73 classes).

Video Coding

Coding procedures were developed to maximize data generated from video coding and were administered by a team of six coders. Coders provided both quantitative ratings of various implementation dimensions (adherence, adaptation, engagement, delivery, and quality) as well as qualitative descriptions of three main aspects of the lessons (curriculum adaptations, use of narrative conventions, and teacher and student engagement). Data for this study come from qualitative descriptions of teacher and student engagement.

Coders participated in training, ongoing coder meetings, and periodic coder reliability checks. Training involved study of the coding manual plus 8–10 hours of didactic and interactive practice and feedback. Coders also underwent practice operating Nvivo 8 analysis software. Inter-coder reliability (Krippendorff’s alpha (see Hayes and Krippendorff 2007) of .80 was reached for quantitative ratings of practice videos and coders participated in recording qualitative descriptions of at least two videos with trainers prior to the outset of coding selected videos. After acceptable reliability was reached, each coder rated videos independently. In addition, coders participated in weekly or bi-weekly meetings to conference findings and settle coding disagreements. On four different occasions during the coding process, reliability of quantitative ratings was re-assessed by randomly selecting videos for double coding (i.e., coding by at least two coders) and computing Krippendorff’s alphas with the following degree of agreement: .94,.93,.84,.92. Together these coding practices helped maintain data integrity, prevent coding drift, and diminish coding bias.

Research Notes and Summaries

Qualitative data for this study took two forms. First, coders watched the video and entered qualitative descriptions (i.e., research notes; Emerson et al. 1995) about three main aspects of the lessons: curriculum adaptations, use of narrative conventions, and teacher and student engagement. Research notes were linked to particular portions of video demarked by beginning and ending time stamps. Coders watched a particular portion of the class, entered the relevant time stamps, and completed notes on that portion of the class before moving to the next portion. In this way, unclear, ambiguous, or inadequate descriptions could be elucidated by re-watching the actual video potion being described in coder observations. This system also limited the amount of video being observed at once. Instead of writing research notes on an entire 40–45 min lesson at a time, coders sequentially described discrete video segments of only a portion of the entire lesson.

Second, coders completed research summaries of the entire lesson. These summaries included information on the physical and social context of the lesson, comments on teacher delivery of content, student engagement with content and behavior during the class session, and any noteworthy or novel aspects of the class. Research summaries provided information about teaching practices and engagement as well as student attention and participation.

Inductive Analytic Strategy

Analysis of qualitative data in this study involved two phases. The preliminary phase occurred during the coding process and the substantive phase took place after completion of all research notes and summaries for selected videos.

Preliminary Phase

As the videos were coded, emerging patterns of implementation were identified in discussions of research notes and summaries during coder meetings. Coders shared observations or noteworthy findings from videos, including salient dimensions of classroom delivery, such as teacher classroom management and student participation tendencies. As more videos were watched, these patterns of implementation were further discussed in coder meetings. Patterns were expanded or revised iteratively to account for new insights. This process continued until patterns stabilized and no new insights were gained (i.e., saturation).

Substantive Phase

Once all videos were coded, authors analyzed research notes and summaries to address the four aims of this study. Categories identified in the preliminary analysis were clarified through examination of research notes and summaries to form a final coding scheme for prominent teacher and student dimensions of delivery. The final coding scheme then was then used to code the research notes and summaries for each class. Rather than look at only one lesson of teacher/student interaction at a time, all four lessons for each class were analyzed together. This procedure helped triangulate the findings and provided a reliable sense of how to categorize each class. In the end, each class was categorized into a typology based on inductive analysis (Thomas 2006) of the final set of research notes and summaries.

Results

Findings are organized around the four study aims. The first section overviews prominent teacher (AIM 1) and student (AIM 2) dimensions of delivery. The second presents a typology of implementation styles based on patterns of interaction between these two dimensions (AIM 3). These analyses use the entire set of coded videos (N = 276). The third section examines what happens when teachers deliver the curriculum to multiple classes of students (AIM 4) using data from teachers who delivered the curriculum to more than one class.

Dimensions of Delivery

Two dimensions of delivery, teacher control and student engagement, emerged in the coding. Examples of classrooms are provided to illustrate variations within each dimension.

Teacher Control

The first key dimension is teacher control, including teachers’ management and power, tolerance for noise in their classroom, comfort allowing students to work in groups, and willingness to allow student role plays. The low end of the teacher control continuum is anchored by passivity and the high end with strict control. In the middle were teachers who coordinated their classroom, integrating elements of both low and high control.

Passivity refers to teachers with low levels of control directing their class, not the student level of activity. This is exemplified by teacher 02Footnote 1 who merely introduced a topic or activity and then allowed student discussion, whether on topic or not. A research summary described: “[The teacher] … read directly out of the curriculum the entire time. … [He] only cared if the students were having fun [during activities], and the main points were completely lost” (0203L9 RS). This teaching was classified as passive because the teacher opted to maximize time spent in activities (whether or not his implementation of them reinforced learning objectives), and tended to abdicate control of the class to his students. Another teacher (01) was described as passive due to his lack of involvement with the class. Summarizing an entire lesson, a coder wrote: “Teacher allotted most of time doing the activity” (0105L9 RS). This teacher typically assigned students to work on one or two selected activities from the curriculum. Although students were attentive to the lesson, the teacher provided minimal direction to the class and was therefore passive. Another class low in teacher control took place in a cafeteria (03). Partly due to the setting (large room, poor acoustics, immovable seating, about 45 students present), the teacher had little control over the class. Student constantly engaged in side conversations while the teacher presented material. When the curriculum called for group activities, the noise level increased dramatically and the teacher had little control over the activities. Passive teachers in this sample did not use either student- or teacher- directed classroom management techniques.

Conversely, teachers at the strict end of the continuum kept control on their classes’ activities at all times. Lecture was the primary teaching method for these teachers, sometimes integrating student responses but always maintaining tight control of the class. For example, one teacher was described in different lessons as using “lecture format most of the time” (04 L8 RN) and “mainly lecture centered” (04 L4 RN). A research summary for another strict teacher records that she held “tight control over the classroom, often interrupting her lesson to call out students for talking or making noise” (0504L7 RS). For yet another teacher, “the main tool of teaching was lecture, and shallow discussion was often integrated with her always leading” (0601L8 RS). The fact that the teacher was “always leading” clearly demonstrates this teacher’s control of her classroom. Teachers toward the high end of the control continuum limited the students’ movement around the classroom as well as their noise level. Strict teachers tended to use teacher-centered management techniques.

Coordinated teachers maintained control over their class while also granting students autonomy to complete activities. Teachers who coordinated their classes allocated time for presentation of material through lecture and discussion and also practice of material through class activities. One teacher who expertly managed class control (07), for example, shifted fluidly among various curriculum activities. She led the class in lecture and discussion, transitioned into group work, and then regained attention to debrief the activity (e.g., 0705L3). Coders described another coordinated teacher this way: “Teacher… read off the [curriculum, but]… rephrased the wording after reading it … and repeated [content]… so that students could better understand the contents” (08L6 RS). Coordinated teachers managed class activities by being well prepared to teach and in control of their classroom without being overbearing. These teachers also tended to use student-centered management techniques (Freiberg and Lamb 2009).

Student Engagement

While teachers clearly played a central role, students’ behavior during the implementation also was considered. Not all students were equally engaged during curriculum delivery. They exhibited differing levels of engagement which ranged on a continuum from disconnected to attentive to participatory.

Disconnected students seemed to be involved in personal amusements, not class or content-related activities. These students often exhibited disinterest in class material, putting their heads down on desks or taking part in off-topic side conversations. They sometimes participated in class activities, especially those that involved group work, but during activities socialized with one another rather than performing assigned tasks. An example of this type of behavior was seen in the class that took place in the cafeteria: “When they began an activity, students were not immediately involved. They simply milled around, talked among themselves, and generally did not pay attention to the teacher” (03L5 RN). In another class (09), students talked among themselves while the teacher organized her paper-work. Research notes record, “Talking continues without teacher intervention. Teacher says, ‘Get this done guys. No talking,’ but students continue speaking at previous volume without even acknowledging teacher’s comment” (0902L5 RN). Signs of disconnection included talking while the teacher was talking, ignoring the teacher’s requests for participation, and general disinterest in curriculum material and class activities.

Attentive students generally focused on the lesson material and the teacher but without being fully engaged. Although at times some of these students seemed to be distracted, across the entire lesson they seemed to respect the teacher and the enterprise of teaching enough to pay attention. Attentive students were described in this way: “Overall, students were attentive and participated, but not at a very high level” (10L5 RS), and “At least two kids [laid] their head on their desks, but there was an average amount of participation” (1103L8 RS). Those in another class were described as attentive because they gradually increased their level of participation during the lesson: “Students at first were not participating. … Towards the middle and end of the lesson more students were willing to volunteer and give an answer” (1201L2 RS). Another class was described as engaging some students, but “a handful of students [did] not raise hands or try to answer questions” (1301L8 RN). Perhaps the quintessential aspect of an attentive classroom was that curriculum activities gained some, but not total, class participation although without the inattention exhibited by disconnected students. In the majority of lessons we observed, students were classified as attentive.

Participatory students were fully engaged in the lesson. These students “were very active in participating and contributing to the discussion” (14L8 RS). One class was described as “one of the best class[es] in terms of the lesson contents and student participation” because “students … well participated in class” (15L4 RN). Most students in another class were described as “very responsive to the discussion and participated in class nicely” (15L6 RS). Having students who exhibited sustained participation in a variety of class activities typified classes categorized as participatory.

Typology of Classroom Environments

This study is not only interested in teacher and student dimensions in isolation but also in how they jointly create and maintain a particular pattern of teacher–student interaction (AIM 3). Conceptually, it might be possible that all types of teachers also encountered all types of student response. However, upon close examination of research notes and summaries and in discussion with coders, a general pattern emerged which best described the observational data. On the whole, students in each of the three teacher control categories varied around one or two of the three engagement categories such that in classrooms where teachers were passive, students tended to be disconnected or attentive. In classrooms with coordinated teachers who managed time well and had high energy, students tended to be attentive or participatory. In classes where teachers were strict students were attentive.

Delivery Patterns across Multiple Classes

We also investigated how teachers and students jointly create classroom environments (AIM 4). Examining this research aim first involved describing the five patterns of teacher–student interaction identified in the typology and then assigning each class into one of the five patterns. The number and percentage of classes and teachers in each category reported in Table 1. Summing across patterns shows the proportion of teachers or classes classified into each teacher or student delivery dimension (e.g., passive, participatory). For example, teachers were normally distributed across their level of control (26 % passive, 48 % coordinated, and 26 % strict) whereas 68 % of classes were in the student attentive category. Individual teachers in each pattern were identified, and, for all cases where teachers delivered the curriculum to multiple classes, teachers were analyzed to learn how they delivered the same curriculum to different groups of students.

Table 1 Number of teachers and classes assigned to each category of teacher engagement/student participation
Full size table

Passive/Disconnected

When teachers passively implemented the curriculum among mostly disconnected students the class was labeled passive/disconnected. Although this delivery pattern was observed for relatively few classes (7 %), it is a particularly dysfunctional environment and thus important. This environment is typified by the two classes taught by one teacher (0901 and 0902).

In this exemplar case, the teacher passively taught both classes where students were described as highly inattentive (e.g., talking while teacher talked, off-topic discussions). Moreover, the teacher seemed unprepared even when teaching the lesson for the second time. This teacher exhibited a consistently passive teaching style with consistently disconnected students. Clearly, future prevention and educational research should establish if this dysfunctional environment reflects teachers who should not be in the classroom or who need further training.

Passive/Attentive

Not all passive teachers had disorganized classes. There also were fourteen passive/attentive classrooms (19 %) taught by five different teachers. This style was typified during one activity when research notes report that the teacher allowed students to “get up and walk around and share with everyone, rather than partnering and sharing between two students. Gets a little loud and disorganized, but the students seem to still be on task for the most part” (0901L8 RN). The teacher’s passivity also was expressed through minimal class discipline or direction to participate or be quiet and also through reallocating time that could have been spent on presentation of material to activities which maximized student participation. For example, one research summary recorded the amount of time one class session spent in lecture versus activity.

Teacher did not spend 10 min [allotted in the curriculum manual] to discuss the concept of risks. He briefly touched on the idea around 4 min but eliminated the discussion of risks (e.g., benefits of identifying risks and risks involved in obvious situations). Teacher spent most of the class time (31 min out of 41 min) for the activity (0105L2 RS).

Another notable aspect of teacher–student interaction was the relaxed and friendly rapport the teacher and students maintained with one another in all sections.

For one teacher (01), the classroom environment did not differentiate substantially among either classes. All classes seemed to be organized such that the majority of class time was allocated to one or two student-directed activities. This teacher consistently called for participation from students, who responded with attentive levels of participation. Thus, the teacher exhibited consistency in teaching style across different classes and each class exhibited a similar level of participation. Comparing the passive-disconnected case with the passive-attentive case, the teacher’s rapport with students seemed to be the primary difference. The positive teacher–student relationship in the latter appeared to be the key factor in facilitating attentiveness in these classes. What is not clear, however, is whether the teacher adopted the more passive style only after establishing the rapport. This has implications for future prevention as well as educational research.

Coordinated/Attentive

The coordinated/attentive group, which included 35 % of teachers in 25 % of classes, is illustrated using an exemplar case of one teacher (12) with two classes. Both classes fit into the coordinated spectrum of teacher control and of the attentive spectrum of students’ behavior.

In both classes the teacher closely followed the curriculum manual engaging in a variety of lectures, discussions, and activities. She gave students freedom to participate in role plays and work independently as well as complete some material as a class. In research notes for both sections, the teacher was described as “thoroughly” covering lesson content (i.e., 1201L9 RN; 1202L2 RS). Apt for coordinated teachers, one coder describe that this teacher “managed the class time and its activities well” (1202L9 RS). The effective “balance” (1202L9 RS) between high and low teacher control was seen equally in both classes. Although coordinating the class, this teacher did not show overt enthusiasm or high energy in either class during curriculum delivery. Research summary 1201L6 related that the teacher “was not enthusiastic about lesson.” Yet, “despite her boredom, students were pretty quick to participate.” In research summary 1202L2, the coder wrote that the teacher “did not look… enthusiastic about teaching the lesson.” This teacher’s enthusiasm and energy during delivery was moderate in both classes despite her undifferentiated, coordinated level of control.

Students in both classes were attentive. Owing in part to the teachers low energy level, students in class one “showed boredom, but for the most part were pretty attentive” (1201L6 RS). In class 2, students “paid attention and complied with teacher instructions” and “responded to teacher questions” (1202L9 RS). Students in another lesson “seem to pay attention, but they hardly participate. They seem sluggish. About one student responds per question” (1202L7 RN). Later in the lesson, while participating in role plays, the class “really engages in a discussion of how other characters feel, what [characters] could do differently [in the situations presented], [characters’] communication styles, etc.” (1202L7 RN). These examples show that, on the whole, students in both classes were attentive.

Coordinated/Participatory

One quarter of the classes we observed, taught by four different teachers, were categorized as coordinated/participatory. This classification was exemplified by a teacher who taught the curriculum to three different sections of students. All sections seemed to participate well and the teacher seemed to exert the same level of control equally for her classes. This teacher was heard praising students in each class with phrases like “good job,” and “excellent.” She empowered students to work together in groups or on paired activities by giving them uninterrupted time to complete activates. Coders also reported that the teacher covered curriculum material, especially discussions, thoroughly. Students in each of the three class sections responded equally well. During one class, the teacher reminded students of a “class rule” that “participation is big thing in our class” (1703L2 RN). Research notes record that the teacher “strongly encourages students to speak up and thank students to participate” and “students actively participated in discussion” (1703 RN). Coders noted that participation in class discussions seemed distributed among “several” students, rather than just a few who participated (1702L6 RN). This kind of involvement was common across all of her classes of students, exemplifying a coordinated/participatory class.

Strict/Attentive

Twenty-five percent of all classes were categorized as strict/attentive. One teacher (05) who delivered the curriculum to five classes was a powerful example of this pattern. Across different classes of students and across lessons this teacher was recorded as holding a “tight reign.” This teacher, compared with others in this sample, was hypersensitive to noise in the classroom, often interrupting her own lecture to call on specific students to be silent (e.g., 0505L7). Students were not highly participatory, the teacher’s classroom management precluded student participation in many instances; however, students did answer questions and comply with the teacher’s directives. Thus, students remained on-task, but never were highly participatory.

Our analysis of research notes revealed that this teacher was undifferentiated in her teaching style—she treated all five classes similarly. Moreover, students in all five classes were attentive throughout the lessons even though not highly participatory. There were some differences in teaching and student participation from class to class, but these were minor variations on the overwhelming theme.

Summary

This analysis revealed overall consistency in classroom environments when teachers taught the curriculum to different classrooms of students. Across different classes of students and across lessons, teachers tended to exhibit the same level of control, seemed to omit or adapt the same curriculum components, and generally taught in a similar fashion. Additionally, each unique grouping of students, although slightly different in terms of class “personality,” generally responded similarly to the constraints of the classroom set up by the teachers. We label this consistency across lessons and across classrooms of students undifferentiated teaching.

Discussion

Findings from this study provide a working typology of classroom environments, suggesting a teacher-driven delivery model of school-based prevention curriculum implementation that may have important implications for educational practices in general. Observational methods provide detailed descriptions of teacher and student behaviors that expand current theorizing about implementation and teaching practices. Analysis identified two salient integrative dimensions of delivery, teacher control (passive, coordinated, strict) and student participation (disconnected, attentive, participatory) that, in combination, reveal five distinct patterns of teacher–student interaction in the delivery of a school-based prevention curriculum. Together, these patterns serve as a typology of possible classroom environments for implementing prevention curricula. This descriptive typology is based on the relationship between the constructs observed, but does not provide information about the directionality or causality of effects. In fact, there may be mutual causation or influence. When teachers were engaged, students tended to participate. When students were inattentive, teachers tended to be less engaged. However, our observations point to consistent teacher deliver styles which were related to student engagement as well as a teacher style across multiple classes suggesting that teachers’ classroom management shaped the classroom environment.

Given previous research findings that student participation is desired (i.e., leads to higher levels of learning), with engagement more desirable than passivity (e.g., Reeve 2012), a clear ranking of teacher styles and classroom environments emerges. The patterns reported in Table 1 show that passive teacher style at best is associated with attentive students and may even lead to passive student behavior. Strict teachers, on the other hand, always seemed to have attentive students while all of the classes we coded as having participatory students were taught by coordinated teachers, who, at worst, experienced attentive student behavior. This suggests a hierarchy in teacher style from coordinated to strict to passive that seems to reflect general educational practices describing authoritative, authoritarian, and permissive teachers (e.g., Paulson et al. 1998; Walker et al. 2006; Wentzel 2002). Our labels, however, refer to general classroom management and are not based solely on dimensions of responsiveness to students and control. For example, our coding of teacher control included teachers’ management and power, tolerance for noise in their classroom, and comfort allowing students to work in groups. While these behaviors may overlap with teaching/parenting style practices (e.g., rule setting; Wentzel 2002), our observation was not conceptually based on these dimensions.

A second indication of the teacher-centeredness emerged from our analyses related to Aim 4. Teachers who taught keepin’ it REAL more than once tended to exert similar levels of control in delivering the curriculum across every class. In addition, their students, for the most part, exhibited consistent levels of participation in every class taught by a single teacher and, as well, based on the teacher’s practices. Despite different lessons and different classrooms of students, teachers created similar classroom environments. These findings make an important contribution to the field of implementation science and school-based prevention, and potentially, to education in general, by illuminating three broader issues facing implementation researchers: interactive delivery, implementer support and screening, and implementation fidelity.

Promoting Interactive Delivery

Research in prevention science as well as education is clear: interactive teaching accrues benefits for students (Brophy 1986; Felder and Spurlin 2005; Resnicow et al. 1998; Rohrbach et al. 1993; Tobler et al. 2000). Our findings, once again support this general observation and, at the same time, suggest how such an environment is created. Based on our observations and analyses, we postulate that when teachers balance student-directed learning (e.g., activities, role plays) with teacher-directed instruction (e.g., lecture, demonstration), students showed a higher range of participation. Conversely, when teachers more fully aligned with either passive or strict teaching, students participated less on the whole. Creating a highly participatory, interactive class happens when teachers exhibited moderate levels of control, allowing students some freedom while coordinating class activities, lectures, and discussions. This style of control mixes teacher-created structure with student autonomy granting in a delicate balance. Less structure with high autonomy tends to create a passive style that may be too chaotic for effective learning while high structure and low autonomy may create a stifling environment for both prevention and general educational practices.

Our descriptive analyses—while precluding causal statements—aligns with existing research. For example, Jang et al. (2010) demonstrate that classroom environments which grant students freedom and structured activities promote student engagement. In general, research indicates that organized and well managed classrooms with reasonably high levels of student autonomy create more productive learning environments (Brophy 1986; Fraser 1998; Pianta et al. 2012; Reeve 2012).

The striking similarities in student participation from one class and the next when taught by the same teacher, suggest to us that the teacher is the dominant influence on the norms and flow of a classroom. The teacher sets up constraints around what behaviors and levels of participation are acceptable. Students appeared to be motivated to comply since the teacher holds authority to levy detentions, assign extra homework, or decrease course grades. Teachers utilize different strategies to enforce these norms. A strategy used most typically by strict teachers in this sample was to censor students who did not comply with class norms. Coordinated teachers were more likely to grant their students reasonable levels of autonomy, which, according to self-directed learning theory enhances not only student participation but also learning (for review, see Reeve 2012). A third strategy was to use teachers’ relational authority (Bingham 2008) to influence students. Research suggests that developing a positive teacher–student relationship is an important aspect of classroom management (Cothran et al. 2002; Emmer and Stough 2001; Pianta et al. 2012) and that if students do not respect the teacher or the material, they likely will not participate (Bingham 2008). Our observations lead us to conclude that this rapport enhances every environment, even for passive teachers although our findings suggest that teachers used strategies to establish norms for participation that tend to align with the level of control they display in the classroom.

In the end, these findings support a teacher-driven model of implementation where teachers heavily influence the environment for learning the prevention curriculum. Future research is needed, however, to statistically examine this over time in order to explore the interactions of student and teacher contributions to classroom environments when implementing classroom-based prevention curricula. At the same time, they suggest a number of additional directions for future research and practice.

Supporting and Screening Teachers

The finding that teachers create a set of constraints within which students vary their engagement adds to extant research demonstrating the crucial role of teachers and classrooms in affecting student outcomes (see Brophy 1986; Pianta et al. 2012). Because research suggests that training alone is insufficient for maintaining program fidelity and improving outcomes, some additional investment in delivery personnel seems warranted. By implication, this finding underscores the importance of staff selection and delivery support (e.g., training, coaching) for improving implementation of prevention curricula (Fixsen et al. 2005).

Training coupled with ongoing support, such as coaching (Dusenbury et al. 2010), is one important way of promoting effective curriculum delivery. This and other studies suggest that training and support should promote interactive and facilitative implementation styles along exerting control through student-centered classroom management techniques. A few practices identified in the current study include promoting participation by selecting students’ names at random from a stack of index cards, transitioning efficiently from one activity to the next, clearly articulating lesson objectives and over viewing lesson content, and sharing personal narratives. These practices were typically, but not exclusively, employed by coordinated teachers. Training and ongoing support might suggest such practices. The video recordings collected in the current study, moreover, provide valuable exemplars of these implementation strategies for future intervention training. Recent work on teacher professional development has shown improvements on specific domains of teaching behavior when teachers are provided personalized coaching based on observations of their teaching performance (for reviews, see Danielson et al. 2007; Kratochwill et al. 2007; Pianta and Hamre 2009). What is less clear is if or to what extent professional development can alter teachers’ style, an issue which needs exploration in future research. Utilizing this method of enhancing delivery requires an ongoing investment in teacher development beyond initial training.

Another possible method for improving implementation is finding teachers who have a proclivity toward interactive delivery. This might be done by developing effective (and efficient) screening tools for existing teachers or by having prevention curricula taught by outside experts already skilled at promoting student engagement (e.g., prevention support personnel, D.A.R.E. officers). For example, a simplified version of the coding system developed for this study might be used as a screening tool if it can be reliably administered by a principal, school counselor, or other third party observer. Using this method of enhancing delivery requires an initial investment of time and resources to screen teachers. Such an investment in staff selection may be prohibitive in some school contexts which are already overburdened; however, this may be a viable option in other cases and may also prove more cost effective or sustainable than allocating resources to ongoing support and professional development. D.A.R.E.’s keepin’ it REAL, for example, allows for screening of officer implementers and the current findings may prove particularly useful in that implementation setting.

In real-world teacher-led drug prevention interventions, teachers are frequently assigned to programs without thought given as to whether they are best suited or even motivated to deliver a particular curriculum. Prevention curricula, like many other educational subjects, are often handed down to teachers based on administrative decisions and there are a number of constraints that can affect curriculum delivery, such as limited time and classroom environment (Miller-Day et al., under review). These organizational factors impact what happens within the classroom and require a holistic, ecological view of school-based intervention efforts (e.g., Durlak and DuPre 2008; Wandersman et al. 2008). Screening teachers as well as providing initial training and ongoing delivery support require organizational and administrative backing and which combinations of strategies are feasible and advisable for a particular situation are best determined by local teachers and administrators. On the other hand, an alternative approach (discussed below) is to create adaptive interventions that can be fit to the teacher style while remaining consistent with program philosophy and delivery strategies.

Fidelity and Adaptation

Our findings suggest that teacher-directed classroom environments also may be linked to issues of curriculum adherence and adaptation. Teachers seem to adapt material, not necessarily because of philosophical differences with the curriculum, but in more practical, systematic ways to accommodate their delivery pattern. For example, a coordinated/participatory class teacher might “thoroughly cover material” because the curriculum is written in line with his or her preferred teaching style, that is, a balance of teacher-directed instruction (e.g., lecture, discussion) and student-directed activities (e.g., role plays, group projects). Another teacher, conversely, might omit (or minimize) teacher-directed instruction in favor of student-directed activities based on his or her preferred method of classroom instruction/management. Perhaps, ongoing support can present alternative curriculum choices in order to provide teachers with practices consistent with their own styles. In other words, rather than providing a single uniform curriculum, the intervention itself would be designed with these teacher proclivities in mind. Such a curriculum design might fit under the rubric of an “adaptive intervention” (Collins et al. 2004) with teachers’ delivery style as a targeting variable. For example, for strict teachers, an intervention curriculum might prescribe an interactive class discussion coupled with a demonstration instead of a small group activity. Doing so might still fulfill the instructional aims of the curriculum, but would accommodate various preferences for classroom control. Future research, however, is needed to examine the role of classroom environments (degree of teacher control along with degree of student engagement) in implementation fidelity and whether alternative practices can be developed and delivered without lessening program impact. Similar practices may be possible in curricula for other subjects.

Limitations

Although description is the purpose of this study and it is not necessarily designed to account for all potential confounds, there are some limitations which should be noted as they inform interpretations and inferences drawn from these data. For example, teachers had different levels of teaching experience, something not considered in these analysis. Another limit to our data is that some teachers (n = 7) delivered the curriculum to only one class, selected at their discretion. It is possible that they selected the class because students participated highly or the teacher performed at their peak. Indeed, we did not control for students’ aptitude or motivation, something which may have varied if teachers selected a “gifted and talented” class, for example. For those teachers who did teach multiple classes—some even delivering the same subject to the entire 7th grade population—we saw remarkable consistency in terms of student participation from class to class which suggests that differing levels of student aptitude and motivation may have had little effect on participation at the class level. Finally, videos were recorded at different times of the day, at different times across an academic year, and in different content courses (e.g., Science, Health, Literature). It is possible, for example, that students in Science class tend to participate more than students in Health classes or student participate more in the morning than in the afternoon. Alternatively, it is possible that all Science teachers use a particular style whereas Health teachers tend toward another teaching style. Any of these unmeasured variables (e.g., teacher experience, student aptitude, time of day, course content) could have impacted what we observed and future research exploring teaching style might do well to account for such confounds.

Another limitation of this study is that it does not account for school level differences, which have been shown to impact the program delivery (for review, see Domitrovich et al. 2008). Previous research has identified that differing levels of administrative support and support among teachers, for example, affect implementation of interventions (Gregory et al. 2007). While in much of our data only one teacher from each school delivered keepin’ it REAL, there was one school in which eight teachers simultaneously delivered the curriculum to their respective homeroom classes. Examining this case allowed us to explore potential effects of nested data. Teachers from this school were categorized into all three levels of control: two were categorized as strict, four coordinated, and two passive. This mirrored the distribution of all 31 teachers. Seven of the eight classes of students were classified as attentive (e.g., 08) with one class considered participatory (14). Given that the overwhelming majority of classes were attentive (68 %), the similarity of students’ attentiveness seemed reasonable to expect from any sampling of eight cases.

These distributions suggest that institutional support may matter little in terms of student engagement and teacher style. Indeed, as observed by Domitrovich et al. (2008), for some research questions, “the classroom, rather than the school, may be the appropriate level of analysis” (p. 20). Although only a single case, our data support this claim tentatively suggesting that teacher style varies within a school but student engagement varies less. One possible explanation for limited variation of student participation is that it is regulated by school or cultural norms for behavior. Students may tend toward a school-wide culture of participation regardless of the teacher or may be regulated by societal expectations for students to pay attention to their teacher. Another likely explanation is that students in a particular school generally come from the same, proximate geographic region and therefore tend to share similar levels of parental education and family income. Students in our sample were all primarily Caucasian adolescents who were in 7th grade in rural school districts in two states, demographics mirroring the rural populations of these states. Such demographic similarities also may influence classroom participation (e.g., Caldas and Bankston 1997).

While this section points out some of the possible limitations of this study, which certainly inform the interpretations of results, we note that these limitations are not unique to this research. Instead, we believe, they provide a glimpse into real-world prevention delivery, which often takes place through teachers with varying levels of training and ability, at different times of the day, and on both good and bad days, in a variety of classroom subjects, with students who have differing levels of aptitude, intelligence, and motivation.

Remaining Questions

Finally, this research is heuristic in the sense that it serves to guide discovery and generates questions for future inquiry. Although beyond the scope of this study which purposed to describe classroom implementation in situ, one important question that arises from these findings is to what extent do implementation patterns moderate youth outcomes? Others have suggested that program effects are almost fully obfuscated by poor implementation (Derzon et al. 2005). However, we wonder if there is a “good enough” level of implementation above which program effects are similar for all teachers? For example, if students are attentive, regardless of teacher style, will they exhibit program effects compared to controls? Perhaps the principal of equifinality (von Bertalanffy 1968) applies here—that there are multiple paths to the same ends? As data become available, future research on keepin’ it REAL will consider this issue.

Implementation of evidence-based practices is consequential and our study suggests, like the interactive delivery system (Wandersman et al. 2008) and ecological implementation framework (Durlak and DuPre 2008), that delivery should be considered during program development, implementer training, and in the measurement of program delivery/fidelity. As prevention research begins to address implementation issues beyond simple coding into high and low fidelity (e.g., the type and degree of adaptation), a more nuanced understanding of these translational processes will emerge. Just as in education in general, we need both general principles for effective learning as well as an understanding of how these principles are implemented in the lives of our nation’s youth.