Introduction

American students evince poor academic outcomes relative to students from many other developed countries. In mathematics, for instance, the United States ranks 25 out of 30 nations belonging to the Organization for Economic Cooperation (Baldi et al. 2007), with 15 year-old American students ranking behind the vast majority of their European and Asian counterparts. Further, the US Department of Education (Planty et al. 2009) reported that only 73% of US students graduate from high school and substantial discrepancies exist between students of different ethnic backgrounds. Failing to graduate from high school can result in a multitude of negative consequences, such as limiting occupational choices to low prestige, low wage jobs and a prolonged dependence on social services (Alliance for Excellent Education 2007). Hence, identifying school policies and conditions that may contribute to the academic success of American high school students may yield immediate and long term gains for the both the students and society.

Academic performance is affected by conditions both at home and at school. Although the potential deleterious effects of a compromised home environment on students’ academic performances have been fairly well-established (Hart and Risley 1995; Sampson et al. 2008), there is comparably less known about which components of the school environment impact high school students’ academic success. This is particularly disconcerting given the existence of many school factors that are amenable to policy interventions. Therefore, to promote and improve American students’ success, school policies and characteristics that contribute most to academic achievement must be identified.

Many school policies, despite their common implementation, have received only limited research attention. In particular, schools’ decisions to enact specific teacher evaluation and reward policies have the potential to positively or negatively influence the climate of their classrooms and subsequently, the achievement of their students. This article evaluates the influence of specific teacher evaluation and reward policies on the teacher–student relationship climate and, subsequently, academic performance of US public high school students while controlling for many of the factors associated with the success of students, teachers, and schools. Specifically, this article examines the role these policies and conditions may have on students’ mathematics achievement and dropout between students’ sophomore and senior years of high school. These outcomes are of particular interest because the previous literature has suggested that high school mathematics scores, as well as high school dropout are two of the best determinants of future economic and social well-being (Hoffer 1995; Rumberger 1987).

Teacher Evaluation and Reward Policies

Teacher effectiveness is generally characterized as improving student learning and may be one of the most important factors contributing to students’ academic achievement (Wright et al. 1997). Methods to improve teacher effectiveness include evaluations (e.g., by school principals, students, or other teachers) and rewards (e.g., monetary stipends, accolades, or allowing teachers their choice of classes to teach). Evaluations of the impact of incentives and rewards on teacher effectiveness, however, have been limited in scope. Furthermore, teachers often doubt the validity, reliability, and fairness of these incentives (Malen 1999).

Considerable controversy surrounds both teacher evaluation and reward policies. In public schools around the United States, the vast majority of teacher evaluations are conducted by school principals who typically use checklists or observations to determine teacher effectiveness (Toch and Rothman 2008). Education scholars have argued that this mode of evaluation lacks validity and reliability for several reasons (Noakes 2009). First, the evaluation instruments rely predominately on subjective observations of teaching. Second, principals are rarely trained to conduct teaching assessments and, because of this, their judgments of teachers’ performance may be contaminated by extraneous factors that are unrelated to teaching performance, such as physical appearance (Jacob and Lefgren 2008; Medley and Coker 1987; Noakes 2009). Finally, infrequent observations may not be indicative of typical classroom behaviors; brief evaluations conducted once or twice a year provide only a snapshot of the actual classroom environment. Given the general limitations of all observation (e.g., reactivity) as well as the specific problems identified by Noakes (2009), principal evaluations of teachers are limited in scope and accuracy.

Whereas virtually all public high schools have policies that allow for principals and administrators to evaluate teachers (99%; National Center for Education Statistics, National Center for Education Statistics, 2010), empirical support for the success of these evaluation policies is mixed. For example, some studies suggest that teachers who receive better evaluations from principals and administrators are more likely to have students with higher levels of achievement (Jacob and Lefgren 2008). Conversely, other research suggests that there is no correlation between teacher evaluations conducted by principals and student achievement (Medley and Coker 1987) and that teacher evaluations have little to no impact on quality of education or student learning (Colby et al. 2002). Taken together, the evidence about the relationship between principal evaluations of teachers and student achievement is unclear.

Despite the limited evidence supporting the utility of evaluations conducted by principals or administrators, only a small percentage of high schools have introduced alternative teacher evaluation methods. According to data from the Educational Longitudinal Study (2002), only 7% of public and private schools have enacted policies that permit students to evaluate their teachers and only 13% of schools have policies that allow teachers to evaluate other teachers. As such, very little is known about the potential impact of these policies on teacher effectiveness and student achievement, although research has found that school policies that allow students the opportunity to assess their teachers are associated with improvements in the teacher–student relationship (Manefield et al. 2007) and increased student engagement in school (Cook-Sather 2007).

Controversy also surrounds the rewarding of teachers for their students’ performance. Rather than base teachers’ salaries on their students’ academic achievement or other performance indicators, teachers are more often paid according to their years of experience and teaching credentials. However, teacher rewards, in particular merit pay, have garnered considerable media attention, in part because of President Obama’s support for the policy (Obama 2009). The idea behind “performance pay,” or merit pay, is that teachers whose students achieve particular benchmarks over the course of the academic year are rewarded with monetary stipends.

Those in favor of merit pay suggest that it will motivate teachers to provide effective instruction, with some research finding that merit pay is positively related to student achievement (Podgursky and Springer 2007). In contrast, many teachers object to merit pay policies because, they contend, that student achievement is multifaceted and complicated by several extraneous factors that do not reflect teaching performance (Ballou and Podgursky 1993). For example, because academic achievement is cumulative, a student’s performance successes or failures might be more attributable to instruction from prior teachers than from their current teacher (Podgursky and Springer 2007). Moreover, students’ abilities are heterogeneous, and, therefore, some students will naturally evidence more progress than others. At a particular disadvantage may be teachers whose schools are located in low socioeconomic areas, because many of these students have home and neighborhood environments that are not conducive to academic achievement (Sampson et al. 2008; Ma and Klinger 2000). Finally, some have suggested that merit pay policies may be counterproductive because they can foster jealousy, competition, and cheating among teachers (Malen 1999), and therefore, may adversely affect the school climate.

Though contentious, the effect of teacher reward policies on student performance has been investigated using data from the National Education Longitudinal Survey in conjunction with a survey that assessed the effectiveness of teacher incentives (Figlio and Kenny 2007). Figlio and Kenny (2007) found a positive association between individual teacher incentives and gains in high school student achievement: standardized test scores were higher in schools in which individual financial incentives were provided for teachers whose students evidenced good academic performances. Moreover, the magnitude of this effect was largest in schools with a low mean socioeconomic status, possibly indicating that high school students who come from more disadvantaged backgrounds benefit most from effective teaching practices. The cross-sectional nature of the data, however, limited causal inference.

Similarly, in their review of the research on the effects of teacher rewards on teacher motivation and student achievement, Podgursky and Springer (2007) determined that of the few studies that have examined these relationships, the majority of them suggest that teacher rewards have positive effects on both teacher motivation and student achievement. Moreover, because the literature suggests that teachers vary widely in their effectiveness, they suggest that adopting teacher rewards into the educational system is a policy change worth considering. Because of the dearth of literature in this area, however, they argue that more studies are needed to determine exactly how teacher rewards influence student performance.

Teacher–Student Relationship Climate

School characteristics also may influence student achievement. Researchers have found that schools that foster positive relationships between teachers and high school students have higher math achievement and higher graduation rates (Muller et al. 1999). Similarly, supportive teacher–student relationships have been associated with improved student academic achievement (O’Connor and McCartney 2007) and lower rates of student dropout (Lee and Burkam 2003). Positive teacher–student relationships may foster students’ sense of belongingness in school and promote a warm school climate, which in turn may facilitate students’ academic success through their association with motivational, emotional, and behavioral factors related to students’ school engagement (Vieno et al. 2005). Vieno and colleagues (2005), working with a sample drawn from middle and high schools in Italy, found that the more students were engaged in their education experience, by participating in processes such as the making of rules and the organizing of school events, the more students’ felt connected to their school. Collectively, research on the teacher–student relationship suggests that the greater the opportunity for students to have a voice, the greater the likelihood for positive relationships, which, in-turn, may potentially lead to greater academic success.

The academic performance of high school students likely results from the interaction of several internal and external factors. Students enter the school setting with differing levels of previous knowledge due to their distinct home environments and varying learning experiences (Davis-Kean 2005; Hart and Risley 1995). Furthermore, high school students experience differing academic motivations and aspirations due to the influence of friends and family (Davis-Kean 2005; Wigfield et al. 2006). For these reasons, it is critically important for studies investigating school-level policies to control for these individual factors along with school-level factors, such as school size, socioeconomic composition, and urbanicity, and graduation requirements, as they also have been found to affect high school student math achievement and dropout (Hoffer 1997; Lee 2000; Lee and Burkam 2003; Weiss et al. 2010). By controlling for a comprehensive set of confounders, researchers are better able to identify which, if any, variations in school policy are associated with changes in student achievement.

Current Study

Prior research has found inconsistent support for associations between teacher reward and evaluation policies, the teacher–student relationship climate and student academic performance. In order to test the potential impact of teacher evaluation and reward policies, this study utilized multilevel analytical techniques with a large sample of students from a nationally representative sample of US public high schools. We used a comprehensive multilevel longitudinal design to estimate accurately the effects of school policies. Specifically, this study was interested in measuring whether evaluation policies that allowed students or teachers to evaluate other teachers, and teacher reward policies, such as paying good teachers more, assigning good teachers better students, or rewarding good teachers with special awards were associated with the teacher–student relationship climate and/or later student academic outcomes.

Based on previous literature, we proposed three hypotheses. First, teacher reward and evaluation policies are related to students’ perceptions of the teacher–student relationship (TSR) climate. Second, the TSR climate, in turn, is associated with longitudinal gains in students’ math scores and risk for school dropout. Third, the TSR climate, in turn, is associated with longitudinal gains in students’ math scores and risk for school dropout. Moreover, our study is interested in determining whether associations between teacher policies and student outcomes are mediated by the teacher–student relationship climate. Testing these associations will aid in determining whether the teacher policies outlined above have any direct or indirect associations with student outcomes after controlling for a comprehensive set of individual and school-level covariates.

Method

Public use data from the Educational Longitudinal Study of 2002 (National Center for Education Statistics 2010) were used. The Educational Longitudinal Study of 2002 (ELS) assessed 16,000 students, 750 school administrators, 750 librarians, and parents associated with one of 750 schools. The public use data set included assessments during the students’ sophomore (spring, 2002) and senior years (spring, 2004) of high school. The overarching purpose of the ELS study, conducted by the US Department of Education, Institute of Educational Sciences, was to monitor the transition of a national sample of young people as they progressed from tenth grade through high school and beyond (National Center for Education Statistics 2010).

Participants

Participant data in this study included 7,779 public high school students in one of 431 schools with complete data on all exogenous variables. Twenty-four percent of the original 580 public high schools in the sample were removed from the analyses due to missing data on any one of the school-level predictor variables. An additional 20% percent of the 9,741 students were then removed due to missing data on any of the student-level predictor variables. The loss of school level was largely due to schools failing to answer questions regarding the lowest pay for a teacher at their school (21%) and individuals removed from the sample was largely due to failing to answer questions about their friends’ aspirations (18%). Despite this, the final sample closely represented the demographics of public high school students in the United States as a whole. The students in the sample were 50% female, were composed of a White majority (53% White, 15% Black/African American, 16% Latino or Hispanic of any race and 11% Asian or Pacific Islander) and were, on average, 16 years of age at baseline (sophomore year). Schools were located in urban (25%), suburban (51%), and rural locations (24%). Due to the greater potential for increased levels of heterogeneity within the sample and the specific research questions under study, only public high schools from the ELS of 2002 were included in the analyses.

Measures

Policy status

Whether schools had specific school policies, such as how teacher performance was evaluated and rewarded, were obtained from the administrator questionnaire completed in the spring of student’s sophomore year. Each of the teacher evaluation and reward policies required only a yes (1) or no (0) response from the school administrator completing the survey. Specifically, administrators responded to the following question regarding teacher evaluations: Does your school currently use any of these forms of teacher evaluation?, with options that included teachers evaluate teachers and students evaluate teachers. Administrators also responded to the following question regarding teacher rewards: Which of the following kinds of recognition are given to good teachers in your school (mark all that apply)?, with options that included, special awards for teaching, assigned better students, and higher pay.

Teacher–student Relationship (TSR) Climate

This measure was composed of three items from the ELS of 2002 student baseline (sophomore year) survey: (a) students get along well with teachers, (b) the teaching is good, and (c) teachers are interested in the students. These items were chosen because they best represented the relationship between students and teachers at the school-level. Each item was measured and analyzed on a four-point ordinal scale with response options of strongly agree, agree, disagree, and strongly disagree, with higher scores representing a better teacher–student relationship climate.

Math achievement

Student math achievement was assessed using the IRT-estimated scores during the sophomore and senior years. The math achievement scores included in the ELS dataset are based on items previously used in other national studies (e.g., the National Educational Longitudinal Study, National Assessment of Educational Progress, and Programme for International Student Assessment; US Department of Education, 2006). Math achievement was chosen over other achievement measures because it was the only measure assessed during both academic years and has been shown to be a strong predictor of college success (Hoffer 1995). The item response theory (IRT) mathematics score represents the sum of the probabilities that a student would have correctly answered each item in the 85-item battery of mathematics questions had they taken the entire test in each of the two observational periods. IRT probability estimates are adjusted for item difficulty and the possibility of a student answering correctly because of guessing. Questions covered math content areas of arithmetic, algebra, geometry, data/probability, and advanced topics. All items were field tested the previous year and evaluated using classical item analysis and item response theory (IRT) to examine each item’s ability to discriminate mathematic proficiency, invariance across respondent subgroups (e.g., male and female), and reliability, and were found to possess excellent psychometric properties (Burns et al. 2003; Ingels et al. 2005).

Student dropout

Student dropout was assessed by observing if the student had dropped out from high school as of the 2004 spring term. Students (7%) were designated as dropouts if they had not received a high school diploma or GED or had missed 4 or more consecutive weeks not due to illness or accident.

Covariates

The school and student-level covariates used in this study included numerous demographic and background variables. These included participants’ ethnic/racial background (coded 0/1), socioeconomic status, friend aspirations, parent aspirations (range 1–7, M = 5.30, SD = 1.32), and school characteristics. To measure friends’ academic aspirations variable, we created a composite variable by summing students’ responses to the question, “How important is getting good grades to this friend?”, which was asked regarding each of their three closest friends (range 3–9, M = 7.23, SD = 1.33). Socio-economic status (SES) was measured by a composite variable available in the data set, and was based on five equally weighted, standardized components: father’s/guardian’s education, mother’s/guardian’s education, family income, father’s/guardian’s occupation, and mother’s/guardian’s occupation (range −2.11–1.81, M = −.08, SD = .72). School-level covariates included urbanicity (urban = 24%, suburban = 51%, rural = 25%), school size (range 1–9, M = 5.22, SD = 2.44), lowest teacher salary (range $13,506–$53,000, M = $29,080, SD = 4,638.53), and whether or not the school required students to pass a test in order to graduate (68%). These covariates were included because prior research has suggested that these factors contribute to the academic success of high school students (Hoffer 1997; Lee 2000; Lee and Burkam 2003; Weiss et al. 2010) and could potentially confound the relations of primary interest in this study.

Results

Data Analysis Overview

Prior to estimating our hypothesized model, we first used multilevel confirmatory factor analysis to examine the factor structure of the TSR climate measure based on the three items from the baseline student survey. Modeling the TSR climate factor on both the student- and school-levels allowed for the decomposition of the variance into both within and between school parts. All items were measured on a four-point ordinal scale, and, thus, analyzed using an ordinal logistic model. Standardized factor loadings ranged from .55 to .81 at the student-level. At the school-level, the residual variances of for the random intercepts were fixed at zero because they are generally very small and estimating them require additional dimensions of integration that would render these analyses computationally infeasible (Heck and Thomas 2009).

After confirming a cohesive, psychometrically strong factor structure, we estimated the full hypothesized model (see Fig. 1) to test our research questions using multilevel (random intercepts) structural equation modeling. Our theoretical focus was directed at the school-level effects. In particular, our research hypotheses were concerned with the direct and indirect paths linking the teacher reward and evaluation policies, through TSR climate factor, with school-level mathematics achievement and student dropout. School-level mathematics achievement and student dropout factors are random intercepts, reflecting the estimated school means in standardized mathematics achievement scores and log odds for school dropout adjusting for student-level covariates, perceptions of TRS climate, and 10th grade standardized mathematics test scores.

Fig. 1
figure 1

Hypothesized multilevel meditational model. Note that all exogenous covariates and predictors were measured and entered into the model independently and appear in the Figure as a single boxes in the figure for visual clarity

Full size image

All path estimates of theoretical interest were adjusted for covariates at both student and school levels to better isolate the variance in student outcomes associated with the teacher reward and evaluation policies. To accomplish this, student ethnicity, socioeconomic status, friends’ interest in school, and parents’ academic expectations were all included as student-level covariates, and, except student race/ethnicity, were grand mean centered. Student race/ethnicity was dummy-coded with White students serving as the reference category. School-level covariates computed as an aggregate of their student-level counterparts (viz., race/ethnicity, SES, friends’ aspirations, parent’s aspirations) were computed as the mean of the student-level responses within each school (i.e., cluster means). Interpretation of these school level covariates is as contextual effects. Additionally, school-level only covariates of whether each participating school was urban or rural (dummy coded, suburban served as the reference), whether students need to pass an exam to graduate, and the base pay for teachers were also included.

Mplus statistical software v5.21 (Muthen and Muthen 1998–2007) was used to conduct all statistical analyses. A full information, robust maximum likelihood estimator (mlr) was employed to obtain parameter estimates and standard errors that account for the nested data structure and are robust to non-normality and missing data under the assumption that missingness is at random conditional on the covariates. Furthermore, to obtain parameter estimates, a numerical integration algorithm was necessary to obtain maximum likelihood estimates given the incorporation of ordinal indicators of the teacher–student relationship factor and a dichotomous dependent variable (dropout). Due to certain technical aspects of this model, model fit tests and indices are not available, nor does the program allow options to obtain bootstrapped or Sobel-based tests of the hypothesized indirect effects. Instead, we used the joint test of significance approach, which evaluates the joint statistical significance of all direct paths along a particular indirect path, to assess the statistical significance of the hypothesized mediated effects (see MacKinnon et al. 2002).

Teacher Reward and Evaluation Policy Effects

Frequencies of the teacher reward and evaluation policies appear in Table 1. The majority of schools in the sample reported giving special awards to high performing teachers (55%) but it was much less common for schools to report rewarding teachers by giving them better students (8%) or providing them with higher pay (2%). Schools also infrequently reported allowing teacher to evaluate other teachers (13%) or permit students to evaluate their teachers (7%).

Table 1 Proportion of schools in the sample with various teacher incentive policies
Full size table

Hypothesis 1: Teacher reward and evaluation policies are related to students’ perceptions of the TSR climate. Adjusting for school-level covariates and student-level effects, results were that (a) schools with a reward policy in which good teachers are assigned better students exhibited a significantly poorer school TSR climate, (b) neither monetary nor award incentive policies were significantly associated with the TSR climate, and (c) schools with policies allowing students to evaluate their teachers exhibited a more positive school TSR climate (see Table 2).

Table 2 The Relationship between teacher evaluation and reward policies and the TSR climate
Full size table

Hypothesis 2: The TSR climate is related to gains in students’ math scores and dropout status. After controlling for sophomore-year math grades, school-level covariates, and student-level effects, a positive TSR climate during their sophomore year was found to be significantly and negatively related to the average log odds of student dropout by their senior year (see Table 3). No significant associations were found between the TSR climate and gains in standardized math achievement by students’ senior year.

Table 3 Associations between teacher policies and the TSR climate, and student outcomes
Full size table

Hypothesis 3: There are direct effects of teacher rewards and evaluation policies on gains in students’ math scores or high school dropout. After controlling for the TSR climate, school-level covariates, none of the teacher evaluation or reward policies were significantly related to school math achievement or average log odds of student dropout (see Table 3).

Taken together, these results suggests that the TSR climate mediates the relationship between the school level independent variables that are significantly related the TSR climate (e.g., students evaluate teachers, teachers being assigned better students) and school dropout by the test of joint significance (Mallinckrodt et al. 2006). Other estimations of indirect effects were not possible due to the need for numerical integration (Muthen and Muthen 1998–2007).

Covariate Effects

The path coefficients pertaining to student-level covariates are shown in Table 4. Adjusting for the effects of other student- and school- level covariates, students from lower SES families were (a) more likely to provide negative assessments of the TSR climate, (b) exhibited lower standardized math achievement gains, and (c) exhibited greater odds of dropping out of school by their senior year than students from higher SES families. Additionally, students that perceived their friends’ as having higher academic aspirations were more likely to report a more positive TSR climate and were less likely to drop out of school by their senior year than those that reported that their friends had low academic aspirations. Similarly, parents’ academic aspirations for their child were found to be positively associated with the student’s assessment of the TSR climate and negatively associated with the students’ odds of dropping out of school. Asian students were found to have significantly greater math achievement gains between their sophomore and senior years than those students identifying as White or Caucasian, whereas students that identified as African American or Latino were found to have significantly lower math gains compared to students that identified as White or Caucasian.

Table 4 Associations between student-level covariates, the TSR climate, and student outcomes
Full size table

Path coefficients pertaining to school-level effects are shown in Table 5. Schools with more enrolled students, higher proportions of students identifying as Black or African American, and with greater proportions of students that identified as an ethnicity or race not otherwise categorized exhibited lower TSR climate ratings. Additionally, schools with higher average student SES also exhibited a more positive TSR climate, controlling for school race/ethnic composition and other school-level covariates. Finally, students attending urban schools (compared to attending a suburban school) and those with the lowest base pay for teachers (as determined by the lowest paid teacher salary within a given school) were each uniquely associated with higher student dropout rates, even after controlling for students’ prior math achievement and other covariates.

Table 5 Associations between school level covariates, the TSR climate, and student outcomes
Full size table

Discussion

Student academic success protects against a multitude of negative consequences, including unemployment and prolonged dependence on social services (Alliance for Excellent Education 2007). Previous research has found that school level factors impact student outcomes (Hoffer 1997; Lee 2000; Lee and Burkam 2003; Weiss et al. 2010). Less is known, however, about the mechanisms through which these structural factors exert their effects.

In an effort to examine individual and school level factors related to math achievement and school dropout, this study investigated whether associations between select school policies and student achievement outcomes were mediated by the TSR climate. The results suggest school policies that encourage rewarding teachers for their performance in the classroom by assigning them higher achieving students negatively affect the TSR climate. Other types of rewards (e.g., merit pay, monetary incentives) have no statistically significant effect on the TSR climate. Similar to Manefield et al. (2007), this study also found school policies that provided students the opportunity to evaluate their teachers were positively associated with the TSR climate. Lastly, we found the more positive the TSR climate, the lower the student dropout rate, even when controlling for prior and current student math achievement. All of these findings are compelling in that several other possible confounding effects were controlled.

The effects of teacher reward policies on the TSR climate are particularly interesting given the controversy surrounding their implementation. Though there are many types of rewards, those examined in this study were not associated with improved TSR climate or student math achievement. In fact, assigning “good” teachers better students seemed actually to erode the TSR climate, possibly because of the inability of average and low performing students to receive “good teachers.” This finding may correspond to the fact that the very students who need the best teachers do not receive them, thus resulting in a net detrimental effect for the school. While top students may benefit from this policy, the majority of students are not receiving instruction from the teachers that are most likely to create a positive TSR climate.

Student math achievement and high school dropout are multifaceted. The findings reported here suggest that explanations for math achievement and high school dropout that rely solely on individual level factors are incomplete; school level factors, such as the TSR climate also influence these particular student outcomes. A positive TSR climate may protect against student dropout, but not contribute to gains in math achievement for several reasons. First, if students get along with teachers, think that the teaching is good, and believe their teachers are interested in them, they may remain in school, even if struggling academically, simply because of their positive relationships with their teachers. Students who are contemplating dropping out of high school may decide to stick it out because their teachers are warm and supportive. Though they may not be engaged with classroom material, students at risk for dropping out may find that a caring and supportive teacher compensates for an otherwise difficult high school experience. Second, although a student may have a good relationship with a teacher, that relationship may not be enough to improve his/her math achievement, particularly at this stage in education. Other factors, for example teacher effectiveness, communication style, teacher education and experience, may have a larger impact on this particular student outcome.

Several other school level variables also were associated with the TSR climate and high school dropout, even when controlling for individual level covariates. For example, SES was related to TSR climate such that the higher the SES the better the TSR climate. Presumably, schools with copious resources attract more qualified teachers and are able to invest more resources into the TSR climate than schools struggling with limited funds. School size negatively affected the TSR climate; as the size of the school increased, the quality of the TSR climate decreased. Additionally, school location and teacher pay were directly related to school dropout. Students attending urban schools were more likely to drop out than students in suburban schools and schools with higher teacher entry-level pay were associated with lower dropout rates, potentially because they were able to attract better-qualified teachers.

It is important to address other significant student-level predictors of school dropout and academic achievement, even though those analyses were not a main focus of our study. Specifically, this study found students who had friends with high educational aspirations and students who had parents with high educational aspirations for them had lower propensities to drop out of high school and generally had better perceptions of the TSR climate. Furthermore, students with parents who had high educational aspirations for them had significant math gains over time, even when controlling for previous achievement levels. Although it is likely that higher achieving students seek out equally achieving peers and parents of high achieving students subsequently set high educational standards, the fact that these effects were maintained even when controlling for achievement measured at the same time as the aspiration assessments is noteworthy.

Strengths and Limitations

Several strengths of this study enhance the confidence and interpretations of the findings. First, we used a large, nationally representative data set to explore our hypotheses. We also were able to estimate both individual and school level effects in our model, thereby more accurately distinguishing individual level effects from school level effects. This also improves the reliability of our estimation of school-level TRS climate. To address other confounding variables and reduce alternative, plausible explanations of our findings, this study also incorporated a wide array of covariates at both the student and school level to better isolate associations between school policies and student achievement. Finally, this study used a longitudinal design that controlled for previous student achievement.

As with any archival research, this study is limited by the original data collected. For example, the TSR climate was measured with only three items. Though these items seem to capture our construct of interest, future researchers interested in the TSR climate should consider a more comprehensive set of questions to measure this construct. The data set used for this study also did not identify the criteria by which teacher rewards were based. As such, we had no means of knowing if schools that stated they gave good teachers higher pay were based on administrator evaluations of the classroom, student evaluations, or student achievement. This information would have allowed for more nuanced data analysis and interpretation. Additionally, only 2% or eight schools in the sample stated that good teachers received higher pay. Additional studies probing associations between teacher merit pay and student outcomes are needed before making any substantial conclusions.

Finally, despite our best efforts to control for a wide array of potential confounds, it is possible that some important covariates were omitted. Though strengthened by a longitudinal design and measurement of many possible confounding variables, the study design is correlational, and causal inferences should be drawn with appropriate caution.

Policy Implications

Given the impact of the TSR climate on school dropout, it seems worthwhile to examine educational policies designed to improve the TSR climate. Previous research suggests that schools that allow students to evaluate their teachers have better teacher–student relationships (Manefield et al. 2007) and that positive teacher–student relationships have been found to be positively associated with a reduced dropout rate (Lee and Burkam 2003). Similarly, we noted that evaluation policies that allowed students to evaluate teachers significantly improved the school’s TSR climate. By allowing students the opportunity to comment on their teachers’ performances, students may actually believe that teachers care about what they think, which may, in turn, improve the TSR climate. However, only 7% of the public schools in our sample had such policies.

Though it is unconventional to promote non-significant findings, we think that given the controversy surrounding merit pay policies the fact that we did not find a relationship between paying good teachers more and the TSR climate or student achievement is worth mentioning. Further research is needed to determine whether teacher merit pay is associated with changes in student achievement when it is based upon specific criteria, such as student evaluations, administrator evaluations, or student achievement.

Collectively, these findings suggest that teacher reward and evaluation policies implemented by schools have the potential to aid or hinder the academic success of their students, either directly or indirectly through the TSR climate. Furthermore, the impact of these policies on the teacher–student relationship may result in an increase or decrease in the US high school dropout rate. However, the impact of teacher reward and evaluation policies will be limited by other school level factors. For example, as we found, average student SES, percent of African American students, and school size were associated with TSR climate. Urbanicity and base teacher pay were directly associated with student dropout rates. Importantly, these associations were detected even after considering the five teacher reward and evaluation policies under study. Therefore, factors such as the ethnic and socioeconomic composition of schools as well as other contextual variables not explored in this study continue to predict scholastic success even after controlling for teacher reward and evaluation policies. School level policies are only a part of the solution to high school failure. Additional research evaluating educational policies is warranted to better understand and identify means to eliminate the disparate impact of educational policies on diverse student populations.