Keywords

Introduction

Politicians have belief systems, often strong ones, congruent with and backed up by the ideology of their political parties, whether they are the parties in power or the ones seeking power. This makes it easy for politicians to fit certain ideas into those beliefs, and to reject others, regardless of whether those ideas are backed by research or not. Rational politicians, of whom there seem to be too few, think scientific findings can sway political opinion. For example, a very powerful politician, the late US Senator Daniel Patrick Moynihan, an academic with a PhD degree, said to someone testifying at a congressional committee hearing “Everyone is entitled to his own opinion, sir, but not to his own facts.” The writer Aldous Huxley held a compatible opinion, saying: “Facts do not cease to exist because they are ignored.” Facts, these rationalists believed, were stubborn things, not easy to deny at all. But they are wrong.

Global warming, evolution, the moon landing, and other near certainties are denied by many individuals. Ignaz Semmelweis discovered how to stop the vast majority of deaths that occurred in childbirth, and was ignored for decades by the physicians of the world (Semmelweis 2014). Sure that they knew best, the medical profession held to its false beliefs and literally killed hundreds of thousands of patients that need not have died. Poor Semmelweis died tragically in a mental institution, apparently driven crazy, in part, by his failure to convince physicians that his research was solid. One person’s surety, no matter how well supported by science, is often doubted by others because of their social, religious, political, or personal beliefs, despite what most fair-minded people would call reasonable evidence. One seasoned politician put it to me this way: “In the legislatures of the world, facts are negotiable, but opinions are rock solid!”

This vein of irrationality flows through much political policy making. This results in our not getting rational policy making by governments, a failure to get decisions based on solid, though imperfect social science research, of which educational research is a part.

The Quality of Social Science Research

Part of the problem in getting research to guide government legislation is that social science evidence is not believed to be “hard.” Rather, such research is seen as “soft,” with the “facts” quite open for the kinds of negotiation my political colleague described (Berliner and Biddle 1995). Physics is the model used to dismiss the research from the social sciences because gravity is gravity in England and Australia, and light and sound travel at the same speed in Europe and the USA. In the social sciences in general, and in education, in particular, findings in one country or a region of a country, cannot always be easily replicated in another setting in or out of that country. Contexts vary so much in the social world. This is because a plethora of unexamined variables interact in odd ways, such that educational findings never have the surety as do finding that come from the physical sciences (see Berliner 2002; Berliner et al. 2014). Some of that faith in the surety of scientific findings, however, begins to fail in the biological sciences. Many drugs taken by many of the world’s legislators have no effect, or negative effects on them, and in a large number of cases, the treatments for many of their ailments are totally ineffective. But still politicians marvel at the research in the biological science. They ordinarily fund that research at relatively high levels because when such research works as intended important effects, literally lifesaving effects, do occur. Then there is research in the social and behavioral sciences, and the most difficult of the subfields in this category is educational research.

Compared to educational research, physics is easy-to-do research, while we in education have hard-to-do research (Berliner 2002). We simply have no scientific projects that impress like space flight and moon landings, bridges, dams, and linear accelerators. And we have nothing to compare to what we commonly call “miracle” drugs and the new medical technology. We simply have no miracle cures for low student performance on the standardized tests so commonly used to judge our students. Politicians around the world are notoriously impatient and want big payoffs, confusing the difficulties of doing science in the social realm with doing science in the physical or biological realms.

What politicians in the USA and elsewhere do not realize, however, is that some areas of our research are almost rock solid, like the research on the effects of early childhood education on later school performance and attainment in life, particularly for a nation’s poorest children. But if that research does not fit a politician’s beliefs, or costs a considerable amount of money to implement, it may be ignored or attacked and even lied about (Berliner et al. 2014). Many politicians do not recognize the almost rock solid research on the deleterious effects of leaving a child back a grade, and in the USA, despite the research, states are recommending that schools do this at increasing rates. The research in this area is not only ignored, but the clear bias that such a decision entails, targeting boys and minorities, is also ignored (Berliner et al. 2014). Many politicians deny the consensus reached in the research community about the effectiveness of private and charter schools. It is convincingly demonstrated that, in general, they are not as good as, or just equal to public schools, as soon as family social class is taken into account (Lubienski and Lubienski 2013; Wenglinsky 2007). Governments and parents will not accept the research on the effects of homework, or how being off from school in summer affects middle class and lower class children differently, and so forth. Researchers in education do have consistent research, and ignoring that body of research is as mistaken as ignoring Semmelweis’s research, though not nearly as costly and dramatic.

Government Misinterpretations of PISA

An example of how governmental irrationality, impatience, and political interests come together is the focus in the USA (and many other nations) on PISA (The Program for International Student Assessment) and other international tests. Our President and Secretary of Education lament America’s poor showing on international tests and thus worry about our economic competitiveness. But both these leaders and the newspapers that report (and support) their views fail to understand a great number of things; four of those misunderstandings follow.

First, they do not understand that in developed nations, PISA and the other international tests of educational achievement have almost no power to predict economic growth. So they are just using the modest mean performance of students on these tests to show they can be tough on teachers. They actually know that economic competitiveness is function of many other factors that are more important to economic health than is education. It is politics, not reliable data, that motivates them to pick on teachers.

Second, mean scores hide variation, But even then, US mean scores on international tests are not awful, merely often about average, which is not something that politicians can live with. Politicians always seek to be number one in every metric used, at least while they are in office. In the more developed nations, they all expect that their country will be the best in education, even though the results of the tests of national educational systems resemble a horse race or an Olympic game. So no matter how good the educational system, the horse, or the athlete is, some educational systems, some horses, and some athletes will come in fourth or sixth or fifteenth. When they are not number one or close to number one in ranking, regardless of the reasons, too many of these politicians turn on their educators.

But more important, and what politicians usually fail to grasp, is what is hidden by the mean scores that determine a nation’s placement in the educational race for the number one position in educational achievement. What is hidden is great variation among the subpopulations that make up the mean. For example, in PISA, American students are among the highest scoring students in the world in science, reading, and even in mathematics, often the weakest subject for US students if they attend schools where fewer than 25% of the families served by the school are in poverty. In TIMSS (Trends in International Mathematics and Science Study), the same pattern emerged. The 15 million or so students in the public schools that serve the middle and upper classes do remarkably well, and even the students in schools where family poverty rates were between 25% and 50% scored well, recently beating the much envied Finnish educational system (Berliner et al. 2014). These public school students, all in schools where poverty rates are under 50%, total somewhere about half of all public schools students in the USA and they do just fine on the achievement tests.

The mean score on these international tests is lower than many other nations because students in schools where poverty rates are higher than 50%, particularly in schools where over 75% of the families served by the school are in poverty, do terribly on these tests. These facts, hidden when only the mean score and ranking is examined, strongly suggest that it is not a problem of teachers and curriculum, but a problem of poverty, that most affects America’s scores on international tests. This same pattern shows up in PIRLS (Progress in International Reading Literacy). On that test, the approximately 15 million American students attending public schools where fewer than 25% of the families were impoverished had a mean score higher than the mean score of every other country in the world. And on that test, Asian-Americans also beat the mean for every other country in the world, indicating that American Asians in American public schools can outperform Asian Asians in their own nations’ schools. This could not happen with a nation that is filled with bad teachers, working for inadequate administrators, using an out-of-date curriculum, and forced to negotiate with obstinate unions, as is so often claimed by government officials working for both former presidents Bush and Obama. Our government officials are deliberately ignoring what appear to be causal factors in determining the unexceptional overall performance of American students on international tests.

Despite the nonsense spouted by politicians as they interpret the mean scores on the international achievement tests, sure in their belief that that the USA does poorly; and despite the problem of American newspapers reporting such nonsense uncritically, convincing many tax payers that such nonsense is true, it simply is not a true statement to say that our American students do poorly. What is true is that some American students do poorly. That is a big difference in both the statement of the problem and, therefore, in how we might address that problem. We now know beyond any doubt that the distinguishing characteristic of those that do poorly in America’s public schools is their poverty, and therefore the kinds of neighborhoods they grow up in, often neighborhoods highly segregated by race as well as by income (Biddle 2014).

From decades of scholarship, yielding highly reliable data whose implications are often ignored by governments, we have learned that the major predictor of success and failure in our public schools is family social class, particularly family income and its correlates and sequelae (Berliner 2013a; Biddle 2014; Perry and McConney 2010) In fact, in one international study where Finland beat the USA handily, the childhood poverty rates for the two countries were statistically swapped. That is, Finland with a low poverty rate for children was assigned the high poverty rate for children in the USA, and vice versa (Condron 2011). The result is completely ignored but showed clearly that if Finland had the same poverty rate for children as the USA did its scores would drop precipitously, and if the USA had the same rate of childhood poverty as Finland, its scores on PISA would rise dramatically. Reducing poverty requires complex and expensive action. Blaming teachers is easy and cheap. Thus teachers are frequently blamed for problems over which they have no control.

Third, our politicians have no clue how to interpret the notion of variance accounted for in either domestic or international testing. When aggregate test results are analyzed, say mean scores, for classrooms, schools, districts, states, and nations, we can partition the variance in the test scores by simple, common, statistical techniques. When we do that, a typical finding is that schools account for about 20% of the variation we see in test scores of students on national tests in the USA, and outside-of-school factors account for about 60% of the variation in the scores we see (Haertel 2013). Any person with basic arithmetic skills can see that the outside-of-school factors are three times more powerful in influencing school performance than are the inside-the-school factors. Thus government policies toward school improvement might better be aimed not at the schools, but at other factors that more powerfully influence school achievement. But governments do not know that their policies are even more off the mark given another fact. That is, while schools account for 20% of the variation we see in test scores, teachers are a part of the schools’ effect. Perhaps teachers are even the most important part of a school’s influence on its students, affecting, perhaps, half of the variance that we attribute to school effects. Thus, teachers probably account for about 10% of the variance we see in students’ test scores, while outside-of-school factors appear to account for 60% of that variance, making the outside-of-school factors 6 times more powerful than teachers in affecting classroom, school, district, state, and national test scores. It appears to be much less likely that we can improve achievement test scores through school reform, than we could through social reforms.

What politicians and the general public fail to understand is that teachers do dramatically affect the lives of their individual students, often influencing their attainments and many other aspects of their later lives (Barone 2001). But teachers rarely have that powerful an influence on classroom mean scores, and even more rarely do they influence school or district mean scores. And it turns out that teachers have virtually no effect on state or national aggregate mean scores. Teachers have no discernable effect on national scores, despite many governments interpreting PISA scores in ways that give credit to, or blame for, those scores to their teachers. Such an inference is quite inappropriate, though it is made all the time.

Most governments avoid facing this quite solid research about the powerful difference that teachers can make in the lives of their individual students and their weak effect on the aggregate test scores obtained from their students. The strong effect on individuals and the weak effect on aggregate scores is a paradox, of sorts. And human beings, especially those who staff government bureaucracies, do not deal well with paradoxical policy. The validity of research demonstrating weak effects by teachers on aggregate test scores was recently confirmed by the American Statistical Association (2014). What we now know is that classroom mean scores are ordinarily more strongly determined by peers in the class (Harris 2010; Berliner 2013a) and not influenced in a major way by the classroom teacher. Grade level mean scores are ordinarily more strongly determined by the cohort of students at the grade level, than they are by the classroom teachers of that grade level. School district mean scores are almost always a function of the social class and income distribution of the neighborhood from which that school draws. Of course exceptions exist. Teachers do, occasionally, affect the test scores, even the lives, of everyone in a class (Pedersen et al. 1978; Barone 2001), and teachers can make a school or a district a great success (Casanova 2010; Kirp 2013). But exceptions do not negate the rule. Exceptions to the evidence that cigarette smoking causes lung cancer exist. So it is not hard to find an 80-year-old lifelong smoker. But that doesn’t change the rule that smoking is harmful, any more than does a highly successful teacher, school, or district change the rule that classroom peers, grade level cohorts, and neighborhood composition are more frequently the most influential of the effects on the mean scores of classrooms, grades, and schools. Exceptions should never be used to make policy. But in the USA, in particular, politicians praise and blame teachers for their influence on classroom or school test scores, when their power is really through their influence over individual student lives and on individual student test scores. Only rarely do teachers have a large effect on aggregate scores such as class, grade level, or district scores.

Fourth, in the USA our politicians have demanded that all states seeking federal dollars follow common rigorous standards. This is intended to eliminate much of the nation’s variability in what is taught, when it is taught, and to make sure that America’s children give up their childhood ways in order to study only what is preparatory for the high-stakes tests that accompany the new standards. This policy, it is thought, will help our nation be first in the world in international tests—we’ll have the winning horse; we’ll take Olympic gold.

But the common standards have attached to them a common test, a part of the two decade long demand by politicians and the business community of the USA for greater accountability by teachers and schools, despite the research cited in the previous paragraphs showing that the effects of teachers and schools on mean test scores is so much smaller than imagined by almost everyone in authority. The testing advocated is high-stakes testing. That is, consequences of importance follow from the testing. Teachers and administrators can be fired or rewarded, and children may be left back, on the basis of test performance. Yet research, history, and anecdote show that invariably, whenever testing is made high-stakes, corruption of the test scores and corruption of the people involved with the testing occurs. Cheating scandals are now a commonplace in the USA and in other nations with high-stakes tests. Cheating on China’s high-stakes tests goes back well over 1000 years (Suen and Yu 2006) and is still common (Moore 2013). The effects of the high-stakes tests to accompany the new US standards are predictable from the research: they will result in cheating and gaming the system so that students and their teachers get high scores; they will narrow the curriculum that is taught; and they will narrow our conceptions of what constitutes a smart child, ignoring all evidence of talent except in those areas that are tested. These ill effects are well documented (Nichols and Berliner 2007), and all are predicted by a well-validated social science law, Campbell’s law (Campbell 1975). But both the documentation of the deleterious effects of high-stakes testing and the validity of the scientific law are ignored by most government designers of educational policy. It is frustrating.

Additional Problems with Politicians, Governments, and Their Relationship to Educational Research

In industry, new ideas are piloted. Tryouts occur before major investments are made. Extensive field testing is often done before settling on a final design. In education this is often not the case. The new Common Core State StandardsFootnote 1 (CCSS), noted above, were developed by non-teachers and never field tested. But adoption of those standards was forced on every state by the federal government without field testing. In their haste to appear to be doing something, anything, this common mistake is made by governments. Field testing is important. But in addition to reasonable evidence that a policy might work as intended, as field testing would reveal, it would also be nice to be sure that the policy is even needed! Many US schools are doing fine in international competitions without using the CCSS, for example, Massachusetts schools and schools in high-income neighborhoods. Since their success is well documented, why would a government require those schools to change?

The No Child Left Behind actFootnote 2 (NCLB), pushed through congress by former president George Bush, did not work as intended. It was rushed into all states without a comprehensive evaluation of its effects in the State of Texas where it was first used. The effects claimed for NCLB in Texas were eventually discovered to be totally false, but it had already been rushed from a state law to a national law, and it has failed again (Nichols and Berliner 2007; Ravitch 2010). Slower implementation of some educational policies are recommended: field testing and exemptions for some schools may be more rational when designing national policy than what was done in implementing NCLB.

But the biggest fault of the NCLB law was something else: it set patently impossible goals to be reached. Politicians had signed into law the requirement that all American children—100% of them—be proficient on tests by 2014, the year in which this chapter is being written. One hundred percent above average would be ridiculous enough, but this law went even further—100% of our students were to be proficient, achieving at some level well above average. This insane law was passed with great bipartisan support by America’s congress. This law, promoted by the younger President Bush, had echoes of his father’s equally ridiculous goal. The older President Bush, in the 1990s, demanded that the USA be number one in the world in math and science by the year 2000. That year also that has come and gone without a hint of a US triumph in assessments of achievement in those fields. It was seen as ridiculous then, and nothing since then has changed researchers’ minds.

The point for governments is to set expectations, and sign into law, that which is possible. Instead, governments too often set impossible goals, and then blame the teachers of America for not reaching them. This is both bad and unfair policy. It is also related to terms in office. Politicians too often set impossible goals because they look forceful when they do so. But sadly they know full well that they will not be in office when those goals are to be met. Blame for the failure to reach those goals is not then attributed to failure in the political system, but easily assigned to others, often teachers and administrators.

Another problem with government and educational policy is that policy makers in all countries favor policies that appear to have simple main effects. They tend to think that if they promote policy A, then effect B will occur. They are ignorant of, or ignore, the complexity of the real world, a world where interactions among a myriad of variables abound. It might be that “If A then B” holds only when X is present. For example, a policy might be designed to let minority students into college, even with lower grade point averages than others, so the college can graduate more minority students. Thus “A” (letting minority students into college), then “B” (more minority college graduates), appears to be a sensible, proactive, and progressive educational policy.

But the college might only graduate more minorities if “X” is present, say the formation of study groups, or the provision of counselors from the same cultural group, or the provision of remedial classes. Policies often work only under some conditions and do not work under other conditions, and that is often overlooked. When that happens, which is frequent, the cost estimates associated with a policy may mushroom. Still another example of this unrealistic “main effect” thinking is that “A” may not produce “B” under circumstances where Y is present. If it is desirous to increase student proficiency in mathematics, currently a concern in the USA, than many policy options are open. But with the vast majority of elementary school teachers having minimum training in mathematics, all such policies are likely to fail. To get mathematically trained teachers into the profession may require much higher starting salaries. Under current employment practices, policy “A” about improving mathematics may never result in the achievement of “B,” higher mathematics test performance, as long as “Y” is present, inadequate mathematics preparation by elementary school teachers. Simple policies for a complex world rarely succeed: extensive modifications are often needed.

A related problem is this: politicians appear to have no idea how hard it is to have what works in one place, work in another place. Because the real world is so complex, with the number of variables interacting so huge, what appear to be similarities in communities may be illusion. In reality, vast differences in local communities exist, and this frequently is the cause of policy failure. For example, politicians may hear stories of someone or something “working” some place, and thus they want to see it transferred to a site over which they have some responsibility. They may even allocate money for a new program or pay a high salary to a new school leader. But it is difficult to transfer successful programs and successful people as easily as it seems. The evidence is overwhelming that many apparently successful reading, science, math, or even sex education programs simply do not replicate at other sites. And highly successful school leaders often fail in new settings too. The impatience of legislators, new boards of education, or new superintendents to put into place a seemingly successful program needs to be tempered by the fact that local sensibilities need to be taken into account, and local adaptations of the program or policy are likely to be needed. In the real world, social and educational findings do not transfer as easily as do physical science findings. Leadership is also very dependent on context and culture, so it too is not easily transferable.

Still another policy problem, often ignored or greatly underestimated by governments, are the costs associated with certain policies. For example, the quality, appropriateness, and philosophy behind the CCSS, and the assessments that accompany them, may be argued about forever. But what is not arguable is that the cost of implementing the CCSS and the associated testing program is huge. Even small states will need to spend hundreds of millions of dollars for computer infrastructure to implement the standards and the accompanying tests, and large districts and states will spend billions of dollars. This is money that will end up in corporate hands, and thus not used for the repair of aging school buildings, or the reemployment of school nurses and librarians, nor the rebuilding of music and art programs, all programs decimated during the recent economic recession. The magnitude of these costs was not mentioned in the initial policies put forth by the US Department of Education.

An additional problem associated with policy costs occurs because many state governments in the USA have also supported vouchers for private schools. In this relatively new educational policy, support is usually in the form of a tax credit, whereby the tuition that is paid to a private school by a particular family is deducted from that family’s taxes that are due to their state. States, therefore, lose revenue. This leads to cuts in all the social programs of the state, including police and fire protection, road construction and public transportation, as well as education. Further, and particularly in the poorer school districts of a state, vouchers reduce the money schools need to support quality education. This is because fewer children attend the public schools, but the loss of a few children in each grade level to a private school results in virtually no cost savings for a public school. Thus, with almost the same expenses, they receive a lower allocation of funds from their state. Furthermore, although many neoliberal state legislators do not care, the evidence is that voucher system are both corrupt and hurting public schools. In addition, voucher schools are usually not any better than the public schools, even though they appear to be biased racially, by social class and by the quality of the students they accept (Berliner et al. 2014; Welner 2008).

Over time politicians and governments have learned to demand summative evaluations. They rightly want to know if a program or a policy is working as intended. But that demand could also be a problem. Programs and policies take time to root. So it would often be smarter to demand formative evaluations of policies and programs for a year or even three, before a summative evaluation is attempted and judgements made about a program’s success or failure.

But even then, after formative assessments and a summative judgment is made, we now know that the likelihood of ever obtaining unambiguous data from our research is quite small. Even if we used a randomized clinical trial (RCT) to assess a program or a policy’s effects, ambiguity in social science findings is common. Politicians want surety. But educational research (the social world) and medicine (the biological world) never really produce the surety that politicians hunger for, such as that which characterizes the physical world. For example, findings from some of the RCTs promoted by the federal government showed statistically significant effects for treatment A over treatment B. But those differences, though statistically significant, were usually remarkably small. So in the hands of a talented or a highly committed teacher, the program or policy that was implemented and found to be ineffective, is likely to be working fine. And in the hands of a skeptical or less talented teacher, the treatment or policy supported by the statistics from the RCT may not work as expected. The social sciences cannot provide legislators making policy with the surety that they seek.

Related to a number of issues raised above is the unrelenting focus by politicians and governments on outcomes, and their frequent lack of concern for the inputs needed to make programs and policies successful. For politicians and governments to be focused on high school graduation, college attendance, job readiness, test scores, and the like is not wrong. But each of these outputs of the education system is strongly related to inputs to the education system, for example, the poverty rates of the families and the neighborhood the school serves. Each of the valued outputs is also empirically related to preschool attendance rates, food insecurity, medical coverage for families, neighborhood drug use, teacher experience, teacher turnover at the school, funding for counselors and librarians and nurses at the school, and so forth. The past 20 years have seen us move almost exclusively to policies related to the outputs of the schools (the achievement gap) and to ignore many of the inputs to the schools (pursuing equal educational opportunities for children). Outcome-oriented policies make it easier to blame teachers and administrators for purported student failures, and these policies are often cheaper to fund than would be the many input variables that are known to affect school achievement. But if the problems of many students and schools are related to inputs, then almost all the proffered output-oriented remedies will fail. If we set policies that increase the rigor and breadth of the outputs of our educational system, which we have recently done, without concern for an increase in the quality of the lives led by the students who come into our educational system, we have the certain makings of a policy failure.

Finally, citizens of a democracy should worry when government policy imposes overly standardized approaches on teachers. Schools are not factories, and teachers are not robots, turning out standardized products. Yet standardization of educational processes and outcomes is often the goal of educational policies such as the CCSS and the many new systems for evaluating teachers. Suppose that a teacher is quite good at teaching, say, Moby Dick as an example of a great nineteenth-century American novel. But suppose also that the standards adopted and the curriculum associated with the standards end up recommending The Red Badge of Courage, or Treasure Island, or The Adventures of Tom Sawyer, or The Adventures of Huckleberry Finn, or The Scarlett Letter, or a dozen other American novels thought to be grade level appropriate. If we force any teacher to teach something they do not care deeply about, and in which they have no great interest, we lose something potentially wonderful. Perhaps teachers should have the right to meet the intent of a policy, say to teach about American nineteenth-century life and beliefs, without being obliged to teach what some distant policy maker or curriculum committee has recommended. It might be better for teachers and the students they instruct if governments treated teachers like professionals who are capable of making their own intelligent choices about curriculum. But government agencies and the politicians who supervise them often treat policy for teachers much like these people and agencies make law: “you will do this, and not that, or penalties will be imposed!” This is a common and career deadening mistake. Education policies ought instead to promote having teachers present things they know well and love to teach, rather than policies that force teachers to accommodate the wishes of a distant bureaucrat or curriculum developer (Berliner 2013b).

Conclusion

We now understand that politicians and the governments they represent hold their positions of influence through the manipulation of symbols. It has become commonplace, therefore, to discover that many of the acts of high-placed policy makers are mere symbolism. Laws and policies are too often put in place not as serious attempts to solve problems, but to placate constituents or gain political backing. This explains the vigor with which new policies are announced, research evidence ignored, and the lack of interest in assessing their eventual consequences. We too often squander opportunity, time, and money on what is merely symbolic politics. We need the courage to face reality, assess what needs to be done, and accept genuine responsibility for improving our world when it does not work as we wish it to. In a better world, the first role of governments and the first responsibility of the politicians and bureaucrats that staff them would be the welfare of the people who they serve. In education, this is clearly not often the case. More openness and realism in setting educational priorities, in monitoring educational programs, and in evaluating their effects would make educational improvement in each nation much more likely.