Keywords

Setting the Context

There had been occasional national and local guidance but no strict requirement about curriculum or pedagogy in English primary schools since 1911. Government ministers started to express concern about national standards of numeracy in the mid-1970s, leading to agreement to recommendations concerning curriculum, teaching methods and assessment in the government-sponsored Cockcroft Report (DfES/WO 1982). A utilitarian curriculum was proposed, supported by more practical work and problem-solving, while students would also undertake mathematical investigations and assume a more active role in classroom discussion. Both curriculum and examinations at the end of compulsory schooling would become more differentiated so as to better meet the needs of students with a wide range of mathematical attainment, and would incorporate coursework to assess practical problem-solving and investigational skills.

The Report had substantial support among policymakers, teachers and educationists; through connected networks of influence, changes were implemented gradually, consistently, and on the whole in a positive spirit, although clearly at secondary level (age 11–16) the changes in national external assessment provided a strong incentive to conform.

It could be said that the Cockcroft Report addressed the more significant problem of designing a mathematics curriculum and assessment for mass education to age 16 and providing an updated definition of numeracy, rather than focusing narrowly on raised standards.

But after the relatively relaxed days of post-Cockcroft reform in the early 1980s, the repeatedly mediocre performances of England in international comparisons like SIMS and TIMSS (Reynolds and Farrell 1996) triggered further, faster and more prescriptive levels of government intervention, starting with a National Curriculum in the late 1980s, then national tests at ages 7, 11, and 14 in the 1990s, to supplement the long-standing external assessment at age 16. Results were then published in school league tables at ages 11 and 16. Finally, when England’s international performance still failed to rise, a new Government decided in the summer of 1997 to introduce a National Numeracy Strategy across all year-groups in all elementary schools in 1999/2000 (Brown et al. 2000).

The next three sections will focus on research relating to the processes and outcomes of the implementation of this major top-down national systemic initiative. The following section of the chapter will contrast this implementation with the development of a curriculum development project, Primary CAME (Cognitive Acceleration through Mathematics Education) through a local researcher-teacher partnership, aimed at building connected mathematical thinking.

The National Numeracy Strategy 1999–2005: Outline

Together with the related National Literacy Strategy which took place a year earlier, the National Numeracy Strategy in England was said by the Canadian team commissioned to evaluate the initiative to be “the most ambitious large-scale strategy of reform witnessed since the 1960s” (Fullan 2000, p. 19). The implementation of the strategy cost around the equivalent of $150m in the first year (1999/2000), with a further $100m per year for the following 5 years, then finally declining to zero within 10 years.

The objective of the reform was to raise standards of numeracy, in particular in national and international tests. Thus the definition of numeracy used (DfEE 1998, p. 11) was in terms of ‘proficiency’ with calculation and solution of word problems rather than that used earlier in the Cockcroft Report which related to the ability to apply mathematics in everyday life, further education and employment.

The Strategy was based on a National Numeracy Project funded by the previous government starting in 1996, which was still large-scale by most standards and had focused on schools in 13 localities selected because of low results in national tests. This fore-runner Project was led by a well-regarded national figure, Anita Straker, who had involved most of England’s elementary mathematics experts in assisting with the detailed curriculum and recommended didactics (Brown et al. 2003). Nevertheless all were aware that the development had to conform to some political requirements, for example by focusing on calculation, being prescriptive about curriculum and didactics, and involving a high proportion of whole class teaching.

The new Government was not prepared to await the full evaluation of the project before going ahead and rolling it out nationally in the form of the National Numeracy Strategy. However, reactions from teachers were positive, and so were early indications of attainment gains. The materials developed and many of the personnel appointed were carried over to the Strategy with minimal obstacles, thus enabling it to be implemented quickly.

Key aspects of the reform were:

  • an increased emphasis on number and on calculation, especially mental strategies for calculation, including new methods of teaching number skills, a delayed introduction of written methods, and an encouragement for pupils to select from a repertoire of strategies;

  • a three-part template for daily mathematics lessons, starting with 10–15 minutes of whole class oral/mental arithmetic practice, then 25–35 minutes of direct interactive teaching, first with the whole class and then with groups, and finally 10 minutes of plenary review;

  • detailed planning using a centrally provided week-by-week framework of detailed objectives, specified for each year group, which introduced many skills at an earlier stage than previously.

The Strategy claimed to be evidence-based, but there was some question as to how many aspects really were underpinned by research (Brown et al. 1998). There was however much use of successful teaching models (e.g. the empty number line) already used by the Realistic Mathematics Education group based at the Freudenthal Institute in Holland (Anghileri 2001). There were also similarities with some aspects of the ‘reform mathematics’ movement in the United States, for example the emphasis on discussion and refining of children’s own strategies and a focus on mental work rather than formal written algorithms. However calculator use was postponed until the final years of elementary school.

Teachers were discouraged from using sets of textbooks and instead encouraged to devise their own detailed lesson plans using the variety of sources they might have access to, in particular, the extensive examples illustrating each of the large number of lesson objectives which were provided in the teachers’ Framework documents (DfEE 1999). In fact the speed of the roll-out meant that it was impossible for publishers to bring out new textbooks matching the Strategy planning templates in time for the first year of implementation. Teachers therefore reported in the first year that they spent many additional hours preparing these new lessons, and found it especially frustrating when some good quality supportive textbooks started to appear in the following years.

The method of implementation was highly systematic and standardised, as it probably needed to be in order to quickly reach more than 100,000 teachers in more than 17,000 schools. It involved a substantial national training programme based on a “cascade” model of capacity-building. This was designed by the national director, together with a group of regional directors, who each trained locally based newly appointed consultants using a standard training package. In turn the local consultants delivered a 3-day training to groups of teachers, one from each school who had been appointed as their school mathematics co-ordinator. Head teachers and school governors (often a parent) attended for part of this time to ensure school managers were properly briefed. Finally each mathematics co-ordinator collected her own package to deliver 3 full days of training, spread over a year, in each school. Each of the training boxes contained videos of several ‘exemplary’ lessons, PowerPoint slides highlighting key features of the Strategy, including recommended methods of calculation, and guidance booklets to demonstrate ‘best practice’.

In addition to the ‘cascade’ training, each local consultant was required to provide additional in-school coaching for teachers in a group of schools. This included running 5-day courses to boost teachers’ subject knowledge. In the first year those schools selected were perceived as needing support because of poor results in national tests at age 11, but over 3 years each school experienced this additional support, and the majority of elementary teachers experienced the 5-day courses.

There were considerable external incentives for schools to quickly and fully implement the National Numeracy Strategy, since it took place within a tight school accountability regime. Statutory testing was in place at ages 7 and 11; at 11 this took the form of externally set and marked tests with results published as league tables in the national press. There was also a strict national inspection regime in place run by Ofsted (Office for Standards in Education). Schools could expect to be inspected at least every three or four years, and even more frequently if they had below average test results. Inspectors observed classes to grade teaching; they had the right to put schools in special measures which required them to improve rapidly or to close. Thus although the National Numeracy Strategy (unlike the National Curriculum) was technically non-statutory, not to implement it thoroughly would have been to risk poor outcomes from inspection.

After the initial implementation, as discussed in more detail in the next section, there were only small rises in national test results, so policymakers felt that a stronger line needed to be taken over control of teaching quality. The central strategy team (national and regional directors, supported by other consultants) were therefore asked to provide a complete year’s set of lesson plans available on the internet to match the objectives specified, for all year groups from Grade 2 to Grade 5. The lesson plans were full but not complete—but they mainly required teachers to supply only additional practice examples. There is no national data but pooled personal experience suggests that local consultants strongly encouraged their use and there was a very high take-up, even among schools which had recently invested in new textbooks matched to the Strategy. Some attributed this to the mathematical insecurity of, and fear of inspectors by, teachers—if the lesson observed followed closely a recommended Strategy lesson then they could not be perceived as non-compliant.

Thus within a period of 20 years England had moved from a position where elementary teachers were free to teach in mathematics whatever and however they wished, via gradually more prescriptive steps of a national curriculum, national tests, accountability measures and a national strategy, to a position where almost all teachers of a given grade were teaching exactly the same centrally designed lesson on the same day throughout the country.

The National Numeracy Strategy: Effect on Attainment

Considerable claims have been made for the success of the reform in raising standards of attainment (e.g., Mourshed et al. 2010). Since the National Numeracy Strategy was a reaction to poor results in international comparative surveys, the ultimate evaluation was whether England’s rankings improved in TIMSS (Trends in International Mathematics and Science Study) and PISA (Programme for International Student Assessment).

At the elementary level, data is available only from TIMSS Grade 4 comparisons. The results for England have risen gradually from 484 in 1995 to 531 in 2003 and 541 in 2007. England was 7th out of 36 countries in 2007 and was only outperformed by four Pacific rim countries, Russia and Kazakhstan (Sturman et al. 2008). (In view of their bottom position out of 65 countries in PISA the Kazakhstan results are unlikely to be valid.) England had drawn ahead of Australia, New Zealand, Canada and the Netherlands, all countries which previously outranked England. This suggests a significant gain across the period of the Strategy, although the actual gain is likely to be only about half as large as this because in 2003 and 2007 the tests in England alone were sat 3 months later in the year.

There has also been a small rise in TIMSS at Grade 8; after remaining pretty steady at around 498 between 1995 and 2003, the score increased significantly to 513 in 2007. This was for the first TIMSS cohort which would have experienced the National Strategy in elementary schools. England was now 7th out of 49 countries, and only significantly outperformed by five countries, Hungary and four from the Pacific rim (Sturman et al. 2008). In contrast the 2006 and 2009 PISA (age 15) results are low compared with 2000, with England at 20th out of 32 countries in 2009 against 8th out of these same 32 in 2000 (OECD 2001, 2010). This contrast in trends between TIMSS and PISA may make some sense in that TIMSS assesses a more traditional curriculum which might be strengthened by a Strategy which favours number skills whereas PISA assesses mathematical literacy, which was probably stronger under the post-Cockcroft curriculum.

A further indication of the effect of the Strategy on attainment should be the changes in the proportion of children reaching the ‘nationally expected’ level in national tests at age 11. Here there has been only a very slow gradual improvement of on average 1 % per year, with 69 % of children achieving the level in 1999 and 76 % in 2006. Remarkably similar trajectories were obtained for Science and English results. It was interesting that Science gains during this period (9 %) were very slightly larger than the Mathematics gain (7 %), since although there were National Strategies for Numeracy and Literacy, there was no such scheme for Science. This suggests that the gains were more closely related to teachers’ growing expertise in test preparation pressured by league tables and inspections than to the effects of the Strategy.

It was fortuitous that a large-scale 5-year (1997–2002) research programme on elementary mathematics, the Leverhulme Numeracy Research Programme (LNRP), coincided with the introduction of the National Numeracy Strategy. The LNRP research involved a longitudinal survey tracking children’s progression in numeracy based on a nationally representative sample of 40 schools, 10 each from four diverse local authorities (education districts). Additional research foci included a detailed qualitative longitudinal study of children’s experiences in mathematics classrooms, an investigation of school leadership in mathematics and numeracy and a study of the effects of the Numeracy Strategy training on teachers. There were two cohorts of children involved, one moving from Kindergarten to Grade 3 and one from Grade 3 to Grade 6. This meant that we had complete Grade 3 (aged 8–9 years) data from 35 out of the 40 schools both in 1997, two years before the start of the Numeracy Strategy, and in 2002, two years after its introduction. The tests used were of the type of numeracy which featured strongly in the Strategy, and the items had been fully trialled in earlier research projects also based at King’s College London (Brown et al. 2008).

This comparison of Grade 3 (aged 8–9) children’s attainment before and after the introduction of the reform shows that the Numeracy Strategy produced an effect size of 0.18 (Brown et al. 2003; see also Tymms 2004). This was consistent whether the gain was measured at the beginning or the end of the school year. Whilst this effect size is relatively modest (and is somewhat smaller than the increases in national test performance over the same period), it is comparable to effect sizes achieved in similar educational systems (e.g., the recent rise in German performance in PISA mathematics). An idea of what this effect size means in practice is that the difference is the equivalent of about 2.5 months’ learning. Alternatively, it meant that just over one in three schools had a lower mean score after the introduction of the Strategy than before, while the remaining two in three had higher mean scores.

Beneath this overall effect, there were differential effects across the attainment range (performance amongst the lowest attaining group of children fell) and in different part of mathematics (attainment on multiplication items did not rise, while number line and addition/subtraction items did). Analysis of the performance of a subset of the children at the end of Grade 6 (their first year in lower secondary school) found that their attainment on the elementary numeracy test was below that at the end of Grade 5, suggesting that the overall gain in attainment was not sustained.

These outcomes suggest that even a carefully developed, well trialled and systematically implemented curriculum change, costing in total around $1billion, may have a relatively small effect on children’s attainment.

The National Numeracy Strategy: Effect on Teachers and Teaching

The official evaluation of the implementation of both the National Numeracy and the National Literacy Strategies was commissioned from the Ontario Institute for Studies in Education (Earl et al. 2003), and there were also reports based on school inspectors’ observations (Ofsted 2002). These demonstrated that the implementation processes had been very thorough and successful in reaching all teachers as well as in giving coherent and consistent messages. The recommendations were being put into practice faithfully by teachers, in relation to the content of the curriculum, the lesson planning and adoption of centrally provided learning objectives, the specific mathematical didactics (e.g. use of the empty number line for addition and subtraction), and the generic pedagogy (the format of each lesson).

The Leverhulme Numeracy Research Project (LNRP) also had a large database of teacher interviews and observed lessons from before and after the Strategy implementation. Particular attention was paid to the Grade 3 lessons, which were taught in the same 35 schools (but rarely by the same teachers) during 1997/8 and 2001/2. These included about 75 lessons, one from each Grade 3 class, in each of the two years.

In particular the LNRP research was in line with the OISE and Ofsted reports in concluding that the more superficial aspects of the reform were implemented conscientiously by almost all teachers and schools (Millett et al. 2004a). Lessons became objective-driven, and lesson structures, pedagogy, didactics, and curriculum were modified in compliance with the guidance provided.

However teachers’ lack of understanding of the mathematics and unwillingness to make independent professional judgements acted as barriers to deeper levels of change. Teachers felt they must stick closely to the objectives and lesson structure since they had been assured that these were research-based and would produce good outcomes, even when they felt that pupils could have benefited from greater flexibility. They rarely felt they had the knowledge or confidence to challenge or adapt the lessons. For example, teachers sometimes expressed a desire to spend longer on a topic or idea until the whole class had consolidated their knowledge, or to omit the plenary session to allow pupils longer to work individually. As intended by the Strategy, the process of teaching seemed to have acquired priority over the process of learning.

We observed some levelling down as well as levelling up of quality in the process of pursuit of compliance. Thus the examples of really inspiring and engaging lessons we had sometimes observed in 1997/8 were no longer there in 2001/2. On the other hand, the fact that teachers were now focusing on prescribed objectives probably explained why by 2001/2 there were far fewer lessons where children’s confusion about mathematics seemed to be a consequence of a teacher’s lack of clarity over what they were trying to achieve.

However, almost all teachers expressed great enthusiasm for the changes. They felt that both they and the children had a better grasp of the mathematics with the new ways of teaching number and number operations. They were convinced that pupils’ achievement was significantly greater than before the Strategy was introduced, in spite of our results which showed that in over a third of schools the results were lower than previously, and in very few schools were they significantly higher. They appreciated the focus on asking children to explain how they tackled problems, although in lesson observations teachers often found it hard to build on children’s responses. Instead, keen to achieve the lesson objective, teachers would often simply then show children how they were expected to solve the problem.

Millett et al. (2004b) note:

In our opinion, the major impact of the Strategy so far on the teaching of mathematics has been in changing the attitude towards mathematics on the part of teachers, and with that the motivation for changing practice … improvements in the quality of mathematical interactions in the classroom are extremely limited. (p. 204)

These observations seem to explain some of the results on attainment reported in the previous section. For example, the drop in attainment of the lowest attaining children seemed to follow from teachers’ reluctance to diverge from whole class teaching on prescribed objectives or to spend longer on a topic than decreed. The fact that there were greater changes in some areas of numeracy than others reflected changes in emphasis and didactics. For instance, more focus on the number line brought significant rises in number line items, whereas some word problem items fell in facility as problem-solving was no longer emphasised. There was not anything like as great an effect size as was anticipated by the politicians because there was not really a change in the quality of classroom interactions between teachers and children.

Primary CAME (Cognitive Acceleration through Mathematics Education): 1997/2001

A significant barrier to the success of interventions at scale is that many of the recommendations made by the mathematics education research community are difficult to communicate to teachers at a distance. For example, systemic interventions face considerable challenges when attempting to encourage teaching that emphasises formative assessment.

We now shift our attention to a curriculum change project, Cognitive Acceleration through Mathematics Education (CAME), which attempted to address this issue (Shayer and Adhami 2007). Specifically, CAME sought to effect “bottom-up” change by working initially at a local level with small groups of teachers, then encouraging their continued involvement through a national support network of teachers.

CAME was one element of a wider programme of research in Cognitive Acceleration that began in Science Education (Adey and Shayer 2002). Central to the CAME approach were lesson outlines in which children were encouraged to grapple with cognitively challenging ideas. Drawing on neo-Piagetian, Vygotskian and other related research into children’s conceptual development (Adhami et al. 1995; Biggs and Collis 1982), the lessons attempted to match the reasoning levels inherent in mathematical tasks with what children might reasonably be expected to achieve, relating this in particular to potential misconceptions and children’s naïve understandings of key mathematical ideas (Hart et al. 1981).

Hence, CAME lessons were designed to provide all students in a typical class with opportunities to engage mathematics just beyond their current level. In doing so, the lessons included explicit attention to the key constructs of concrete preparation, construction, cognitive conflict, metacognition and bridging. These ideas were drawn from the mathematics education literature and are described in some detail elsewhere (Shayer and Adhami 2007).

For the purposes of this chapter, however, we emphasise two key design features of the intervention with teachers. The first important feature relates to conceptual teaching. CAME lessons were introduced to teachers as “Thinking Maths” lessons to supplement (and not replace) regular mathematics teaching and to be taught every two or three weeks. This reduced the conflict that teachers often feel in novel approaches between curriculum coverage and covering an issue thoroughly. Thus, it offered teachers the opportunity to explore and think in depth about the CAME approach, while mostly maintaining their previous practice.

The second feature relates to collaboration. Teachers’ professional development was built around the teachers doing the CAME mathematical activities themselves, then planning, team-teaching and reflecting on the relevant lesson, before teaching the lesson to their own classes. Hence, collaboration was designed into the professional development specifically around teachers’ central professional interest—teaching the lessons. Teachers are often extolled to collaborate, but they need both an opportunity and a reason to collaborate.

CAME has been developed for lower elementary, upper elementary and lower secondary education. The upper elementary work, of Primary CAME was part of the Leverhulme Numeracy Research Programme (LNRP) and hence, although this was not intended, went on alongside the National Numeracy Strategy. This made it difficult to analyse the results, especially while using national test results where some schools focused more than others in coaching their pupils for tests.

The work started with 2 teachers in each of two schools, working as teacher researchers with a local primary mathematics advisor and a group of four university researchers. The aim was to design, trial and refine Grade 4 lessons as explained above. In the second year, the teacher researchers inducted teachers in seven main study schools from the same local district, who trialled the lessons while the research group developed and trialled Grade 5 lessons, which were then introduced to the additional seven schools in the third year.

Hodgen and Johnson (2004) have described the significant changes in beliefs that occurred in some, but not all, of the teacher researchers involved.

Evaluation results for CAME in different phases all showed positive effects. In lower secondary (equivalent to Grades 6 and 7, ages 11–13), gains on an immediate post-test show an effect size of 0.34 on a test of conceptual understanding. In addition and significantly, public examination results at age 16 indicate a “far effect size” of 0.44, three years after the intervention took place (Shayer and Adhami 2007).

In lower elementary (equivalent to Kindergarten and Grade 1, ages 5–7), a group of teachers from 8 schools participated in professional development led by the researchers themselves, whilst a further group of 10 schools participated in professional development led by others (Shayer and Adhami 2010). On an immediate post-test, the group taught by the researchers showed gains equivalent to an effect size of 0.71 with gains for the additional group of 10 schools at 0.60 using a test of conceptual understanding. On a national test conducted five years later, both groups showed gains in comparison to the national sample equivalent to effect sizes of 0.24 and 0.22.

Finally, CAME in upper elementary that is described here was evaluated alongside the introduction of the National Numeracy Strategy (Adhami 2002). These indicate an effect size gain of 0.26 of the intervention classes over the control. Although this is a more modest gain than in lower elementary or upper secondary, we note that this gain was in addition to the effect of the introduction of the National Numeracy Strategy.

Thus there is reasonably good evidence for the efficacy of the CAME intervention both in elementary and in secondary education. Of particular note are the “far” effects indicated by the lower elementary and the secondary evaluations, showing that the effects of the interventions appear to be sustained. However, scaling up and sustaining the approach remains a challenge, an issue that we return to in our concluding discussion.

It is worth noting that in a recent report on good practice in elementary mathematics teaching by the inspectorate, based on observations in many schools (Ofsted 2011), a Primary CAME lesson taught by a teacher who had not been part of the original research was featured as an example of an outstanding lesson, and indeed the chief inspector for mathematics explained that it was the best lesson she had ever observed.

Conclusions

The local project, Primary CAME, had significantly larger effect sizes than the systemic reform. However this may only reflect the smaller scale of these projects, bringing about a greater personal commitment and a potential Hawthorne effect often associated with early adopters. As in the case of other CAME projects it was also clear that not all teachers bought into the system (non-implementation was not a realistic option for the National Strategy).

Nevertheless, Primary CAME is part of a wider and mature cognitive acceleration programme that has shown effects can be sustained (Shayer and Adhami 2010). In contrast to the deep change in teachers’ beliefs that can result from these local projects where teachers develop positive relationships with the project leaders and with other teachers involved, commit to the project and play an active role in the development, and often therefore experience a sense of shared ownership of the work, the National Numeracy Strategy produced rather superficial changes, but on a far wider scale. It seems likely that the marginal costs per teacher might be similar, but further work is needed on comparing the relative costs of local and systemic reform (Brown 2010).

We argue that two key deficiencies of systemic development are that it fails to encourage the development of authentic teacher professional networks (which Spillane 1999, argues is key to successful professional change), and it tends to discourage teacher exploration and experimentation (thus discouraging change, Cuban 1993). We are now seeing the lack of long term effect in that since the removal of the National Strategy infrastructure, many of the curricular and didactic features (such as methods of teaching calculation) seem to be fragmenting, leaving a lack of coherence in approach, both between and within schools. Only the more simplistic pedagogic features which probably have a lesser effect on outcomes (like the ‘three-part lesson’) seem to have survived as part of the nationally agreed definition of ‘good practice’. In contrast there are certainly still networks of Primary CAME teachers sustained by a small number of enthusiasts, but it is unclear how long these will survive.

These two case studies suggest that there is no clear winner between local and systemic innovation in mathematics curriculum; they have different development paths and effects. It may be that a system should alternate; for example, after a period of closely prescribed systemic change like the National Numeracy Strategy in England, a “let a thousand flowers bloom” approach of encouraging a wider number of small scale local projects would achieve gains equivalent to that achieved by systemic reforms initiative at a roughly similar cost, but would have the advantage of encouraging a revival of teacher creativity and producing a wider variety of approaches to the teaching of mathematics. The most promising might then be carefully evaluated and considered for wider, maybe even systemic, implementation in the next phase.