Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Background to the Taiwanese Educational System

Before 1967 only elementary school education, for children from 7 to 12 years old, was compulsory in Taiwan. In order to gain entry to a favoured junior high school, many 11 and 12 year old students attended after-school classes, often colloquially called the cram-schools. In Taiwanese, these are called buxiban. In order to release students from the pressures of competitive entry to limited places in junior high schools and thereby postpone the time for attending buxiban, in 1968 compulsory education was extended to 3-year junior high schools (12–15 years old). Besides, the public believed that the length of compulsory education reflects the modernity of a country. Therefore, some politicians extended compulsory education to attract votes. Now, because of the educational policies of our current president, Ying-jeou Ma, a 12-year compulsory education program that integrates primary education, junior high, senior high or vocational education will be implemented in 2014.

The Taiwanese Mathematics curriculum for the years of compulsory education has undergone three reforms in the last three decades. Based on the shift of focus towards students as knowledge constructors, the first reform was to revise the 1975 Standards of School Mathematics Curriculum to emphasise the manipulation of concrete materials. Nevertheless, the revised Standard of School Mathematics Curriculum in 1993 resulted in having the two systems of algorithmic mathematics and mathematics with manipulatives coexisting in classrooms: the formal taught methods alongside the child-invented methods, as described by Booth (1981). In order to complement the defects of previous curriculum, further reforms were still required.

The second and third reforms included the 2000 Nine-Year School Curriculum, issued in 1998 and then revised in 2003. The 2000 Nine-Year School Curriculum was proposed with a basic philosophy of constructivism and it emphasised the value of children’s own methods. However, after the implementation of this new curriculum in 2002, the seventh grade students did not perform well on the first mathematics examination. Thus some scholars, especially the mathematicians, asked the Ministry of Education to revise the 2000 mathematics curriculum and a reform group was formed (Leung et al. 2012). The major differences between the Nine-Year School Mathematics Curriculum of 2000 and 2003 lay in the quantity and sequence of the content. The 2000 Mathematics Curriculum expected 80 % students to keep up with the scheduled content, so the content was less, simpler, and flexibly divided into four learning stages. However, the 2003 Mathematics Curriculum presupposed that no more than 50 % students would be left behind the scheduled content. Consequently, the content sequences for each year were listed according to ability indicators. However, the content was more inflexible and more difficult than that of 2000.

Buxiban are private schools that offer out-of-school instruction to improve students’ achievement scores. In Taiwan, buxiban are so popular that they have become a sunrise industry. The major function of these buxiban is to increase the possibilities of getting into desirable high schools and universities. These schools mainly focus on enhancing students’ academic abilities in mathematics, science, English, and Chinese writing. For Taiwanese students of Grade 11, it has been found that attending a buxiban improves educational achievement, and 49 % of eleventh graders reported they spent several hours each week in buxiban (Chen and Lu 2009). The main reasons why students go to buxiban include (1) following established customs of going to buxiban, (2) as a way to make friends, (3) their fears of getting academically behind classmates who go to buxiban, and (4) unsatisfactory performance in school examinations.

Taiwanese Students’ Performance in PISA

Taiwan has participated in PISA since 2006. In mathematics, it was ranked 1 in 2006 (mean score 549) and 5 in 2009 (mean score 543). Taiwanese students were relatively better in mathematics and science (ranked 12 in 2009) than in reading (ranked 23 in 2009) although the correlations between the three scores at the level of the student are very high (0.81–0.85). Although Taiwanese students’ performance in mathematics and science literacy is internationally ranked at the top level (OECD 2009, 2012), the achievement gaps between high and low achievers are larger than in many other countries, so this is something that needs attention. Table 14.1 shows the percentage of students at each level of mathematical literacy in Taiwan and the other countries ranked in the top six for PISA 2009, and also for Taiwan in PISA 2006. The percentage of students at and below level 1 is larger for Taiwan than the other high-performing countries. First results from PISA 2012 indicate that this pattern continues.

Table 14.1 The percentage of students from high performing countries at different levels of mathematical literacy in PISA 2009
Full size table

Although Taiwanese students’ average performance in mathematical literacy is internationally top-ranked, it is still profitable to study their weaker areas to guide further improvement. By analysing Taiwanese students’ responses, several places where improvements might be made have been identified. Firstly, some students were not familiar with connecting given situations and their descriptions with figures to make reasonable assumptions, e.g. statistical graphs. Secondly, although the mathematical models behind problems situated in real-world contexts were not hard for students, some students were distracted by superfluous but related information in a problem. This implied that they were relatively weak in discriminating relevant and irrelevant information to solve problems in real-world contexts. Thirdly, some students did not correctly answer estimation problems, which may result from lack of familiarity with estimating large numbers, computing with calculators, or thinking of tolerable errors. All of this may arise because Taiwanese students are often ‘stuffed with a standardised answer’. Fourthly, some students tended to provide personal interpretations rather than evidence-based explanations and then their over-inference caused wrong answers. This showed that they did not understand that valid information would be the basis of strong explanations. The weaknesses of Taiwanese classroom teaching were revealed by the above-mentioned features and partially resulted from the strategies teachers used. Due to the prevalence of multiple-choice tests, many Taiwanese teachers teach students strategies of deletion and substitution, which can be used for quickly isolating a correct answer.

Two Literacies for Selecting High Achievers

Taiwan is going to implement a 12-year compulsory education program in 2014. In general, senior high schools, being compulsory, should have open admission for junior high school students if they meet the minimum test score and other relevant requirements. However, the reality is more complex. Currently, senior high schools are ranked hierarchically according to their students’ entrances scores in the national examination. The most desirable high schools have the highest scores. Consequently, the top 15 % of students, in particular, are nearly all gathered into specific schools. It is not yet certain whether this will continue in the future, or whether they will be spread among many different schools that all may offer a special curriculum for high achievers.

Although there is an examination to evaluate the competency of students in Grade 9, the main assessment goal has traditionally been students’ mastery of textbook content rather than their ‘learning power’, which is more important. Learning power is based on thinking and reading, and therefore mathematical literacy and reading literacy should be assessed. In this context, mathematical literacy is defined as using mathematical knowledge and skills to identify and solve situational or mathematical problems, and understanding written text to reflect on mathematical knowledge included in the Taiwanese curriculum. It is a new challenge to identify the high achievers in the top 15 % according to their learning power rather than just their content knowledge. This definition of mathematical literacy has been inspired by the PISA definition and adapted to suit the purposes of assessment in Taiwan.

Alternative Assessment Goals and Framework

When we acknowledge that “mandated assessment mediates between the expectations of the system and their embodiment in classroom practices” (Barnes et al. 2000, p. 626), we come to realise that the alternative assessment goals for high achievers should be a tool to reshape school practices. Instead of considering only a selection function, they should also consider the key purposes of teaching and learning in compulsory education.

Accordingly, Lin (2012) analysed the consequences of deciding that a major educational goal was to enhance students’ learning power. He elaborates learning power in three dimensions: tools, learning methods and dispositions. The three dimensions of learning power support an analytical approach to assessment reform. Language and thinking are two necessary tools, while reading and inquiry are two main learning methods. Dispositions refer to learners’ emotions, attitudes, and beliefs. This is in accordance with the definition of learning power as

a complex mix of dispositions, lived experiences, social relations, values, attitudes and beliefs that coalesce to shape the nature of an individual’s engagement with any particular learning opportunity. (Deakin Crick et al. 2004, p. 247)

Both language and thinking are required in learning different subjects. On the one hand language, especially as reading literacy, is an interdisciplinary competency. On the other hand, mathematical literacy supports logical thinking and forms the basis for pursuing advanced knowledge. Therefore, the assessment goals are to measure mathematical literacy (how students use the knowledge and skills they have acquired at school to solve open-ended and reasoning problems) and reading literacy (how to gain knowledge from reading text in multiple disciplines including history, geography, civics and science). Students’ dispositions are not included in this assessment reform because they cannot easily be objectively evaluated and ranked through a time-limited, paper-and-pencil test.

For the proposed assessment of mathematical literacy, we adopted three components from the PISA Mathematics Framework. The first component was mathematical content organised around overarching ideas such as Quantity, Space and shape, Change and relationships, and Uncertainty (OECD 2004). The second component was the use of context so that problems are set in various real-world situations. The third component was mathematical competencies. The mathematical competencies were considered to be more critical than the other two components in order to discriminate the level of mathematical literacy of high achievers. For this assessment, the most important competencies are problem solving, reasoning and proof. (Note that this notion of competencies draws on but is not the same as that described in Chap. 2 in this volume). Taiwanese students’ performance in PISA placed about 30 % of students at levels 5 and 6 of mathematical literacy. The feature of these two levels, as described in the Mathematics Framework (Taiwan PISA National Center 2011) is being able to handle complex problems and advanced reasoning. In order to provide an assessment for selection of the very best students, we only focused on problem solving and reasoning and proof, which also correspond to the top level of mathematical competence delineated by Jan de Lange (1999).

Feasibility of this Framework

The feasibility of this framework will be verified by its relevance to the Taiwanese School Mathematics Curriculum and the empirical validity of selecting high-achievers. Ideally, assessment tasks should match the expectations of curriculum documents, syllabuses, or courses of study. Although the framework above is not directly related to the national curriculum, two components of the framework, the overarching ideas (now called content categories as in OECD 2013) and the mathematical competencies fit the spirit of the curriculum. To be more specific, the national curriculum aspires to connect different mathematics units to different learning domains inside and outside of mathematics, applying mathematics to daily life, appreciating the beauty of mathematics, and further cultivating interest in exploring the essence of mathematics as well as other related disciplines (Ministry of Education 2003). The PISA content categories are not directly drawn on but are in reasonable correspondence with the Taiwanese School Mathematics Curriculum. The mathematical competencies just match its spirit. Only the mathematical models underlying task situations that are directly within the Taiwanese School Mathematics Curriculum will be included in the PISA-like assessment. This is so that it can assess all high-achievers equitably. Even though the mathematical content of the reformed assessment is constrained by the national school curriculum, the scope of mathematics is deeper and broader than PISA in order that it can validly select the top-achievers.

Before proposing this framework, Lin (2011) invited 25 mathematics educators to help 180 junior high school teachers understand mathematical literacy from the perspective of PISA and also using the ideas on mathematical proficiency expressed in the book “Adding It Up” from the United States (National Research Council 2001). The first aim of this collaboration was to help teachers clarify the difference between mathematics for the promotion of mathematical literacy and the mathematics of examination; a distinction that should be well-known by teachers but might be easily confused. The features behind the mathematics of examination include closed problems, one problem with one predetermined answer, precise information, and problems posed in decorative (but not necessarily realistic) situations. On the contrary, the features behind mathematics to promote mathematical literacy included open-ended problems; problems with more than one possible approach; problems with multiple plausible answers; problems with superfluous information and ‘productive situations’. Productive situations give clues for students to connect a situation with various related mathematical concepts and then to produce multiple assumptions or a required transformation between situational and mathematical worlds.

During the workshop for designing assessment tasks for mathematical literacy, Lin posed several problems to enhance teachers’ perception of the difference between mathematics of literacy and mathematics of examination. For example, he posed the question of estimating the lowest threshold to win an election if eight representatives are to be chosen from 200 members. In general, teachers automatically answered this question based on the assumption that one member only had one vote. The discussion of this problem was used to highlight the fact that assumptions about situations could be implicit and multiple, and that different answers were plausible depending on the assumptions made.

Then, they cooperatively designed about 180 problems that aimed at assessing students’ abilities of using conceptual understanding, procedural fluency, strategic competence, and adaptive reasoning (the elements of mathematical proficiency from the report of Kilpatrick et al. (National Research Council 2001)) to formulate mathematical models, provide mathematical answers, explain mathematical answers in situations, and critique mathematical models or answers. The scope of the 180 problems differs in several ways to the set of PISA problems. One difference is that they deliberately connect different mathematical units, for example the distance between two points at rectangular coordinates and the Pythagorean Theorem. To score each question, PISA uses at most three levels (0, 1, 2) whereas the PISA-like assessment scores are classified into more levels because the mathematics content is much more complex than PISA. Moreover, although the scoring rubric for the assessment is precisely described, it is still a challenge to get consistent assessment across a large number of teachers.

Another consideration is that mathematical proof is not specifically included in the PISA items (given its focus derived from the OECD mandate for life skills), but needs to be included in the reformed assessment in Taiwan. In mathematics, proof is the rigour and logical connection among mathematical knowledge and this greatly differs from proofs outside of mathematics. Assessing proof is essential when the purpose is not only to select the top 15 % of achievers but also to rank them. Mathematical proof is a special text genre in written discourse (see Pimm and Wagner 2003), and the ability to read a mathematical proof requires both mathematical knowledge and deductive reasoning (Lin and Yang 2007). On the contrary, but in accordance with its definition of mathematical literacy, PISA mainly considers plausible reasoning. In Taiwan, a research study showed that 5.7 % of Grade 9 students could give complete arguments to prove that ‘the sum of any two odd numbers must be even’ and a further 37.2 % could give partial arguments that included all information but omitted some reasoning (Lin et al. 2004). Around 36 % of Grade 9 students could construct a correct proof, which required combining several geometric arguments (Heinze et al. 2004) and 18.8 % of Grade 9 students were scored in the top level of reading comprehension of geometric proof (Lin and Yang 2007). Thus, in the Taiwanese situation, items requiring constructing or comprehending proof were also considered to be a necessary and viable part of assessing mathematical literacy of the highest achievers.

Problems Exemplifying Mathematical Literacy

In this section, we provide three problems to exemplify the assessment of mathematical literacy for high achievers. Figure 14.1 shows a problem about the geometry underlying antique architecture. Students are required to actively use rulers to figure out the scale of this picture and then to estimate the length of the diagonal line in the innermost layer of the octagon. An adequate solution is to measure the length of the long diagonal line in Fig. 14.1 with a ruler, then calculate the proportional scale using the known measurement of 5.5 ft. Measure the length of the short diagonal line, then calculate the real length using the scale. Figure 14.2 shows an uncertainty problem concerned with data about buxiban students. Students need to actively identify one advantage with regard to each buxiban and represent this advantage with a suitable statistical chart. For one buxiban, the pass rate (as a percentage) is the highest, for another the absolute number of students passing is the highest, and the third shows steady improvement. Figure 14.3 shows a paper-folding problem where students need to prove the obtained triangles are equilateral.

Fig 14.1
figure 1

Eight Trigrams shaped ceiling and a problem for Grade 9 students

Full size image
Fig. 14.2
figure 2

Buxiban advertisement problem

Full size image
Fig. 14.3
figure 3

Paper-folding problem

Full size image

The content of these questions is included in the Taiwanese junior high school curriculum, and the situations come from students’ life experiences. Nonetheless, our students are unfamiliar with these kinds of questions due to the need to identify that some information is superfluous, the need for to make assumptions and the openness of the potential problem solving strategies.

PISA: Insanity and Retreat

PISA-like assessment reform has become a storm in Taiwan. Three different types of ‘PISA insanity’ are illustrated by the news report in Fig. 14.4, which has been translated by the authors. The news item shows that the effects of PISA-like assessment are found on parents, on buxiban, and on governmental policies. Parents are worried about their children’s failure in the entrance examination. Buxiban are sensitive to the disturbance and take advantage of the assessment reform to make money. Whether there is any positive effect on students’ learning is still questionable. The government is advancing several programs for high school teachers to better understand the PISA-like assessment and to design tasks for developing students’ mathematical literacy. In addition, it is suggested that PISA-like problems should be included in each regular test. Some teachers agree with this reform but others do not. The voices querying the reform are continuously represented in mass media by professors, teacher representatives, parent representatives, and ordinary people; in particular, some of them expressed concern that different scoring criteria would result in unfairness. As to the traditional examinations, the scoring codes referred to one standardised answer with several key steps. The more different students’ answers and the key steps are, the lower scores would be obtained. In PISA-like assessment, the scoring codes refer to multiple plausible answers. The more plausible answers completed, the higher scores would be given until full marks. That is to say: traditional examinations mainly tested what students had not comprehended, but PISA-like assessment focuses on what students should have learned. As a consequence, mathematics teachers may spend more time discussing their ideas about mathematics, its learning and teaching with each other.

Fig. 14.4
figure 4

Article from the China Times of 1 January 2013 (Lin et al. 2013)

Full size image

Several months ago, we were all preparing for the PISA-like assessment. However, in the Taipei City Council in April 2013, regional delegates rejected the proposed assessment reform based on mathematical literacy and reading literacy for selecting the top students, and we suppose there may be progressive transition to the PISA-like assessment. There were multiple factors in the opposition. Most people felt the move was too hasty and there had been inadequate support. When news of this assessment reform was publicised, the database of sample questions was embryonic, and the scoring criteria and the exact time for executing the assessment were still uncertain. Some people oppose the whole idea of selection to the ‘star schools’ and think students should attend local schools in the compulsory years of education. As students concentrate on only mathematics and reading literacy, teachers of other subjects in the buxiban have fewer students and so oppose the reform, and even teachers of the newly assessed subjects are against it because it does not match their regular teaching. The last straw was doubt about the fairness of the reformed assessment. Taiwanese are used to being ‘force-stuffed’ with a standardised answer to a standardised problem; we did not believe open-ended problems could make a fair assessment. In a diploma-driven traditional society like Taiwan, there is always great public concern about assessment.

Reflection

After reflecting on the failure of the assessment, we agree that it is important to align assessment with classroom teaching and learning. However, the premise for such incorporation lies in whether classroom teaching and learning are appropriate to enhance students’ learning power. Based on the fate of this reform, we confess that the national assessment reform was not supported across the system. Failure demonstrated that sometimes political issues are much more influential than the assessment, teaching and learning. Assuming optimistically that the PISA-like assessment has been postponed rather than cancelled, our preparations for it continue. For example, the National Academy for Educational Research, the Curriculum & Instruction Consulting Team of Ministry of Education, and the National Science Council will continue with projects to develop students’ mathematical literacy. Through longer term projects of developing and implementing a new educational system in harmony with the goals of assessment reform, we believe the concern about reform will be eased, the beliefs about the fairness of non-traditional assessment will build up, and the uncertainty surrounding the new assessments will be eliminated. We are also confident that the emergence of the alternative assessments will be beneficial to improve and not undermine our classroom teaching and learning. Hopefully, after a further 3 years of effort in preparation for PISA-like assessment, it will be successfully implemented and it will stimulate our education to focus on developing students’ learning power rather than teaching just for examinations.

In 2009, Grek wrote

The construction of PISA with its promotion of orientations to applied and lifelong learning has powerful effects on curricula and pedagogy in participating nations, and promotes the responsible individual and self-regulated subject. (Grek 2009, p. 35)

She noted that PISA data were applied to justify changes or provide support for domestic and European policy-making processes to different extents in different countries: from the PISA-promotion of the UK, the PISA-shock of Germany to the PISA-surprise of Finland. Like these Western European countries, Taiwan is experiencing the effects of PISA. In the past, PISA data was applied to check whether Taiwanese students were retaining their top ranking. Now, PISA’s theoretical background and assessment Framework strongly influence thinking about examinations and teaching in schools.