Introduction

In recent years, there was a public demand for the revision of the mathematics textbook in Iran for Grade 9 (students about 15 years old). The main reason for this demand was an unwritten consensus that the previous mathematics course, compulsory for all students, was responsible for the very high drop-out rate of high school students at this grade level. Many critics believed that mathematics was difficult and unpopular because of the high degree of abstraction and the lack of real world contexts in the previous Grade 9 mathematics textbook. In response, some stake holders advocated changing the book significantly, not simply revising it and they proposed that the content of Iranian mathematics textbooks needed to change in a way that could meet the needs of all secondary students for both their further schooling and future lives in many different types and levels of occupations. Such direction for change is in line with mathematical literacy, the construct measured by PISA, the Programme for International Student Assessment of the Organisation of Economic Cooperation and Development (OECD). This coincidence provides the rationale for this paper to examine the new Iranian textbook that was produced in response to these concerns. Because Australia has shown consistently high levels of mathematical literacy in comparison with other counties (OECD 2004), the authors decided to compare the newly published Iranian textbook (Bakhshalizade et al. 2008) with two Australian mathematics textbooks, to examine features related to the development of mathematical literacy. The PISA results are additionally relevant because the international study is also conducted with 15 year old students. Iran has not participated in PISA, so there is no formal assessment of Iranian students’ mathematical literacy. However, Iran is a country with a very strong record of achievement in the International Mathematics Olympiad, so it was both unexpected and of great concern when it was found that Iranian students’ performance in TIMSS in 1995,1999, 2003, 2007 was well below the international average. This led to the hypothesis that the education system, with its emphasis on abstract mathematics, does well for the best students but needs to change to meet the needs of most students for their schooling and for their future lives.

The education system in Iran is centralized. Mathematics education goals are set at the national level and the Ministry of Education develops the syllabi and textbooks (Kiamanesh 2005). All students (of the approximately 70 % of the age cohort still at school) learn mathematics at Grade 9 and mixed ability classes are the most common student grouping, before formal tracking begins at Grade 10. In Iran, students use the government-subsidised textbook for every subject including mathematics. As noted above, many mathematics teachers and educators had strong criticisms of the previous 1992 grade 9 textbook, regarding it as inaccessible to the majority of students. One could speculate that the idea behind this textbook was the widespread myth that there is no need to study applications of mathematics since learning abstract mathematics well is adequate to equip a person to apply it in real world situations. This myth has been challenged by de Lange (2003) who indicates that although pure mathematics is essential for doing mathematics in the real world, it is not enough. Many other researchers have taken the same position in this regard (e.g. Niss et al. 2007). While there were some attempts to write a new mathematics textbook for grade 9 from 1998 to 2004, no progress was made until 2008 when the Ministry of Education decided to launch a new textbook. As a result, in less than a year, the new textbook was written, reviewed by a group of teachers, and replaced the previous one in the school year 2008–2009. In the preface of this textbook (Bakhshalizade et al. 2008) the authors indicated that the syllabus of the book had not changed significantly, but the focus had changed and the new direction aimed to include more applications to the real world and broadly speaking, to develop the ability to use mathematics in everyday life. The preface also stressed that the target audience of the textbook is all students. These changes in orientation lead us to conclude this new mathematics textbook was written with mathematical literacy for all students as a goal. A comparison with Australia, with a longer and more established orientation towards mathematical literacy for the whole school population, was therefore likely to be instructive.

The education system in Australia is different in many respects, but there are sufficient commonalities with Iran to support an instructive comparison. Education is considerably less centralised than in Iran. Every state has its own curriculum documents and standards, which give only general guidance about the sequence or content of lessons. In every state of Australia, there are several textbook series produced by commercial companies. Schools and teachers can use any textbook, draw from several, or use none at all. However, nearly all schools select a textbook for secondary students and these are used as the primary or secondary resource for teaching by nearly all teachers (Mullis et al. 2008). In Australia, almost all students are at school in Grade 9, and mathematics in some form is expected for every student with mixed ability classes being the most common student grouping. These are similarities with Iran. Australian curriculum and standards documents (e.g. New South Wales Syllabuses in http://www.boardofstudies.nsw.edu.au/syllabus_sc/#mathematics, and Victorian Essential Learning Standards in http://vels.vcaa.vic.edu.au/support/progression/maths.html) indicate that Grade 9 emphasises mathematics for all students and is oriented towards mathematical literacy. These documents also showed that the mathematical content of Grade 9 in Australia and Iran is sufficiently similar to support a comparison.

There are many aspects of an education system that are likely to contribute to the development of mathematical literacy, including teaching in subjects other than mathematics. However, given the focus in Iran on the new textbook, it was decided to investigate only the textbooks and within textbooks to investigate only the problems that are provided for students to work on. Other aspects, such as the textbook explanations, illustrations etc. and classroom learning activities beyond the textbook are also relevant to developing mathematical literacy, but are not considered in this study. Therefore, we selected two Australian textbooks for the comparison, and then conducted a content analysis of the problems in three chapters that addressed comparable topics. The specific research question was: What are the characteristics of the sets of mathematics problems in the Australian and Iranian textbooks regarding mathematical literacy and how do they compare? Note that the word ‘problem’ is used loosely to refer to the various short and long mathematical tasks and exercises that are provided for students to do; in this paper, a problem may be highly routine or of a challenging nature.

Literature review

The review of the literature begins with mathematical literacy, and presents an overview of the process of mathematical modelling (the key process involved) which will form the basis of the theoretical framework of this study. This is followed by a section reporting studies which have analysed textbooks, especially in response to the results of international studies of students’ mathematical achievement. Further relevant literature is used in the later section describing the framework.

Mathematical literacy

Many countries have participated in the PISA international survey which focuses on the assessment of mathematical literacy as defined in the 2003 PISA mathematics framework (OECD 2003):

Mathematical literacy is an individual’s capacity to identify and understand the role that mathematics plays in the world, to make well-founded judgments, and to engage in mathematics in ways that meet the needs of that individual’s current and future life as a constructive, concerned and reflective citizen (p. 76).

PISA therefore aims to investigate the mathematical learning of students that prepares them for their future lives and work. Mathematical literacy is the central concept of PISA mathematics (de Lange 2003), and mathematical modelling or what de Lange (2003) calls mathematisation, is the key process (OECD 2006). In short, the modelling process starts with a problem situated in the Extra-Mathematical World (an EMW problem) – perhaps a problem concerned with an everyday situation or a problem from another discipline such as physics or biology. The modelling process continues with formulating the EMW problem in mathematical terms. The problem now formulated mathematically is solved by the application of mathematical concepts and procedures. Finally the mathematical solution must be interpreted to provide an answer to the EMW problem, and then checked for its adequacy in answering the original question. If it is not adequate, a new cycle of formulation might begin to improve the model. The problem begins situated in a real context, and in the formulation phase the problem solver gradually trims away aspects of reality, recognizing underlying mathematical relations, organising according to mathematical concepts, and describing the stripped down problem in mathematical terms (OECD 2003). In the interpretation phase, the problem solver considers the mathematical result(s), and uncovers their meaning in terms of the real context and checks their adequacy against the requirements of the real problem (OECD 2003). The diagram in Fig. 1 captures the essence and is taken from the 2006 PISA framework (OECD 2006). Similar descriptions of the modelling process have been used, with variations, by researchers around the world. Verschaffel (2002) explains that there are many different descriptions of the modelling cycle with only slightly different interpretation, and commonalities among them are strong. Examples include Kaiser and Schwarz (2006), Borromeo Ferri (2006), Pollak (1979), Stacey (2002), Verschaffel (2002) and Niss et al. (2007). In Figure 1, the labels have been changed to match the terminology of this paper, but there is no intention to change their meanings. This description of modelling forms the basis of the theoretical framework used in the present study.

Fig. 1
figure 1

A model of the modelling cycle (adapted from OECD 2006)

Full size image

The PISA concept of mathematical literacy is closely related to several different terms, used around the world. To give some examples, in Australia the term numeracy is often used synonymously with mathematical literacy, but in fact its meaning ranges from a narrow ability with number to ability to use mathematics in different situations to ability to understand the meaning of mathematics (Willis 1990). In the United States, there is widespread use of the term quantitative literacy popularised by Steen (2001). Niss (2003) has commented that this is the same as mathematical literacy. Fujita (2004) also claims that he introduced the notion of mathematical literacy as a target for mathematics education in Japanese mathematics curricula in 1980s.

To sum up, there is a widespread effort throughout the world it to equip high school graduates with a kind of mathematics that is needed for life in the new millennium. Wu (2008) has noted the effort of some countries to make necessary alignments between their curriculum and their assessment demands to better meet this need. This and similar studies reinforce the view that if mathematical literacy is regarded as a valuable outcome of schooling, then it should be well represented in school mathematics curriculum and textbooks. This explains our focus on textbooks and introduces a brief review of the studies into textbooks in school mathematics.

Studies of textbooks

Recently, many studies have compared aspects of mathematics textbooks of different educational systems hoping to shed more light to local practices and goals. In this brief summary, we only refer to a few studies that focus on textbook problems. Bao (2004) compared new and old Chinese mathematics textbooks on dimensions which had been identified by Nohara (2001) to predict difficulty of items used in international achievement studies. Nohara (2001) had, for example, found that PISA items (i.e. those which claim to test mathematical literacy) more often use extra-mathematical contexts and have more multi-step reasoning than others. As a conclusion, Bao (2004) pointed out the importance of alignment of the problems in textbooks to desired outcomes in students’ learning. Yeap et al. (2006) compared mathematical problem solving in Singapore and United States textbooks. They classified questions as problems versus routine exercises; whether or not these problems demonstrate Polya-style heuristics; or if, they were primarily intended for the teaching of, for, or through problem solving; and the propensity to model reality.

Studies of comparing textbooks and their problem sets have become popular since the 1990’s when there was a noticeable attention to the first findings of the TIMSS study. For instance, Fan (1999) compared United States and Chinese mathematics textbooks quantitatively. He classified arithmetic addition and subtraction problems into applications and non-applications and found that both series of texts paid almost no attention to applications and instead, concentrated on certain types of addition and subtraction situations. Later, Li (2000) compared all problems relating to the addition and subtraction of integers in several United States and Chinese Grade 7 textbooks. For doing the comparison, he analysed the problems using a three-dimensional framework including mathematical features, contextual features, and performance requirements. The results showed that American textbooks’ problems were more varied in terms of problem requirements and focused more on conceptual understanding than Chinese textbook problems. More recently, Vincent and Stacey (2008) classified textbook problems in several Australian Grade 8 mathematics textbooks by adapting the classification system of the TIMSS video study (Hiebert et al. 2003). This study showed that the characteristics of textbook problems were broadly aligned with the characteristics of the problems as enacted in the random sample of Australian lessons observed in the Video Study, and highlighted the strengths and weaknesses of the textbook problems. These studies show that comparing problems in textbooks can give new insights into characteristics of what students are learning. It is also evident that the comparison needs to be done according to a framework which captures essential features of the textbook which are likely to promote the curriculum goals under study.

Theoretical framework for mathematical literacy

As the above review demonstrates, a content analysis of textbooks and their problem sets requires a specific theoretical framework that focuses on characteristics of problems that are germane to the aims of the research. We propose that each of the elements of the framework identifies an aspect of textbook problems which is relevant to the development of mathematical literacy. We propose that students who regularly encounter problems which rate highly on these characteristics (whether from a textbook or otherwise) are likely to develop better mathematical literacy than others, although testing this hypothesis is beyond the scope of the present study.

This section describes the framework developed for the present study, which has been refined and extended from previous work of the first two authors (Rafiepour and Stacey 2009). In the previous study, the authors used a framework that concentrated on the phases of the modelling cycle where the real world context impinges, namely formulation, interpretation and checking (see Fig. 1). That framework analysed only problems that had an extra-mathematical context (i.e. EMW problems). Given the centrality of problems set within the mathematical world to textbooks, the framework used in current study applies to all kinds of mathematical problems and codes all phases of the mathematical modelling cycle.

In this section, we define the framework components and sub-components, drawing on further relevant literature. The discussion is illustrated with sample problems given in Appendix A. The framework is summarised in Table 1. Appendix B gives the classifications of all the sample problems. The textbook names are abbreviated to Ir (Iranian textbook), Aus1 (first Australian textbook) and Aus2 (second Australian textbook). For conciseness and to make clear the links between the source literature and the final framework, this section also describes some outcomes of the pilot study (see below) which was used to refine the coding and framework.

Table 1 Components of the Framework for Mathematical Literacy and sample problems*
Full size table

Problem context

The first component of the framework classifies the context of the problem, according to whether the statement of the problem includes a real-world context or whether it is entirely within the mathematical world (i.e. its statement only references mathematical objects). To be classified as an EMW problem, the problem text must include extra-mathematical elements and as a consequence, the solution begins and/or ends on the left hand side of Fig. 1.

This framework component began with the PISA classification of the problem situations in which four groups of contexts (personal, educational/occupational, public, and scientific) are distinguished (OECD 2003). In this 2003 PISA framework, problems which lie entirely within the mathematical world are classified as scientific. This classification is viable for PISA coding because there are very few such items in PISA, since the intention is to test mathematical literacy (OECD 2006). However, for good reasons, mathematics textbooks have many problems set within the mathematical world, and therefore identifying these clearly was essential for this study. Extra-mathematical world problems were initially additionally coded according to the PISA context categories above; some observations on this are reported in the results section. However, this analysis was too fine grained and the scheme was simplified. Examples of problems coded as EMW (see Appendix A) are Aus1_1 which related to walking in mountains and Aus2_1, which is concerned with electric charges. An example of a problem entirely within the mathematical world is Ir_2 (solving linear equations).

It is important to note that the classification as EMW problem is judged only on the presence of a context, and not on the way in which the context is required in the solution of the problem. Stillman, as described in Goos et al. (2007), distinguishes three ways in which context may be involved in a solution. There are border problems where the real world context is irrelevant to the solution, wrapper problems where the mathematics is hidden within a context which can be stripped away and then discarded, and tapestry problems where considerations of the context is entwined with the mathematical solution, and there is continual reference to both. Stillman contends that the tapestry problems are more likely to promote mathematical modelling competence. These would therefore be most relevant to mathematical literacy. However, the present classification does not distinguish border, wrapper and tapestry problems, only the presence of the EMW context.

Mathematical problem solving

In the intra-mathematical problem solving phase of the modelling cycle depicted in Fig. 1, a mathematical solution is found within the mathematical world. All of the textbook problems we coded required this phase, although this need not be the case for textbook exercises, which could theoretically require only formulating or interpreting. The mathematical problem solving (MPS) requirements within the mathematical world are analysed with three separate sub-components:

  • Multi-step MPS

  • Connection in MPS

  • New MPS

A problem is classified as multi-step mathematical problem solving if students are to solve one part of a problem (a sub-goal) and use it in other steps to solve the problem. Nohara (2001) noted this as a characteristic of PISA items. This sub-component of the framework is similar to procedural complexity, used by Hiebert et al. (2003). Hiebert’s definition of multi-step problems included a count of the number of mathematics steps as well as the presence of sub-goals, but here only the presence of sub-goals is coded. Frequently textbooks divide multi-step problems into a chain of single-step problems which lead to the overall solution. In such cases, the definition of the unit of analysis is critical. This is discussed in the methodology section below. The problem Aus1_1 which requires a series of trigonometric calculations is a clear example of a multi-step problem. The problem Ir_2 is not coded as multi-step MPS: whilst each of its separate parts could possibly be regarded as a sub-goal, these are ends in themselves and the solutions are not used in later steps to solve any problem.

The second coded sub-component of MPS relates to connections, which has been a major theme in recent discussions of good mathematical learning (e.g. Hiebert et al. 2003). This sub-component records whether the mathematical ideas, procedures and facts from more than one mathematical topic are required to solve a problem. This is often required in solving problems from real life. Students with good mathematical literacy have knowledge which can cross chapter boundaries. Ir_1 is coded as a problem involving connections because it requires use of Pythagoras’ theorem and trigonometry.

The sub-component New MPS is intended to give a measure of the different types of mathematical problem solving that students meet within the textbook problems. Being able to deal with a variety of problem types relates positively to mathematical literacy because students need to deal flexibly with a wide range of problems. This sub-component is based on the definition for repetition by Hiebert et al. (2003) and adapted by Vincent and Stacey (2008) to classify textbook problems. A textbook problem is regarded as a repetition in MPS when its solution requires the same or almost the same solution steps as an earlier problem or worked example. The numerical or algebraic expressions involved will be similar (for students at that level), rather than identical. Vincent and Stacey (2008) concluded that repetition in this sense is too frequent in Australian lessons and textbooks although they acknowledged its important role for improving skills in mathematics. The problem Aus1_3 is a problem with repetition in mathematical problem solving because at least one previous problem (e.g. Aus1_2) has the same mathematical solving process. Aus1_2 (first part) requires solving the equation 3.25 + 0.72x =12 and Aus1_3 requires solving the equation 1200 + 0.95x = 2100. Hence, Aus1_3 is coded ‘no’ for the component New MPS.

Formulation

The third component of the framework applies only to problems in an extra-mathematical context and has four sub-components. The first sub-component, New context, records whether the context is new or a repetition i.e. if it has been used before in the same chapter of the textbook. Being exposed to a variety of contexts in which mathematics is applied seems to be a useful preparation for using mathematics in these contexts and hence for mathematical literacy. The problem Aus2_3 is coded as ‘no’ to New context because there is at least one earlier problem in the chapter about a ladder leaning against a wall. In coding this sub-component, we do not regard it as repetition if the context is taught or used in other chapters.

Complexity of formulation is the second sub-component of formulation and it is judged on the presence of one or more factors that make the formulation process complex: (a) having too much or too little data, (b) requiring multiple steps of formulation, or (c) involving a situation where it is difficult to identify the relevant mathematics. For instance Aus2_2b is classified as having complex formulation, especially because of the multiple steps involved in describing the positions of both riders at a given time. The formula distance = speed × time applies to both riders, but the position of rider Y has to be adjusted because his starting point is at the far end of the track and he travels towards the origin of co-ordinates.

The third sub-component is whether the formulation is done by students. For instance Aus2_2b is classified as a problem to be formulated by students, because it is the student’s task is to set up the mathematical equations from the real context. If the text of this problem had included formulas such as x = 30 t and y = 40 – 20 t then the problem would not be coded as requiring formulation by students. Textbook problems frequently give such assistance.

Finally, the fourth sub-component, New formulation, examines the variety of formulation tasks that students see. Does the textbook provide different opportunities for students to find relevant mathematics, sort relevant from irrelevant data and connect them together to formulate the contextual problem as a mathematical problem? Again a variety of such experiences would seem to strengthen mathematical literacy. For example Aus1_2 and Aus1_3 (see Appendix A) both show repetition of formulation. In Aus1_2, cost is a linear function of number of kilometres (initial cost and cost per kilometre specified) and the central task is to find the distance for a given cost. In Aus1_3, cost is a linear function of number of units (initial cost and cost per CD specified) and the task is to find the number of units for a given cost. Aus1_2 is not coded as a new formulation because a demonstration problem (worked example) with this mathematical structure was solved earlier in the chapter. Aus1_3 is a repetition both of the demonstration problem and of Aus1_2. Thus, both problems Aus1_2 and Aus1_3 coded as ‘no’ for New formulation.

Interpretation and checking

A final component of the framework examines whether interpretation and checking is required or not. Some problems set in an extra-mathematical context require non-trivial interpretation of mathematical answers back into the context of the problem to fit the real situation. For example, there may be more than one correct answer to a mathematical problem but not to the real world problem, or the problem may have one mathematical answer which needs to be modified to fit the real world situation (for example, by non-routine rounding). In the coding process for this study, we have tried to distinguish between mere comparison of results and active interpretation. We combined the interpretation and the checking into one component after analysing the data from piloting our framework for the final study, because only a few problems in our sample required this active interpretation and/or checking.

To give an example, consider the problems Aus1_2 and Aus1_3. These two problems are coded as ‘yes’ for requiring interpretation and checking since after finding a mathematical answer, students have change that mathematical answer to adjust it into the real world answer. On the other hand, Aus1_1 has been coded as ‘no’ because the mathematical answer is the final answer of the problem. We do not know of previous attempts in the literature to code for these phases of the mathematical modelling cycle.

Methodology

Content analysis methodology has been used for the study of textbooks for many years. Pierce (1930) is an early example. Krippendorff (2004) has described content analysis as a research technique for making replicable and valid inferences from a body of written or other meaningful matter to the contexts of their use. For conducting the content analysis, we developed the framework that was introduced in last section and used it to analyse the content of the selected textbooks with regard to fostering mathematical literacy.

Sampling: Textbooks, chapters and problems

The sampling for the study was a goal-directed process focusing on mathematical literacy. For comparison with the Iranian Grade 9 mathematics textbook, we first selected the two grade 9 Australian mathematics textbooks that were the best selling in the two most populous states of Australia, using available sales data. By making this choice, we selected textbooks that are commonly used, and well regarded by those who select textbooks for schools. We do not make further claims about how representative they are of textbooks in Australia.

The selection of textbook chapters was based on four following factors:

  • Topics that have the potential to be supportive of mathematical literacy;

  • Topics that are introduced for the first time - not continued from previous years;

  • Similarity of content in the chapters in all three textbooks;

  • Suitability of topics for further international comparisons.

Taking these factors into account, we were able to find three chapters from each textbook which contained topics which we judged to be sufficiently comparable for the study. For the purpose of mathematical literacy, all of these topics have the potential for numerous real-world applications, including many within the experience of 15 year old students. The approximate size of the sample of each textbook was one quarter of the 678 pages of Aus1, one fifth of the 520 pages of Aus2, and one third of the 186 pages of the Iranian text. Although in this selection process, the exact match of the mathematical content was not possible, the overlap was roughly 90 %. Table 2 contains detailed information. The trigonometry chapters were each about 7 % of the total book, but the two algebra chapters were a greater proportion of the Iranian book than for the Australian books.

Table 2 Chapters selected for analysis with number of pages and percentage of total textbook pages
Full size table

Within the chapters, we also needed clear, common decision rules about which problems were coded. We coded the problems that appeared in the exercise sets of each section of the nine selected chapters. It is important to note that short exercises (for example, Ir_2) were considered as problems, not only complex questions that richly deserve the label. Since we intended to study the problems that students tackle, we did not code worked examples. To avoid potential different purposes amongst the problem sets, we excluded problems identified as revision of earlier work and in tests. Puzzles or challenges that are labelled separately, or problems discussed in specially identified sections such as the history of mathematics were also excluded. In the Aus2 textbook, the selected chapters contain a section named Working mathematically where problems are more oriented towards modelling. This section was not included in the coding because it did not appear to be an expected part of a regular teaching sequence. One reason for this decision was that other sections were numbered 8.01, 8.02 etc., but this section had no numbering. Hence we excluded it from the coding, although noting that it may well play an important part in the development of mathematical literacy.

For content analysis, it is important to define the unit of analysis. The units of analysis are either (a) individual problems (e.g. Aus1_1 in Appendix A), (b) problems of a similar nature according to the framework and grouped as one in the textbook (e.g. Ir_2, Appendix A), or (c) separately identified parts of a numbered problem (e.g. Aus2_2a, in Appendix A). Multiple-part problems have been carefully identified as either one or multiple units of analysis. For example, the problem Ir_2 is labelled as one in the textbook and its several parts each ask for solving an equation of the same form so it is therefore one unit of analysis. However, parts (a) and (b) of Aus2_2 are treated as separate problems (i.e. 2 units of analysis) because these parts have a different context and/or method and therefore have different coding. Hereafter the word ‘problem’ applies to these ‘units of analysis’.

Framework refinement and coding

The data consists of classifications of the problems in the selected chapters according to the framework. The coding process required two coders who had good understanding of the framework, solid knowledge of mathematics, and reasonable ability in both languages of English and Persian (Farsi). One of the coders was the first author, and the other one was trained in the framework by him. Initially, the two coders began coding in order to refine the framework and criteria. The outcome of this first stage was refinement in the definitions of the components, but also several components of the initial framework (not listed in Table 1) were deleted because of the low inter-rater reliability or lack of use. To give an example, the coders initially aimed to code the degree and the nature of interest of the problem contexts. However, there was not even a slight agreement on this issue between two coders. For instance, one coder regarded football contexts as interesting and the other did not agree. Consequently this component was removed from the framework. Another example is that interpretation and checking were merged because of low frequency of coding and difficulties in distinguishing them. After this refinement stage, the two coders independently coded all 9 chapters according to the final framework and definitions, producing the data presented and analysed in the next section.

Results and discussion

The results of the coding are shown in Table 3. Together, the three textbooks had 302 mathematical world (MW) problems and 159 extra-mathematical world problems (EMW). Mathematical world problems were coded for MPS features only, whereas EMW problems were coded for all components of the framework. Thus, we had 1208 decisions to make for mathematical world problems and 1431 decisions to make for extra-mathematical world problems. In the final coding, there were only 15 disagreements between coders for the 2639 decisions that were made. Table 3 reports both absolute numbers and percentages, to give a picture both of the opportunity to engage with problems of a particular type (absolute) and the balance of the program (percentage).

Table 3 Number and percent of problems by framework components
Full size table

The first observation from Table 3 is that there are many fewer problems in the Iranian textbook even though the 3 chapters make up a larger proportion of the Grade 9 textbook chapters compared to the corresponding parts of the Australian textbooks. This structural feature has many consequences, discussed below. When considering the percentages of problems in each classification, there are many similarities between the Australian and Iranian textbooks and some differences. On some components, there was greater Australian intra-country variation than between-country variation. This reflects the diversity and nature of textbook provision in the Australian educational system where there are curriculum differences between different educational jurisdictions, schools have considerable freedom in selecting resources, and textbooks can be designed to meet the needs of different student cohorts and the preferences of different teachers. For the purpose of the inter-country comparison, we do not focus on these differences, but use the two Australian percentages as a guide to the Australian range.

Problem context

Close to a third of the classified items in each textbook were extra-mathematical world problems. As noted above, initially problem contexts for the EMW problems were classified according to the PISA classification, but it did not fit well. Most EMW problems had a context in everyday life and school activities, usually personal to the immediate experiences of students and also usually social in nature often involving money, games, travel, public transport, sport and such. There were very few occupational or scientific contexts. In total, there was 1 problem in a scientific context in Aus1, 7 in Aus2 (including the sample problem Aus2_1) and 2 in Ir. Although science is a major area of application of mathematics, including the newly developing Year 9 topic of algebra, this context was rarely used. We speculate that authors consider that the mathematics might be made more difficult by requiring an understanding of scientific topics. This may imply that a teacher who seriously feels a responsibility to prepare students to use mathematics in other subjects including science, will need to do this outside the provision of any of these textbooks.

At least partly as a consequence of the small number of problems, the context variation of the Iranian textbook was severely limited in comparison to the Australian textbooks. Across the 3 chapters, the Iranian text had 8 different real world contexts in total whereas the Australian textbooks included about 50 each.

Mathematical problem solving

The three sub-components of mathematical problem solving taken together indicate that the Iranian textbook uses at least as high a percentage of problems requiring demanding mathematical solutions as the Australian textbooks – indeed arguably higher on balance. The Iranian textbook uses a slightly higher proportion of multi-step problems, considerably more new types of MPS and a relatively high proportion of problems where connections need to be made. Both Australian textbooks show a high degree of repetition of MPS, with under 10 % of problems requiring a new method. We conjecture that this is driven by a need to provide for a wide range of abilities of students, including those who need much more practice with simple topics than other students. However, the Iranian textbook does not make this provision. The somewhat lower proportion of multi-step problems in the Australian textbooks might also be seen as further evidence that the Australian textbooks provide more material for less able students. Taken over the three measures, the Aus1 textbook appears to require the least demanding mathematical problem solving.

Formulation

On the four formulation sub-components, the Iranian textbook is within or nearly within the Australian percentage range. However it is again important to consider the absolute numbers, both for diversity of context and of formulation. Each of the Australian textbooks provided about 50 problems with new contexts giving a diversity of contexts where using the targeted mathematics is required. However, there are only 8 problems with new contexts in these chapters of the Iranian textbook. In the content analysis, we observed that in some instances in all three textbooks, all problems in a section of a chapter use the same formulation processes. Furthermore, in the three chapters of the Iranian mathematics textbook, no problems are coded as requiring a new formulation. This means that the Iranian textbook always demonstrates the mathematical model in advance (e.g. by worked example). In contrast, both Australian textbooks include some problems where the formulation has not been explicitly demonstrated. Over all four aspects of formulation, in percentage terms, the Aus1 textbook seems to make fewer demands on student formulation than the other textbooks.

Interpretation and checking

All three textbooks had similar proportions of problems requiring interpretation and checking and the number and percentage of such items was low in all textbooks. One of the features of the type of problems that students will face in their future lives is the need for interpretation of mathematical results and checking against the real world requirements. These findings indicate that these textbooks provide few resources to develop such competence and teachers need to find other resources to teach them.

Implications and conclusion

In summary, the results of the content analysis indicated that there are similarities and differences of the three textbooks and two countries with respect to the criteria that we presented for mathematical literacy. The great difference in the absolute number of problems means that the Iran textbook provides a limited range of problems and problem contexts in comparison to the Australian textbooks. This is quite possibly due to the different economic circumstances of the two countries impacting on textbook size, and its compulsory use. However, the very large number of problems in the Australian textbooks enables a very wide variety of contexts to be encountered by students and also enables provision for students who need much more practice with simple topics than other students. This orientation could be indicated by the generally less demanding mathematical problem solving and formulation of the Aus1 textbook. The structural feature of there being many problems in Australian textbooks implies that Australian teachers must carefully select the problems for their students so that more able students do not waste time over-practising skills which they have already mastered. In contrast, Iranian teachers may well need to supplement the textbook problems so that students encounter a variety in contexts of application, they cover different aspects of mathematical problem solving and the needs of different student groups are met.

Whilst, in terms of percentage, the other formulation measures for the Iranian textbook fell between the two Australian textbooks, it was notable that there was no requirement in the Iranian problems for students to formulate a new mathematical model themselves. This shows that all the EMW problems in the Iranian textbook could be regarded as standard applications (rather than as mathematical modelling) which only partly fulfil the mathematical literacy requirement for modelling. Both of the Australian textbooks provided some experience of this. All three textbooks had very low absolute and relative numbers of problems requiring either interpretation or checking. Teachers in both countries who are committed to mathematical literacy would therefore need to supplement this aspect, so that students have some exposure to all parts of the modelling cycle.

The study has several limitations that are important to acknowledge in avoiding inappropriate over-generalization of the findings. It focused only on textbooks and within textbooks only on the problems which students do. However, many planned and incidental learning experiences beyond work on textbook problems contribute to the development of mathematical literacy. We selected only two textbooks from Australia, which may not be representative of the diverse mathematics textbooks in that country. We limited the study to Grade 9 mathematics textbooks because it is the target grade for PISA, even though the teaching of mathematical literacy is a multi-grade enterprise. The study was limited to the content analysis of only three comparable chapters from each textbook - not all chapters are similarly comparable. We also note that students may not do all of the problems in the textbook, and so the selection of problems which they encounter may have a different balance to that of the whole book. Our classification of problems simply as EMW or MW on the simple presence or absence of real world context could, in a future study, be refined to distinguish Stillman’s border, wrapper and tapestry problems. Finally, only the problems of the main sections of each textbook were coded and we did not consider either the whole approach of the textbook nor of supplementary materials that the Australian teachers, in particular, might use. For example, the Aus2 textbook has a section entitled Working Mathematically and another entitled Physical Phenomena which are both likely to develop mathematical literacy but were not included in our analysis since the other two textbooks did not have similar sections. Even without including these special sections, in both absolute and relative terms, the Aus2 textbook had the most number of problems that required demanding formulation, the most number of problems that need to be formulated by students, the least number of repetitions of formulation and the greatest percentage of problems that require making connections. Therefore, according to our criteria, and especially with its additional sections, it seems that the Aus2 textbook is the one most likely to prepare students for mathematical literacy.

Based on the findings of this study, we suggest that authors of mathematics textbooks similar to the new Iranian textbook might further encourage mathematical literacy by:

  • Including more diverse contexts for textbook problems;

  • Including problems without repetition in formulation, and more which require formulation by students;

  • Including some problems which are closer to genuine modelling, especially tapestry problems;

  • Further considering whether the textbook is accessible to all, because the aim of teaching for mathematical literacy applies to students of all abilities.

Additionally, we suggest that curriculum developers, authors of mathematics textbooks and teachers in any country might use at least the four top-level components of the theoretical framework (Context, Mathematical Problem Solving, Formulation, Interpretation and/or Checking) as a checklist to examine textbooks for their capacity to promote mathematical literacy and to select the subsets of problems which students are asked to work on. Countries with a system for approving suitable textbooks might employ the framework to identify how each textbook is likely to promote mathematical literacy.

Finally, even though mathematical modelling is at the heart of the mathematical literacy (OECD 2006), we must acknowledge that neither in this study nor in our other experiences with school textbooks in Australia or Iran, have we seen many examples of problems that really meet the criteria for a genuine modelling problem. For example, with context, although this study did not formally classify problems as Stillman’s border, wrapper or tapestry problems, we saw that there are few of the latter group. As Galbraith and Stillman (2001) have explained: “genuine modelling problems do not have a ready mathematical formula, do not give explicit clues for solving, have a situation which has not be ‘cleaned’ for mathematisation, and do not have explicit mathematics in the problem statement” (pp. 302–303). Nevertheless, as Verschaffel (2002) has pointed out, there are many problems that do not meet all these criteria that can contribute to the development of mathematical literacy in students. Full growth of mathematical literacy requires genuine and diverse experiences far beyond the textbook problems of any of the textbooks reviewed, but a textbook still has an important contribution to make.

Appendix A. Sample problems

These problems have been selected to illustrate the coding scheme. Full codes are provided in Appendix B. Problems from the Iranian textbook have been translated into English by the first author. Problems are named according to the source. Diagrams are omitted.

ABBREVIATIONS: Ir: Iran textbook; Aus1 & Aus2 are the Australian mathematics textbooks.

Aus1_1: Elena and Sonja were on a camping trip to the Grampians, where they spent their first day hiking. They first walked 1.5 km along a path inclined at an angle of 10° to the horizontal. Then they had to follow another path, which was at an angle of 20° to the horizontal. They walked along this path for 1.3 km, which brought them to the edge of the cliff. Here Elena spotted a large gum tree 1.4 km away. If the gum tree is 150 m high, what is the angle of depression from the top of the cliff to the top of the gum tree? (Diagram supplied by textbook shows where tree is growing)

Aus1_2: The Green Cab taxi company charges $3.25 plus $0.72 per kilometre. Michael has $12 left after going out for dinner. How far can he go in the taxi?

If he lives 13 km from the restaurant, will he make it home in the taxi? If not, how far will he have to walk?

Aus1_3: The cost of producing computer CD-ROM disks is quoted as $1200 plus $0.95 per disk. If Maya’s recording studio has a budget of $2100, how many CDs can she get made?

Aus2_1: The formula \( F = \frac{{k{q_1}{q_2}}}{{{r^2}}} \)gives the force between two point charges of q1 and q2 coulombs that are r metres apart. If two equally charged balls are placed 0·1 m apart and the force between the balls is 9·8 × 10−4 newtons, calculate the charge on each ball. (k = 9·0 × 109)

Aus2_2:

  1. a)

    Six kilograms of an inferior tea is mixed with 3 kilograms of tea that costs $2 a kilogram more. The total price of the mixture is $24. What was the price of the inferior tea?

  2. b)

    Two bike riders X and Y both start at 2 pm riding towards each other from 40 km apart. X rides at 30 km/h, Y at 20 km/h. If they meet after t  hours, find when and where they meet.

Aus2_3: A ladder leans against a wall so that the angle it makes with the ground is 52° and its base is 4 m from the wall. How far does the ladder reach up the wall (to the nearest centimetre)?

Ir_1: A person 1.70 m high wants to raise a 3 m bar to an angle of inclination of 60°. At first she is standing with one end of the bar on the floor and against a wall, holding the other end exactly up to her height. Then she walked toward the wall raising the bar until the angle of inclination became 60o. How many metres did she walk to the wall? (Diagram supplied)

Ir_2: Solve these equations: (parts d, e, f, g omitted)

$$ {\text{a)}}\;16 - 4y = 0\quad \quad {\text{b)}}\;100 = 50 - 4d\quad \quad c)\;\frac{{\left( {a - 3} \right)}}{4} = 2 $$

Appendix B

Table 4 Sample coding for Appendix A problems*
Full size table