Competence Models as a Basis for Defining, Understanding, and Diagnosing Students’ Mathematical Competences

Abstract

Enhancing students’ mathematical competences and supporting their development are the major goals in the mathematics classroom. However, evaluating to what extent these goals are reached is a complex task for teachers because it requires a profound definition of what mathematical competence actually means at various levels. It also requires diagnostic skills to adequately interpret students’ responses to certain problems and to identify their problems in learning, understanding, and applying mathematics. Competence models could be a way to assist teachers in their diagnostic processes. However, current models lack information relevant to the diagnostic process, such as successful learning trajectories or students’ misconceptions and typical errors. In this chapter, we discuss how competence models can be seen as useful tools for defining, understanding, and diagnosing mathematical competence. We also provide suggestions on how to extend current models to make them more powerful for teachers’ diagnostic processes of their students’ mathematical competences.

Competence Models as Normative Definitions of Educational Goals

What students should learn in the mathematics classroom and, in particular, what they should understand and be able to do has been discussed intensively for many years. While in former years curricula focused mainly on the mathematical contents as input of instruction, the attention shifted to its outcome more recently. In consequence, standards for school mathematics were implemented in numerous countries in the last years (e.g., Kultusministerkonferenz, 2003, 2004, 2012, in Germany; Common Core State Standards Initiative, 2012, in the USA). Standards are normative tools in education. They describe the aims of schooling and illustrate what students are supposed to understand and to achieve. Moreover, they define the mathematical problems students should be able to solve.

Educational standards typically address students’ competences. The concept of competences encompasses content-related knowledge as well as ways and means to apply this knowledge within a subject or in a general context. In this sense, competences have been defined by Weinert (2001, p. 27 f.; original citation in German, translation see Klieme et al., 2004, p. 16) as “cognitive abilities and skills possessed by or able to be learned by individuals that enable them to solve particular problems, as well as the motivational, volitional, and social readiness and capacity to utilize the solutions successfully and responsibly in variable situations.” Competences are according to this definition neither personal traits nor general characteristics. They may be regarded as domain-specific requirements in order to solve a problem and may be acquired by an individual via learning. Standards thus also reflect mathematical literacy as defined in PISA, the Programme for International Student Assessment that emphasizes “an individual’s capacity to formulate, employ, and interpret mathematics in a variety of contexts” (OECD, 2013, p. 25). In PISA, as in most other contexts where applicable knowledge is addressed, motivation, volition, and social readiness do not play a prominent role. In the following we will therefore concentrate on the cognitive aspects of competences.

To define what competence means in a particular domain, standards rely on competence models. In the case of mathematical competence, these models include descriptions of mathematical contents like numbers and operations, algebraic thinking, functions, geometry, statistics, and probability as well as mathematical activities like problem-solving, reasoning and argumentation, modeling, use of tools, communication, and identification of mathematical structures. For example, the German standards for school mathematics at the primary level (Kultusministerkonferenz, 2004) state that with respect to numbers, students should acquire a variety of abilities: An important aspect is to understand place value and numbers up to 1,000,000 and their properties. Students should also be able to add, subtract, multiply, and divide whole numbers both mentally and in written form and recognize the relations between these basic arithmetic operations. They should develop different solutions to arithmetic problems, identify errors, control results, and use arithmetic rules. In addition, students are supposed to apply their knowledge in different contexts. They should be able to solve real-world problems using exact or approximate calculation and verify the results.

With such definitions, educational standards can provide guidance concerning the goals of learning. However, intending to define mathematical competence from a normative perspective, standards often seem to presuppose that teaching and learning take place under good or even optimal conditions, for example, in well-appointed rooms, with well-educated teachers, and in front of attentive students (e.g., National Council of Teachers of Mathematics, 2000; cf. Reiss, 2009). Thus, standards do not provide information about results of less successful learning and especially not about the knowledge acquisition of students with learning difficulties. Furthermore, educational standards specify the goals of classroom instruction but usually do not give recommendations or show ways how teachers should actually reach the goals in the classroom (Klieme et al., 2004). Accordingly, standards lack information about concrete steps leading to students’ competences. For doing so, more fine-grained models would be necessary that describe mathematical competence on various levels and provide information on possible learning gains and learning gaps. Such models may also be used for empirical evaluations of students’ competences and should be apt for understanding successful as well as ineffective learning processes.

Competence Models to Understand and Evaluate Students’ Learning

In order to evaluate to what extent students meet the goals described in educational standards, the introduction of standards has often been accompanied by the implementation of testing procedures. Assessment instruments were based on models of mathematical competence that describe this competence in a hierarchical manner. Competence models that have been used for international comparison studies such as TIMSS (the Trends in International Mathematics and Science Study) for primary school students (e.g., Mullis, Martin, Foy, & Hooper, 2016; described as “international benchmarks”) or PISA for secondary students (e.g., OECD, 2016; described as “levels of proficiency”; see also Reiss, Sälzer, Schiepe-Tiska, Klieme, & Köller, 2016, p. 226, for a more detailed report on competence levels in PISA) characterize mathematical competence based on empirical data. Accordingly, they do not aim at describing desirable knowledge as educational standards do but realistic and mostly empirically confirmed knowledge. In particular they reflect the important differences in students’ performance and allow the appreciation of high-achieving students as well as the assessment of performance at a lower level and of students with learning difficulties.

These descriptions of proficiency or competence levels in the large-scale studies mentioned above were presented first in the late 1990s. They were accompanied by sample tasks and turned out to be useful for getting an idea of what students’ performance at a certain level really meant. However, the levels of proficiency within these models were not sufficiently “fine-grained” but lacked details of mathematical processes and their products. The models mentioned above provided only rough information and, in particular, could not be used to explain how students would proceed from one level of proficiency to the next.

As a consequence, Reiss and Winkelmann (2009) presented a model of competency for the primary mathematics classroom (grade 4), which took into account more details of the students’ actual problem-solving behavior. The model was based on data of a representative sample of students and of test items and was particularly used in the course of further test development. Moreover, it was extended and refined by Reiss, Roppelt, Haag, Pant, and Köller (2012) based on a larger number of test items and of participating students, thus using more representative data. The model includes descriptions of levels of competence with respect to different mathematical topics, such as numbers and operations or geometry or probability. The different levels of competence within the model were defined in a way that each level covered the same range of test points. In the following, the levels are described for competences concerning numbers and arithmetical operations.

Level I (Lowest Level): Basic Technical Knowledge (Routine Procedures Based on Elementary Conceptual Knowledge)

At this level, students know the basic structure of the decimal system such as the classification of numbers into ones, tens, hundreds, etc. Students are familiar with basic single-digit multiplication and addition problems. Subtraction and addition of lower numbers can be completed in partly written form. While doing this, students are able to check for the accuracy of their solutions. Written addition can be utilized correctly if two summands are involved. Written subtraction can be utilized if the carry is less than ten. In simple problems, students make use of the relationship between addition and subtraction. Strategies that students have learned during their first years at school – such as doubling a number – are applied to larger numbers. One-digit numbers or numbers below 1000 with last digits 0 or 00 can be placed on a number line with appropriate scale. Such numbers can be compared according to their size.

Level II: Basic Use of Elementary Knowledge (Routine Procedures Within a Clearly Defined Context)

Students use the structure of the decimal system when dealing with various representations of numbers. They recognize ordering principles and utilize these principles when continuing number patterns or during structural counting. Simple problems related to basic types of calculation are conducted mentally but also in a partly written or fully written form; occasionally, students find the solutions through systematic trial and error. During such trials, students make rough estimations and use them to determine the value range of their solutions. They correctly utilize fundamental mathematical terms (such as “sum”) as well as basic mathematical procedures to solve simple word problems.

Level III: Recognition and Utilization of Relationships Within a Familiar Context (Both Mathematical and Factual)

The numbers that were taught as part of the curriculum are securely read and written in various representations (such as in a place value panel). Also, the number zero can be assigned correctly. Students are proficient in every type of a partly written or of a fully written calculation procedure that is part of the curriculum, but division is limited to single-digit divisors. They can use basic procedures of mental arithmetic even in unfamiliar contexts. They can transfer the multiplication table to a larger range of numbers, perform rough estimations, and round the results meaningfully, even with large numbers. Students recognize the relationship between addition and subtraction, as well as between multiplication and division. They can recognize and communicate simple structural aspects (e.g., in relation to sequences of the multiplication table) if the contents were practiced before. In addition, they model simple object matters and find solutions – as long as the numbers used are within the number range covered by the curriculum.

Level IV: Secure and Flexible Utilization of Conceptual Knowledge and Procedures Within the Curricular Scope

Students solve problems securely using all types and variations of the calculations taught as part of the curriculum. In particular, this includes written division. During calculations, students systematically utilize the attributes of the decimal system and relations between operations. They also apply this knowledge when investigating number sequences, for example, when finding incorrect numbers in a sequence or when explaining the underlying procedures for the sequence. Different calculation procedures are combined flexibly, and solutions are estimated or rounded appropriately. Students use solution strategies such as systematic trial and error even for more complex problems. Students are familiar with rules for calculation, and they can apply these rules meaningfully. Students are able to adequately model, and correctly work on, complex situations, and to present their solutions appropriately. Students’ conceptual knowledge also includes special technical terms they can use and communicate appropriately.

Level V: Modeling Complex Problems and Independent Development of Adequate Strategies

Difficult mathematical problems can be solved correctly using various strategies. Relations between numbers are recognized according to the situation. Mathematical rules, such as the factorability of natural numbers, are utilized in problem-solving processes. Based on basic mathematical principles, even difficult solutions can be worked on and are solved utilizing procedures such as systematic trial and error. Special aspects such as calculations with fractions or numbers in decimal notation do not pose any problems. Moreover, students are able to comprehend and describe different solution approaches.

The model covers the key topics of numbers and operations and includes computation, estimation and number sense, word problems, and the structure of the whole-number system, which may be regarded as important aspects of this knowledge domain (Verschaffel, Greer, & DeCorte, 2007). Moreover, it takes into account that regarding products and processes, respectively, conceptual and procedural aspects of knowledge interact in problem-solving processes and complement each other (Hiebert, 1986).

According to Pant, Böhme, and Köller (2012), students performing at level II are regarded to master a minimum standard in mathematics at the end of grade 4. These students should be able to successfully participate in further instructions in the next grade. Students performing at level III and IV perform on average or slightly above; students at level V show outstanding mathematical competence. Thus, the model may help to identify the individual level of performance and may be suitable to describe gaps of knowledge and competence.

Competence Models to Better Understand the Difficulty of Mathematical Problems: Examples

Teachers’ diagnostic proficiency encompasses knowledge about the competences students need to have in order to solve specific mathematical problems. The competence model presented above can be used to describe these competences. This way, the model may help teachers to classify the requirements of a particular task and the proficiency of their students in solving this task. The following examples will illustrate how these aspects complement one another. The items shown below were used in a nationwide mathematics test for German primary schools. This test was completed by nearly all students and administered by teachers. The data presented below come from a pilot study administered by professional test personnel. The study yielded data on item characteristics like difficulty and solution rates as well as written solutions of students.

The first item presented here addressed the place value of whole numbers (Fig. 4.1; see also Obersteiner, Moll, Reiss, & Pant, 2015). For a correct solution, students were supposed to argue why the place value table did not represent the number 370. The item asked for a basic understanding of the place value system and was thus regarded at competence level II from a theoretical point of view. The empirical data substantiated the classification in level II as 56% of the children gave a correct solution. The information that theoretical and empirical difficulty were identical does not only verify the model but may also help teachers in understanding what low performance means with respect to students’ knowledge.

Fig. 4.1

Sample problem “Place Value” (translated from the German original)

Moreover, the artifacts as part of the empirical data provided information on students’ errors or erroneous strategies. Obviously, a dichotomous coding may cause a loss of important details about individual solution processes. It is accordingly more apt for evaluating the performance of groups like schools or classrooms and less apt for understanding the individual need for support (see also Klieme et al., 2004) whereas looking at the solutions in detail provides relevant information. In this specific task, as mentioned above, 56% of the students gave the correct answer, but 23% of students did not answer at all. The remaining 21% of (wrong) solutions could be analyzed in depth. They showed that most of the students who did not succeed but tried a solution had at least rough ideas about the place value system but were not able to give a coherent argumentation. The problem lied in formulating and presenting the mathematical claim and not so much in understanding place values as such. From the mathematics education point of view, this information is helpful in particular for teachers. As level II is regarded the minimum standard, students’ wrong or missing solutions are particularly important to know. They provide evidence why students fail in answering correctly and thus precisely identify their learning problems.

Another item aimed at the knowledge of number patterns (Fig. 4.2). In order to solve this item correctly, students needed to understand that all pairs but one added up to 100. From a theoretical point of view, the item was assigned to competence level III, namely, “recognize and explain the principles in number patterns if numbers are used that are part of the curriculum.” However, 34% correct solutions showed that the empirically verified difficulty was higher and placed the item at level IV. Children who were not able to give a correct solution often referred to irrelevant aspects and stated, for example, that the number 5 was missing in the pattern or that the number pair given in the question was part of the set of pairs. Some tried to apply operations other than addition to the number pairs (e.g., multiplication: “93 is not a multiple of six.”). None of these solutions provided a consistent pattern and could not be rated correct from a mathematical point of view. Presumably, the correct solution did not only presuppose an understanding of patterns but asked for a specific kind of number sense (Dehaene, 1997), which was an obstacle for many students. As mentioned above, all ideas – whether correct or incorrect – are valuable information for classroom work and might particularly lead to an explicit understanding of deficits and errors.

Fig. 4.2

Sample problem “Number Patterns” (translated from the German original)

Identifying students’ learning progress in detail relies on knowing when they succeed as well as when and where they fail. These aspects are part of a teacher’s knowledge on the diagnosis of learning. It is important that teachers are able to correctly interpret the test results in order to take advantage of them. This is a challenging task as educational standards in mathematics and mathematical competence are closely related concepts, but knowing one does not necessarily imply knowing the other. Research suggests that a profound knowledge of mathematics is the basis for teaching, but this content knowledge is not sufficient for being a successful teacher and should be accompanied by pedagogical content knowledge (Kunter et al., 2007; Shulman, 1987). Accordingly, teachers should not only learn whether a student’s answer is right or wrong but they should also be assisted in understanding these answers in more detail. In particular, it is not only the product that counts in the classroom but also – and probably much more – the process leading to a correct or erroneous product.

Competence Models as Tools to Support Teachers’ Diagnostic Processes

Understanding students’ mastery of mathematical topics and evaluating their difficulties with mathematical problems are most challenging for teachers (Baumert et al., 1997). However, diagnosing students’ learning processes is a task that teachers face in their everyday classroom. It is important that they fulfill this task according to high standards as it is the basis for adaptive teaching and thus affects the overall instructional quality (Helmke & Schrader, 1987). Diagnosing presupposes to systematically collect useful information in order to plan and initiate appropriate interventions (Hoge & Coladarci, 1989). Accordingly, diagnosing is based on data and the proper reflection of these data (Helmke, 2010; Herppich et al., 2017).

As part of their diagnostic activities, teachers should be able to evaluate students’ learning processes and the requirements of specific contents of teaching (Helmke, Hosenfeld, & Schrader, 2004; Hill, Rowan, & Ball, 2005; Lorenz, 2011; Schrader, 2009). Diagnosing requires diverse professional competences of teachers and asks for content knowledge, pedagogical content knowledge, as well as for pedagogical knowledge (Shulman, 1987). All these components are regarded to be important in order to understand a students’ behavior in the mathematics classroom (Helmke, 2010; Weinert, Schrader, & Helmke, 1990). However, some authors emphasize the role of pedagogical content knowledge (Brunner, Anders, Hachfeld, & Krauss, 2011) because a sound diagnosis will often be based on students’ solutions to mathematical problems. Accordingly, teachers need to choose adequate tasks, to assess their difficulty, to identify errors, and to judge possible reasons for faulty solutions.

Obviously, competence models provide rather general ideas about students’ knowledge and skills and describe outcomes but do not include ways how to acquire a specific type of knowledge or how to solve a certain mathematical problem. Still, competence models can support teachers in diagnosing their students’ competences in at least three ways.

First, competence models may support teachers in understanding the structure and composition of their students’ mathematical knowledge. Classroom instruction usually follows a domain-specific arrangement taking into account the organization and logic of a specific subject. In mathematics, for example, addition and subtraction of whole numbers or fractions are taught in parallel as they are regarded to be complementing operations: subtraction is the inverse operation of addition. Multiplication is taught at a later point in time as the definition of multiplication asks for the definition of addition: multiplication is regarded to be repeated addition (e.g., Common Core State Standards Initiative, 2012). The way in which these arithmetic operations are seen from the mathematics point of view and accordingly instructed in school does not necessarily reflect the views of children on the subject. Many children perceive addition and subtraction as different operations with different degrees of difficulty or miss the linking of addition and multiplication. This means that their knowledge structure does not necessarily correspond to the structure of the curriculum or of the subject as a scientific discipline. Moreover, children might have prior knowledge on a specific topic from everyday experiences, making a seemingly more difficult topic easier for them to understand. For example, children might encounter fractions much earlier than fractions are introduced at school (in German classrooms, for example, fractions are mostly part of the grade 6 curriculum). These views are reflected in competence models for the early grades (cf. Reiss, Heinze, & Pekrun, 2007). They provide evidence that the structure and composition of mathematics cannot be easily transformed into the structure and composition of students’ mathematical knowledge. Teachers’ understanding of their children’s views may be enhanced by a comparative analysis of competence levels.

A second way in which competence models may support teachers’ diagnostic processes is through their functioning as tools for classifying, evaluating, and interpreting empirical results. Understanding children’s mathematical competences is not only important with respect to an individual but also with respect to schools, school districts, or even countries. Teachers as well as the general public are therefore confronted with empirical studies describing the results of tests and give evaluations and interpretations. Competence models can help interpret results from empirical studies. For example, the German national assessment of mathematical competence in third grades (VERA; https://www.iqb.hu-berlin.de/vera) is based on the competence model suggested by Reiss et al. (2012). The results of this assessment are reported back to the teachers. These test results are not sufficiently elaborated for diagnosing individual students, but they will give an overview on the level of classrooms.

Third, competence models can support teachers’ diagnostic processes by providing detailed information about students’ competences based on theoretical considerations and empirical data. Competence models can thus serve as a reference point to which a specific students’ performance can be compared. The reference point provides more information than teachers usually receive when taking the average performance of their classroom as benchmark for individual achievement. This way, models may help to initiate more accurate and theoretically as well as empirically substantiated judgments of students’ competence and will thus support a general comparison of students with their peers. The absence of reference points has often been an issue in research on teachers’ judgments of their students’ competences (Südkamp, Kaiser, & Möller, 2012).

Diagnosing is a process that encompasses a number of steps with varying demands (Fischer et al., 2014). When teachers diagnose their students’ mathematical competence based on students’ written work to a specific problem, competence models can be beneficial at several steps of the diagnostic process. At first, the teacher has to understand the affordances of the particular problem. This includes knowledge of the mathematical content but also knowledge about whether and why a problem is generally difficult for students. As described above, competence models can provide guidance for this judgment. As a second step, the teacher needs to identify possible mistakes in the students’ work. Doing so is a more challenging task than it may seem at first sight. As we will discuss in more detail in the next section, whether a student’s solution to a problem should be considered correct or incorrect is not only a matter of the content itself. Rather, this judgment depends on many other factors, particularly at the primary school level. Once the teacher has identified faulty solutions, he or she needs to be able to understand the nature of the mistakes and hypothesize about potential reasons. In particular, it is of interest whether student errors are of a systematic nature. To examine whether a student consistently shows a specific error pattern, the teacher should ask the student to solve another problem. The proper selection of this problem is critical in order to be able to actually capture a hypothesized error pattern. At this stage, a competence model can be useful because it helps selecting a problem at a competence level that is just suitable for the particular student. Eventually, this iterative process may end when the teacher is convinced of the student’s error pattern that may correspond to a certain level of competence according to the model. Suitable interventions should follow this process with the aim of helping the student reach the next level of competence.

This detailed description of a diagnostic process has revealed where competence models can be useful. However, our description also points to limitations of current competence models: They describe what students know at certain levels, but they do not describe what students do not yet know or what typical mistakes at a certain level might look like. Integrating this information may, however, improve teachers’ understanding of their students’ learning.

Advancing Mathematical Competence Models: The Role of Student Errors

As already mentioned, diagnostic processes require an understanding of what students know but also what they do not know at a specific level of competence. This information is relevant to identify error patterns that students might have with regard to a certain problem. More fundamentally, as errors are an essential part of learning, understanding student errors and misconceptions is required in order to describe their learning progress and development.

Errors are sometimes regarded as interference of a learning process that should be avoided if possible. However, constructivist theories of learning suggest that errors should be regarded as fruitful learning opportunities (Bodemer & Ruggeri, 2015; Oser, Hascher, & Spychiger, 1999). This view is particularly important because it is not always feasible to precisely define what an error is. Even in mathematics and above all in primary school mathematics, it may depend on the context whether a student’s answer is rated as correct or incorrect (Beitlich, Moll, Nagel, & Reiss, 2015). In general, the answer to an arithmetic problem will be true or false, but if a solution requires reasoning, it is not always self-evident which argument is acceptable at a certain stage of learning and which is not. Likewise, depending on the specific problem and its requirements, the mathematical language in general may be rated as correct or incorrect. For example, if a problem primarily asks for a numerical result, faulty argumentation or an inaccurate use of the terminology may play a minor role. As a consequence, regarding an error as divergence from a given norm (Oser et al., 1999) is a useful approach also in mathematics.

When teachers diagnose student’s mathematical competences, they need to be able to identify whether a student’s response deviates from the norm given by academic mathematics. However, they also need to consider whether it fits into norms developed in the classroom, and these norms are difficult to define and to evaluate. Accordingly, if competence models would include information about which sort of mistakes are to be expected on the various competence levels, it would be easier for teachers to define the norm.

There is a further facet of knowledge, which has been introduced by the group of Oser (e.g., Oser et al., 1999). They defined the concept of negative knowledge: in order to solve a mathematical problem correctly, students need specific knowledge, such as the rules of mental and written calculation or properties of the decimal system. However, in many situations, it is also useful to know which methods or contents will not help solving the problem. An example is knowing that specific operations like ignoring brackets or mixing up addition and multiplication will generally lead to a wrong result. This knowledge may come from experiences when application led to a wrong solution or no solution at all. As acquiring negative knowledge can support conceptual learning (Heemsoth & Heinze, 2016), teachers should not only know about the facets of (positive) knowledge that constitute mathematical competence but also about the negative knowledge that may support students’ development. Although it is probably a challenge to integrate negative knowledge into competence models, an explicit knowledge of what does not work should be helpful for students. There are typical errors in mathematics that could be part of competence models. Moreover, a better understanding of this view on knowledge might be enriching for teachers’ diagnostic competence.

Desiderata

Mathematical competence is a complex construct, and diagnosing students’ mathematical competence is a complex task of teachers. Models of mathematical competence that are based on theories and empirical evidence can provide guidance because they help in understanding what mathematical competence means and how it develops. Moreover, models that include fine-grained descriptions of competence levels can be used as reference points and thus support teachers in diagnosing students. Empirical research is needed to evaluate the effectiveness of using competence models during diagnostic processes.

Research into teachers’ diagnostic processes should also assess the role of different types of knowledge that are most relevant to support these processes. Although research has identified gaps in teachers’ diagnostic competences (Heinrichs, 2014; Ostermann, Leuders, & Nückles, 2015), it is to date unclear which knowledge components teachers actually rely on and should rely on when diagnosing students. Knowledge about students’ errors and misconceptions might be just one facet that has as of yet received little attention.

Research suggests that errors play an important role for successful learning. Accordingly, models of competence should be accompanied by information about typical errors and misconceptions students might have. Such information may help teachers in getting a clearer picture of their students’ potentials and limitations. This information might also help in recognizing developmental steps and in defining supporting steps. In particular, models that describe the development of competences (e.g., Fritz, Ehlert, & Balzer, 2013; Reiss et al., 2007) might benefit from such a broader perspective.