1 Introduction

In recent years, the importance of metacognition for working on complex modelling problems successfully and in a goal-oriented manner has been confirmed several times. Blum (2011) even summarizes that for developing modelling competencies, metacognition is not only helpful, but crucial. However, despite the presumed importance of metacognition for modelling, only a few studies have been conducted for answering how metacognition influences the development of modelling competencies and the modelling process, and how to foster students’ metacognitive modelling competencies best, for several reasons:Footnote 1

For one, although, for example, Maaß (2007) includes metacognitive modelling competencies as a crucial part of modelling competence, until now there is a lack of a conceptualization of metacognition for modelling.

In addition, a main point in the discussion on research on metacognition has been the question of how to measure metacognition most appropriately (Veenman 2005). Different methods such as questionnaires, observation and interviews were used, and their appropriateness was investigated (Schellings and van Hout-Wolters 2011). However, until now, no sufficient instrument for measuring the metacognitive modelling competencies of a large sample of students has been developed, even though this is a necessity for evaluating teaching units for fostering students’ metacognitive modelling competencies across several classes and several schools. In this paper, a questionnaire for measuring students’ metacognitive skill while modelling is introduced (see Sect. 3.3), as research results must always be discussed with regard to the measurement instrument that was used.

Another reason and point of discussion was whether metacognition is to be assumed as domain specific. Flavell, for instance, stated metacognition to be rather domain-specific (Flavell et al. 1993). Veenman (2011), on the other hand, reported general metacognitive skills that are used by students whenever they encounter a new learning task, across domains. Thus, results from other domainsFootnote 2 concerning metacognitive skills can be transferred to modelling. But as meta-knowledge about characteristics of modelling problems as well as requirements for solving those kinds of mathematical problems surely play an important role (and differ from other kinds of mathematical problems), these special aspects must be taken into account (see Sect. 2.5). However, in order to teach metacognitive knowledge and strategies useful for a specific domain, one has to analyze what is important. An example of necessary knowledge and strategies for working on a special task is given in Sect. 2.4. Another important point resulting from the fact that metacognition was investigated in different domains and with different goals, is the fact that there is not only one conceptualization of metacognition, but several (see Sect. 2.2). Most of these conceptions were developed theoretically. An empirical investigation on the structure of metacognition was done by Scott and Levy (2013), confirming the assumption that there are at least two different components: metacognitive knowledge and metacognitive skills. However, this study was not conducted in the domain of mathematical modelling. Thus, it is still an open question whether this conceptualization is transferable to other domains and whether it can be elaborated.

Furthermore, group dynamics and social metacognition (also called “team cognition”) should be considered, since working on modelling problems is often done in groups. As a result, the distributed cognition of the group is regulated and monitored in a metacognitive way. Furthermore, working collaboratively in groups requires joint planning processes. Thus metacognition has to be considered, not only applied by individuals, but also on a group level (see Sect. 2.3).

This paper focuses on metacognitive strategies and not metacognitive knowledge,Footnote 3 so only procedural aspects of metacognitive modelling competencies are investigated in the research reported. The paper addresses definitively the question of how metacognitive strategies can be conceptualized, considering all the aspects mentioned above. Especially, the underlying structure of metacognitive strategies, that is the components that can be discretely identified and how these go together to form metacognitive strategies useful within metacognitive modelling competencies, is analyzed. For this purpose, in the following, some theoretical aspects of modelling, metacognition and their connection are outlined as well as related research results. After that, design and methods of the project “MeMo” that aims at developing a conceptualization of metacognitive modelling competencies as well as fostering students’ metacognitive modeling competencies are presented. Results of the study concerning the structure of metacognitive strategies are presented and discussed.

2 Theoretical background

In the following, theoretical conceptualizations of modelling competency and the connection to metacognition as well as theoretical conceptualizations of metacognition itself will be presented, together with a short digression on the importance and problems of social metacognition in connection with metacognition. Based on these theoretical assumptions, examples of useful and important metacognitive strategies for modelling as well as research results will be presented. The section ends with the formulation of the research questions that are addressed.

2.1 Modelling competencies

In recent years, how best to foster students’ modelling competencies is one of the main research aims in the field of research on the teaching and learning of mathematical modelling. Efforts aim at analyzing most appropriate classroom settings and teacher behavior as well as developing teaching units that promote modelling competency or special sub-competencies (Greefrath and Vorhölter 2016). Kaiser and Brand (2015) describe the different understandings of modelling competencies and their development since the early eighties of the last century and distinguish different strands of the modelling competency debate. Their report indicates that in most strands the ability to work on a modelling problem is seen as composed of different sub-abilities. For example, according to Maaß (2006), “[m]odelling competencies include abilities and skills to conduct modelling processes adequately and in a goal-oriented way; as well as the willingness to put these abilities and skills into practice” (p. 139). The single abilities and skills are differentiated in different ways by different authors. Those competencies that are necessary for getting from one step of a modelling process to another are often referred to as sub- or partial competencies (see, for example, Greefrath and Vorhölter 2016). The definition given above indicates that for successful goal-oriented working on a modelling problem, appropriate beliefs and insights as well as further competencies such as working cooperatively in groups, communicating with each other and metacognitive competencies, are important (see, for example, Kaiser 2007).

2.2 Metacognitive competencies

The concept of ‘metacognition’ was introduced in the 1970s by John Flavell and Ann Brown (Flavell 1979; Brown 1978). From that time on, it has been taken up in many disciplines. Used in very many domains and for different purposes, it became a rather ‘fuzzy’ concept. All definitions of metacognition have in common, that metacognition is considered as consisting of several components. However, the components of different definitions are overlapping or even differentiated into different components in other definitions (Veenman 2006). Flavell (1976), for instance, characterizes metacognition as:

one’s knowledge concerning one’s own cognitive processes and products or anything related to them […] Metacognition refers, among other things, to the active monitoring and consequent regulation and orchestration of these processes in relation to the cognitive objects on which they bear, usually in the service of some concrete goal or objective. (p. 232).

In this definition, it becomes clear that Flavell focusses on monitoring and regulating a learning process when referring to metacognition. In his model, he differentiates between four classes of phenomena: (a) metacognitive knowledge, (b) metacognitive experiences, (c) goals (or tasks), and (d) actions (or strategies) (Flavell 1979). Metacognitive knowledge (often called declarative metacognition as well) is taken as explicit knowledge or knowledge that can be made explicit, subdivided into the knowledge of tasks, knowledge of appropriate strategies and the knowledge of one’s own skills and competencies as well as those of other persons involved. Metacognitive experience comprises any conscious cognitive or affective experience that controls or regulates cognitive activities. Every metacognitive activity aims at achieving a metacognitive goal for which the use of metacognitive strategies is useful or even necessary.

In other definitions, metacognition is differentiated into declarative meta-knowledge and procedural metacognition (see, for example, Schraw and Moshman 1995). Declarative meta-knowledge is taken as in the conceptualization of Flavell presented above. Procedural metacognition (often referred to as usage of metacognitive strategies) consists of orienting, organizing and planning of the work, monitoring and regulating as well as evaluating the whole process. Metacognitive strategies can either be used consciously while working on a task, or automatically run in the background (Veenman 2011). The declarative facet of metacognition is considered as a basis for the procedural facet; declarative metaknowledge therefore is rated as required for the usage of metacognitive strategies (Artelt 2000). These two aspects are sometimes complemented by a conditional (Schraw and Moshman 1995) as well as a motivational facet (Sjuts 2003). Conditional metacognition refers to knowing when and why to apply cognitive or metacognitive strategies; the motivational facet comprises the motivation and willingness to use metacognitive strategies, based on metacognitive experiences and awareness. Both facets are often subordinated to the declarative aspect of metacognition. Veenman (2005) points out that the use of metacognitive knowledge depends on different motivational, cognitive and depositional aspects. For example, metacognitive strategies are used only when working on problems that are of medium difficulty for the one trying to solve the problem. Furthermore, the metacognitive strategy must be well known by the students, otherwise the usage would cost too much cognitive capacity (Hasselhorn 1992). These aspects therefore need to be considered, although they may not be easily evaluated empirically.

Comparing the different classifications, it can be summarized that all of them differentiate between (at least) one declarative component as well as procedural aspects. This two-component-structure was confirmed by the study of Scott and Levy (2013). In many classifications, a motivational or affective component is considered as part of metacognition as well. Having Weinert’s (2001) definition of competence in mind (i.e., competence comprises not only knowledge and skills, but also related motivational, volitional and social willingness), one can also speak of metacognitive competence instead of metacognition when referring to all components of metacognition mentioned above. However, the following descriptions focus on metacognitive strategies.

2.3 Metacognition in group work

Research on metacognition in the past mainly focused on individual processes. According to Goos (2002), there has been little research on the characteristics of collaborative metacognitive activity during group work. “By focusing on the individual student, researchers have failed to address the dynamics required for progressive knowledge building by collaborative learning groups” (Chalmers 2009, p. 105). In recent years, there has been growing attention to social metacognition, often termed “team cognition” (Baten et al. 2017).

Considering the work on modelling problems as group activities, it becomes obvious that not every group member, but the group as a whole, has to have the modelling competencies mentioned above. Furthermore, the necessity of metacognitive strategies that ensure that all group members can take part in the modelling process and work together becomes clear, as well as the fact that not all group members have to propose metacognitive strategies on their own. But “team cognition is more than the sum of the cognition of the individual team members. Instead, team cognition emerges from the interplay of the individual cognition of each team member and team process behaviors.” (Cooke et al. 2004, p. 85). Thus, teams benefit as well from individual as from inter-individual learning (Siegel 2012). To solve a modelling problem successfully, not the individual, but the group competencies are crucial. Students have to share their knowledge and their competencies, explain their ideas to each other and externalize their thoughts (Artzt and Armour-Thomas 1992; Goos 2002). Therefore, they have to communicate with each other and work collaboratively. At this level of verbalization, most acts can clearly be distinguished as metacognitive or cognitive strategies. Those used for planning together, monitoring and explaining to each other, are of a metacognitive nature, as they aim at monitoring progress; whereas making progress would be an indicator of cognitive strategy use.

2.4 Useful metacognitive knowledge and strategies for modelling: an example

The well-known (at least in Germany) modelling problem “hot-air balloon” (Herget et al. 2001) is a less complex one. Nevertheless, it contains some key aspects of metacognitive modelling competencies for working on the problem in small groups and is therefore used to illustrate what kinds of metaknowledge and metacognitive strategies are useful or even necessary to work on this modelling problem in a group goal-oriented and successful manner. The text (Fig. 1) is accompanied by a picture of a hot-air balloon with a person atop. For getting a solution, the group has to approximate the hot-air balloons’ shape by a solid figure such as (roughly) a sphere or a hemisphere and a cone. Furthermore, they have to estimate the values of the dimensions by using the picture of the hot-air balloon as well as the man atop it. In this manner, one can calculate the volume of the hot-air balloon.

Fig. 1
figure 1

Modelling problem “hot-air balloon” (Herget et al. 2001)

Full size image

As mentioned above, metaknowledge is related to the task, to persons and to strategies. Concerning the task aspect of declarative metaknowledge, students can experience (and must be willing) to estimate the values of dimensions by using comparable figures, to be able (and have to) develop a model on their own, to judge the model and if necessary build and work on another model. The personal aspect of declarative metaknowledge becomes important when deciding on one model (Who knows how to calculate the volume of a chosen figure? Who is able to identify the necessary values? Who can look up a missing formula?), for work planning, time management and for presenting the working process in class.

This metaknowledge leads to the procedural aspect of metacognitive modelling competencies, the usage of metacognitive strategies. Necessary metacognitive strategies are getting to a common understanding (the volume of the hot air balloon is asked for) and planning the work together in the group (which includes the decision to split the group and work on two models separately and simultaneously or to work on different models one after the other). Not all students of the group must be able to manage and plan the solution process, but at least one must be able to. The others at least have to agree with his or her plan. Furthermore, monitoring the working process (especially the mathematical part) by posing questions on the one side and explaining the procedure on the other side to each other are useful metacognitive strategies for working on this problem successfully. This way, a common failure (conversion of units) can be prevented or at least detected early. In addition, the evaluation of the working process and the working behavior of each single group member is part of metacognitive modelling competencies and especially important, if this task is used as a first task of a whole teaching unit that aims at fostering students’ metacognitive modelling competencies.

2.5 Research in metacognition and modelling

Three different approaches of metacognition in modelling processes that originally came from the modelling discussion (and not from other domains) influenced the development of the study presented in this paper. In the following, their design and results are presented briefly.

First, Maaß (2007) identified in a qualitative study, misconceptions concerning modelling as part of (inappropriate) meta-knowledge about modelling processes. Students’ meta-knowledge was measured by analyzing interviews and concept maps that students had to create at two times during the study (in the middle and at the end) on their own. For creating the concept map, different terms related to the modelling process as well as to the teaching unit were given to the students and they were asked to develop a visual representation of these terms. Maaß differentiates between misconceptions concerning (a) setting up a real world model, (b) setting up a mathematical model, (c) the mathematical solution, (d) the interpretation and validation, as well as (e) general misconceptions. She revealed a relation between meta-knowledge about modelling and modelling competencies: Misconceptions about real models were related to deficits in setting up a real model, misconceptions in validating with deficits in doing so. Furthermore, she identified parallel development of both meta-knowledge about modelling and modelling competencies, whereas the quality of meta-knowledge in most cases was related to performance in modelling.

By doing so, Maaß (2007) focused on metacognitive knowledge, not strategies. Thus, her study gave important insights into the importance of metaknowledge for the development of modelling competencies. However, knowledge and skills are important for solving a problem successfully, but not sufficient. They can be effective only if used. When considering metacognition, the application of knowledge and skills in the form of metacognitive strategies is crucial. As presented above, several aspects influence the usage of metacognitive strategies during group work: students’ motivation, task difficulty, group members, etcetera. Thus, if one wants to make inferences from metacognitive strategies used, to procedural aspects of metacognitive competencies, one has to take these aspects into account as well. Connected with this point is the fact that one has to make sure one is measuring strategies used–not only knowledge about useful strategies. One possibility for doing so is measuring metacognitive task-related strategies, for example by analyzing (a videotape of) the group working on the task, or via self-reports during, or just after, the working process.

In their approach, on the bases of a cognitive-metacognitive framework of Garofalo and Lester (1985), Stillman and Galbraith (1998), reconstructed from videotapes and interviews, cognitive as well as metacognitive strategies used by students while working on an application task. They differentiated between strategies in five different phases: (1) strategies to aid understanding of the problem, (2) strategies to organize information, (3) strategies for developing and executing plans, (4) strategies for monitoring progress and (5) strategies for verification of the final result. In a following study, Stillman (2004) used a different kind of cognitive-metacognitive framework for analyzing application tasks. With the help of this framework, she identified cognitive as well as metacognitive strategies that students used for working on application tasks. In addition, she indicated relations between the usage of cognitive and metacognitive strategies. In a further study, Stillman (2011) focused on students’ metacognitive strategies while working on modelling problems. Based on Flavells’ (1979) classification, she identified productive metacognitive acts on three levels: (1) recognition that particular strategies are relevant, (2) choice of strategy for implementation and (3) successful implementation. Based on the results presented by Goos (2002), she focused on students’ responses to red flag situations and differentiates between appropriate and inappropriate use of metacognitive strategies. Appropriate strategies are called productive metacognitive activities; their use depends on the students’ experience and knowledge about metacognitive strategies. Inappropriate strategies are those of metacognitive blindness, metacognitive vandalism, metacognitive mirage and metacognitive misdirection (the first three types of strategies were adapted from Goos 2002). For developing those differentiations, Stillman used transcripts of students’ working processes.

Also based on the work of Goos (2002), Ng (2010) analyzed group dynamics as a mediator for the impact of metacognitive judgements. In particular, she elaborated the concept of metacognitive blindness for groups by introducing the term “partial metacognitive blindness”.

All those studies presented above have in common that they use videotaping and intensive coding of just a few tasks. But measured strategies should not be special for one or a few modelling problems, but useful for working on modelling problems in general. Adapting the design of the studies presented above, would be very costly in terms of time and money. When weighing up the effectiveness of different methods for measuring metacognitive modelling strategies,Footnote 4 one has to consider that these strategies are sometimes consciously used, but at other times they are used unconsciously, for example because they are used automatically. Thus, students may not be aware of using them. In this case one would prefer to have the measurement done by an outstanding rater. On the other hand, a rater can only measure those strategies that are verbalized or shown in related actions. However, because modelling is usually done in groups, students cannot verbalize all their thoughts, so self-reports may be more reliable for this point. In addition, self-reports in the form of a questionnaire allow the possibility of bigger samples, because they are less time and cost consuming.

Schukajlow and Leiß (2011) used questionnaires for investigating the influence of students’ metacognitive competencies on the modelling process: In a quantitatively oriented study, 86 ninth graders from 10 different classes were asked to report on their use of learning strategies and metacognitive strategies while solving modelling problems. For measuring students’ metacognitive strategies for planning and monitoring, they used questionnaires in the following way: They showed the students a known modelling problem and asked them to fill in to what extent they would use various metacognitive strategies. In addition, the students’ modelling competencies were tested. No significant correlation between metacognitive self-reported strategies on the one hand and mathematical modelling competence on the other hand were found. In a study conducted similarly, Schukajlow and Krug (2013) analyzed the positive influence of developing multiple solutions on students’ planning and monitoring activities.

The studies of Schukajlow and Leiss (2011) and Schukajlow and Krug (2013) have in common that they focus only on planning and monitoring activities, neglecting regulating and evaluating strategies. Furthermore, no differentiation between strategies used by individuals and strategies used on the group level were made. Therefore, the development of a questionnaire adapting the existing questionnaire and considering the aspects mentioned above seems to be worthwhile.

2.6 Research questions

There is no doubt about metacognition being not only helpful, but necessary for solving modelling problems successfully and in a goal-oriented way. Until now, there have been qualitative studies identifying students’ metacognitive knowledge and strategies used while solving modelling problems (Maaß 2006, 2007; Stillman 2004, 2011; Stillman and Galbraith 1998) as well as studies on the influence of meta-knowledge on performance when working on modelling tasks (Schukajlow and Leiß 2011; Schukajlow and Krug 2013). However, as presented above, several concepts of metacognition exist separating different components of metacognition (Sect. 2.2) in the domain of mathematical modelling, as well as across domains. Therefore, it is unknown how metacognition for modelling can be empirically divided into different components and how these components are connected. As this paper focuses on the structure of metacognitive strategies, the main research question is as follows:

  1. 1.

    Into which components can metacognitive strategies be separated?

    In the literature on metacognition, strategies are differentiated into strategies for goal-setting, orienting, organizing, planning, monitoring, regulating, evaluating and elaborating (e.g., Baten et al. 2017; Stillman and Galbraith 1998); furthermore, Stillman (2011) differentiates routine and non-routine metacognition. It is assumed that these different phases are aggregated to fewer in the case of modelling, but the open question is in which way they can be combined and how many components can be distinguished empirically. In addition, as social metacognition must be considered as different from individual metacognition (Sect. 2.3), strategies at individual level must be analyzed independently from those at group level. Thus, the research question can be divided into the following two:

    1. (a)

      Into which components can metacognitive strategies at the individual level be divided?

    2. (b)

      Into which components can metacognitive strategies at the group level be divided?

    As stated above, one must distinguish between metacognitive strategies at individual and at group levels. Nevertheless, the following question arises:

  2. 2.

    Do metacognitive strategies at the individual level reveal the same structure as those at the group level?

As research on social metacognition shows that there exist several aspects influencing social metacognition, it is possible that strategies at the group level reveal a different structure from those at the individual level. It might also be possible that the structure is the same.

3 Methodical approach

In the following, the design of the study as well as the sample and methods for collecting and analyzing data are outlined. Furthermore, the development of a questionnaire used for measuring metacognitive strategies in the study is presented.

3.1 Design of the study

The data for this paper were collected within the project MeMo. The central aim of the project is to evaluate a teaching unit designed for fostering students’ modelling competencies, including metacognitive modelling competencies as well as students’ and teachers’ perceptions of application of metacognitive strategies. The whole project was carried out from October 2016 to July 2017 (Fig. 2). During this time, students in grade nine were asked to work on six different modelling problems. Participating classes were divided into two groups: In the first group, after solving the modelling problem, metacognitive strategies used and those that were useful were reflected upon in teacher facilitated discussions. In the other group, mathematics used was reviewed. The classes were divided into two groups in order to be able to analyze if the teaching approach has an influence on students’ acquisition of metacognitive modelling competencies. Participating teachers–divided into two different groups as well—took part three times in a teacher training program. Each time, the following two modelling problems were introduced, possible difficulties were discussed, and strategies for helping students according to the principle of minimal help (Aebli 1997) were imparted. Especially, teachers were asked to refer to the modelling cycle and to use structural help. Students’ modelling competencies were measured using a test adapted from Brand (2014) some weeks before and after working on the first and last modelling problem. They were asked to work in the same groups on each modelling problem and after working on the first and the last modelling problem, they were asked to fill in the questionnaire on metacognitive modelling strategies presented above. Furthermore, 57 of the groups were videotaped and 57 students of 17 different groups were interviewed about their perception of the metacognitive strategies they used, following the three-step-design of Busse and Borromeo Ferri (2003). In addition, 15 teachers were interviewed following the three-step-design as well. Within this study, several research questions are tackled.

Fig. 2
figure 2

Design of the study MeMo with two comparison groups

Full size image

3.2 Sample

Altogether, 18 classes of grade nine from 11 different schools from Hamburg participated in the study. The questionnaires of 431 students (48% girls, 52% boys) from the first measurement period were analyzed. The students worked in 138 small groups, mainly in groups of three (33%) or four (58%). 83% claimed to be (highly) satisfied with the group work. However, only 67% of the students indicated that they had worked together as one group, 31% stated they had split into sub-groups (Table 1).

Table 1 Cross table of students’ satisfaction with group work and kind of cooperation while working in the group (in number of students)
Full size table

Teachers claimed that students had nearly no experiences in working on modelling problems. Nevertheless, 85% of the students judged the given modelling problem as having average difficulty. In total, in the beginning 67% of the students were motivated or highly motivated to work on the modelling task, and 64% claimed to be motivated at the end of the working process. Thus, the conditions for using metacognitive strategies concerning motivation and task difficulty (see Sect. 2.2) were met.

3.3 Questionnaire for measuring metacognitive modelling strategies

As shown above, different possibilities for measuring metacognitive strategies exist and have been used before. But as Blum (2015) pointed out: “One of the problems in these empirical studies is: how to measure strategy knowledge, on the one hand, and strategy use, on the other hand, and another problem is how to reliably link students’ activities to their strategies.” (p. 88). For overcoming these problems, a questionnaire for measuring metacognitive modelling strategies was developed in several steps. Based on a literature review on metacognitive modelling competencies (like those presented above) as well as considering existing questionnaires on declarative aspects of metacognition of other domains (like Lingel et al. 2014; Rakoczky and; Klieme 2005), a first version of a questionnaire was developed.

It was piloted and revised several times with different groups of students, slightly different designs and different modelling problems. Some of the questions addressed were as follows: do students understand the items as intended? Are there any strategies that are useful only for some problems? If so, how can they be reformulated or summarized with others? Do experts rate the videotaped working processes the same way students themselves do? [for an overview see Vorhölter (2017), Vorhölter (accepted)].

The observation of several modelling processes results in a differentiation of metacognitive modelling strategies into three steps: before working on the problem (understanding the task, organization of information, rough planning of the working process) while working on the problem (detailed planning or re-planning, monitoring and regulating the working process) and after working on the problem (evaluating the working process). Furthermore, items were differentiated into strategies on an individual level (used by the student herself/himself) and on a group level (strategies that were shared in the group). The questionnaire includes the following:

  • strategies for organizing and planning the solution process in consideration of the

    • task that has to be worked on,

    • the involved persons,

    • specific circumstances

  • strategies for monitoring and, if necessary, regulating the working process, such as

    • using the modelling cycle as a tool,

    • applying strategies systemically and in a goal-oriented way

    • realizing of cognitive barriers

  • strategies for evaluating the modelling process, such as

    • individual grade of participating in group work,

    • evaluating the cooperation within the group,

Examples of items in the questionnaire, allocated to the phases of before, while and after, as well as to the separation of individual and group items, are shown in Table 2.

Table 2 Item examples
Full size table

Students had to judge their usage of metacognitive strategies at individual as well as at group level on a five-point Likert-scale (1 = no agreement to 5 = full agreement). As mentioned above, complementary to strategy use, they were asked about task difficulty, their motivation at the beginning as well as at the end of the working process, in which way they worked together and about their satisfaction with group work.

3.4 Methods for collecting and analyzing data

In every class, the lesson in which the first measurement took place followed the same structure: In the beginning, the students were introduced to a modelling cycle (Kaiser and Stender 2013) and then they worked on the same modelling problem in groups. After claiming that they had completed working on the problem, students were asked to fill in the questionnaire on metacognitive modelling strategies without speaking to each other. Beneath items such as those presented in Sect. 3.3, students were asked to fill in a personal code as well as the personal codes of their group members. When all students finished the questionnaire, teachers asked them to present their work in class.

For analyzing data, the items of the questionnaire were divided into items at individual level and items at group level. Both groups of items were analyzed separately: at first, statistical characteristics like means and standard deviations were determined. After that, items of every group were analyzed using principal component analysis with promax-rotation, because the aim was revealing the underlying structure of the items of each group of items. Furthermore, a correlation between the different components could not be ruled out. The number of factors was determined using parallel analysis, Kaiser criterion as well as analysis of the screeplot (Bühner 2011).

4 Results

In the following, the results of the principal component analysis are presented, separated into results concerning items at group level and items at individual level. After that, the results are compared and discussed.

4.1 Items at group level

Principal component analysis with promax-rotation of the items at group level of 431 questionnaires led to a three-component solution. By these three components, about 54% of the variance could be explained in the data. The minimum factor loading of λ = 0.40 was in each case exceeded. All items could clearly be assigned to one of the three components.

  • The first component consisted of five items (factor loadings 0.54 < λ < 0.81). All items refer to strategies used for planning the working process (e.g., capturing important information out of the task) and ensuring working on the problem without greater difficulties (e.g., asking each other to explain ideas). This component therefore can be called “strategies for proceeding”.

  • The second component comprised four strategies that can be applied if different ideas were mentioned (e.g., joint decision about how to proceed) or if a problem was recognized (e.g., joint discussion if difficulties appeared). The factor loadings of this component were 0.52 < λ < 0.91. Thus, this component is indicated as “strategies for regulation”.

  • The third component included three strategies for “evaluating the modelling process” (e.g. wondering what to do better next time). The factor loadings were 0.45 < λ < 0.86.

The results are summarized in Table 3.

Table 3 Item characteristics of the three components of group level
Full size table

Item means of the components differ: Students used fewer strategies at group level for evaluating the working process (2.0–2.6); strategies most used were for planning the working process and taking care of it (3.5–3.9). The standard deviation of all components were similar.

4.2 Items at individual level

Principal component analysis with promax-rotation of the items at individual level of 431 questionnaires led to a three-component solution as well. By these three components, about 41% of the variance could be explained in the data. The minimum factor loading of λ = 0.40 was exceeded in most cases, thus most items could clearly be assigned to one of the three components. They can be described similarly to those of the items at group level:

  • The first component consists of eight items (factor loadings 0.39 < λ < 0.71). All items refer to strategies used for planning the working process (e.g., becoming aware of possible steps) and ensuring working on the problem without greater difficulties (e.g., checking whether we are still on the right track). This component therefore can be called “strategies for proceeding”. The strategy “checking whether we are still on the right track” is the one with a loading of only λ = 0.39 with another loading on the second component of λ = 0.37. As this strategy clearly indicates a monitoring behavior, it was included in this factor.

  • In the second component four strategies were included that could be applied if a problem was recognized (e.g., asking for help). The factor loadings of this component were 0.47 < λ < 0.70. Thus, this component is indicated as “strategies for regulation”.

  • The third component included three strategies for “evaluating the modelling process” (e.g., wondering about a better solution). The factor loadings were 0.53 < λ < 0.83. The strategy “checking of errors” of the component “strategies for process” has a loading of λ = 0.50 on the proceeding-component, as well as a loading (λ = 0.38) on this third component. As checking of errors is also a habit of smoothly proceeding as an overall check at the end of the working process, this may not be astonishing.

The results are summarized in Table 4.

Table 4 Item characteristics of the three components of individual level
Full size table

When looking at means and standard deviations of the components, different to the means of the items at group level, one cannot clearly identify a range of strategy use according to components. However, four strategies for proceeding had means over 3.5, the other had means below. Nevertheless, the means of the strategies for proceeding were (with one exception) higher than those for regulation and evaluation. The standard deviation was lower for strategies for proceedings, so almost all students used more strategies for proceeding than other strategies.

4.3 Comparison of the results and discussion

As presented in Sect. 2.2, procedural metacognition is differentiated into different facets. Depending on the conceptualization, strategies for understanding, organising, planning, monitoring, regulating and evaluating are differentiated (for example, Stillman and Galbraith 1998). The results of principal component analysis of items at group level and those at individual level clearly show the same structure: One component can be characterised as proceeding, the second as regulation, and the third as evaluation. Thus, in a sense, Stillman’s (2011) differentiation in routine (strategies for proceeding) and non-routine (strategies for regulation) metacognitive strategies appeared, complemented by strategies for evaluation.

Comparing these results with those six components mentioned above, strategies for orienting, organising, planning and monitoring are summarised in one component, termed “proceeding”. Thirteen of the 27 items (eight at individual, five at group level) build this component. Since the strategies of this component have means above the average, both at individual as well as at group level, this may indicate that students of grade 9 are familiar with those strategies. As planning and monitoring are habits of good strategy users (Pressley et al. 1989), this result also indicates that participating students are good strategy users, at least concerning these strategies. Analyses of the interviews with the students support this assumption: Students recognized processes of planning and monitoring and rated them as relevant for their solving processes. Thus, students already used strategies for planning and monitoring without being requested to do so by their teachers (Vorhölter et al. submitted). Nevertheless, analysis of the videos of the pilot study shows that students used strategies for planning and monitoring–but sometimes they used strategies, but the outcome was incorrect or the outcome was not used (Vorhölter accepted). So in terms used by Stillman (2011), students recognized that particular strategies were relevant (level 1) and chose strategies for implementation (level 2), but the implementation was not successful (level 3).

Strategies with the second highest means, at least at group level, were those concerning regulating. Whereas these were mostly above average, strategies for regulating at the individual level were rated below average. This indicates that students regulate mutually more often than a student does individually. In addition, the lower mean of strategies for regulating (in comparison to those of proceeding) may point out that students trust in their teachers for regulating if problems occur. For overcoming such behaviour, teachers should be aware of the principle of minimal help and concepts like scaffolding. On the other hand, the results clearly indicate the importance of collaborative group work for modelling. In accordance with findings of Goos and Galbraith (1996), students benefit from each other (as long as they respect each others’ perspectives).

Often, strategies for monitoring and regulating are seen as two parts of one component, because monitoring is a condition for regulating. Indeed, one of the most important strategies for monitoring on the individual level, checking if the group is still on the right track, loaded on the proceeding as well as on the regulating component. The mean of this item is 3.2, which corresponds to the mean of the scale of 1–5. This may indicate that students did not see any difficulties when monitoring, so regulation was not necessary. In other cases, difficulties were detected and regulation strategies were used.

Strategies for evaluating at an individual level were rated similarly to those of strategies for regulating at individual level, both below average. Strategies for evaluating at group level were rated even lower. One reason may be that evaluation takes place after getting a solution, at the end of a lesson. Often, there is not enough time for evaluating or students simply are not motivated to do so. “[S]tudents usually do not reflect upon their activities and, closely related to that, are not able to transfer their knowledge and skills from one context or task to a different context or task, even if there are structural similarities” (Blum 2015, p. 80). Further on, he advises: “All activities ought to be accompanied by reflections and ought to be reflected in retrospective, with the aim to advance appropriate learning strategies” (Blum 2015, p. 84).

Summing up, strategies for proceeding were claimed to be used to a greater extent at the individual and the group level. Strategies for regulating especially at individual level as well as strategies for evaluating, should be improved, in order to foster students’ metacognitive strategies for modelling.

5 Conclusion and outlook

Summarizing, the results presented clearly show that metacognitive modelling competencies consist of different components. Items at group level showed the same structure as items at individual level. Different to other conceptualizations, strategies for understanding the task, planning, and monitoring, were subsumed under one component. Because this component includes only strategies used for making a process smooth, it was termed “strategies for proceeding”. The second component consists of strategies used if difficulties occur, that is, if students have to regulate their behavior. It was highlighted that students regulate each other’s more often than their own behavior. Strategies for evaluating build the third component.

As mentioned above, data for evaluating the structure of metacognitive strategies came from the first measurement period within the MeMo project, before intervention started. In the first step, a principal component analysis was done for exposing an underlying structure. In the next months, the second measurement period, i.e., measurement after the intervention, will take place. It will be worthwhile to analyse whether the data of the second period of measurement reveal the same structure. Furthermore, the revealed components should be validated using confirmatory factor analysis. Furthermore, it will be interesting to differentiate between motivated students and those who were not as motivated, as well as between those who rated the modelling problem rather easy or too difficult with those who rated it of medium difficulty. At least the influence of satisfaction with group work on the judgment of items at group level would be interesting. As the data of this questionnaire are not the only data measured in the project, a triangulation of different data sets, especially those of the test of modelling competencies and declarative metacognition is planned. This way, questions such as the influence of metacognition on modelling competencies can be tackled. Especially questions such as the difference between the judgements of group members concerning the items of group level, as well as a comparison of students’ judgements and that of experts rating students’ metacognitive strategies used in the videotaped lessons, will be worthwhile. This way, a comparison of students’ self-reports and experts’ ratings (i.e., a comparison of different methods for measuring students’ metacognitive strategies used while modelling) would be possible and worthwhile, taking into account that research results may depend on the method used for measuring. Against this background, one has to formulate the outcome of this study the following way: Students show different rating behaviours for strategies for proceeding, regulating and evaluating, both at individual and at group level.