1 Rationale

Today’s dynamic, knowledge-based society requires students at school to develop competences in such areas as attaining new knowledge, creative problem solving and critical thinking. Inquiry-based learning (IBL) can support the development of such competences. In this paper, the term “inquiry-based learning” refers to a teaching culture and to classroom practices in which students inquire and pose questions, explore and evaluate. Learning is driven by open questions and multiple-solution strategies. Teachers are proactive: they support pupils who are struggling and challenge those who are succeeding through the use of carefully chosen strategic questions. In the classroom, a shared sense of ownership exists. IBL aims to develop and foster inquiring minds and attitudes that are vital for enabling students to face and manage uncertain futures (Artigue and Blomhoej 2013).

In recent years, IBL has been increasingly promoted—in particular through funding streams of the EU. This situation is due partly to the findings in the 2007 Rocard Report (Rocard et al. 2007), which included criticism of the “deductive approach”—a teaching in which “the teacher presents the concepts, their logical-deductive implications and gives examples of applications.” This so-called “top-down transmission” method is considered to be one factor which leads to students’ lack of interest in science and mathematics and their considering these subjects to be fixed systems which are extremely difficult and have little or no connection to out-of-school contexts (Rocard et al. 2007, p. 9).

Achieving large-scale implementation of IBL in day-to-day teaching requires devoting considerable attention to teachers’ (initial) education and to their professional development. Since the 1990s, enormous developments have been made in research related to teacher education. These include the increased integration of pedagogical, psychological and social aspects into the didactic discussion, and the shift from seeing the teacher as a provider of knowledge to a facilitator engaging students in mathematical activities (e.g. Wood et al. 2008; Krainer 1998). The shift therefore turns to teacher educators: How can they best educate pre- and in-service teachers in order to provide support for these more student-centered ways of teaching?

To date, however, research studies on the professional education of teacher educators have been rare (Krainer 2012), with research on professional development being dominated by small-scale, qualitative research conducted by teacher educators studying the teachers with whom they are working (Adler and Jaworksi 2009). Achieving a widespread implementation of IBL, though, asks for a process of scaled-up professional development initiatives.

The cascade model—educating multipliers who in turn lead professional development courses themselves—is one example of running professional development courses on a large-scale basis. Despite the concern of how much can actually be handed down through use of this model (OECD 1998), the training of teacher educators (multipliers) seems to be crucial for a large-scale uptake of IBL in day-to-day teaching in various countries and different school contexts. This implies that a large network of teacher educators is required. The issue of how to train teacher educators becomes even more complex when taking into account that the teacher educators themselves often have very different backgrounds. For example, the trainers may be specialized in the area of mathematics education research, or teach specific subjects at school or be generally educated trainers (Krainer 2012). Unfortunately, not much is known about what qualifies someone to be a teacher educator and there is little support for their ongoing learning (Ball and Even 2009; Robert 2009; Krainer 2012). Subsequently, the question of how to educate the teacher educators becomes vital for any initiative aiming to use the cascade model to help obtain a widespread implementation of IBL.

Further, the question of scaling up is closely related to the question of how programs for professional development can be implemented practically within different contextual settings (Adler and Jaworksi 2009). How far can a program which has proven efficiency in one setting be extended to other settings? An important factor for successful professional development programs is their flexibility on the school level, meaning that the program should be adaptable and include as a basis possibilities that take into account and incorporate teachers’ needs in a given context (e.g. Krainer 1998).

These insights show the necessity of designing a model for dissemination and implementation that both addresses core principles of IBL and has the flexibility for implementing and scaling up professional development in various national contexts. In this paper, we present the design of a model for dissemination and implementation as developed within the international project PRIMAS (14 universities from 12 European countries, 01/2010–12/2013).

The design, evaluation and refinement of the PRIMAS model follow principles of design research. This methodology offered us opportunities to optimize the model during the iterative process of theory- and experience-based improvements.

2 Design research

We will first outline the guiding principles of design research and then describe how the design of the PRIMAS model was based on them.

2.1 Theoretical background

The aim of design research is to find solutions for open questions and problems in complex, real contexts. Its emphasis is on solution finding. A main feature of design research is that it does not strive for “context-free” claims; rather, it sees context as central to its conceptual terrain (Kelly 2006).

Altogether, design research can be characterized by the following aspects (van den Akker et al. 2006; Kelly 2006):

  1. 1.

    Interventionist: it aims at designing an intervention in the real world;

  2. 2.

    Utility-oriented: the merit of a design is measured, in part, by its practicality for users in real contexts;

  3. 3.

    Iterative: it incorporates a cyclic approach of design, evaluation and revision;

  4. 4.

    Theory-based: it is based upon theoretical propositions;

  5. 5.

    Context-oriented: it considers the context as central for the intervention;

  6. 6.

    Process-oriented: a black box model of input–output measurement is avoided; the focus is on understanding and improving interventions;

  7. 7.

    Theory-oriented: field testing of the design contributes to theory building.

As educational research in general has long been criticized for its weak link with practice, including design research into educational research can contribute to more practical relevance of the latter (van den Akker et al. 2006). This is because design research offers opportunities to systematically design products (materials, strategies or models) relevant to teaching practice.

The iterative research cycles in design research encompass an evaluative element. Evaluation in a traditional sense often means a carefully planned, long-term study in which all variables are controlled; however, in design research, this must be tackled in a different way due to the complex situations dealt with and the emphasis on solution finding. The aim of evaluation in design research is not hypothesis testing (Edelson 2006), but finding out to what extent a design works and developing an understanding of how it does so (Gravemeijer and Cobb 2006).

Criteria for evaluation are: design relevance; consistency (the intervention is logically designed); practicality (the intervention is usable in the settings for which it has been designed); and intervention effectiveness (using the intervention results in the desired outcomes) (Nieveen 2007).

Methods to evaluate design can include, among others (Nieveen 2007):

  • Screening: members of the design team check the design with checklists containing important characteristics of components of the prototypical intervention;

  • Expert appraisal: an expert group reacts to a prototype of an intervention, usually on the basis of a guideline;

  • Walkthrough: the design researcher and one or more representatives of the target group jointly go through the intervention set-up;

  • Micro-evaluation: a small section of the target group uses parts of the intervention outside its normal user setting;

  • Try-out: a limited number of the user group utilizes the materials in the day-to-day user setting. Evaluation methods may include observation, interviewing and administering questionnaires.

Summative evaluation methods, such as (quasi-) experiments and surveys, can also be used once the intervention has been implemented in educational practice (Nieveen 2007).

Retrospective analyses, carried out both during and after the iterative cycles of improvement, impact on design refinement because the research process allows for framing the model and examining how it works in a broader context. While it is speculative to generalize, an accumulation of related cases can form the basis for building an empirically supported theory. Therefore, it is important to recognize the value of an individual, innovative case for a larger design research and theory development agenda (Edelson 2006).

2.2 The PRIMAS design research approach

We refer here to the above-listed features of design research (at the beginning of Sect. 2.1) and explain how the design and refinement of the PRIMAS model of dissemination and implementation follows design research.

PRIMAS is interventionist as it aims at a widespread uptake of IBL through use of interventions in day-to-day teaching across 12 European countries. It is also utility-orientated: all the PRIMAS activities need to be practical for users in local teaching contexts as otherwise the partners across Europe would not be able to implement them. Its iterative character follows four research cycles (see Fig. 1).

Fig. 1
figure 1

Iterative cycles of improvement of the PRIMAS model

Full size image

In the first cycle of evaluation, the consistency and practicality was checked by members of the design team. The checklist included scientific quality, logical structure of the approach, and relevance and practicality within each national context (November 2008–January 2009). In the second cycle, the relevance, consistency and practicality was further checked by a group of experts who wrote a report on the scientific content, the implementation plan, the management and the potential impact of the project (February–June 2010). This cycle resulted in an international, theory-based model for implementation. In the third cycle, at the beginning of the PRIMAS funding period, a walkthrough was carried out with a group of target users, institutionalized within a so-called National Consultancy Panel (NCP) in each country (comprising, for example, heads of schools, teacher trainers, school authorities). Here the main focus was on the practical, context-specific adaption of the overall model to the national requirements (February–June 2010). These national adaptations of the common international model were implemented in the partner countries and evaluated during the fourth cycle of evaluation.

Further features of design research as listed in Sect. 2.1 will be elaborated in the following section: first the theoretical basis of the PRIMAS model and the model itself (after the second cycle of improvement); then the context-orientated implementation of PRIMAS through use of two examples; followed by the process-orientated evaluation of the try-out, which forms the fourth cycle of our iterative proceeding; and finally we will reflect on how PRIMAS contributes to theory building.

3 The PRIMAS model and its theoretical basis

What can a theory-based model for the dissemination and implementation of IBL in different national settings across Europe look like? In the following, we will look at three different aspects:

  • What should professional development courses look like? (Sect. 3.1)

  • How can professional development courses be scaled up by using multipliers? (Sect. 3.2)

  • How can teachers be supported in their efforts to implement IBL? (Sect. 3.3)

For all three aspects, we will first outline the theoretical background and, using this as a basis, then explain the PRIMAS approach.

3.1 Professional development

3.1.1 Theoretical background

The term “teacher professional development” relates to changes in the teachers’ professional knowledge and competence.

Teachers’ professional knowledge is multi-layered. Shulman (1986) distinguished various types of knowledge, among which are content knowledge (knowledge about the subject, CK), pedagogical content knowledge (knowledge about how to teach the subject, PCK) and pedagogical knowledge (general pedagogical knowledge, PK). Professional competence goes beyond professional knowledge and includes a person’s ability to transfer knowledge into action (Frey 1999, quoted in Jäger 2001). Professional competence is influenced by: beliefs (long-lasting subjective knowledge of certain objects, as well as the attitudes linked to that knowledge—Pehkonen and Törner 1996); motivation; and teachers’ competence in self-reflection (Baumert and Kunter 2013).

Consequently, teachers’ professional development needs to address issues that include: cognitive potentials in learning activities; students’ cognitions and their dispositions towards the discipline; discipline-specific methods of teaching; teachers’ beliefs and motivation; and their self-reflection competence.

In a meta-analysis of research in relation to professional development, Lipowsky and Rzejak (2012) distinguish four aspects of professional development effectiveness: (1) teachers’ opinions; (2) teachers’ knowledge; (3) teaching; and (4) effects on students’ performance.

In regard to aspect (1) teachers’ opinions, professional development initiatives are considered to be effective if they have clear relevance for day-to-day teaching. In addition, teachers appreciate the exchange of experiences with colleagues (Wood et al. 2008). Networks of teachers with similar objectives can support teachers in their efforts to understand, value and seize opportunities for experimenting with IBL-related classroom situations that ask for new teaching competences (Tirosh and Graeber 2003; Hart 2002; Wilson and Cooney 2002).

Turning to aspect (2) professional knowledge and competences, research shows that during reflection, teachers should be requested to deal with their own beliefs about the nature of mathematics and mathematics teaching and learning (see also Davis et al. 2009; Joubert and Sutherland 2009; Tirosh and Graeber 2003; Hart 2002). Putnam and Borko (2000) recommend that teacher educators should treat teachers the way they expect teachers to treat students (“teach what you preach”). This seems to be an extremely important factor when it comes to IBL. If teachers are to encourage students to participate in IBL, they must first have experienced inquiry themselves.

When it comes to aspect (3) teaching, professional development interventions have proven to be effective if: the interventions are long-term and intensive; they combine learning-off-job in courses with learning-on-job in school as discussed above; and teachers are given feedback about their teaching. Examples of effective, long-term courses are PFL (Pedagogy and Subject-Specific Didactics for Teachers) in Austria (Müller et al. 2011) and MANOR (a national mathematics teacher center in Israel; see Even 2005).

Speaking to aspect (4) effects on students’ performance, the Lipowsky meta-analysis (2004) shows that professional development courses seem to be effective if they have a clear focus on a certain aspect of teaching.

Finally, when aiming at changes it is important to recognize that all professional development takes place in a setting which is culturally and temporally situated (Joubert and Sutherland 2009). Learning is rooted in socio-cultural settings such as school communities and societal norms (Davis et al. 2009). The attitudes held by heads of schools, as well as students’ and parents’ reactions, are factors that influence teachers’ teaching practices and their willingness to change (Tirosh and Graeber 2003; Maaß 2004). External evaluation, assessment and time on task available are also important factors which may influence teachers when they are intending to experiment with new pedagogies (Long 2004; Brown and McIntyre 1993).

In general, it should be recognized that changes in teaching practice are gradual and need time, since integrating new ideas and practices into ongoing practices is not simple or straightforward. Several years may be necessary to bring about permanent changes in day-to-day teaching (Tirosh and Graeber 2003).

3.1.2 The PRIMAS model for professional development

The PRIMAS professional development model draws on the insights mentioned above. In particular, the PRIMAS professional development program offers:

  • support for all components of professional competence (beliefs, motivation and self-reflection);

  • relevance for day-to-day teaching;

  • opportunities to experiment with new pedagogies in seminars and in daily practice;

  • chances to discuss and exchange experiences with new pedagogies;

  • possibilities for teachers to connect reflections with beliefs by presenting and discussing example opinions in favor or against IBL aspects;

  • a structured combination of seminars (learning-off-job) with phases of practice (learning-on-job) allowing teachers to experiment in between the seminars;

  • long-term courses within a timeslot of 2 years;

  • a clear focus on IBL;

  • a close connection to the cultural context: every session suggests an inventory of teachers’ needs on the topics being addressed (which can differ between and within countries).

The professional development included in PRIMAS consists of alternating meetings and lessons in which teachers experiment with the implementation of IBL. During the meetings, they have the opportunity to reflect on their practice and experiences using structured resources. They are explicitly encouraged to analyze new teaching practices, implement them and reflect on the growth of new practices and beliefs (of both teachers and learners). These phases of analysis, implementation and reflection are repeated over the long-term. A range of different pedagogical foci are included and each is tackled using a wide variety of problem types. This approach can be regarded as a spiral model in which teachers gradually develop their teaching practice towards IBL (see Fig. 2a, b).

Fig. 2
figure 2

a One circle of the spiral model. b The spiral model in long-term professional development

Full size image

The overall focus of the professional development in PRIMAS is to enable teachers to implement inquiry-based learning in their daily classroom practices. Addressing IBL in daily classroom practices raises the following issues for teachers:

  • organizing student-led inquiry;

  • helping students to tackle unstructured problems;

  • promoting concept development through inquiry;

  • asking questions that promote reasoning (and include all students);

  • supporting collaborative work;

  • building on what students already know;

  • using self- and peer assessment to promote learning.

The resources for professional development are structured in seven modules and each module focuses on one of the above issues. A module consists of guidelines for teacher educators, handouts for teachers, video-examples of lessons (originally videotaped in England) and example lesson plans. The PRIMAS modules are published on the PRIMAS website.Footnote 1 All partners used these resources for setting up long-term courses in their respective countries.

Schedule and numbers to be reached

Each of the 12 partner countries was given the goal of educating at least 100 teachers within the years 2012 and 2013. The courses designed are long-term, meaning that they must allow for repetitive circles of analysis, implementation and reflection (spiral model). Another relevant factor here is that PRIMAS does not strive to achieve context-free solutions (see Sects. 2 and 4). Therefore, in some countries, the course could run over 1 year; in others over several weeks. PRIMAS aims to allow for courses that last as long as possible within the given contextual framework.

After the lifetime of PRIMAS, follow-up projects will eventually allow teachers to continue their IBL professional development. An example of this is project Mathematics and Science for Life (MaSciL), which is being funded by the EU from 2013 to 2016.

3.2 Scaling up with multipliers

Long-term professional development courses that reach out to a larger number of teachers have high demands in terms of personnel costs. It is not financially—or practically—feasible for each partner to directly give long-term development courses to the target amount of 100 teachers per country. Therefore, the project follows the cascade model (see Sect. 1). We educated teacher educators; they, in turn, are our multipliers who “multiply” their knowledge to teachers. In order to develop effective courses for a large-scale uptake of IBL, the project needed a strategy for educating multipliers.

3.2.1 Theoretical background

Müller (2003) suggests that the education of multipliers should consist of three strands:

“Learning-off-job” the first strand is a basic qualification in seminars and tailored to the needs of participants. In it, multipliers learn and experience a topic and learn to reflect on their practice. These seminars should follow the same principles as outlined in Sect. 3.1 on professional development. In addition, they need to include reflections on the role of a multiplier and how to develop and run a professional development course (Ball and Even 2009; Zaslavsky and Leikin 2004).

“Learning-by-job” multipliers are supported by colleagues or experienced teacher educators when running professional development courses. Learning-by-job can be provided using different means: (a) groups of multipliers can meet regularly to discuss cases from their professional development courses and to reflect on them; (b) supervision; and (c) observation, in the sense that colleagues or experts can sit in on multipliers’ professional development courses, observe them and then give feedback to the multipliers (Müller 2003).

“Learning-on-job” this refers to multipliers’ self-education and is an important supplement to learning-off-job and learning-by-job. Formal seminars and meetings with colleagues can support the development of the competences of a multiplier, but, in the end, it is the multiplier’s self-education that leads them to develop their competences. Self-education includes individual reflection on: their role as a teacher educator; the ways they teach in their classrooms and when leading professional development courses; and their own needs regarding competence development. Once a need is discovered, the teacher educator must search for related information to improve their knowledge and carry out “experiments” by asking questions about their way of teaching or running professional development courses, by conducting “experiments” in class or in courses, and by reflecting on them. This is known as “action research” (Altrichter et al. 2008).

3.2.2 Multiplier education in PRIMAS

Multiplier education in PRIMAS was based on the Müller model (2003) as outlined above. It consisted of the three strands: learning-off-job, learning-by-job and learning-on-job.

Within the seminar, teacher educators were introduced to IBL. They experienced the modules designed for our professional development courses themselves (see Sect. 3.1.2) and reflected on their use on a meta-level. To support their changing role from a teacher taking part in a professional development course to a multiplier educating other teachers, multipliers were given the chance to simulate their role in courses for multipliers.

In order to support their work in practice (learning-by-job), multipliers worked in pairs, had project team members observe their professional development courses, or were encouraged to exchange ideas and experiences about their practice with other multipliers.

Finally, they were gradually introduced into learning-on-job, for example by giving them literature to work with and by encouraging them to reflect on their competences in running professional development courses and their needs in further education.

Schedule and numbers to be reached:

Each country had the goal of educating a minimum of 20 multipliers who in turn were to educate at least 100 teachers (see Sect. 3.1.2). The multipliers’ education was scheduled to take place in 2011 so that they would be ready to start their work as multipliers at the beginning of 2012.

3.3 Supporting teachers in a complex system

3.3.1 Theoretical background

As we have seen above, professional development takes place in a setting which is culturally and temporally situated (Joubert and Sutherland 2009). Learning is rooted in socio-cultural settings such as school communities, as discussed in Sect. 3.1.1 (Davis et al. 2009). The so-called socio-ecological approach from community psychology (a part of psychology which deals with individuals acting in their communities within their geographical and social contexts) provides a model for this situation.

The teacher is part of a bigger system that influences their actions (Dalton et al. 2007, p. 17, based on Bronfenbrener). Figure 3 shows the different systemic levels that an individual is connected to. The closer a system is to an individual, the more immediate will be the influence of factors of this system on the individual, and vice versa. In micro-systems (e.g. family, colleagues), members share interpersonal relationships and carry out activities. Micro-systems are seen as being social resources that provide social support (e.g. in different situations of life, change of role). The meso-level is formed by organizations, such as schools (Dalton et al. 2007), and by localities, as for example the neighborhood and town in which an individual lives. The macro-system level refers to the level of society or culture in which the lower levels are embedded. Here cultural, political, economic and structural factors can be identified which influence individual members of the system.

Fig. 3
figure 3

Socio-ecological system levels (Dalton et al. 2007)

Full size image

All these systems influence what a teacher does in their classroom and should thus be taken into account when aiming at a widespread implementation of innovative pedagogies.

3.3.2 Use of the socio-ecological approach in PRIMAS

Following the socio-ecological approach of Dalton et al. (2007), PRIMAS was designed to work with several layers:

  • We took into account the macro-system by analyzing contextual working conditions for teachers (macro-system and organization—for details see Dorier and Garcia 2013).

  • We initiated a dialogue process with policy makers at national and EU levels to identify obstacles, and then work to remove them; and to seek possibilities of cooperation and synergies between research, policy and practice.

  • Our dissemination plan addressed all the layers as described by Dalton et al. (2007). Apart from running professional development courses, PRIMAS includes carrying out a variety of dissemination activities (conferences, workshops, talks, exhibitions and publications) addressed to, for example, teachers, parents, students, teacher educators, heads of schools and policy makers.

  • We actively involved stakeholders in the project work by setting up a national consultancy panel (NCP) in each country comprised of key actors such as representatives of school authorities, schools, parents associations, industry and out-of-school learning institutions. These panels advertise the project and its activities.

Schedule and numbers to be reached:

Each country had to set up its NCP at the very beginning of the project and require the NCP to meet on a biannual basis. Further, each country was assigned the task of carrying out at least four bigger dissemination activities (i.e. conferences, workshops) targeting various layers in the years 2012 and 2013.

4 National adaption and implementation (try-out) of the model

As has been outlined above, design research does not strive for context-free solutions and it sees context as central to its conceptual terrain (Kelly 2006; see Sect. 2). In order for PRIMAS to succeed, the 12 different national contexts involved had to be taken into account.

Firstly, we carried out an analysis of the context in each country in which PRIMAS was to be implemented. This included analyzing curricula, examining means used to assess student performance, and looking at the existing conditions for teachers to participate in professional development (Dorier and Garcia 2013). Secondly, a baseline study was conducted which informed the project about teachers’ beliefs in regard to IBL and their teaching practice (Engeln et al. 2013).

In the first meeting of the NCP, the PRIMAS model, as well as the results of the analysis of context and the baseline, were presented to the panel members who then developed concepts for the adaption of the international model and implementation in the given setting. Subsequently, the national implementations differed from the common concept to some extent. In all our work, we maintained a balance between following the international concept and adapting it to the national contexts.

To exemplify this adaption process, we will look at the implementation of the professional development courses in the Netherlands and in Germany.

4.1 National adaption in the Netherlands

4.1.1 Education of multipliers

In the Netherlands, it turned out to be difficult to reach potential multipliers from commercial teacher training institutes. For this reason, multipliers were recruited from neighboring teacher colleges. This created a specific situation as there was only a limited need to educate these multipliers because they already knew a lot about inquiry-based teaching and its implementation in day-to-day teaching and about educating teachers. Following the model described above about multipliers’ education (Sect. 3.2), their professional development consisted of learning-on-job phases. Here, they tried out PRIMAS materials in their pre-service education after they were familiarized with the materials in short workshops (learning-off-job). A few multipliers from further regions joined the project at a later stage and were educated with a 1-day course. They were then assigned to start running professional development courses jointly with experienced teacher educators (combining learning-off-job and learning-by-job).

4.1.2 Professional development courses

The Dutch professional development courses followed the spiral model and the principles outlined in Sect. 3.1.2. The first step before being able to run the courses was to translate the PRIMAS resources and adapt them to the Dutch school system. The modules contain enough activities for seven 1-day professional development events (including reflection on experiences). Nevertheless, in order to ensure a perfect fit to the national context, we added activities based upon tasks from Dutch textbooks that we considered to be too structured. This supported the connection between the activities in the modules and Dutch teaching practice and, thus, ensured relevance for day-to-day teaching.

Next, in a pilot course at one school, four professional development events were assembled from the seven modules in cooperation with the involved teachers (8 teachers from mathematics, physics and chemistry). This course was restricted to four events as the school authority had allocated four dates during the school year that had to be used by the teachers for professional development. During these four dates, the school authority offers a mix of courses to their teachers and the PRIMAS course was one of the options for mathematics and science teachers. The situation at this school might sound rather specific, but such structures for supporting professional development can be found at many schools in the Netherlands. As we had to restrict the course to four events, we assembled four modules from the seven modules that are available.

To support teachers’ professional development, they were asked to work in pairs. The idea of pair-work was initiated by the school authority, which also provided handheld video cameras to support peer observation.

4.2 National adaption in Germany

4.2.1 Education of multipliers

All professional development courses in Germany were held by multipliers. These multipliers were normal teachers who had had almost no prior experience in providing teacher education. They were trained especially for this task in order to be PRIMAS multipliers. The German model of educating the multipliers followed the three-strand model described above: learning-off-job, learning-by-job and learning-on-job. The education of the 26 multipliers began in February 2011, 1 year before they started running courses themselves (February 2012), and ran on 5 days spread over the year 2011. While working as a multiplier, they were to receive 3 days of further, learning-off-job training per year (6 days in total). Additionally, the PRIMAS team was to study their work by sitting in on courses and giving advice (learning-by-job). From year three on, they were introduced to the practice of self-education (learning-on-job).

As a means of supporting them, the multipliers were all assigned to work in teams of two during their task of educating up to 15 teachers (learning-by-job). It was decided that multipliers would meet their groups on three to four afternoons in each year of training (2012 and 2013). Additionally, two big events were planned to bring together all multipliers, their teachers and the PRIMAS team as an opportunity for them to exchange experiences (in December 2012 and by the end of 2013).

In the basis qualification, we used the seven modules provided by the international project, as well as some new modules that were specially developed by the German team. Multipliers had asked specifically for topics such as: “How to support students systematically in developing competences in inquiry?”; “How to assess written class tests?”; and “How to use experiments in sciences for inquiry?”

In all meetings with the multipliers, reflection on a meta-level played a crucial role. The multipliers reflected on how to use the modules in their courses and what problems might arise. Further, professional development courses were simulated.

4.2.2 Professional development courses

The professional development courses in Germany were designed to run over 2 years, starting in February 2012 and ending in December 2013. Altogether, about 100 teachers participated in the PRIMAS courses in Germany during the project lifetime. The German professional development courses were set up to follow the above-described principles and the spiral model (Sect. 3.1). The seven provided modules were used in their existing form; however, supplementary modules, as discussed above, were added.

Between sessions, teachers were assigned the task of trying out some aspect of IBL and then to reflect upon it during the following session.

4.3 Reflection on the two country adaptations

The main goal of PRIMAS is a widespread, European implementation of IBL. This calls for a focused, yet flexible professional development program. The Netherlands and Germany developed and implemented the PRIMAS strategy with common aspects and with different elements.

First, as the two cases show, the materials for professional development are essential elements of the implementation model. High quality materials support a common understanding of IBL and the pedagogies needed for implementation. Second, in both cases strategies were found to minimize potential risks when working with multipliers. In Germany, the potential risks of offering professional development courses with an insufficient quality was minimized by a very intensive education of the multipliers following the Müller model (2003). In the Netherlands, the same risk was minimized by working together with experienced colleagues from teacher colleges.

The Dutch and the German cases illustrate the flexibility of the model and the potential to adapt the professional development resources to national contexts.

5 The evaluation of the PRIMAS try-out in 12 countries

The aim of PRIMAS is to contribute to a widespread implementation of IBL in day-to-day mathematics and science teaching. As every design should be measured against its initial starting point (Kelly 2006), a baseline study (Engeln et al. 2013) and an analysis of contexts (Dorier and Garcia 2013) were carried out in each of the 12 partner countries at the beginning of the try-out.

The main criterion evaluated in the try-out (fourth cycle of our iterative improvement, see Sect. 2.2) was the effectiveness of the PRIMAS dissemination and implementation model. Four aspects were related to this evaluation: (1) the effectiveness of the professional development courses; (2) the effectiveness of scaling up; (3) the effectiveness of the supporting measures; and (4) the effectiveness of the national adaptation and the implementation in the 12 countries. In the following we will outline our evaluation approach in regard to these aspects.

5.1 Effectiveness of the professional development courses

Research on how professional development courses have an impact on teaching practice is considered to be complex (Lipowsky 2004). In order to capture as many aspects of this complex enterprise as possible, we planned to follow the four levels of effects of professional development from Lipowsky (see Sect. 3.1) and used a mixed-method design that is also recommended for design research (Kelly 2006). Our evaluation consists of both a formative, qualitative evaluation and a summative, quantitative evaluation.

5.1.1 Summative evaluation

When completed, the summative evaluation should provide us with an overview of the changes that occurred with all teachers that participated in PRIMAS across 12 countries. It will report on teachers’ beliefs and teaching styles with the help of teachers’ and students’ questionnaires. The summative evaluation has been designed to provide answers to the following questions:

  • What beliefs do teachers have about IBL?

  • What problems do they see in relation to the implementation of IBL?

  • What way of teaching do they use?

  • What beliefs in relation to science and mathematics do the students have?

The development of items for the questionnaires was based on our definition of IBL (Sect. 1). Whenever possible, tested items, for example from PISA, were used.

Teachers completed the questionnaires before they started participating in the professional development courses at the beginning of 2012 and should complete them again upon finishing their training course (target: by the end of 2013). Congruently, each teacher’s students also completed questionnaires at the time their teacher began participation in the development courses (and were to do so again at the end of their teacher’s IBL training). The questionnaires for the students addressed their activities in the classroom, their teachers’ activities and their beliefs about mathematics and/or science. Evaluation of the responses to the questionnaires will allow reconstruction of each teacher’s beliefs and their way of teaching, as well as students’ beliefs. These questions will be evaluated in regard to differences between the pre-test and the post-test and under consideration of subject-specific and cultural differences.

Each country is expected to participate in the summative study and to have gathered 50–100 questionnaires from teachers and up to 1,500 from students (in the pre- and post-test) by project end.

5.1.2 Formative evaluation

The aim of the formative evaluation is to give a deeper insight into the effects of the professional development courses on the individual teacher. The guiding question is:

How does a teacher evolve during their participation in the PRIMAS professional development courses?

We also plan to obtain answers to the following more detailed questions:

  • How do the teachers’ beliefs about IBL and on effective teaching evolve?

  • What impact does the context have?

  • How does the teacher’s teaching practice evolve?

Each country is supposed to analyze two cases (teachers) taking part in the professional development courses. The schedule of the evaluation follows our spiral model (see Sect. 3.1). Apart from interviews with teachers to obtain insight into their perspectives, observations in classrooms and the development courses should provide insight into how aspects of the development courses are transferred into practice (see Fig. 4). For the evaluation, we developed common interview and observation schedules for the professional development courses and the lessons.

Fig. 4
figure 4

Concept of formative evaluation of the professional development

Full size image

Due to constraints of time and money, a translation of all qualitative data in an international project is practically impossible (Baistow 2000, p. 11). Additionally, there is always the risk of changes in meaning and nuance when data are translated. For these reasons, it was decided that each country would evaluate its own data. In order to do this, we developed a common evaluation framework with guiding questions and agreed upon a common method for data evaluation (qualitative analysis of content; Mayring 2003). Each partner country then follows the guiding questions to summarize its results in two country case studies (in English) and report on them in December 2012 and September 2013.

5.2 Effectiveness of scaling up with multipliers

The effectiveness of the cascade model had to be measured in terms of: (a) the number of teachers reached by multipliers; and (b) the quality of the training the multipliers then provided. Point (a) was quite easy to achieve. As required by the EU and by design research (Edelson 2006), every country carries out a systematic documentation of the teachers taking part in every professional development course (participation lists). Therefore, it could be seen immediately whether the multipliers were reaching the required numbers and also quickly find out the rate of teachers who might have stopped participating. As the numbers are always provided regularly, this allowed us to react in cases where the rates were not on target (for example by announcing a new course).

Regarding point (b), in order to evaluate the quality of the courses provided, we implemented observations of the professional development courses chosen for each country’s two case studies (see Sect. 5.1.2). The guiding question used for this purpose was:

Was the professional development delivered following our concept?

In order to answer this question we decided to look for the following during our observations: whether the course allowed the participants to reflect on their teaching and their beliefs; to what extent the course followed the spiral model of analysis, implementation and reflection; and to what degree were the multipliers using the modules provided.

Of course, the quality of the professional development course that the multipliers provided was connected to the training they had received themselves. (Based on the Müller model (2003) and discussed in Sect. 3.2.) For this reason, we needed to find out, for example: whether the multipliers had gained enough knowledge about IBL and its use in class; if the multipliers knew enough about teachers’ problems and questions when trying to implement IBL; and if they had profited from their training based on the Müller model.

As the success of the model stands and falls with the successful education of the multipliers, the multipliers have a very central role within the complex structure of the model. For this reason, it was highly important to obtain meaningful and constructive answers to these questions directly when running the courses for the multipliers. This enabled us to improve the training if the multipliers did not feel that they were being given sufficient support. Thus, each country selected an evaluation method that was best suited to its national context and specific evaluation culture (Hantrais 1995); these methods included questionnaires, group discussions and group interviews.

5.3 Effectiveness of the supporting activities

Following the socio-ecological approach of Dalton et al. (2007), the question of how to evaluate the effects of this approach arises. In PRIMAS, we followed two strategies: (a) quantifying the number of people reached by counting; and (b) writing case studies.

(a) One quick method is to simply count dissemination activities, participants, publications, people who have subscribed to the newsletter and so on. However, this does not necessarily inform us about the impact of a single activity, or the overall effect that the combined activities have had, or will have. For example, if a conference attendee is a person responsible for all teachers’ professional development in a certain area—and if this person shows a high interest in a project and its materials—the impact is very different from the participation of a “normal” teacher. This question needs to be discussed further in light of the fact that more dissemination projects have been planned.

(b) In addition, each country was assigned the task of writing case studies about two relevant dissemination activities which should provide a deeper insight into how the dissemination activities worked. These cases will help us to analyze and compare dissemination activities and learn to what extent certain features work within a certain context.

5.4 The effectiveness of the implementation in 12 countries

Our explanations above show that by the end of the PRIMAS project every country involved will have collected an enormous amount of data which will in turn provide insight into how the PRIMAS model worked within its particular context as analyzed in our context analysis (Dorier and Garcia 2013). This data helped countries to correct their proceedings if deemed necessary, and were thus used in a process-orientated manner. In addition, by the end of the project, each country will have summarized and analyzed the results of the evaluation in a country case study. These country case studies will show which aspects proved to be more—or less—successful. Further, they will reveal the contextual factors that turned out to be supports or impediments—and provide information about why this was so. These country cases will be based on all the data collected in each country as outlined above, documentation of all activities carried out and expert discussions in the NCP. An analysis and comparison of the 12 country cases will allow us to understand how far the PRIMAS model worked under the different contexts.

6 Conclusion

In this paper, we have presented a complex model about how to implement IBL in day-to-day teaching. The design of this model follows the principles of design research. It tries to provide an answer to a relevant question, namely “How can inquiry-based learning be implemented in day-to-day teaching across Europe?” The model’s design model was iterative as outlined in Sect. 2.2 and based in theory, as our discussion in Sect. 3 shows. The PRIMAS dissemination model was also context-orientated, as the international model was nationally adapted (see Sect. 4). At the time of writing (April 2013), we are currently conducting and evaluating a try-out in 12 countries. This evaluation was planned to be mainly process-orientated. During this try-out, the formative and process-orientated evaluation allows us to correct single aspects if deemed necessary, but not the overall model: for example, if a dissemination activity addressed to heads of school should not be very successful due to low turnout, the partner would think about other events aimed at supporting teachers considering the socio-ecological model. However, a partner country could not, for example, decide to just stop running professional development courses and only provide materials instead. Such a decision would conflict greatly with the overall model.

Following the project’s active phase, it may be necessary to refine the overall model. In Sect. 5, we have shown that the try-out was designed to help us collect an enormous amount of data which can be used to give us information about the overall effectiveness of the model and how it worked in the different national contexts. This information will feed a new try-out of an implementation model which is to be carried out in the follow-up project MaSciL. This project will allow for a try-out in more—and in some cases different—countries than those involved in PRIMAS.

As design research should not only be based in theory but also contribute to theory-building, the following question arises: “What can we learn from PRIMAS for research?” The main intent of PRIMAS is to design a model for the dissemination and implementation of IBL. When the retrospective analysis of the overall data is completed, it will allow us to see which aspects of the model worked in which context and how. We will analyze data from 12 different countries and thus 12 country cases, which will enable us to set up a design framework (Edelson 2006). This will provide guidelines for achieving a particular set of goals—in relation to dissemination and implementation—within a particular context. In our dynamic, globalized society, such guidelines are of great importance for improving and innovating education.

Note

This paper is based on the work within the project PRIMAS—Promoting inquiry in mathematics and science education across Europe (http://www.primas-project.eu).

Project coordination: University of Education, Freiburg (Germany). Partners: University of Genève (Switzerland), Freudenthal Institute, University of Utrecht (The Netherlands), MARS—Shell Centre, University of Nottingham (UK), University of Jaen (Spain), Konstantin the Philosopher University in Nitra (Slovak Republic), University of Szeged (Hungary), Cyprus University of Technology (Cyprus), University of Malta (Malta), Roskilde University, Department of Science, Systems and Models (Denmark), University of Manchester (UK), Babes-Bolyai University, Cluj Napoca (Romania), Sør-Trøndelag University College (Norway), IPN-Leibniz Institute for Science and Mathematics Education at the University of Kiel (Germany).