1 Introduction

The question of how to promote mathematics teachers’ PD is of great interest and has been discussed in various papers (e.g. Krainer and Zehetmeier 2013; Loucks-Horsley et al. 1996; Maldonado 2002; Sowder 2007; Zehetmeier 2010, 2014a, b; Zehetmeier and Krainer 2011). In this context, the question of impact is of particular relevance. Evaluations and impact analyses of PD programmes are mostly conducted during or at the end of a project and exclusively provide results regarding short-term effects. These findings are highly relevant for critical reflection of the terminated project and necessary for the conception of similar projects in the future (Fullan 2006). However, apart from and beyond that, an analysis of sustainable effects is crucial (Loucks-Horsley et al. 1996). Despite its central importance for both teachers and teacher educators, research on sustainable impact is generally lacking within teacher education disciplines (Datnow 2006; Rogers 2003). This kind of sustainability analysis is often missing because of a lack of material, financial and personal resources (McLaughlin and Mitra 2001; Hargreaves 2002). This paper addresses the factors which influence the sustainability and scale-up of a particular professional development programme’s impact.

2 Theoretical framework and research questions

In this paper, theoretical models and empirical findings from impact research (e.g. Zehetmeier and Krainer 2011) and innovation research (e.g. Cobb and Smith 2008; Rogers 2003) are combined, with the aim to use them as a theoretical framework for the analysis of data. In particular, this framework is used to discuss the questions concerning sustaining and scaling up the impact of mathematics teacher PD programmes.

2.1 Impact research

We use a comprehensive theoretical model covering the issue of impact of professional development programmes, the IPD (Impact of Professional Development) model (Zehetmeier 2008; Zehetmeier and Krainer 2011), which combines and integrates theories and results of previous research activities on this topic. It was developed based on a literature review (Zehetmeier 2008) and offers a structured overview regarding existing knowledge and concepts of the topic. Within this model, core elements constituting PD activities and central levels of possible impact are juxtaposed; the impact of PD programmes can be regarded as changes or innovations within the respective levels, which are influenced by fostering factors (Fig. 1).

Fig. 1
figure 1

The IPD model

Full size image

In this model, the following elements are used to describe teacher PD programmes (see also Borko 2004): participating teachers, participating facilitators, the programme itself, and the context in which these are embedded. On the other hand, Zehetmeier (2008) introduces three major levels for describing teacher PD programmes’ impact to be used in the IPD model: Zehetmeier (2008; see also Zehetmeier and Krainer 2011) highlights that the levels knowledge, beliefs and practice are suitable to cover the impact not only regarding the participating teachers, but also concerning other parties; for example pupils, colleagues or principals. This paper’s focus, however, is on the participating teachers’ levels.

2.2 Sustainability

Within the literature, there is a remarkable number of (to some extent disputed) concepts and definitions of “sustainability”. The notion of sustainability is mainly part of ecological and economic terminology, but is also increasingly used in the instructional and educational realm. Without using this notion explicitly, as early as 1657 Comenius dedicated a chapter of his opus Didactica Magna to the “foundations of lasting teaching and learning”. In the twentieth century, the notion “institutionalization” was used to describe temporally stable changes. During the 1970s, institutionalization was seen as the third and final phase of change processes in schools: (1) mobilization, (2) implementation and (3) institutionalization (Anderson and Stiegelbauer 1994, p. 280). Anderson and Stiegelbauer (1994) define institutionalization as “a phase after initial implementation when an innovation either got ‘built in’ to ongoing use and organizational structures or was discontinued due to such factors as the loss of funding, staff turnover, competing practices, and low administrator or teacher commitment” (p. 280). Within the teacher education disciplines, other models also are used which highlight that sustainability might refer to individual, group or system level, for example the “hierarchy of sustainability” model (Seufert and Euler 2004): at stage 1, an innovation is sustained due to a social group’s own interest, to obtain the material benefits of the programme (“project-oriented sustainability”); at stage 2, an innovation is not only maintained by those directly involved, but leads to an efficiency increase of the entire system (“system-oriented sustainability”); at stage 3, the innovation leads to behavioural changes, which allow the involved individuals or organizations to respond flexibly and appropriately to environmental conditions (“potential-oriented sustainability”).

Fullan (2006) defines sustainability when discussing educational change as “the capacity of a system to engage in the complexities of continuous improvement consistent with deep values of human purpose” (p. 114). Another definition says: “a reform is considered institutionalized when it becomes a taken-for-granted feature of life in a school” (Datnow 2005, p. 123). Hargreaves and Fink (2003) refer to the individual level and claim that “sustainable improvement requires investment in building long term capacity for improvement, such as the development of teachers’ skills, which will stay with them forever, long after the project money has gone” (p. 3).

To sum up: the expected outcomes of PD programmes are not only focused on short-term effects that occur during or at the end of the project, but also on long-term effects that emerge (even after some years) after the project’s termination. Effects that are both short-term and long-term can be considered to be sustainable. Sustainability may refer to both system and/or individual level. Sustainability can be defined as the lasting continuation of achieved benefits and effects of a project or initiative beyond its termination (DEZA 2002).

2.3 Scale-up

Empirical evidence concerning the question of scaling up the impact of PD programmes points to the finding that “prior large-scale improvement efforts in mathematics and other subject matter areas have rarely produced lasting changes in either teachers’ instructional practices or the organization of schools” (Cobb and Smith 2008, p. 232). Thus, it seems reasonable to focus on factors which might foster the broad effects and scale-up of PD programmes’ impact. Cobb and Smith (2008) highlight networks, shared vision and mutual accountability as key factors for the scale-up of changes and impact in mathematics teacher education.

Teacher networks are described, for example, as groups of colleagues who provide social support in developing demanding instructional practices; this affords time built into the school schedule for collaboration among mathematics teachers and access to colleagues who have already developed relatively accomplished instructional practices.

Moreover, a shared vision of high quality mathematics instruction fosters the scale-up of impact: this includes a shared vision concerning the question of instructional goals (what pupils should know and be able to do mathematically) and the question of how pupils’ development of these forms of mathematical knowing can be supported.

Another key factor which fosters the scale-up of changes and impact in mathematics teacher education is mutual accountability. This means, for example, that if school leaders hold mathematics teachers accountable for developing high-quality instructional practices, then—in turn—school leaders are mutually accountable to mathematics teachers for supporting teachers’ learning. This factor also includes, for example, district superintendents who hold principals accountable for supporting mathematics teachers in improving their instructional practices; these superintendents—in turn—are mutually accountable for providing and organizing school-based PD possibilities.

Rogers (2003) highlights that the diffusion and scale-up of innovations and impact depend on several characteristics: relative advantage, compatibility, complexity, trial ability and observability. Fullan (2001) describes similar characteristics (need, clarity, complexity, quality and practicality) that influence the acceptance and impact of innovations. Relative advantage includes the perceived advantage of the innovation (which is not necessarily the same as its objective). An innovation with greater relative advantage will be adopted more rapidly. Compatibility and need denote the degree to which the innovation is perceived by the adopters as consistent with their needs, values and experiences. Complexity and clarity include the teachers’ perception of how difficult the innovation is to be understood or used. Thus, more complex innovations are adopted rather slowly, compared with less complicated ones. Trial ability denotes the possibility of participating teachers being able to experiment and test the innovation (at least on a limited basis). Innovations that can be tested in small steps provoke less uncertainty and will be adopted as a whole more rapidly. Quality and practicality also influence the process of scaling up: high-quality innovations that are easily applicable in practice are more rapidly accepted. Observability points to the claim that innovations which are visible to other persons (e.g. parents or principals) and institutions are more likely to be rapidly accepted and adopted.

2.4 An exemplary professional development programme in Austria: PFL

PFLFootnote 1 is an Austrian PD programme, which started in 1982, has undergone several adaptations, and is still running (for more detail, see Rauch et al. 2014). The programmeFootnote 2 is designed for teachers from all types of schools across the nation, including all age groups of pupils. The overall focus of PFL is on the PD of teachers in the fields of content, didactics and pedagogy. School development plays a central role without losing sight of classroom instruction. The major goals of the teaching process (e.g. heeding the learner’s individual background, promoting self-reliance and independence, as well as cooperation) should be primarily achieved through—and not detached from—the subject-related design of teaching and learning. PFL takes 2 years and is organized in three 1-week seminars with workshops, input, discussions and group work; in addition, five 2-day meetings for regional groups are arranged at participants’ schools with lesson observations, analyses and discussions of the participants’ work. The focus is on the individual teachers’ own reflective practice using action research methods (Altrichter and Posch 2007). By the end of the course, each participant is obliged to write a reflective paper using the data he/she has gathered throughout the process through the use of qualitative and quantitative research methods. Participants are part of a community of practice (Wenger 1998), since their work is embedded in a structure of mutual assistance and external support.

2.5 Research question

The research question dealt with here is: which factors influence the sustainability and scale-up of the PFL programme’s impact?

3 Methodology

In this paper different types of data sources (document analyses and interview series) are combined in order to describe as well as explain the PFL programme’s various impacts on different levels, according to the theoretical framework. This methodological design was set up with the aim of identifying hypotheses concerning the research questions, using a qualitative explorative case study setting. We present the cases of two former participants of PFL who provided data concerning their ex-post perspectives regarding the sustainability and scale-up of the PFL programme’s impact. Data was gathered by teachers’ self-estimation. Though there might be limitations due to issues of self-disclosure, Lam and Bengo (2003) highlight that retrospective processes provide high validity. The study presented in this paper includes data from various sources and time periods to gain validity by “convergence of evidence” (Yin 2003, p. 100).

3.1 Document analysis

In a first step, a document analysis was carried out to get insight into participating teachers’ perspectives during and/or at the end of the PD programme. Thus, the reflective papers (written by the teachers at the end of their participation in PFL, see Sect. 2.4) were analysed. This analysis aimed at gathering data concerning short-term impact which (a) occurred during and/or at the end of the teachers’ PFL participation and (b) might hold the possibility of sustainability and scale-up. Subsequently, this data formed the basis for the interviews series.

3.2 Interviews

In a second step, a series of in-depth interviews was designed to gather additional data concerning the participating teachers’ perspectives from an ex-post perspective (up to 20 years after the end of their participation in the PFL programme). In 2013, former PFL participants were interviewed to gather data concerning the diffusion of impacts and innovations.

The interviews were semi-structured, since they were based on the analysis of existing data (document analysis), which identified various levels of short-term impact which occurred during and/or at the end of teachers’ participation in PFL. The interviews were designed accordingly (a) to gather data concerning the sustainability and scale-up of impact and (b) to reveal other types of impact which were not already coded. Therefore, the interview questions were both closed (e.g. could the impact you described in your reflective paper be scaled up?) and open (e.g. what further impact of PFL is still effective?). To avoid possible bias (e.g. due to social desirability), teachers were explicitly asked to give frank and accurate answers. All interviews were audio recorded and fully transcribed verbatim.

3.3 Case study design

This research makes use of a case study design (Stake 1995; Yin 2003), which is particularly suited for impact analysis (Rogers 2003). Here, the aim is not to elaborate the number (and variance) of impacts, but to describe and explain what kinds (and variability) of impact occur. In this sense, Hancock and Algozzine (2006) state: “Through case studies, researchers hope to gain in-depth understanding of situations and meaning for those involved” (p. 11).

This paper’s case studies can be classified as historic, since they analyse changes over time (Merriam 2001).Moreover, they are intrinsic case studies, since they focus on the particular teachers’ cases (Stake 1995); and since they aim at the respective developments’ fostering conditions, they are explaining case studies (Yin 2003).

The cases were selected to represent extreme cases with rather high and low impact (concerning sustainability and scale-up). Thus, they are able and suited to highlight and clarify the variability of possible impacts. The sampling is an information-oriented convenience sample, since it is based on the possibility to gather rich data and to have access to former PFL participants.

3.4 Data analysis

Both inductive and deductive elements of data analysis are used (Altrichter and Posch 2007). In a first step, data from teachers’ written documents was coded inductively, according to the research question. In the next step, the interviews were planned and conducted. This data was coded both inductively and deductively in order to be able to combine and contrast the interview results with those of the document analysis. Data was analysed according the theoretical framework to understand both the impacts’ sustainability and scale-up, as well as the respective influencing factors. Data analysis was based on principles of qualitative content analysis: data summary, data explication and data structuring (Mayring 2003, p. 58). In particular, inductive category building was used to identify common topics, to elaborate emerging categories, and to gain deeper insight into the impacts’ development over time.

3.5 Validity

To ensure the validity of findings, various verification procedures were used: Creswell (2007) identifies several procedures for qualitative studies and recommends that at least two of them should be given to ensure validity. Five of these verification procedures (triangulation, member checking, researcher bias, peer review and rich description; Creswell 2007, p. 209) were present in this study. (a) The data used for analysis of the research questions came from a variety of sources which were gathered at various time periods (triangulation by convergence of evidence, see above). (b) Preliminary research results were refined by use of member checking methods with regard to disconfirming evidence until any disagreements (between researcher and teachers) among the findings were eliminated. (c) The role, position, and possible bias of the researcher are laid out within the study. (d) The case studies were developed within peer review settings: the researcher was supported by and discussed the studies with critical friends of his research institution. (e) The use of case study design enables detailed information on all persons and activities relevant for this research (rich description).

4 Results

Here, the cases of EveFootnote 3 and Paul, two former participants of the PFL programme, are provided. These case studies aim at providing close-up perspectives on the sustainability and scale-up of impact by qualitative data analysis.

4.1 The case of Eve

4.1.1 Eve: background

Eve participated in the PFL programme for two school years, starting in September 2000. At that time, she was a mathematics teacher in a lower secondary school and had 20 years of teaching experience. One of her classes was an 8th grade class of 27 pupils, which was the focus of her PFL activities. The school staff consisted of 16 teachers. Since 1997, open learning had been part of the school’s daily routines. Within these settings, pupils could choose their individual working pace, task sequences and social forms. Moreover, pupils bore responsibility and control concerning their learning efforts. The school’s mathematics teachers began to prepare working plans and various teaching materials.

4.1.1.1 Eve’s goals

Against this background, Eve had the goal to promote these open learning settings by implementing new teaching approaches in her mathematics classes. She aimed at enhancing pupils’ self-directed and inquiry-based learning opportunities. Moreover, she wanted to test the limits of such settings. In particular, her objective was to foster her 8th grade pupils’ (who would leave the school after this grade) individual responsibility concerning their mathematical learning’s process and outcome. Thus, Eve intended to accomplish these goals by introducing open learning environments and using working plans in her classes.

4.1.1.2 Eve’s research question

During her participation in PFL, Eve was interested in the following question: “Are my pupils able to acquire mathematical knowledge by using self-directed learning settings?” (Eve’s reflective paper, p. 3). Here, self-directed learning settings included (a) searching and working with information from various sources, (b) creating and maintaining a personal learning diary, (c) choosing both challenging and feasible tasks, and (d) estimating one’s own competences.

4.1.1.3 Eve’s methods

Eve used various methods of action research to find answers for her question. On the one hand, she developed observation sheets to gather data regarding classroom activities. On the other hand, she kept a research diary in which to save personal experiences and thoughts. Further, Eve prepared various tasks and conducted interviews with individual pupils to assess their mathematical learning progress. At the end of each semester, pupils were asked to complete a questionnaire concerning their learning and skill development, as well as their evaluation of the open learning settings.

4.1.1.4 Eve’s findings

Eve’s observations portrayed a picture of pupils with “eagerness, motivation, calmness, curiosity, and autonomy; however also scepticism, uncertainty, and questions” (Eve’s reflective paper, p. 20). All pupils gained positive achievements in the assessment tasks, which was “really surprising” for Eve: “All without exception were able to develop knowledge and skills in this self-directed and open learning environment” (Eve’ reflective paper, p. 21). The questionnaire’s results revealed 22 (out of 27) positive ratings regarding the open learning settings; 3 ratings were neutral, 2 ratings were negative.

4.1.2 Eve: impact

The following paragraphs provide data concerning this paper’s research question. In particular, various impacts which were sustainable and scaled up, as well as their respective fostering factors, are described.

4.1.2.1 Implementing innovative teaching approaches

During her participation in PFL, Eve implemented new and innovative teaching approaches to enhance her pupils’ self-directed and independent learning. This impact was sustained and is still persisting today (12 years later). She describes this impact as follows: “Of course, I still want to try something new, something better; again and again. I really need innovative ideas for both myself and my pupils” (interview, p. 3); “When participating in PFL, I learned about the benefit of innovations and how to tackle doubts” (interview, p. 9).

This impact was (and still is) fostered by the school principal’s support. Eve states: “This is very important for me and my teaching practices. Every idea I want to try is welcome and gets support by the principal. And by now, there are many things I have integrated into my teaching routines” (interview, p. 12).

Eve also describes another factor that fostered the sustainability of this impact. She highlights that within the school there is a high level of mutual appreciation: “All of us, teachers, pupils, administrative staff, we have a culture of appreciation. This allows me to update my teaching practices without being impeded by colleagues or pupils; rather, everyone seems curious” (interview, p. 18).

Eve points to one more fostering factor: her pupils’ benefit. In particular, Eve highlights that the pupils’ joy and success are core reasons for her to keep this impact sustained: “I can see very clearly that our lessons run both smoothly and goal-oriented if pupils are allowed to work on their own. They really learn mathematics. Thus, I want to push further this teaching approach” (interview, p. 8).

4.1.2.2 Using many books and materials

Within her PFL project, Eve started to make use of additional books and teaching materials. She stated that this impact was sustained: “Since then, I use more books than just one or two, which would be the usual and common way. Rather, I began to collect books and materials. This is the reason why I feel free und able to change and adapt my teaching practice whenever needed” (interview, p. 7).

The main factor which supported the sustainability of this impact is the pupils’ benefit. Eve points out: “The pupils have to choose from various books and materials. They have to learn to decide which books, which tasks, are suited for them, and which are not. Thus, beyond their mathematical learning, they develop an autonomous and self-directed working habit” (interview, p. 9).

4.1.2.3 Planning lessons

In the course of her PFL project, Eve paid particular attention to lesson planning. She invested additional time and other resources to explicitly planning a lesson’s goals, methods and tasks to challenge pupils’ thoughts and learning: “During PFL, I could see the importance and purpose of such kind of lesson planning” (p. 10). This impact could also be sustained. The main factor that fostered the sustainability of this lesson planning was the pupils’ advanced learning: “If I did not plan which problems could possibly arise, then it would not be possible to maintain the lessons’ goals and outcomes or to support pupils’ mathematical learning” (interview, p. 11).

4.1.2.4 Reflecting on teaching activities

A major focus of the PFL programme is on the teachers’ reflective practice (see above). Thus, Eve conducted various self-evaluations during her participation in PFL and gained knowledge concerning action research methods: “This knowledge is really an advantage which I very much appreciate” (interview, p. 18).After the end of her participation, she continued to reflect on her teaching practices. This impact was sustainable: “On a regular basis, I still reflect on my lessons; either on my own, or together with my pupils; sometimes verbally in a kind of group discussion or by means of a questionnaire” (interview, p. 19). This impact’s sustainability was fostered by several factors. On the one hand, Eve experienced direct advantage by getting information on her classroom performance: “These evaluations tell me what is running well, and what should be changed. This is very important for me” (interview, p. 15). On the other hand, the school’s principal is convinced that reflections and self-evaluations are important steps on the journey to school quality: “He is continuously reflecting on his leadership role and evaluates various projects on a regular basis” (interview, p. 16). Besides this, Eve highlighted that she is not only seeking new ideas for her teaching; rather, she disseminates and shares her experiences and expertise with colleagues: “I see myself as someone who likes to learn; not only during PD programmes, but also in my everyday teaching” (interview, p. 19); and she stated: “teaching without learning is not possible” (interview, p. 21).

4.1.3 Eve: scale-up

Some impacts were not only sustained, but also scaled up: they had broad effects within Eve’s school.

4.1.3.1 Implementing innovative teaching approaches

It was not only Eve who started and continued to implement innovative teaching approaches to enhance her pupils’ self-directed and independent learning. In addition, her colleagues also began to work in this way: “It is not only my personal goal and purpose to support the pupils’ autonomy; after some rather short time, the whole school staff began to see the benefit and integrated this idea into their classrooms. Every subject team developed new ideas and materials, and informed the other colleagues about their implementation success or failure. Today, this is rather a part of our school culture than something special” (interview, p. 14).

4.1.3.2 Using books

Similarly to Eve, her colleagues also started to use more books and additional materials in their lessons: “This is a great development. The colleagues really use them. In every school term, we decide to dedicate some extra budget for supplementary books and materials” (interview, p. 14).

4.1.3.3 Influencing factors

The scale-up of impact was fostered by several factors. On the one hand, the teachers engaged in joint reflection and communication: “We have a lot of discussions; we meet after the lessons or during weekends to reflect on our practice and to support each other when implementing new ideas. For example, when I designed the mathematics working plans, all other mathematics teachers were keen to join and engaged in lesson planning. Whenever I have any question concerning mathematical content or pedagogical issues, I ask my colleagues; and vice versa” (interview, p. 4). Moreover, teachers cooperated across the school subjects: “Of course, the colleagues teaching music or science were involved, since these subjects are easy to combine” (interview, p. 13). On the other hand, the teachers hold similar values and standards concerning pedagogical or subject-related issues. Some of them also participated in action-research-based PD programmes which were similar to PFL: “We have many really innovative and motivated colleagues in our team. Thus, most ideas and developments get rather broad support” (interview, p. 5). The school’s principal showed great interest in, and provided support for, the teachers’ activities. Eve states: “He also is a former PFL participant. Since he also conducted similar action research projects, he really understands our ideas and supports our activities. He participates in our school’s mathematics study group and shares his perspective with us” (interview, p. 20). Another factor which fostered the scale-up of impact was teachers’ voluntariness. Each teacher was free to (not) adopt innovative practices. In particular, even if a teacher had already started to implement pupils’ self-directed learning, he or she was able to cancel at any time. Eve pointed out: “Everybody was free to start or stop whenever and for whatever reason. However, nobody stopped” (p. 17). Moreover, the teachers established a system of mentoring new colleagues: every new teacher joining their team was introduced by the subject coordinator who informed him or her about actual issues. “In particular, they are told that they are free to both share and adopt ideas; they are brought in very smoothly; this is very helpful whenever there is staff turnover” (interview, p. 17).

4.2 The case of Paul

4.2.1 Paul: background

Paul’s PFL participation took four semesters. During his participation, he was a secondary teacher in a rural grammar school.Footnote 4 Paul was highly interested in cooperation and team teaching.

4.2.1.1 Paul’s goals

Paul’s goal was to foster cooperative practice (such a steam teaching or mutual classroom observations) within his school. His objectives were to inform about and—if possible—to convince some of his colleagues to engage in such practices. Thus, he introduced his ideas and goals at a “pedagogical day” (a meeting of all teachers and the principal) in his school, and stated his willingness to plan and implement cooperative practices together with interested colleagues.

4.2.1.2 Paul’s research interest and methods

Since Paul was interested in cooperative practices, he used methods of action research to gather data concerning this topic: he evaluated his colleagues’ perspectives on cooperative practices via an anonymous questionnaire and wrote a research diary to collect and save personal data. In particular, he wanted to find out, whether his colleagues were willing to cooperate and—if yes—which kind of cooperation(s) they would practise.

4.2.1.3 Paul’s findings

Paul found that there was significant interest in engagement within the group of his colleagues: “Many colleagues stated that they were interested, at least at the beginning” (interview, p. 4). Sixty-two percent of Paul’s colleagues participated in his questionnaire survey: 91 % of them stated that they were interested or highly interested in cooperative practices (such a steam teaching or mutual classroom observations). However, almost none of his colleagues eventually engaged in such practices: “Nobody dares to open his/her classroom door” (Paul’s research diary; quoted in the interview, p. 9). During his participation in PFL, Paul cooperated with one colleague on a regular basis: “We tried to visit each other’s classrooms at least once every two months; and most of the time, we managed to do so” (interview, p. 6). However, this practice could be sustained for no longer than 1 year: “In the second year, my colleague was not able to continue our cooperation, since he changed school. This change was good for him and his career, but it was not good for our cooperation” (interview, p. 7).

One further colleague observed Paul’s teaching practices twice, “then this cooperation discontinued” (interview, p. 6). Further mutual observations or team teaching with other colleagues did not take place.

4.2.2 Paul: impact

The following paragraphs provide impacts concerning Paul’s colleagues and his own interest in cooperative practices, as well as their implementation. Further, the respective influencing factors are provided.

4.2.2.1 Paul’s colleagues’ interest in cooperative practices

During his participation in PFL, Paul intended to engage his colleagues in cooperative practices. His colleagues’ interest in such practices was high at the beginning, but decreased “within weeks” (interview, p. 4) and did not lead to concrete engagement. This impact was short-term and neither lasted until the end of Paul’s participation in PFL, nor afterwards.

Concerning the reason for this, Paul claimed that “maybe the idea was too new; maybe the colleagues were anxious because they were not familiar with this idea” (interview, p. 9). From an ex-post perspective, Paul supposes that more support and detailed information provided to the colleagues could have been helpful.

4.2.2.2 Paul’s own interest in cooperative practices

Paul’s own interest in such practices decreased over time, particularly after the end of the project. During his participation in PFL, he maintained his interest and intended his colleagues to cooperate with each other. In particular, he tried to get information from literature and practice examples from other schools’ teachers. Paul used the opportunity to network and exchange experiences with teachers from other schools and other regions within the group of PFL participants. Paul states: “This was like a virtuous circle: exchange led to interest, and interest led to exchange” (interview, p. 14). These networking practices also had an effect on Paul’s affective level: “I gained self-confidence. Colleagues from other schools confirmed that my ideas were great” (interview, p. 14). However, after the project’s termination, both his efforts and motivation vanished: “It was the PFL project which gave me some motivation to try further. However, after the project’s end, I realized that it was just my project, and not one of my colleagues. It was my interest, and not my colleagues’ or the principal’s” (interview, pp. 11–12). In particular, the possibilities to network with teachers from other schools were limited: “During the project, I met the colleagues on a given regular basis. Afterwards, we tried to meet in a self-organized manner; but this did not really work, since we all have so many obligations in our respective schools” (interview, p. 15).

4.2.2.3 Implementation of cooperative practices

During his participation, Paul engaged for 1 year together with a colleague in cooperative practices. This impact was short-term and could not be sustained. The main factor influencing this impact was staff turnover: Paul’s cooperating colleague left the school.

No further colleagues were willing or able to engage in cooperative practices. Concerning the reason for this issue, Paul states that the principal did not explicitly or directly hinder cooperative practices, but “… he made it difficult to realize” (interview, p. 12). For example, sometimes it would have been necessary to change teachers’ classroom schedules to allocate time for collaborative practices, “but several times, this was not possible; due to reasons, I could not fully comprehend” (interview, p. 12).

Here, a further influencing factor becomes obvious: the support provided by the professional development project’s support structures. This factor influences the project’s impact both directly and in directly: on the one hand, Paul had the possibility to network during and between project meetings. On the other hand, Paul had “certain freedom” in his school, because he was “the ‘PFL-teacher’ who is allowed to do some unusual things” (Paul’s research diary; quoted in interview, p. 2).

4.2.3 Paul: scale-up

Paul’s activities were not scaled up: they had no broad effects within Paul’s colleagues’ cooperative practices.

5 Discussion

5.1 Sustained impact

In the case of Eve, the implementation of innovative teaching approaches, the use of many books and materials, the planning of lessons, and the reflection of teaching activities were sustainable. The factors that fostered this sustainability were the school principal’s support, a high level of mutual appreciation and communication within the staff, Eve’s experienced advantage, and pupils’ benefit (regarding advanced learning, joy and motivation). PFL enabled community building, mutual appreciation, joint reflection and networking (Lerman and Zehetmeier 2008) and fostered school-based support, in particular by the principal (Fullan 2006; Krainer 2006; Owston 2007). Eve could develop ownership of the proposed change (Peter 1996) and was empowered to influence her own development process (Harvey and Green 2000). PFL provided opportunities to develop both content and pedagogical content knowledge and skills (Loucks-Horsley et al. 1996; Mundry 2005). Moreover, PFL activities focused on content knowledge and used content-specific material (Garet et al. 2001; Ingvarson et al. 2005; Maldonado 2002). The focus was on student-centred learning activities, and on learner-centred implementation (Farmer et al. 2003). Eve demonstrated an “executive use” (Joyce and Showers 2002) of the PFL programme’s content and methods, since she selected and implemented specific concepts according to her or her pupils’ needs.

Farmer et al. (2003) identified three levels of appropriation in order to characterize how mathematics PD interacts with the teachers’ professional selves. At level 1 (“concrete activity and content”) teachers use appropriate content such as specific mathematical skills or concepts. They look for specific mathematical problems, tasks or games to use with their pupils. At level 2 (“professional principles and understandings; attitudes and beliefs”) participants view themselves as professionals who are gaining additional knowledge. Regarding content, they look for and construct mathematical ideas that will allow them to integrate, connect and explain the mathematical concepts. Regarding pedagogy, they attempt to gain an understanding of strategies that can be useful in mathematics instruction, such as cooperative learning or challenging mathematical tasks. Teachers who are constructing knowledge at level 3 (“teaching as inquiry”) see themselves as learning from and in the process of teaching (in addition to being able to use and adapt concrete elements, learning mathematical ideas, and applying general principles for mathematics teaching). They view themselves as mathematical learners alongside their pupils and as learners about their pupils’ cognition. PFL provided opportunities for Eve’s active and inquiry-based learning (Garet et al. 2001; Ingvarson et al. 2005; Maldonado 2002), thus she took an “inquiry stance” (Farmer et al. 2003): data clearly indicates that she understood both herself and the pupils as learners in her own teaching processes.

Eve’s case study clearly shows the influence of her pupils’ success, joy and mathematics learning on the sustainability of impact. This opens the scope for an advanced perspective of the relationship between impact and its fostering factors: in Eve’s case, the PD programme initiated the implementation of student-centred open teaching settings; this activity led to the short-term impact of pupils’ benefit; this impact (pupils’ benefit) then, in turn, served as a fostering factor for the long-term continuation of the original activity; this sustainable impact (implementing open teaching settings) led again to pupils’ benefit. In sum, a kind of virtuous circle was unfolded, containing a reciprocal relationship of impact and fostering factors: fostering factors led to impact, which led to fostering factors, which led to impact.

5.2 Scaled-up impact

In the case of Eve, the implementation of innovative teaching approaches and the use of many books and materials were scaled up. The factors fostering this process were joint reflection and staff communication, similar values and standards, leadership content knowledge, teachers’ voluntariness, and a system of mentoring new colleagues.

These factors relate to several research findings. Eve’s colleagues engaged in teacher networks which provided communication and social support in developing and reflecting instructional practices; thus, there was exchange of expertise (Cobb and Smith 2008) between colleagues who had already developed accomplished instructional practices. This was perceived as an important relative advantage (Rogers 2003) which met Eve’s and the other teachers’ needs (Fullan 2001). Eve was not only seeking new ideas and opportunities to learn; rather, she also started initiatives and shared her experience. This identified Eve as an “omnivore” (Joyce and Calhoun 2010) who proactively seeks opportunities for development for herself and others, since she “draws information from her environments and integrates it into her conceptual systems” (p. 22). Moreover, a shared vision (Cobb and Smith 2008) of values and standards regarding high quality mathematics instruction (concerning the questions of instructional goals and how pupils’ development of these forms of mathematical knowing can be supported) was established. The school’s teachers could experiment and test the innovative teaching practices in small steps and had the possibility to cancel at any time. This trial ability (Rogers 2003) provoked less uncertainty and, thus, fostered the adoption of new ideas. Another key factor was the principal’s support and knowledge. This leadership content knowledge (Cobb and Smith 2008) was particularly enhanced, since the principal was also a former participant of PFL and was part of the school’s mathematics study group. The results of several studies point to the central influence of school leadership to the sustainable and scaled-up impact of school innovation initiatives (e.g. Fullan 2006; Owston 2007). Fullan (2006) proposes a direct correlation between the impact of innovations and the role of school leadership. Owston (2007) distinguishes three types of administrative support: neutral leaders (who meet innovations rather passively without promoting or prohibiting); supportive principals (who create and support beneficial environments for innovations); and actively involved leaders (who are driving visionary ideas, identify personally with innovations and motivate other teachers for the innovation). Eve’s school’s principal turned out to be of the third type, while Paul’s principal belongs to the first type. His school’s principal showed no interest in, and provided no support for, the teachers’ activities. In particular, there was no time provided within the school schedule for collaborative practices. Thus, the factors “teacher networks” and “mutual accountability” (Cobb and Smith 2008) were not present.

Moreover, the case of Paul points to further factors which influenced the scale-up of impact. Paul’s innovative ideas were not picked up by his colleagues since there was a lack of information; thus, factors such as relative advantage, complexity, clarity, compatibility or need (Rogers 2003; Fullan 2001) could not become effective, even though they may have been present. In particular, Paul’s colleagues did not develop a shared vision (Cobb and Smith 2008): many colleagues did not “dare” (Paul’s research diary, see above) to open their classrooms to discuss pedagogical or subject-related issues.

5.3 Implications

The case studies highlight that some influencing factors are within the PFL programme’s realm, while some other factors lie beyond the programme’s direct influence. It seems reasonable for PD programmes which aim at sustaining and scaling up the impact to focus not only on the “inner” factors, but also on the factors “beyond”. In particular, the case studies’ data points to context factors as being crucial for sustainability and scale-up. However, these context factors may not be under the control of the particular PD programme. In some cases, context factors may serve as a kind of mediator between teachers’ intended and implemented change processes. Thus, it would be reasonable to integrate each programme’s context factors explicitly into its conception, realization and evaluation. If some of these factors are dependent on the PD programme’s existence, then these factors may be substituted or backed up with alternative ones that are less or not at all connected to the programme’s existence.

Moreover, for programmes aiming at sustainable and scaled-up impact, it seems indicated that they should foster a supportive and actively involved school leadership; particularly regarding related hindering factors such as staff turnover (Scheirer 2005), this issue becomes highly relevant. Slavin (2004) states: “Innovations are often brought in or championed by … a small number of staff members, and a program may disappear when these people move on” (p. 61). In the case of Eve, the sustainability of impact as well as the institutionalization of innovations and the ongoing use within the organizational structures could be continued since the teachers developed a reasonable system for dealing with staff turnover (Anderson and Stiegelbauer 1994). In the case of Paul, both staff turnover and lacking interest led to the termination of cooperative practices.

Moreover, the goals of PD programmes should be clearly formulated and consciously differentiated concerning sustainability and scale-up. This distinction seems to be important and reasonable for PD programmes’ conception, implementation and impact analysis.

Another issue should be kept in mind when planning and researching the impact of PD programmes: the reciprocal relationship between impact and influencing factors (see, e.g., the central role of pupils’ benefit as fostering factor in the case of Eve). Moreover, the cases particularly highlight that some influencing factors might be present (e.g., the school principal’s support in the case of Eve), not present (e.g., mutual accountability in the case of Paul), or latently present (e.g., relative advantage in the case of Paul). Knowing these factors and being sensible of them seems to be prerequisite for PD programmes which aim at sustainable and scaled-up impact. Moreover, it appears to be of particular importance to support the latently present factors to become effective.

5.4 Limitations

The findings and implications of these case studies should be read in the light of possible limitations. Firstly, the studies’ data is gathered within a PD programme which is framed by certain characteristics (see Sect. 2.4). Thus, any attempt to transfer the cases’ findings should be carried out carefully, and preferably within similar PD contexts. Secondly, this case studies’ sampling might be biased, since data is provided by former participants who volunteered to be part of this research. The provided findings are rather exploratory in nature and thus are hardly suited to be generalized. Thirdly, the studies’ findings are based on teachers’ self-reported data, which may be susceptible to social desirability or lacking accuracy. Even if the teachers stated that each impact is a function of PFL, the exploratory findings cannot show without any doubt that impact is due to PFL only. Upcoming analyses of sustained and scaled-up impact might collect data of teacher’s changed knowledge, beliefs and practice which (a) is not (as much) biased by self-reports, and (b) is able to control other factors beyond the particular PD programme.

5.5 Causality, conditionality and contribution

Van den Berg (2005) discusses the issue of causality, which is central to establishing impact: “The model that evaluations have used in this regard is that of causal linkage, allowing for attribution of observed changes to the intervention” (p. 30). He suggests carefully analysing what causal linkage may mean: the common concept of cause in grounded in physics, where a causal relationship refers to a linkage that has been established both theoretically and empirically. However, within teacher education there are too many activities and circumstances for any general causality to be established: “We seem to lack general causality and we need to restrict ourselves to specific causality” (p. 31). If research on programmes’ sustainability uses a rigorous approach which considers many factors contributing to impact, then therefore the term contribution may be more suited (rather than causal linkage). In this regard, van den Berg (2005) proposes that research “should move from the concept of linear causality to the concepts of conditionalities (necessary but not sufficient conditions for changes to occur). Furthermore, it should be made clear that these necessary but not sufficient conditions contribute to, rather than cause, the change to take place” (p. 34).

This perspective seems to be an appropriate framing for these case studies’ findings.

5.6 Future research

Impact analyses that combine and compare various cases and bigger samples are currently being conducted (see Zehetmeier, in preparation) and will help in answering the following (and further) questions: Do different PD programmes show different sustainable and scaled-up impact? Are there any patterns? Are there any hierarchical structures within the different levels of impact? Does one level require another one to occur? Are there any factors that promote certain levels of impact in a particular way?

To answer some of these questions, an online questionnaire was designed to gather data concerning the teachers’ ex-post perspectives regarding the sustainability of the PFL programme’s impacts. This questionnaire contains 32 questions concerning the programme’s impacts on their knowledge, beliefs and practice, as well as concerning the corresponding fostering factors. One particular aim of this current research is to provide differentiated evidence from various perspectives. A synopsis of these varied findings should provide new insights on different levels: the questionnaire’s results should provide an overview of the sample; while the interviews’ findings should produce a mosaic of particular cases. In sum, these results (overview and mosaic together) can help to develop a broader understanding of PD programmes’ sustainability and scale-up; the findings may in particular allow deeper insights concerning the respective fostering factors (see Zehetmeier 2015). Upcoming impact analyses dealing with these and similar questions appear to be necessary and promising; from the perspective of both scholarship and practice.

From a theoretical perspective, this study supports that the IPD model is basically well suited to both describe and explain a PD programme’s impact. However, the case of Paul also points to colleagues’ beliefs being rather an influencing factor than a concrete level of impact. This issue highlights the deep interconnection between, and interdependency of, influencing factors and levels of impact; thus the conception of the IPD model might be revised based on broader as well as deeper research of PD programmes’ impact and their respective fostering factors; this appears to be promising from both scientific and practical perspectives.

From a methodological perspective, this research may be suitable as a trigger to further critically discuss the question of how to research the sustainability and the scale-up of PD programmes.