Keywords

Introduction

Teachers often participate in traditional professional development events which are of short duration and communicate abstract knowledge; however, these have not only been criticized frequently by participants, but they have also demonstrated little overall effect (see Altrichter, 2010; Lipowsky, 2010; OECD, 2009; Scheerens, 2009).

It appears that the possibility to examine one’s own profession or teaching in a self-directed manner and during exchanges with colleagues, is a key to professional development. This research-based learning approach has been reflected for some time in theoretical and conceptual considerations, as well as in practical applications for a variety of professionalization measures (e.g., Altrichter, 2002; Dirks & Hansmann, 2002; Feindt, 2009; Hollenbach & Tillmann, 2009; Horstkemper, 2003; Roters, Schneider, Koch-Priewe, Thiele, & Wildt, 2009). In particular in the last two decades, a variety of innovative models of teacher professional development have been designed, implemented and evaluated all over the world. In the field of mathematics teacher education, for example, such models are discussed not only in research papers (like in the Journal of Mathematics Teacher Education, launched 1998) but also in the International Handbook of Mathematics Teacher Education (Wood, Jaworski, Krainer, Sullivan, & Tirosh, 2008) and in several specific books (e.g. Even & Loewenberg Ball, 2009). Studies analyzing research on mathematics teachers’ professional growth (see e.g. Llinares & Krainer, 2006) show that teachers’ learning is not only promoted by meaningful activities, but also by teachers’ (oral and written) reflections on these activities, in many cases related to students’ or teachers’ own mathematical learning. Sustained and intensive professional development, often designed as teachers’ participation in a “community of practice” (more and more also virtual communities using new technological tools such as videopapers, blogs, etc.) and integrated into the daily life of the school (see e.g. Krainer & Wood, 2008; Sowder, 2007 or Wood et al., 2008) is more likely to be effective than short-term- and practice-distant professional development activities that address teachers mainly as “single fighters”.

Concurrently, the degree of attention given to these innovative approaches is not reflected in the amount of empirical research available. There are only few large-scale findings relative to the conditions, processes, and effects of research-based learning approaches for professional development (see e.g. Adler, Ball, Krainer, Lin, & Novotná, 2005).

There are some indications regarding the effects of approaches similar to action research. As part of a broadly based meta-analysis, Cordingley, Bell, Thomason, and Evans (2003) were able to show that a collaborative Continuing Professional Development approach (CPD) had positive effects on confidence, feelings of self-efficacy, motivation to work in a team, and willingness to change one’s own actions in a teaching setting (see also Cordingley, Bell, Thomason, Rundell, & Firth, 2005). As far as the students are concerned, the research overview revealed indications about the differences between collaborative and individual professional development. These differences refer to outcomes, such as learning motivation, performance, and attitude towards subjects, as well as active participation during the course. It is also evident that the research overview offered by Cordingley and colleagues does not refer exclusively to action research as a method of professional development, but to professional development and developmental measures that consider several aspects of action research. Other overviews and individual studies report similar findings, although these can vary significantly depending on the study and type of professional development. Nevertheless, recent research shows that collaborative and reflective teacher professional training impacts teacher cognition and partly students’ characteristics (e.g., Gärtner, 2007; Gough, Kiwan, Sutcliffe, Simpson, & Houghton, 2003; Gow, Kember, & McKay, 1996; for a summary, see e.g., Benke, Hospesová, & Tichá, 2008). On a global basis, however, research efforts still remain too incomplete to allow valid statements to be made.

This article introduces findings from monitoring research conducted for the university courses “Pedagogy and Subject Didactics for Teachers” (PFL) at the University of Klagenfurt (Austria), which generally follow the action-research approach. Initially, the theoretical background and concept behind the PFL courses will be described, followed by a report and discussion of the research design and selected results.

The starting point for learning-based research approaches, which extend beyond teacher education and professional development, is based on the scientific insight into and practical experience with professional development, which is not only an intellectual and academic process, but is also an active practical, emotional, and social process (Altrichter, 2002). A central consideration in this context is that the simple transfer of scientific concepts and innovations is very difficult and at times, impossible. There are many reasons for this; for example, theoretical knowledge is often inert and was not obtained in authentic, complex and team orientated learning situations.

Research-based learning aims at reducing the gap between knowledge and action by focusing on one’s own actions. In this context, one of the most prominent approaches taken to teacher education is so-called “Teacher Research” or action research (see e.g., Burns, 2007; Altrichter, Feldman, Posch & Somekh, 2008; Elliott, 1991; Hollingsworth, 1997; Wittwer, Salzgeber, Neuhauser, & Altrichter, 2004; Posch, Hart, Kyburz-Graber, & Robottom, 2006), which finds its theoretical basis in the action theories of Schön (1987) and Stenhouse (1975), among others. In this vein, professional development should be conceptualized through a repeated cycle of action and reflection. Here, teachers systematically investigate their own teaching practices, interpret the insights they gain, and create new action ideas (reflection), which are then implemented (action) and evaluated. Relative to this, Schön (1987) notes the “reflective adoption of practical solutions to problems,” which is based on having an experimental attitude regarding real life practices. Reflection is viewed as one of the main competencies of those in the teaching profession, and not just in action research (see e.g., Bromme, 1994). This ability is of particular consequence relative to the implementation and objectives of teacher professional development.

Another core concept of action research that is in line with Elliott (1991) is that individual research and the further development of one’s own teaching practices or those of the school, are embedded into a professional community. The significance of professional communities in action research has been pointed out by Altrichter (2002), who also established a systematic relationship between the concept of “situated learning” (see Lave & Wenger, 1991), and teacher education.

To overcome the risk of excessive self-referencingFootnote 1 of teacher groups (because collegial cooperation can also prevent learning; see e.g. Corcoran, Fuhrman, & Belcher, 2001), the support and intervention offered by outside colleagues or experts can be integrated into action research projects. These act as a corrective factor or “critical friends”.

The Philosophy of the PFL Programme

The University of Klagenfurt has been offering the 2-year university programme “Pedagogy and Subject Didactics for Teachers” (PFL) since the early 1980s. The PFL programme consists of several courses, each dedicated to one or more subjects (Krainer, 1999). The courses are based primarily on the concept of action research (Posch, Rauch, & Mayr, 2009). One maxim for these courses is the close linking of pedagogical knowledge and pedagogical content knowledge (Shulman, 1987), and identifying the teacher’s own actions as the starting point for teacher professional development. In this context, academic issues are, at least initially, of secondary importance. PFL courses initiate personal teaching-related development projects, which the participants typically implement at their own school. The projects are supported by experts from research and practice, and the intensive exchange of information among teachers in terms of collegial advisory services forms a part of the PFL course. During the course, participants prepare on average two reflective papers that document the development process of teaching projects, the objective of which is to evaluate one’s own teaching actions. The courses also integrate the school environment by focusing not only on researching one’s own teaching actions, but also observing the projects of colleagues and school development initiatives (see e.g. Krainer, 2001). Beyond the project phases, as part of three, 1-week module workshops held during the course, participants also receive information on subjects, such as evaluation methods or new trends in pedagogical knowledge and pedagogical content knowledge, which they can link with their teaching projects. Additional work group meetings offer more opportunities for exchange with colleagues.

The following box offers some competences the PFL courses aim at.

At present, four parallel courses for secondary teachers are offered which include mathematics, sciences, English, a cross-subject course for art, history and German (ArtHist), as well as a course designed specifically for the primary level including the issue of integration.

The following section outlines the concept of monitoring the research for PFL courses. It will present and discuss the results of this research for the four teaching courses held from 2006 to 2008 (respectively 2007–2009 for the PFL mathematics course).

Research Design

Theoretical Background

The monitoring study follows a longitudinal design which focuses on the present courses with regard to input, ongoing processes, and outcome. The theoretical background is a concept which is based on the association between the experiences of teachers, the opportunities to learn in the courses and the uses of the learning environment (see Helmke & Weinert, 1997), which can be presented in a simplified form as follows:

Participants enter the course with specific input conditions (expectations, interests, competencies, etc.), and encounter specific learning opportunities (in the form of individuals, information, etc.). They employ these depending on their input conditions and the quality of the learning environments (such as by applying specific learning strategies). The learning benefits drawn from each course at the individual level (expanding knowledge, changes in thinking, etc.) are viewed as being dependent on the aforementioned input conditions, learning opportunities, and learning strategies. The competencies developed as part of the course (in the widest sense, see Allemann-Ghionda & Terhart, 2006) should have an impact on teaching actions, so that the course contributes to a further development of teachers’ practice.

Research Plan and Questions

The research design (see Fig. 1) identifies four dates during which data collection took place. (1) Prior to each PFL course, participants were surveyed using an online questionnaire that covered previous professional experience, interests, self-assessed occupational competencies and existing knowledge, and the reasons for participating in the course. (2) At the beginning of each course a video test was conducted in order to measure teachers’ competences in analyzing lecture units. (3) At the end of the first year, participants were asked to complete a questionnaire regarding their assessment of the learning environments and their utilization (learning strategies). (4) All measurements were repeated at the end of the course and supplemented with several additional questions regarding satisfaction with the PFL and scales measuring the learning motivation of the participants. This article reports on the findings of the initial survey, the two video tests, and the final survey.

Fig. 1
figure 7_1_162454_1_En

Research design

With regard to this article, the following questions are considered leading components of the research:

  • What are the motivating factors behind a decision to participate in the PFL-programme?

  • What learning strategies do participants use during the course?

  • Do the participants’ interests in the professional activities, and their own self-assessment of competences and knowledge, change during the course?

  • Is there a change in the ability to analyze lecture units with regard to the learning opportunities of students (teaching video on the topic of geometry)?

  • Are any differences noted between mathematics and other courses?

Instruments

The following section outlines briefly the instruments used for collecting the survey data, and it introduces the instruments used in the video tasks. Detailed statistical information about the scales and items are described elsewhere (Müller, Andreitz, & Mayr, 2010).

Questionnaires

  • Motivation to participate in a course (six scales)

  • The first two scales that focus on motives for participating in the course (see also Fig. 1) were constructed on the basis of the self-determination theory by Ryan and Deci (2002, 2008). Notably, a distinction was made between the aspects “self-determined” motives (item example: I am taking the course because I enjoy learning something new) and “controlled” motives (because of the high prestige associated with university courses) (see also Müller, Palekčić, Beck, & Wanninger, 2006). Further, scales were added that capture the interest in “Development of the school system” and “Development of classroom teaching” as a motivation to take the course. In addition, scales regarding the participation motivation of “Maintain and promote professional motivation” and “Making social contacts” were prepared.

  • Job-related interests and competencies (eight scales each)

  • Slightly adapted versions of the six dimensions of teacher interest scales (LIST; Mayr, 1998) were used prior to and at the end of the course to assess the interests and competencies for a teaching career (e.g., “Teaching” or “Address specific needs”). In addition, two areas that refer specifically to the content of PFL courses were included: “Reflect on own actions” and “School development”. Each item referred to the teachers’ interest in the relevant activity and the competence for conducting that activity.

  • PFL-specific knowledge areas (four scales)

  • The teachers were asked to assess the state of their knowledge before and after professional development with regard to aspects that are considered to be essential in the PFL courses. These are: (1) methods of promoting teaching (measures of inside differentiation and individualization in the classroom); (2) pedagogical content knowledge and pedagogical knowledge (newest concepts); (3) knowledge about performance standards (educational standards); and (4) management and evaluation (evaluation methods).

  • Learning strategies (five scales)

  • The three items with the highest factor loading for the scales “Create associations,” “Critical review,” and “Repeat,” and the scale “Effort,” were used from the surveys on learning strategies (LIST; Wild, 2000). In addition, the scale “Reflection” was created (item example: I think about how I can improve my actions), since there were some indications that this learning strategy is initiated particularly through the conception of PFL courses.

  • Satisfaction with the course

  • One item was formulated with regard to overall course satisfaction, two other items referred to the satisfaction with the 1-week modules, and the working groups.

Video Task

At the beginning and the end of the course, participants were asked to analyze videographed sections of problem- and action-oriented mathematics lessons on the topic of the “Pythagorean theorem” (from the DVD “Introductory Sequences” by Reusser, Pauli, & Krammer, 2004). They were asked to identify learning opportunities that activate the students at a cognitive level and then substantiate their answers (see Table 1). Exactly the same video scenes were used at the beginning and at the end of the course.

Table 1 Tasks related to the video sequences

The research design was adopted from the work group “Bi-national video-supported professional development for teachers in Germany and Switzerland”Footnote 2 (Krammer et al., 2006, 2008, 2009). In this study, however, the instructional video was used only as a diagnostic instrument to measure the participants’ analytical competence. The objective of the video task was to evaluate whether participants were using the pedagogical knowledge and pedagogical content knowledge addressed in the course as they analyzed a video sequence, in addition to self-assessing their knowledge and competence.

The open answers provided by participants were subjected to a content analysis (Mayring, 2000) in which the category system by Krammer and colleagues (2009) was adopted. This category system contains seven main categories: (A) Characteristics of instruction design with regard to content; (B) Characteristics of instruction relative to interaction and social forms; (C) Characteristics of instruction with the objective of achieving an active examination of the content by the learner; (D) Comprehension orientation; (E) Behavior characteristics/Characteristics of the teacher; (F) Observation of student behavior; and (G) Learning atmosphere. In addition, the categories Direct Instruction by Teacher (as the main category I) and Reciprocal Teaching (as subcategory A1) are also formed, since information was frequently provided for these aspects, particularly in t2, but no provision was made for these categories in the German-Swiss research group.

Sample of the Study

Participants in the four PFL courses were surveyed between 2006 and 2009 (questionnaire: Nt1 = 131; Nt1 and Nt2 = 84; video task: Nt1 and Nt2 = 54). The average age of teachers is 46 years (SD = 8.6); they teach at university entrance secondary institutions (Gymnasien) (30%), vocational middle and secondary schools (28%), general secondary schools (Hauptschulen) (27%) and other types of schools (15%). Thirty-three teachers took part in the PFL mathematics course. This group will be analyzed consistently relative to teachers who chose another PFL course (ArtHist, English, Sciences) covering the same school types.

Results

Questionnaire Analysis

Table 2 provides an overview of the motivation for participation and illustrates that the self-determined motivation towards taking the course is significantly higher than controlled motivation. In this vein, the course is selected due to intrinsic motivation, an interest in the topic, curiosity, and the desire to develop one’s own competence in the area. Extrinsic motivation, such as obtaining a certificate, the prestige of the course, or “feelings of guilt” if one does not participate in professional development played a subordinate role.

Table 2 Motives for participating in the course: non mathematics, mathematics

However, with regard to the controlled motivation (extrinsic motivation), 20% of participants demonstrated decidedly stronger values (means of greater than 3.5). These participants had already attributed more importance to the significance of tests, certificates, and associated career opportunities at the beginning of the course. In this context, Posch et al. (2009, p. 212) point out that the courses can also represent the “starting point for an exit from the teaching profession,” and that they can stimulate new career ideas and open new career options (see also Benke et al., 2008, p. 289).

The primary motivation (relative to content) for selection of the course is the development of one’s own teaching. All other kinds of content-related motivation are ranked below.

No significant differences were found between PFL mathematics participants and those of other PFL courses.

The high degree of self-determined motivation and the content’s focus on teaching development are good motivating criteria for sustained learning processes in the course (see also Smith & Gillespie, 2007). These concur as well with the alignment of the PFL courses relative to content. It has been shown elsewhere that the participants in such courses remain highly self-determined over the 2-year period (Müller et al., 2010).

Overall, the participants are very satisfied with the course (see Table 3). This applies particularly to regional groups in which the participants work intensively on their individual instructional project in cooperation with their colleagues and support staff.

Table 3 Assessment of satisfaction with course

An observation of the learning strategies used by the participants shows that different approaches are used to various extents (see Table 4). In particular, the in-depth learning strategies of reflection, creating associations, and critical review are utilized. Relative to the research-based learning conception of professional development, these represent results that correspond with expectations. Notably, repetition strategies are used less frequently. Similarly, the Effort scale also features a significantly lower mean.

Table 4 Learning strategies

In the PFL mathematics course, the two latter learning strategies of repetition and effort are used significantly more often than in the other courses. However, it remains to be seen whether these differences can be traced back to the learning environment in the course or to the differences in the culture relative to the various subjects.

The interest shown for different activities of teachers, and the competencies and knowledge assessed by the teachers themselves were investigated in a longitudinal study (see Table 5). That fact that interests changed little during the 2 years was to be expected, even though a significant increase is noted regarding participant interest in the “Development of school” scale, which was created specifically for the PFL courses. This applies as well to the non-mathematical PFL courses regarding the aspect “Reflect on one’s own actions.” These two aspects, school development and development of own actions, are essential objectives of the PFL courses.

Table 5 Change in interest, competence and knowledge

With regard to the competencies experienced, three scales indicate substantial changes over the 2 year period of the study. For example, participants assess their teaching competence somewhat higher after 2 years, along with the competence to reflect on their own actions and conduct school development. With regard to PPL mathematics, these changes are visible only for the “Development of the school system” scale. There is no change for the scale ‘adress specific needs’ of students.

All self-assessments of the knowledge areas increased significantly. In particular, the scales “Pedagogical content knowledge and pedagogical knowledge” and “Management and Evaluation” achieved higher self-assessments at the end of the course than at the beginning. These knowledge areas also featured the lowest values at the beginning of the course, which indicates significant development potential. Similarly, knowledge of methods for promoting learning and performance standards also increased.

With regard to the scale “Performance standards,” a noticeable and highly significant difference between PFL mathematics (mean 4.1) and other PFL courses (mean 3.4) is observed for t2. This development stems primarily from the fact that the issue of performance standard established a focus of the PFL mathematics course.

Video Task

The verbal responses for the video task were categorized initially by three people independently, based on the category system of the research group “Bi-national video-supported professional development for teachers in Germany and Switzerland” (see section “Instruments”). Subsequently, the category assignments were validated within the research group as part of a discussion. This article reports on the results of the partial task “Identification of learning opportunities/cognitive activation in the classroom”.

Figures 2 and 3 shows the average number of entries for some categories (‘G: Learning atmosphere’ and ‘I: Direct instruction’) and subcategories (like ‘A4: Situating’ or ‘A4: Reciprocal teaching’) by way of example. Because of the large number of categories, the average number of entries for observation units in the individual categories is low.

Fig. 2
figure 7_2_162454_1_En

Average number of entries for the categories of the video task (non mathematics)

Fig. 3
figure 7_3_162454_1_En

Average number of entries for the categories of the video task (mathematics)

The sum of all entries for cognitive activation does not reflect any significant increase for both groups in the longitudinal study (Mt1 = 5.08; Mt2 = 5.93; t-Test: t = 1.2, p = 0.11). Hence, the changes refer to individual aspects:

  • PFL Non-Mathematics

  • Significant increases (t-test, double sided; N = 34) are observed for the aspects of “Individual support” (Mt1 = 0.22, SD = 0.77; Mt2 = 0.54, SD = 0.63; p = 0.05), “Motivate students to think” (Mt1 = 0.38, SD = 0.99; Mt2 = 0.60, SD = 0.95; p = 0.04), and “Direct teacher instructions” (Mt1 = 0.08, SD = 0.40; Mt2 = 0.32, SD = 0.51; p = 0.02). No significant changes were observed for most of the other categories; e.g., “Reciprocal teaching” for “Situating,” “Inclusion of student comments” for “Cooperative learning,” or “Learning atmosphere.”

  • PFL-Mathematics

  • Only the “Direct Instruction” category has significantly more entries at the end of the course (Mt1 = 0.00; Mt2 = 0.45, SD = 0.54; N = 20). The category of individual support is rarely mentioned for t2 and decreases significantly compared to t1 (Mt1 = 0.45, SD = 0.60; Mt2 = 0.09, SD = 0.30). None of the other categories undergo a significant change. This lack of significant differences can be traced back to the small sample.

In addition to the number of individual entries, the “quality” of analyses was also analyzed. In this case, quality has been defined as the reasoning for each entry including an indication of associations to other passages of the teaching sequence. Figure 4 denotes that the number of reasons provided increased significantly (t-test, double sided, p = 0.02, d = 0.33). The correlation of the quality of analysis between t1 and t2 is r = 0.57 (p < 0.01). Overall, however, teachers gave few explanatory statements at both points in time. The number of reasons provided is lower for t1 with PFL mathematics than for other courses.

Fig. 4
figure 7_4_162454_1_En

Change in analytical competence: quality of analyses

A differentiated look at the quality variance in the analyses shows that the number of explantions provided and the associations only increase to a significant level, from t1 to t2, if the participants analyzed and evaluated some in-house or external teaching videos as part of the 2-year course (see Fig. 5). Nine of all 54 participants in the video study were involved in this process as part of or external to the course. The correlation between the quality of analysis and experience with video analyses is r = 0.41 (p < 0.01).

Fig. 5
figure 7_5_162454_1_En

Change in analytical competence: quality of analyses in partial groups N = 54 (math and non math PFL courses)

No differences were found between mathematics and other courses when experience with video analyses was considered.

  • Predicting changes in analytical competence

  • In addition to the experiences acquired by participants with regard to the analysis of teaching sequences during the course, two other aspects can also be used to explain the increase in reasons provided. On the one hand, analytical competence develops more strongly if the participants receive explicit instructions from PFL-trainers regarding how these teaching issues are analyzed (even if no teaching videos are used). The correlation between this assessment and the quality of video analysis is r = 0.39 (p < 0.01). On the other hand, a strong interest in course content as a motive for selecting the course is a good prerequisite to expand one’s teaching-related analytical competence. No other indications for predicting differences for analytical competence were observed.

Summary and Discussion of Findings

Professional development opportunities such as the PFL course, which focuses on a research-based learning approach and on a close link between theory and individual teaching approaches as well as supports teachers’ learning processes over a long time period, are assessed postively by teachers. This applies particularly to working in small groups (regional group meetings), which allows for in-depth discussion of the development of one’s own teaching as part of a collegial exchange along with the intensive support provided by experts. Of course, the high degree of self-directed motivation, the strong interest in linking pedagogy and subject didactics, and an interest in personal development of competences at the beginning of the course also play roles in the high level of satisfaction and acceptance of these courses.

The participants primarily apply elaborative learning strategies, such as “critical review” and “reflection.” However, it is not only the learning strategies employed that correspond with the leading theme of “reflecting practitioner” in the PFL courses, since the longitudinal study also shows that interest and self-assessed competence regarding reflection on one’s own practice increase as well. Yet it is also the periodic self-assessment of competence for teaching and a desire to participate in school development processes that increase at the end of the course. The same applies to course-specific knowledge areas such as pedagogical content knowledge and pedagogical knowledge, management and evaluation, methods for supporting learning processes, and knowledge of learning standards.

Given a careful interpretation, the change in interests and competencies can be viewed as evidence of the effectiveness of the courses. However, it is not known whether these subjective assessments also correspond with changes in behavior in the classroom setting. To look more closely at this item, the research design must be expanded to include the corresponding activities (e.g. teaching observations in the PFL participants’ classrooms before and after the course).

In this study, the video task serves as a diagnostic instrument to measure the teaching-related analytical competencies of the participants and to validate their self assessed competencies. The categorization and number of entries indicate that at the end of the course, teachers have become more sensitive regarding individual aspects of learning situations. Overall, however, the difference in aggregated entries in the video task is not significant between the two test dates.

An analysis of the quality of answers for the video task, which considers the reasons provided and the associations made in the responses, results in increases from t1 to t2. In this vein, mathematics teachers are not superior to other PFL course participants. On the contrary, it is evident that the number of contributions (explanatory statements and associations) for both t1 and t2 is less than the quality of contributions by PFL participants from other subjects. Finally, additional clarification to this result can only be obtained through studies that involve larger samples. In the course of such a study, factors such as the motivation of participants during their work on the video task can also be taken into account, since they may influence the result.

The teachers who increased in their analytical competence were mainly those who worked with teaching videos as part of the course, as well as participants who indicated that they were given explicit information on how experts analyze teaching-related issues. The finding that an examination of teaching videos will increase competence for video-based teaching analyses seems almost trivial, yet it shows again that active application and the idea of practice are indeed significant factors in professional development (see also Neuweg, 2010).

Outlook

Until now, externaly conducted research on university courses has been limited to assessments by course participants with regard to their interests, competencies, motivation, or learning strategies, as well as the illustrated video task that records the competence found in analyzing one’s teaching. The effects on teachers’ actions; i.e. on teaching practice as well as on students’ learning attitude and performance have been only refined to teachers’ own investigations and some analyses by PFL staff. Similarly, the effects of courses on individual schools and other colleagues have not been investigated on a large scale.

Therefore, it is still an open question to what extent the results of this study are really relevant to the teaching practice. Irrespective of this uncertainty, the findings suggest that practice sequences should increasingly be built into research-based learning approaches for teacher professional development.