Keywords

Introduction

According to the Dutch Inspectorate of Education, inspection should connect to educational practice and meet the expectations of parents, students and society (Dutch Inspectorate of Education 2016a). In light of this the inspectorate is introducing a new mission: improving education in all Dutch schools (Dutch Inspectorate of Education 2016a). The new inspection framework is designed to promote further improvements to school quality overall, not only the weak schools, in order to improve education more broadly.Footnote 1 Given that the Dutch educational system and society are changing, the inspectorate is changing its policies and practices in response. Current inspection aims to maintain quality in all schools, and has a particular focus on the improvement of (very) weak schools. Although number of weak and very weak schools has declined in recent years, most Dutch schools meet basic quality standards (98% in primary education and 94% in secondary education; (Dutch Inspectorate of Education 2016b),’ there is still room for improvement. Most schools with basic quality, may also possess any number of shortcomings that quality initiatives so far have failed to improve (Dutch Inspectorate of Education 2013). Therefore the inspectorate is introducing a new mission to: improve education in all Dutch schools (Dutch Inspectorate of Education 2016a). The new inspection framework is designed to promote further improvements to school quality, not only in weak schools, but in order to improve education more broadly.

In order to realize this new mission, the Dutch inspectorate outlines three categories of inspection activities: (1) monitoring basic quality of schools, (2) stimulating school improvement, and (3) thematic and problem-oriented inspection (Ehren and Janssens 2014). The first two activities are part of a form of differentiated supervision: In schools in which basic quality has not been established, the focus is on correction, whilst in schools that meet the criteria for basic quality, the inspectorate acts more as a critical friend: the focus is not on evaluation but on further improvement. In order to effect this, a new framework for inspection has been developed, one which allows inspectors to differentiate between schools and provide feedback on a more broad range of quality criteria. The third supervision activity, thematic and problem-oriented inspection, implies contributing to the identification and answering of research questions at the system level by thematic and problem-oriented research. All three inspection activities are designed to stimulate all schools to, (further), improve (Dutch Inspectorate of Education 2013). Inspection activities are directed towards making a greater contribution to the development of a culture of improvement within schools and school boards.

This renewal of the inspection methodology includes four elements/components:

  1. 1.

    A renewed evaluation framework

The new evaluation framework has several new properties compared to the current one. Firstly, it has a similar structure for all educational sectors. In addition, it has to provide more space for input by schools, school boards, and the inspectors. This permits school boards and schools to provide insight into the education context of the schools, but also allows the inspector to make decisions that are suited to the specific situation of the school. The framework should also provide better opportunities for a professional dialogue.Footnote 2

  1. 2.

    A stronger focus on quality care by schools and school boards

The second aspect of the new inspection system relates to the focus on the quality of care in the school. The inspection will, based on the quality of information made available by the school and schoolboards, in order to make choices on the design of the inspection at the school (e.g. more or fewer classroom observations). Besides that, the caliber of information from the school is also an important source for the assessment of school quality. In particular, self-assessments will be relied upon to a far greater extent. Such self-assessment includes a questionnaire which schools use to assess themselves according to the standards that will also be used for inspections. The school principal also fills completes the questionnaire prior to inspection.

The school boards are responsible for the quality that their individual schools deliver. Each has its own process to ensure the quality of education and development.Footnote 3 In addition, they possess a great deal of knowledge and information about their schools. The Inspectorate looks to connect more closely with this information in order to make better use of it. And as time goes by and schoolboards and schools possess better processes of control and more sophisticated information systems, the inspection will undoubtedly benefit.

  1. 3.

    Differentiated school quality judgements

The third element of the renewed supervision is that it is designed to work with differentiated judgments (ratings). Until now it was customary to express the overall quality of a school in a so-called ‘supervision arrangement’. The Inspectorate currently distinguishes between three criteria: “basic quality” (basic criteria satisfied.), “weak school”, and “very weak school”. In the renewed supervision framework, for schools with basic quality, a distinction will be made between “satisfactory” and “good”. This will be done not only in terms of the overall conclusion on the quality of the school, but also for the different quality areas (and the underlying standards) of the new developed evaluation framework.

  1. 4.

    Inspection should promote quality improvement to a greater extent

Finally, an essential part of the new system is the intended effect of inspection to drive quality improvement. This will be achieved by involving other groups in the school in the inspection activities. Besides the usual interlocutors, students and parents are also invited to input. This should provide more variation in the feedback to the schools by the inspector. This is an important aspect of the new inspection methodology, and meant to strengthen the quality promotion effect of inspections.

It was already normal practice to have a meeting with the management (head) and school board at the end of the school inspection; a meeting in which the inspector explained the findings and conclusions. As part of the new approach inspectors have a feedback meeting with a representative of the school board, the management, the special needs coordinator, and teachers. The main purpose of this feedback session is to stimulate staff to improve the quality of education at their school. The inspector will discusses the opportunities for improvement of the school. This feedback will be provided orally to all visited schools, no matter what their quality may be. In addition to this, written feedback will be provided in a report including a quality profile which informs on student achievement, teaching quality, finance, and administrative agency of the school board that matter most for school quality.

It is assumed that in providing (better) feedback, and by publishing school profiles, schools will be encouraged to work on school improvement. The underlying assumption is premised on the belief that this feedback by inspectors will help schools to improve.

A meta-analysis carried out by Kluger and DeNisi (1996) showed that feedback in general can indeed improve performance, however it also showed large variability in feedback effects and in over one third of all cases it even reduced performance. For this reason, in this chapter we will take a closer look to the assumption that inspection feedback can help a school to improve. Firstly, we will look at what we know about the effectiveness of inspection feedback by using a review of the literature on this topic. In the second section of this chapter we will describe two pilot studies by the Dutch inspectorate which provide insight into the quality of feedback by Dutch inspectors and the perception of the recipients. Whilst in the final discussion we will bring this information together and evaluate the role of feedback by Dutch inspectors in order to improve education.

Effectiveness of Feedback According to Feedback Literature

The Dutch inspectorate is developing the feedback in the new inspection with the assumption that, among other interventions, it will help schools to improve. This section refers in part to research summarised in a previous literature review (Nelson and Ehren 2014).

The Effects of Inspection Feedback

All information provided by inspectors during or after an inspection can be considered to be feedback. Inspections can provide a rich source of feedback to schools, including for example: information about pupil attainment and progress; feedback to teachers given after lesson observations, feedback on a school’s own internal documents, policies and procedures. This feedback may take the form of oral feedback to school staff given during a school inspection visit and through the publication of a report sent to the school and/or made available to the public. Overall, feedback from inspections can work:

  1. (i)

    By impacting on school improvement initiatives. This could involve actions or decisions taken by all or any members of staff but is likely to be strongly influenced by the commitment of school leaders and also the extent to which the inspectorate follow up or support such efforts.

  2. (ii)

    Through changing the behaviour of teachers and school leaders. The effect of teacher quality is often cited as the most important influence on overall pupil achievement (Barber and Mourshed 2007), the former may be considered to be particularly important.

  3. (iii)

    Indirectly through affecting school self-evaluation. Inspection feedback can comment on, validate or suggests ways to improve the school’s own self-evaluation processes. This can lead to the school enhancing its own capacity to make improvements.

(Adapted from Nelson and Ehren 2014).

We can distinguish this from the feed-forward effect of preparation by schools for an inspection visit. A number of studies have commented on positive and negative effects of this (Matthews and Smith 1995; Chapman 2002; Gaertner et al. 2014).

Multiple studies show a positive effect of inspection feedback, for example, Chapman’s (2002), case study of five schools showed that 20% of teachers reported that inspectors’ feedback led to changes in teaching practice. Brimblecombe et al. (1996) found 38% of teachers (in England) intended to make changes shortly after an inspection visit, particularly in relation to the way they teach and organise classes. The tendency to change increased the higher up the teacher was within the organisation. Studies have also shown that teachers and school leaders welcome feedback from inspectors, that they find it useful and feel disappointed when it is not given (Dean 1995; Brunel University Centre for the Evaluation of Public Policy and Practice and Helix Consulting Group 1999; Kelchtermans 2007). In a report commissioned by Ofsted (2015), post-inspection surveys were sent out to school leaders, staff and governors for their views on a range of issues, including the quality and conduct of the inspection, its likely impact on their school and what changes they intend to make. 22, 800 responses from between 2009 and 2014 were analysed for this section of the report. In 2013–14, Ofsted also surveyed 829 school leaders four months after an inspection had taken place (reported in the 2015 article). The key findings were:

  • Almost all respondents said that they would use the inspection recommendations to improve their school (98%).

  • Nearly nine out of 10 school leaders (88%) reported that they had made changes to their school as a result of inspection.

  • Most leaders (81%) said that inspection helped them to improve by providing an accurate analysis of their strengths and weaknesses.

  • A large majority of leaders (79%) had found inspection helpful in confirming that they were taking the right actions.

  • Around seven out of 10 school leaders (73%) agreed that the inspection report would help their school to improve.

  • Over half of school leaders (56%) identified that ongoing professional dialogue with inspectors was the most useful aspect of the inspection process (pp. 1–2).

Examples of changes implemented provided in the Ofsted report include “improvements to feedback and marking strategies, specific mathematics interventions and enhanced programmes of professional development” (p. 15 in outstanding schools). Where schools needed to improve, most changes were identified in areas of management and teaching and learning, for example, in monitoring and evaluation, use of data and tracking, improving professional development and in behavior management.

However, Klerk (2012) cites a range of research that shows very minor effects for the impact of inspection on student achievement, suggesting that overall the results are inconclusive (see Luginbuhl et al. 2009; Shaw et al. 2003; Rosenthal 2004; Matthews and Sammons 2004).

Factors Mediating the Impact of Inspection Feedback

The quality of feedback given to schools about their performance, as well as their capacity to improve their own work using this feedback, appears to be key to its success (Nusche et al. 2011). Good quality feedback should:

  • Contain specific and clear recommendations to refocus leadership and to have an impact after the inspection (Matthews and Sammons 2004).

  • Point out the specific weaknesses of school (Ouston et al. 1997).

  • Be clear to assess these weak points as unsatisfactory (Ehren and Visscher 2008)

  • Involve direct inspector feedback to the teacher (McCrone et al. 2009)

  • Contain specific improvement suggestions and agreements on school improvement (Ehren and Visscher 2008).

Responses to inspection feedback can also be determined by how negative the report is. Looking across inspectorates in 17 countries, Penzer and CfBT Education Trust (2011) found that schools are generally disinclined to accept unfavorable inspection conclusions. However, the credence, communication skills and demeanor of the inspectors have an effect on this aspect.

Another important aspect that determines the effect of inspectors’ feedback on teachers and school leaders, is the personal communication skills of the inspector and their perceived expertise (Dean 1995; Brunel University Centre for the Evaluation of Public Policy and Practice and Helix Consulting Group 1999; Kelchtermans 2007; Bates 2013). For example, in interviews with 11 headteachers and 37 teachers at 14 schools in England Dean (1995), showed that where pre-visit contact was made and where inspectors were reassuring teachers, that this helped allay some anxiety. Feedback tended not to be received well when inspectors lacked credibility, for example when secondary school teachers were tasked with inspecting primary schools.

Kelchtermans (2007) found that inspectors with an authoritarian attitude triggered reactions of resistance and rejection. Yet that more positive, constructively critical approaches made it more likely that teachers would use advice for improvement. An independent evaluation of extensive evidence on the Ofsted inspections in England (Brunel University Centre for the Evaluation of Public Policy and Practice and Helix Consulting Group 1999) also found that the relationship established by the lead inspector and staff was very important to the process and reaction to it; it helped establish trust and also credibility of the team. While the perception of schools to the feedback provided by inspectors appears very positive, the extent to which these schools improved on objective measures of performance however, remains largely undetermined by these studies.

Jones and Tymms (2014) suggest that there is limited empirical evidence on the link between feedback and improved performance from inspections and that there is also evidence of unintended effects, such as teaching to the test and short termism. Kelchtermans (2007) notes that the audit procedure communicates a particular view of a ‘good’ education, so that schools react either with compliance or through an awareness that they needed to justify a different approach. Negative inspection reports can also be used to justify controversial decisions about reassignment of staff and to strengthen the personal authority of the school leader (ibid).

A relationship between inspection feedback and a school’s own self-evaluation (SSE) has been found to be important in a number of studies. (See also chapter four, this volume). For instance, one study shows that the sensitivity of stakeholders to inspection reports and the acceptance of feedback changes following inspection. This can then lead to schools improving their capacity for change via improved SSE (Altrichter and Kemethofer 2015; Ehren et al. 2015a, b). Whitby’s (2010) analysis of high performing education systems showed that external inspection is most likely to be effective when the content and focus of the review were agreed with the school. The report recommends that the school’s own self-evaluation and external evaluation should be complementary, as happens in Scotland. The amount of guidance and support that schools have for SSE and external inspection also appears to affect the impact of inspection systems on school improvement. Similarly, commenting on the inspection system on Kingergartens in Hong Kong, Wong and Li (2010), concluded that effective quality assurance mechanisms should maintain a balance between external and internal evaluations and should work toward school empowerment and improvement. However, there can be a tension between SSE and inspection and a risk that the latter may simply be written to comply with expectations of the inspectorates (Whitby 2010). Supporting this point, Ferguson et al. (2000) found that most head teachers they interviewed (in England) were very good at predicting the issues that would be high on the inspectors’ agendas.

The Response to Inspection Feedback

Some authors have shown that schools need time, skills and support in order to respond effectively to the type of performance feedback that external inspectorates often refer to Verhaeghe et al. (2010). This can include training and support to use data to identify and implement improvement actions (Vanhoof and Petegem 2007). However, Vanhoof et al. (2011) found no difference in use of data by principals who had participated in training on interpreting data, they found that the collaborative culture and good professional relationships among staff determined greater use of school performance feedback. In England, Learmonth (2000) and Ouston et al. (1997) found that, in general, younger, less experienced school principals were more likely to implement changes as a consequence of inspection visits than older, experienced school principals.

The response to feedback has been shown to be mediated by school type. Looking at the German context of school supervision, Wurster and Gaertner (2011) identified four different schools types in terms of responses to inspection feedback. 25% of schools were ‘active’, getting good inspection results, and were high in activity level and involved in extensive communication and reflection following inspection. 29% were ‘unsatisfied schools’, who received poor inspection results, engaged in low activity and had a negative perception to the outcome; 26% were ‘(self-) satisfied schools’ who engaged in low activity having received a very good inspection result and 21% were ‘reactive schools’—low activity apart from high preparation for the eventually substandard result. However, in a follow up survey one year later, looking at a wide variety of outcome measures, very little change was found in overall school quality. Hopkins et al. (1997) suggest that “different improvement strategies and types of intervention are needed for schools at different stages of growth.” (p. 401). The extent to which the school has its own capacity to change or requires external steering, support and guidance is a key aspect to their framework and this has a strong bearing on whether a school can respond well to inspection feedback.

Some schools may respond poorly to feedback because they deem the inspection framework to be inappropriate to their educational ethos. In one landmark case in England, Ofsted inspected a distinctive private boarding school, Summerhill, (Keeble-Allen 2004). Their report raised a number of issues, including low standards of literacy and numeracy. A later follow up visit by inspectors concluded that there had been very little change to these aspects. The school leaders and parents argued that due to their uniquely personalised and pupil-driven approach the inspectors were not able to capture its strengths. The school made the case that Ofsted was trying to enforce a particular narrow, neo-conservative approach to education and idea of effective teaching. Later on, they won a legal case that exempted them from future action by the inspectorate.

The national context of the school system also mediates the effect of feedback. One aspect is the extent to which inspections operate on high or low stakes systems. High stakes systems make feedback more threatening. Altrichter and Kemethofer (2015) rated countries on a scale from 5 to 0 of high to low accountability as follows: The Netherlands 5, England 5, the Czech Republic 3, Sweden 2, Ireland 2, Austria 0, and Switzerland 0–1. Approximately 2300 principals in 7 European countries were surveyed. The authors found that nearly two thirds of respondents felt pressure to do well on measures of inspection, and this perception of pressure was strongly correlated to the accountability ratings. Those principals who reported feeling pressure agreed that the inspection process set expectations for performance, however they also reported more unintended consequences such as discouraging new teaching methods; narrowing the curriculum and instructional strategies.

One further factor to consider is the ‘Maturity’ of the national school system and the extent to which it is capable of ‘self-improvement’ (Hargreaves 2010). Barber and Mourshed (2007) suggests that in a ‘matured’ system, with a climate of ‘informed professionalism’, top down improvements are no longer as essential. Hargreaves describes a self-improving system as working in clusters of schools to find local solutions through co-construction, driven by systems leaders (Hargreaves 2010). Ehren et al. (2015a, b) suggest that in mature systems, inspection feedback needs to take into account the school’s own SSE and ability to interpret and use data and build its capacity for change. Furthermore, if feedback may need to be directed more at systemic level in a ‘self-improving system’, to take into account how schools work in partnership to drive improvements (Hargreaves 2012). At systemic level the actors need time to compare feedback directed at school level to local and institutional level and put in context for it to be effective (Coe 2002).

The extent to which the Dutch system has reached the type of maturity described above, has been questioned elsewhere (e.g. Ehren and Honingh 2011). In particular, the authors refer to annual reports of the Inspectorate of Education and Hooge and Honingh’s (2011) report on governance in primary schools.

However, according to the Inspectorate’s latest annual report, this situation seems to be improving rapidly. More and more schools and school boards are developing and using a system of quality assurance and they focus on improving the quality of education of their schools. Almost all schools evaluate pupil achievements and the teaching and learning process. Most schools use this information for work directed at improvement (75% in secondary education, 80% in primary education) (Dutch Inspectorate of Education (2016b).

Pilot Study 1: Quality of the Feedback by Dutch Inspectors

As already described in the previous section, the quality of the feedback as well as the personal communication skills of the inspector are important factors in order for the feedback to have impact. In research by the Dutch Inspectorate of Education Dobbelaer et al. (2012) looked to measure the quality of the feedback provided by Dutch inspectors. The aim of the research was to explore whether oral feedback provided by Dutch inspectors to teachers is adequate to support the professional development of teachers in primary education. The study also explored the effect of a training aimed at improving this quality and teachers’ perception of the feedback.

Feedback Framework

In order to measure the quality of the feedback provided by inspectors, Dobbelaer et al. (2012) developed a quality framework. Based on a literature review, they developed 19 indicators of a feedback conversation that were expected to contribute to feedback effects. The indicators formed four subscales:

  1. 1.

    The Start of the feedback conversion.

Feedback is mediated by a range of variables including task complexity, time constraints, mode of feedback, timing of feedback, frequency of feedback and whether it is positive or negative (Kluger and DeNisi 1996). Many of these aspects are fixed during an inspection and inspection might therefore not be the most ideal situation to provide feedback. For example, feedback is best provided by someone with a similar status to the recipient (Ilgen et al. 1979) and it is better if feedback is not provided by an assessor (Brinko 1993). These factors are not the case when an inspector provides feedback. Therefore a scale was included which looked at how feedback conversation begins within the context of the framework. In order to start the conversation off in a positive manner, this scale measures if the goal of the conversation is discussed as well as the classroom observation instrument on which the feedback is based. The indicators on which the feedback is based should be clear to both the feedback provider and the recipient (Hattie and Timperley 2007).

  1. 2.

    The Content of the feedback.

The content scale includes indicators that measure whether the feedback complies with characteristics of the most effective feedback, according to the literature. Feedback is generally ineffective when delivered at the level of the ‘self’ (Kluger and DeNisi 1996; Hattie and Timperley 2007; Dweck 2008). This type of feedback does not usually lead to improvements in performance because it does not contain sufficient information to guide the recipient towards improved performance. What this means in reality is well demonstrated through the use of praise: While individuals generally find praise re-assuring, this is not generally translated into increased learning or performance (Hattie and Timperley 2007). Often, praise is better seen as a reward or reinforcement rather than specific information on processes or performance. Where praise is directed towards improving a learner’s self-regulation or choice of strategy used to achieve a task, this is likely to be more effective (Hattie and Timperley 2007).

If feedback is too specific, this may have the effect of the individual focusing too much attention on a specific short-term goal rather than the strategies used to attain the goal. Hattie and Timperley (2007) suggest that such feedback “can lead to more trial-and-error strategies and less cognitive effort to develop informal hypotheses about the relationships between the instructions, the feedback, and the intended learning” (p. 91). Successful feedback on task performance is thus most effective when it is directed at correcting incorrect hypotheses about cause and effect. Where feedback corrects an individual’s faulty conceptions, and especially where the individual has a high confidence in the correctness of their modus operandi, this is when feedback is the most effective (Hattie and Timperley 2007). Therefore inspectors need to be confident and well-informed enough to challenge the false assumptions of teachers and school leaders.

The aim of feedback is to provide insight into the gap between the current situation and the intended learning goals, and what can be done to reach the desired improvement, (Black and William 1998; Hattie and Timperley 2007; Sadler 1989, 2010). Feedback should also be aligned with the professional development of the recipient. When the gap between the desired situation and the current situation is too wide, chances are higher that the feedback will not lead to action (Sadler 1989). It is not useful to focus the feedback on a skill on a higher level if skills on a lower level are not yet acquired (Van de Grift 2001). Therefore, an inspector needs to be able to make a good estimation of the level of the teacher and his professional development based on a single observation. This requires expertise of the inspectors, since teaching and learning processes are often highly complex.

  1. 3.

    The Dialogue during the feedback conversation.

In the dialogue scale, one indicator is focused on the type of dialogue during the conversation. Askew and Lodge (2000) have distinguished three models for exchanging feedback:

  • Feedback as a gift, where the expert provides information to the feedback recipient with the primary aim of evaluation. Feedback in this model is often too overwhelming and insufficiently aligned to the recipient’s knowledge and skills (Hargreaves et al. 2000).

  • Ping-pong model, which refers to the nature of the communication process during a conversation. Performance is described and discussed, while the feedback provider is still in control. This could lead to a lack of alignment between the feedback and the recipient’s knowledge, skills and goals.

  • Co-construction, in which feedback and reflection are intertwined, connecting the process of learning to a context and to previous experiences.

Askew and Lodge (2000) and Prins et al. (2006) consider this third model the preferred one. This seems also the preferred model when an inspector provides feedback to teachers, in order to make sure that the inspector’s feedback aligns with the teacher’s knowledge, skills, goals and professional development.

  1. 4.

    The Closing of the feedback conversation.

The literature on feedback (Kluger and DeNisi 1996; Hattie and Timperley 2007) suggests that it is most effective when the recipient of the feedback:

  • is committed to achieving the goal

  • believes that effort, rather than ability determines success on learning tasks

  • has a high degree of self-efficacy

  • has sufficient knowledge, skill or understanding on which to build on the feedback provided.

The fourth scale is about closing the feedback conversation. Often, the perception of the feedback recipient and the feedback provider differ (Carless 2006; Gil et al. 1984; Irby 1994). By discussing this perception, possible barriers that can interfere with the effect can be eliminated. Feedback can only lead to action if the recipient understands the feedback (Sadler 2010; Hyatt 2005; Gibbs and Simpson 2004). According to Sadler (2010), feedback is often not understood by the recipient. Therefore an indicator in the framework assesses if it is clear that the teacher understands the feedback or if not, if this is checked by the inspector.

Method

In the research by Dobbelaer et al. (2012), 15 primary school inspectors provided feedback to 40 teachers after a lesson observation during a regular inspection. All feedback conversations were audio-taped and its quality was assessed on the basis of the framework. The reported inter-rater reliability of the framework was 0.73. Cronbach’s alpha for the total scale was 0.81, sub-scales ranged between 0.66 and 0.80. In order to explore the effect of a training on the quality of the feedback as assessed by the framework, nine inspectors were trained before they provided feedback (experimental group) and six inspectors were not trained (control group). In the two-hour training, the feedback framework and the underlying literature were discussed.

Results

The mean quality of the feedback provided by the inspectors was on average sufficient to good (3.23 on a scale from 1 to 4 with a standard deviation of 0.56). More than 50% of the feedback conversations could be characterized as co-construction, however in 25% of the conversations there was not enough time for teachers to give their opinion. Trained inspectors scored significantly higher than untrained inspectors both for the whole framework and for the subscales (except for the dialogue scale) of the feedback by untrained inspectors was also greater, more feedback conversations were considered insufficient than those of trained inspectors.

Dobbelaer et al. (2012) also measured teachers’ immediate perception of the feedback with a questionnaire. The questionnaire included scales about the perception of affect (how did the feedback made the teachers feel), cognition (what did the teachers learn from the feedback), behavior (how do teachers intend to change their behavior) and the quality of the feedback (how did the teachers perceive the quality of the feedback). Most teachers experienced the feedback conversation positively. About 66% of the teachers indicated that the feedback gave them insight into their strengths and weaknesses, and 75% indicated that the feedback motivated them to improve. A correlation of 0.53 was found between the quality of the feedback, and teachers’ perception of the feedback. Teachers who received feedback from a trained inspector seemed more positive than teachers who received feedback from an untrained inspector, however this difference was statistically only marginally significant.

Pilot 2: Evaluation of the New Inspection Approach

In the next paragraph, we discuss the evaluation of the new inspection approach in which primary schools have been visited in line with the new policy (Dutch inspectorate of education 2015, 2016c). This particularly evaluation provides insight into the perception of school leaders, special educational needs coordinators and teachers of the feedback by the inspectors after the school visit, and is key to understanding how individual inspectors affect the quality of feedback and its impact.

Set-up of the Pilot

In school year 2014–2015 the Dutch Inspectorate of Education began several pilots in the context of the innovations of the inspection approach. Before the pilot visits took place, central school boards and school leaders were informed about the characteristics of the new supervision and the pilot visits. Inspectors had short meetings about the feedback dialogue and practiced providing feedback in short role plays. During pilot inspections, ten standards of the new evaluation framework were used, namely: the standards for cognitive outcomes, curriculum, insight into development of pupils, teaching methodology, support, school climate, safety, evaluation and improvement, quality culture and accountability/dialogue. Besides the new evaluation framework some innovations of the inspection approach have been piloted. In the various conversations with the head, the staff, parents and pupils, mind maps have been used (instead of structured conversation guidelines) to increase the contribution of the interlocutors. The mind maps consists of an overview of the topics that can be discussed. The meeting participants have an influence on the choice of topics.

To build up experience with all the new aspects of the innovations of the inspection system, the inspection used three types of school inspections:

  • Method 1: The inspectorate and self-assessment.

The starting point is the school self-assessment. Prior to the inspection visit, the director received a questionnaire with which the school had to assess itself on the ten standards that were also assessed during the visit by the inspectors. The literal text of the standards used by the inspectors was made available together with the questionnaire. Based on interviews and classroom observations, inspectors build up a picture of the quality of the education of the school. At the end of the school inspection the inspectors leave behind a list of preliminary judgements. In the feedback session, often on a different day (usually one or two days after this school inspection), the images are shared in a dialogue with the management, (part of) the team and the board. Inspectors also examined what the conclusions could mean for subsequent development in the school.

  • Method 2: The inspectorate and the school look together.

The starting point is again the self-assessment, to be completed in advance by the school, with now extra emphasis on the way the school evaluates the quality of the teaching learning process in the classes. The classroom observations during the inspection are performed together by the inspectors and members of the school staff. The classroom observation form (checklist) of the school is being used. Furthermore, there is a lot of room for meetings and at the end of the inspection there is a feedback session.

  • Method 3: The inspectorate looks at how the school looks.

In this method at the beginning of the inspection the school presents its own quality evaluation to inspectors. The portrayal of the school within this document, forms the starting point for the agenda and the meetings during the investigation. Classroom observations are not a standard element of these inspections. Within this process there is a good deal of room for discussions with stakeholders and at the end of the inspection a feedback session is held.

It was not the purpose of these three methods to test the three operating modes as a whole, but rather to try out the new framework and the innovations within the inspection methods, such as; completing a self-assessment; joint classroom observations; a presentation by the school; interviews with the team, parents and students; the feedback session etc.

The (Postponed) Feedback Meeting

This section mainly focuses on the experiences of the feedback meeting in the pilot. The data used as a basis for this part of this chapter stem from the pilots carried out in the period February to October 2015 at 772 schools in primary education (726 primary schools and 46 special schools for primary education). Only schools of basic quality were visited during this pilot study. The three methods were distributed to the schools, taking into account the stability of the student achievements and the picture of the quality care of the school (based on the latest inspection of the school).

During the pilot visits, the inspectors had a feedback meeting with a representative of the school board, the management, the special needs coordinator, and some teachers. The purpose of this feedback session; to encourage the staff to improve the quality of education at their school. The inspector discusses opportunities for improvement alongside the conclusions of the report. An important part of this conversation is formed from the self-assessment by the school. The meeting is mainly held to engage the school in conversations about the possible significance of the judgment for the school development. After the feedback session the inspectors prepare a report on the investigation.

Opinions of the Schools About the Feedback Meeting

After the school inspection, the schools were questioned about the various components that were tried in the pilots, including the feedback meeting. The head, the special needs coordinator and a number of teachers were given a questionnaire to record their responses.

Figure 5.1 shows that almost all respondents experienced the feedback session (very) positively. Most respondents felt that the feedback provided clues on how to improve school quality (on average 97%) and that the feedback conversation was a stimulant to improve (on average 93%). Most school leaders thought that the feedback meeting was an improvement on previous forms of the meeting (Dutch Inspectorate of Education 2016c).

Fig. 5.1
figure 1

Perception of the feedback conversations by school leaders, teachers and special educational needs coordinators (Dutch Inspectorate of Education 2016c)

Full size image

Teachers were new participants in the conversations at the end of inspection. They were largely positive. A quote from a teacher:

A very pleasant way of communication. Bright, clear. With a good ‘accountability’ both gentlemen succeeded in describing our school. The finger was put on the sore spot, but in a respectful way. Good that we ourselves were made responsible for our upcoming improvement process. By challenging questions we in turn were challenged to look at our education from helicopter view and to contemplate. This gave us strength and new insights (Dutch Inspectorate of Education 2015, p. 34)

Almost all school leaders perceived the self-assessment before inspection as useful. Two quotes from school leaders:

Completing a self-assessment prior to inspection gave us a good overview of how we are doing and forced us to critically review our own quality

We perceived the process of handing in the self-assessment prior to inspection very positively. We appreciated the motto ‘tell your own story as a school’ (Dutch Inspectorate of Education 2015, p. 33)

Some head teachers, special needs coordinators were less positive; they appeared to believe that the inspectors were not capable of sustaining a dialogue with the school. Certainly in schools with a number of failures, and that the inspectors must continue to seek to avoid the conversation is one-way direction in which only the judgements were explained. The inspection will continue to focus on the dialogue with the school during training and collegial consultation (Dutch Inspectorate of Education 2015).

There were slight differences in the level of enthusiasm for the feedback session between the three methods. The feedback session in method two scored slightly lower. This could be related to the fact that the visit day in method 2 is very long. The feedback session then begins after 4 pm. In method 1, the feedback meeting is on another day and in method 3 around the sessions started at 3 pm (Dutch Inspectorate of Education 2016c).

Inspectors Opinion on the Feedback Meeting

Inspectors completed a questionnaire about the school visits as well. They were generally positive about the feedback discussion, they especially appreciate the presence of the teachers. The current meetings after school inspection always take place without the presence of teachers, with only the school board and the head. Inspectors found that the feedback session at the end of the examination day (in method 2 and 3) generally makes sense. Several inspectors report that the feedback session promotes ownership of issues among both school and the team. They also responded that they find it difficult to support their judgments if the school does not agree with the judgement. In this situation it is not always possible to resume a dialogue.

Inspectors made a few specific suggestions: That the feedback meeting would be less suitable for schools that are not so capable of self-reflection. In these schools they felt that it would not be possible to establish a good enough dialogue to effect improvement. Some inspectors noted that they needed to more effectively communicate the purpose of the meeting before inspections take place in order to clarify what feedback will occur.

Discussion and Conclusions

The Dutch Inspectorate of Education is innovating within their inspection process. Their new mission is to improve education in all Dutch schools (Dutch Inspectorate of Education 2016a). One of the chosen mechanisms to stimulate school improvement is the provision of feedback by inspectors. Feedback will be provided by the inspector orally to a representative of the school board, the school leader, and teachers. After the inspection, a report is produced in which a quality profile of the school is presented. The whole idea behind the inspection feedback is that it is designed to stimulate school improvement. Multiple studies show that teachers and school leaders welcome feedback from inspectors, that they find it useful and feel disappointed when it is not given (Dean 1995; Brunel University, Centre for the Evaluation of Public Policy and Practice 1999; Kelchtermans 2007). The two pilot studies described in this chapter suggest that this also applies to the Dutch context. The two pilots in this chapter also show that school leaders, teachers and special educational needs coordinators are positive about inspection feedback. However, the research synthesis in this chapter showed that the results of research into the effect of inspection feedback are mixed. The effect is, among other things, highly dependent on the inspector, the feedback provider. In the next section, we will focus on this role of the inspector.

Improvement Through a Dialogue by Trained Inspectors

To make sure that the feedback is understood, aligned with the recipient, and correctly perceived; a dialogue during the feedback conversation is absolutely essential. In the revised inspection system, the dialogue between the inspector and the school is allocated a bigger role than in the previous framework. In the present form, inspectors mainly presents their findings to the school leader after their school visit. This will be different in the innovative inspection, where the starting point of the inspection will be the schools’ perception of its own quality, which is presented via a self-assessment.

With this new form of inspection, in schools that meet basic quality criteria, the focus is on school improvement rather than accountability. This also means that the role of the inspector is changing and it requires different skills. The literature review and the pilot study by Dobbelaer et al. (2012) both show that the communication skills of inspectors are key to the whole process, as other inspectorates using similar forms of inspection (see for example chapters three and four this volume) have shown. If the attitude of the inspectors is perceived as authoritarian, this could trigger reactions of resistance and rejection by teachers (Kelchtermans 2007). When the inspector is too dominant during the conversation and the recipient does not get enough time during the feedback conversation, the feedback might be overwhelming and may well not align with the development or the context of the school.

Training of inspectors for this new role may be needed. The pilot by Dobbelaer et al. (2012) showed that even minimal training could increase the quality of the feedback. This is supported by other research into the training and development of inspectors (see for example chapters three, four and eight, this volume). Both pilots of the Dutch inspectorate described in this chapter were conducted at schools that meet basic quality criteria. Schools that do not meet basic quality criteria are in greatest need of improvement and good quality feedback is especially important. However, at those schools the inspector cannot only focus on school improvement, but has to hold the school accountable for its shortcomings as well. This creates a high stake situation in which it is even more difficult to provide feedback: one where more factors come into play which might interfere with the effect of the feedback. Research also shows that schools are generally disinclined to accept unfavourable inspection conclusions and that communication skills have an effect on this aspect (Penzer and CfBT Education Trust 2011). Training if inspectors in providing feedback could be extra important for these inspections.

Feedback to Individual Teachers

Multiple studies show a positive effect in terms of inspection feedback. However, many of those studies (e.g. Chapman 2001; Brimblecombe et al. 1996) are about feedback to individual teachers: precisely the one form of inspection feedback that the Dutch inspectorate is not incorporating. Is this a missed opportunity? When feedback is directed towards an institution or a group, other mediating variables also come into place. For instance, Hattie and Timperley (2007 citing Nadler 1979) explain that feedback directed to a group or institution would be diluted. In essence, it could apply to the person reading (or listening to) the feedback, the institution (or group) as a whole, or to other people in the group or institution (excluding themselves). Thus the extent to which it leads to individual changes in behaviour is likely to be mediated by the perception that the feedback is relevant to them. Kelchtermans (2007) found that if teachers were not able to recognise themselves in reports, they were more likely to react superficially rather than by making any real change in classroom practice. Overall this might suggest a limited role for the feedback in inspection reports to drive improvement and the need for inspectors, wherever possible, to give direct feedback to teachers.

If the Dutch Inspectorate provided feedback to individual teachers, the feedback would be based on a single observation. The unusual circumstances of the inspection could influence the teaching quality and the teacher might feel that the lesson is not generalizable to other lessons. This could interfere with the effect of the feedback. In Lithuania, this issue is resolved by observing teachers at least twice by different evaluators during a 3–5 day visit in which 75% of the time is spend on teacher observation. Feedback to teachers is provided straight after the lesson, in which both strengths and areas for improvement are provided (Kamenskienė and Kardelytė 2016).

A key reason for the Dutch Inspectorate to not incorporate feedback to teachers is that teacher evaluation is seen as a task of school boards. Another element is the role of the inspector as both critical friend and assessor. If an inspector makes recommendations to a teacher and the teacher follows up on them, the inspector will need to assess his own recommendations the next time they observe the teacher. This is not only applicable when an inspector provides feedback to an individual teacher, but also when an inspector provides feedback with recommendations to schools. Combining those two roles as an inspector would appear to present a great deal of challenge. For this reason the Dutch inspectorate chose not to provide suggestions on how to improve during the feedback conversation, but rather focus on discussion as to how a school aims to develop from the current performance to the desired one.