Keywords

Introduction

In most countries, it is clear that the quality of schools should be monitored in some way. Most often, an Inspectorate is established in order to inspect schools on a regular basis. Notwithstanding the differences between different educational contexts regarding the operationalisation of inspections and the way inspections are conducted, there is generally a remarkable similarity between methods used and the aspects focused on during an inspection (Ehren et al. 2013; OECD 2013).

During the last two decades, there have been numerous studies into the impact of inspections on schools—for an overview of impact studies, we refer to Chapman and Earley (2010) or Penninckx and Vanhoof (2015). Each of these studies describes the effects of inspections on the processes and/or the output of schools. These studies, however, do not evaluate these effects with regard to a prescriptive frame of reference, as they do not include an assessment of the extent to which inspections fulfil their aims. In other words, these studies do not make an analysis of the effectiveness of inspections. An effectiveness analysis implies that one resolutely assesses whether or not an inspection meets predetermined expectations or aims (Scheerens 2011). Most studies are indeed limited to a description of effects. It is not sufficient to describe the effect of an inspection to make an assessment; instead the described effects should be weighed against what is considered to be ‘desirable’ (Biesta 2009). The question of which aims inspections should pursue, has only very rarely been explicitly responded to. Clarifying these aims is a necessary step in order to assess whether or not the actual effects are reflected in desirable effects (Rossi et al. 1999).

The study presented in this chapter aims to explore a frame of reference in which these desirable aims of inspection are defined. Theoretically, one could try to derive these desirable aims from the legislative framework. However, this is problematic because of two concerns of both a pragmatic and a fundamental nature. The pragmatic element relates to the finding that, in the Flemish educational context, the Decree regarding the Quality of Education (2009) does provide an enumeration of the tasks of the inspection, but it gives little information regarding inspection aims. The fundamental concern in this case is the idea that the aims of inspection should not be unilaterally defined by the authorities competent to issue decrees. Instead the desirable aims of an inspection should be determined by the shared expectations of different stakeholders with regard to these aims (Faubert 2009). The aim of an inspection is to be found in the realization of these expectations (Harrington and Harrington 1994).

This chapter aims to provide an inventory of desirable aims that inspections should pursue. These desirable aims should be supported by different stakeholders in the field of Flemish education. The research question is: “What are the aims that inspection should pursue, according to different stakeholders in the field of Flemish education?” It is important to note that this study is related to an assessment of desirable aims, and it is not about an inventory of the actual aims of inspection as they currently stand. To our knowledge, this kind of research has not yet been conducted. It therefore concerns an exploratory study in a new area of research.

Depending on one’s position in the field of education, one may hold different understandings of the term ‘quality of education’ (Hoy and Miskel 2008). Representatives of different interest groups may also assign different meanings to the evaluation of educational quality (O’Hara et al. 2007). It is therefore to be expected that obtaining a consensus between various stakeholders regarding some of the desirable aims of inspection may well be difficult, as there may be strongly polarized points of view. By mapping the different arguments in these fields of tension, this study also contributes to the policy and academic debates regarding the aims of inspections.

Conceptual Framework

The aims that are assigned to inspection, are most often related to one of two perspectives: either accountability or development (Earley 1998; Janssens and Van Amelsvoort 2008; Van Bruggen 2010). With regard to the accountability-oriented aim, it is expected that the inspection will help to control the school’s quality, and in so doing, to open up “judgemental” information for the authorities, as well as for pupils, parents, and the school’s own school board, in order to endorse the legitimacy of the school (and, by extension, of the whole education system) (Ng 2011; Penzer 2011). The accountability functions regarding the government; pupils and parents; and stakeholders within the schools, are referred to respectively by the International Institute of Educational Planning as: “contractual accountability”, “public accountability”, and “professional accountability” (IIEP 2011, pp. 6–7).

With regard to the development-oriented aim, there is an expectation that inspections will contribute to the quality of schools (Vanhoof and Van Petegem 2007). In most countries, both aims are assigned to the inspection, albeit often implicity (Van Bruggen 2010). There are, however, slight differences between countries, as inspections in some education systems are predominantly accountability-oriented, while others focus more on development aims (Ehren et al. 2013; van Amelsvoort et al. 2006). Nevertheless, it is unclear how these functions can be combined.

The roles of ‘inspector’ and ‘counsellor’ have been perceived as incompatible in several education systems (Ehren and Honingh 2011; Waterman 2013). Different scholars have argued that inspected schools do not open up for development due to the fear caused by the inspection experience (Faubert 2009; MacKinnon 2010). It is assumed that in order assume a developmental perspective, it is crucial that there is some kind of trust between the inspector and school, and that the school has the courage to show itself in full view for inspectors (Macbeath 2006; Ouston and Davies 1998). According to several scholars, this is only seldom really the case, due to the position of the school with regard to inspectors, who have the power to decide the school’s fate (Ferguson et al. 1999; Lonsdale and Parsons 1998), and also due to the potential sanctions after a negative judgement (Ferguson et al. 1999; Woods and Jeffrey 1998). Moreover, imposed changes cannot include real school development due to frequent lack of support within the school (Faubert 2009; MacKinnon 2011; Perryman 2010). McNamara and O’Hara (2008) have argued that the more an inspection system leans towards accountability-oriented aims, the less useful it is in orchestrating development. Some scholars have noted that the accountability-oriented aim may not only inhibit the development-oriented aim, but that it may even have a negative impact on school development, as schools are seduced to engage in practices that are deleterious for their own development; such as window dressing activities, giving up their own priorities and reflections, or placing undue attention on the “measurable” aspects of the school (Ball 2003; Ozga 2009; Perryman 2009; Suspitsyna 2010).

In this study, we define the development-oriented aim in a broad sense. The concept encompasses the “conceptual effect” (“The extent to which inspection influences the thinking of decision-makers (and practitioners) and as such may have an impact on their actions”), the “instrumental effect” (“The decisions taken as a result of inspection and the actions that are based upon these decisions”) (definitions based on Visscher 2002), as well as the results of these decisions and actions.

Apart from the accountability-oriented and development-oriented aims, several scholars have assigned a third purpose to inspections: Inspections are expected to support national education policy by collecting and analysing data from different inspections, which is then considered to contribute to an information-rich environment for policy-makers and other stakeholders in the field of education (Matthews and Smith 1995; Scheerens et al. 2003).

Research Context: The Flemish Educational Accountability Context

Given the context-inclusive nature of this study, it is important to describe accountability mechanisms in the Flemish educational context. An important feature of this context is the absence of central examinations—or any other kind of imposed standardized tests—in the Flemish education system. The absence of central examinations is a subject of perennial debate. Until now no initiatives have been taken to implement (obligatory) central examinations, mainly because of the fear of negative side effects such as the possibility of ranking schools based on pupils’ learning outcomes (Andersen Consulting 2002), and because of the idea that central examinations would affect the constitutional principle of ‘freedom of Education’ (Shewbridge et al. 2011). Because of the current absence of central examinations, inspections are the only measure of school accountability in terms of educational authorities (Standaert 2001).

Remit of Inspections

Flemish schools enjoy a relatively large degree of autonomy to set up processes according to their traditions and beliefs on how to achieve ‘attainment targets and development goals’ [“Eindtermen en ontwikkelingsdoelen”] imposed by government (OECD 2013). The Decree declaring the Quality of Education (2009), stipulates that the inspection controls to what extent schools have made informed choices that ensure that pupils achieve these attainment targets and development goals. The inspection also assesses whether or not schools systematically monitor their own quality. Finally, inspection controls whether or not the ‘habitability, safety and hygiene’ of the school infrastructure meets the legal requirements.

Like many Inspectorates in other education systems, the Flemish Inspectorate pursues both accountability- and development-oriented aims. Its ambition is to monitor schools, but also to contribute to the quality of the education provided by schools (Michielsens 2008; Onderwijsinspectie 2015). However, in terms of its development function, the Inspectorate is limited by the Flemish legislation’s strict distinction between inspectors (assigned to control schools) and school counsellors [“pedagogische begeleidingsdiensten”], who are assigned to provide advice and support to schools. This distinction implies that inspectors are expected to analyse and report on the schools’ strengths and weaknesses, but that they need to refrain from making recommendations to the schools on how they can enhance their quality. The Inspectorate’s operating assumption in terms of pursuing its development-oriented function is that the insights into the schools’ own strengths and weaknesses provided by the inspectors (uncovering the ‘blind spots’ that schools are unable to detect through self-evaluation) will serve as an impetus for the schools to secure their strengths and address the identified weaknesses (Vanotterdijk 2008).

Methods and Instruments of Inspection

The inspectors employ the CIPO-model during inspection. The acronym CIPO stands for context, input, process and output (Vlaamse Regering 2010). Each of these four components is further divided into several indicators, which are assumed (based on research or experience) to have an impact on the quality of education. Although the primary focus of inspection is on the output delivered by schools (the extent to which the attainment targets and development goals are achieved), the absence of central examinations implies that no standardized data about school performance regarding pupils’ learning outcomes are available. The lack of reliable output data constrains the inspection, requiring the inspectors to adopt a process-oriented approach (Shewbridge et al. 2011; Van Bruggen 2010).

During the inspection, which lasts approximately three to five days, the inspectors’ data collection methods include interviews (mostly including principals, members of the management team and teachers), document analysis and lesson observations.

The inspection adopts a ‘differentiated approach’ which implies that it does not carry out ‘full inspections’ of all areas. Instead, two or three education areas (e.g. mathematics) or particular departments, and a number of process indicators (e.g. pupil evaluation, human resource policy), are selected for detailed inspection. The inspection judges purely at the school level. On the few occasions on which individual teachers are mentioned, the inspection report is required to ensure strict anonymity. ‘Teacher evaluation’ is the exclusive responsibility of the school board and the principal. Inspectors are therefore not allowed to give formal or informal individual feedback to teachers or principals (Ministry of Education and Training of the Flemish Community 2010).

Consequences of the Inspection

The inspection leads to two independent judgements—on educational matters and on school infrastructure. These judgements are in effect, advice to the government about prolonging the school’s “recognition”, a condition for the receipt of public funding. The judgements given to schools are either ‘positive’, ‘restricted positive’ or, in a few cases, ‘negative’. The ‘positive’ judgement means a school is considered to have the competencies and preparedness to continue working in an optimal manner, and that no follow-up needs to be scheduled. The ‘restricted positive’ judgement denotes that a second inspection is required three years later to determine whether or not identified shortfalls have been adequately addressed. Schools that show structural deficiencies are given the judgement ‘negative’. Policy-makers and the Inspectorate are convinced that pupils in malfunctioning schools do not benefit from an eventual school closure, but neither are they helped by a soft approach with regard to their school. Therefore, although the judgement ‘negative’ officially means that the recognition of the school is being revoked, in reality the school is allowed to set up a plan for improvement by which it is obliged to be monitored by an external agency—mostly the school counselling services that are already in charge of providing advice to schools (Ministry of Education and Training of the Flemish Community 2010). Therefore it is highly unlikely that a school be closed down as an immediate result of an inspection. This explains why the Flemish inspection system is generally seen as ‘low-stake’ compared to other educational contexts (OECD 2013; Van Bruggen 2010). During the school year 2014–2015, the judgements ‘positive’, ‘restricted positive’ and ‘negative’ were given respectively to 51.5, 45.5 and 3.0% of the inspected primary and secondary regular schools (Onderwijsinspectie 2015). Due to the differentiated approach, different departments in the school may receive a different judgement. For instance, it may be the case that while one department in the school is assessed as ‘positive’, the school in general receives a ‘restricted positive’ judgement.

In sum, in spite of several similarities between the Flemish context and other countries regarding accountability measures in education, there are some notable features in which the Flemish context differs from that of some other inspection systems. These features are the absence of central examinations, the perceived lower inspection stakes, the strong process-oriented approach of inspection, the relatively low frequency, the long notification period, the strict distinction between inspectors (for control) and school counsellors (for advice) and the restriction on inspectors with regard to giving individual feedback to teachers or principals.

Research Design and Methodology

The Delphi Method: Argumentation and Constitution of the Expert Panel

Due to the explorative nature of this study within a research area in which only a small amount of evidence is available, we selected the Delphi method. The Delphi method is a technique that allows the exploration and confrontation of the points of view of different experts through several sequential research phases (Dalkey and Helmer 1963; Day and Bobeva 2005; Okoli and Pawlowski 2004). The Delphi method is used to structure a process of group communication in such a way that individual experts can examine a complex matter as a group (Day and Bobeva 2005; Linstone and Turoff 1975). By using an iterative feedback technique, this method collects the opinions of different experts, rather than objective facts (Schmidt 1997). One of the major benefits of this method is that there is no direct confrontation between different experts. The approach therefore implies that the method supports independent reflection (Dalkey and Helmer 1963; Dalkey and Rourke 1971; Okoli and Pawlowski 2004), that it avoids a scenario in which experts join too easily with a stance taken earlier by another expert (Dalkey and Helmer 1963), and that experts do not experience pressure to conform with others’ points of view, which may particularly be the case when there is some kind of (implicit or explicit) hierarchy between the experts (Van de Ven and Delbecq 1974).

One of the most critical demands for the validity of a Delphi study’s results, is in the selection of appropriate experts (Chong et al. 2012; Custer et al. 1999; Okoli and Pawlowski 2004). It is recommended that the panel not be limited to experts at policy level, but rather that it should also include experts that are directly touched by the subject of the study, in this case experts from inspected schools (Hsu and Sandford 2007). Considering functions and expertise, after careful selection a panel of fifteen experts was constituted. The selection process’ point of departure was the three actors in quality assurance in the field of education, as defined by the Decree regarding the Quality of Education (2009): schools (n = 5), inspection (n = 3), and counselling services (n = 4). In the category ‘school’, we selected representatives of different sections within the school: two principals and one member of a school board, one teacher representative, and one parent representative, all having been recently affected by inspection. The panel was then completed with the inclusion of policy-makers (n = 3), for whom it was necessary that their position allowed them to discuss inspection from a well-underpinned position. The anonymity of each of the experts was guaranteed throughout the entire process of this study.

Research Development

This study is built on three research phases. As is customary in Delphi studies, the first research phase consisted of an open examination of the experts’ ideas regarding our subject (Day and Bobeva 2005). A written questionnaire was developed with open questions and several sub-questions, e.g. ‘How would you describe in your own words the function(s) of an inspection?’, ‘Supposing that you were asked to evaluate inspections, which criteria would you use to evaluate whether or not an inspection is effective?’, and ‘Which effects should an inspection lead to?’ This questionnaire allowed the researchers to probe the opinions of the experts with regard to the aims that an inspection should pursue. In this first research phase, the experts were encouraged to give as many responses as possible (Schmidt 1997).

The second research phase consisted of individual semi-structured interviews with a dual aim: It allowed different experts to clarify their earlier responses, and to comment and react on the responses of other experts. During these interviews, the different aims for inspections were expanded upon further, along with examination of the fields of tension that had come out of the first research phase.

Both the written responses from the first research phase and the transcribed interviews were encoded. We first used an ‘open’ coding, creating ‘nodes’ during the process. These nodes included ‘general aims of inspection’, ‘school areas on which effects should show up’ as well as combinations of both. While coding the data into nodes, we carefully documented which information was delivered by which expert, in order to examine the extent of consensus and support, as well as the comprehensiveness of the data. The open coding was followed by ‘axial coding’, which included consolidating (partially) overlapping or related nodes into one node. Finally, the sources were re-analysed on the basis of the remaining nodes (‘selective’ coding) (Gibbs 2002).

During the third research phase, a closed, written questionnaire collected data on the extent to which the experts attached importance to the various stated aims that were derived from the results of the first research phase. Each of these stated aims was scored by the experts on a Likert-scale ranging from 1 (an unimportant purpose) to 4 (a key purpose). This written questionnaire allowed every expert to react on each of the responses that different experts gave in the first research phase. We received a response on this closed written questionnaire from 13 out of 15 experts. The parents’ representative stated that she—given her lack of vision regarding the subject of ‘inspections’—was not able to give valid responses to the questionnaire: “as an organization, we do not have a formal point of view about the inspection”. In addition to this the teachers’ representative did not return the questionnaire.

As mentioned earlier, it was expected that some discord would manifest between the points of view of various experts. In this respect, the present study deviates from the traditional aim of a Delphi study, namely to obtain a full consensus (Hsu and Sandford 2007; Linstone and Turoff 1975). The aim of this study was rather to identify different aims and the assessment of these aims by the individual members of the expert panel. This ‘looser’ application of the Delphi method has been applied before by other studies with the intention to inform an inventory of opinions regarding a certain subject (Okoli and Pawlowski 2004). Even the founders of the Delphi method have indicated that “it cannot even ideally be expected that the final responses will coincide, since the uncertainties of the future call for intuitive probability estimates on the part of each respondent” (Dalkey and Helmer 1963, p. 459).

Results

We first discuss the general aims that inspections should pursue, according to the expert panel. Then we examine the school areas to which these general aims apply in order to arrive at specific aims, these areas are investigated under the headings:

General Aims of Inspections

Accountability-Oriented Aims of Inspections

Every expert agreed that inspections should have the purpose of making schools account for their actions. It was stressed that schools need to account to the authorities, as well as to the parents of (future) pupils. Internal stakeholders were only rarely mentioned. As schools are financed or financially supported by the authorities, every expert thinks that it is logical that inspection leads to the school accounting for their use of public money: “If any organization receives financial funding from the authorities—and for such a substantial remit at that—it is just plain logic that accountability is expected” (Policy-maker C).

This does not, however, relate to financial control, but rather a quality control on standards of education within schools. Furthermore the expert panel agreed that the accountability-oriented purpose also includes a sanctioning role, namely the revoking of the recognition of schools that are not able to sufficiently account for their quality.

Development-Oriented Purpose

There was less agreement amongst the experts regarding the development-oriented purpose of the inspection. Some of the experts (from the category ‘counselling services’) thought that it cannot be the explicit purpose of an inspection, in that it needs to result in schools taking the initiative for its own improvement (the above-mentioned “instrumental effect”). Neither should it be an explicit purpose of the inspection that schools improve their output (learning results, or pupil well-being) as the result of an inspection. Two arguments were mentioned by these experts to support this position: on one hand, the instrumental effects or output effects cannot be directly linked to the inspection, as they are, by definition, the result of a complex interrelation between different factors, some of which are not under the direct control of the inspectors. On the other hand,—and more importantly—it was argued that if this purpose was to be expected from an inspection, it would require that inspectors be assigned to provide concrete and constructive advice to the school, and to offer concrete solutions for deficiencies identified in the school. Such a requirement—starkly contrasting with current policy in Flanders—is considered as undesirable, not only by the experts from the group that includes ‘counselling services’, but is also by several (not all!) other experts from ‘schools’, ‘inspection services’, and ‘policy-makers’. The main argument mentioned by these experts was that “one body cannot be judge and party at the same time” (School counsellor B). Furthermore, allowing the inspectors to provide concrete advice for improvement, would imply that certain teaching methods may be encouraged by the inspectors, a facet which is seen as a threat to the independence of the Inspectorate, as well as to the schools’ autonomy. Another problem that may be expected if inspectors are allowed to provide advice, according to these experts, is the lack of alignment in educational ideas between inspectors: “And then you see, due to a lack of coherence between the inspectors, then you see that the inspectors’ subjective ideas regarding education play far too large a part” (School counsellor A).

Notwithstanding the above arguments, a minority of the experts (three experts from the category ‘school’, one inspector and one policy-maker) were convinced that inspectors should be allowed to provide constructive and guiding advice to schools. The arguments to support this statement were threefold: that inspectors, through their regular monitoring of many schools, may be a key source of ‘good practices’; that when schools receive criticism, they also desire constructive feedback from the body that has voiced the critique; and that the inspection may have a larger impact on school development by providing direct advice. Although the latter argument was supported by most of the experts, most of them remained convinced that the arguments against such a direct contribution towards school development, outweigh the positive arguments: “I think that we have the task to ensure that the report is something that helps the schools to develop, even in cases of a positive evaluation. But concrete recommendations, that would of course be going too far” (Inspector A).

From a confrontation of points of view during the second research phase, it was clear that consensus could be obtained on a “conceptual effect” with regard to the development-oriented purpose: an expectation shared by each expert was that inspections need to pursue the aim of staff members in inspected schools gaining more insight into their own functioning, and increasing levels of reflection on their own quality. The inspection report and/or the inspector’s informal feedback should serve as a basis from which schools can look critically at their own quality and thereby come up with new ideas, or result in the confirmation of existing ideas.

The experts thought that the accountability-oriented and development-oriented aims could be combined. Although three experts (two policy-makers and the representative of parents) recognized there may be some tension between these two aims (for instance because the school needs to present itself in an open and vulnerable manner in order to attain development), there was a general consensus that the two aims can be reconciled. Three experts (one policy-maker, one inspector and one principal) suggested that both aims can even strengthen each other, particularly when school development is the consequence of being held to account.

Eight experts thought that the development-oriented purpose was as important as the accountability-oriented purpose. Six experts, evenly divided over the categories ‘inspection’, ‘counselling services’ and ‘school’, thought that the accountability-oriented purpose was more important: “Quality assurance and monitoring whether or not the goals are obtained is their first task, so accountability is the key task!” (Member of a school board).

In contrast, one school principal indicated that she thought the development-oriented purpose of the inspection should be the predominant purpose.

Policy-Informing Purpose

Next to the accountability-oriented and development-oriented aims, providing relevant information for policy-making should be a third purpose of inspections. The information generated by inspections needs to lead to a coherent view on the quality of the Flemish education system, which is seen as a support for development at policy level. Inspections also have a signalling purpose to help the authorities determine priorities and to assess the attainment targets and development goals.

For me, [the inspection] includes that the authorities can ameliorate their expectations, and through the inspection it should be clear that certain attainment targets and goals need readjustment. It is also important that the authorities see that certain expected improvements are only feasible when the financial means of a school are in line with these expectations. (Principal A)

In contrast with the accountability-oriented and the development-oriented aims that apply to individual school inspections, the policy-informing purpose only applies to the wider picture-inspections of many schools taken as a whole. There was consensus amongst all the experts regarding this third purpose.

Stated Aims of an Inspection

The accountability-oriented and the development-oriented general aims of inspections were elaborated in more detail in the experts’ responses. A stated aim entails that expectations of an inspection are that it should lead to the school accounting for a certain school area (e.g. the teaching methods applied by the teaching team, or the personnel policy in the school), or that it should lead to development in the school regarding these school areas. During the first research phase, 68 of these concrete school areas were identified by the experts. After ‘axial’ and ‘selective’ coding, the number of school areas was reduced to 31. These 31 school areas are enumerated in Table 11.1. They are divided over one school area related to infrastructure, 23 school areas related to the schools’ processes, and seven school areas related to the school’s output.

Table 11.1 Importance attached by the different groups of experts to the different stated aims(1/2)
Full size table

After the first research phase, 31 potential stated aims were identified with regard to the general accountability-oriented purpose, and 31 stated aims with regard to the general development-oriented purpose. The policy-informing purpose of the inspection was not made specific, as this purpose applies to many inspections taken as a whole and as such, is not related to specific school areas.

Based on the second and third research phases, an indication can be provided as to the extent to which these 62 stated aims are supported by the expert panel. In view of the small number of respondents that participated in the third research phase (n = 13), we need to be careful with interpreting the quantitative data. Nevertheless, the mean scores in the four categories of experts (schools, inspection, counselling services, policy-makers) provide an additional indication of the extent to which these stated aims are considered to be important in the Flemish educational context. These mean scores are listed in the second column of Table 11.1.

In the remaining part of this section, we will discuss the importance that the experts attach to each of these stated aims. We first discuss the stated aims related to infrastructure, next the process-related stated aims, and finally the output-related stated aims.

Stated Aims Relating to ‘Infrastructure’

According to the expert panel, an inspection should make schools account for their compliance with regulations regarding the school infrastructure:

An inspection has to examine whether the school complies with the demands regarding hygiene, tidiness, safety, toilets, classrooms, fire safety, safety of the outdoor equipment, … (Parents’ representative)

The data collected during the third research phase indicated that, on average, one of the two stated aims of the inspection viewed as most important, was that inspections should make schools account for the hygiene, safety and habitation conditions of their infrastructure. The development-oriented purpose with regard to hygiene, safety and habitation conditions of the school’s infrastructure was also rated as important. From the interviews it became evident that there was, indeed, a consensus that it is an important stated aim of the inspection that the school gains insight into, and improves, eventual deficiencies in their infrastructure, particularly when these deficiencies relate to the pupils’ safety. The difference between the importance attached to the accountability-oriented Stated aim, and the development-oriented Stated aim, may be explained by the fact that adaptations to the current situation are not easily realized on short notice (e.g. building new classrooms, increasing the safety of the school building), but also because experts assume that schools are commonly already strongly aware of eventual deficiencies with regard to infrastructure.

Process-Related Stated Aims

In total, 23 school areas with regard to process features of the school were identified, including seven school areas related to the education process, four school areas related to personnel matters, and twelve areas related to the school policy. This figure resulted in 46 stated aims that needed to be assessed by the experts, as both the accountability-oriented and the development-oriented purpose were applied to each of these 23 school areas.

At first, some of the experts in the category ‘counselling services’ took a critical stance towards these process-related stated aims. They claimed to adhere to an output-oriented approach to inspections-meaning an inspection should only aim to fulfil the legislative obligation to monitor whether or not pupils achieve the expected learning outcomes (attainment targets and development goals). Process areas belong, according to these experts, strictly to the autonomy of the school, and therefore inspections should not pursue aims related to these process areas. However, due to the absence of an objective measure for output (there are no central examinations, for instance), these experts also admitted that inspectors inevitably have to take these process areas into consideration; however this should only be in order to judge whether or not these processes sufficiently guarantee that attainment targets and development goals are being achieved. These stated aims are therefore, according to these experts, not a goal as such, but an intermediate goal to reach a higher purpose, namely accountability and development with regard to the output of the school.

Stated Aims with Focus on the Education Process

In this section, it is important to note that four stated aims are considered by Flemish experts as being very important (see Table 11.1): an inspection should make a school account for, foster school development in the areas ‘teaching methods of the teaching team’, and ‘pupil assessment’. Regarding the stated intention to, ‘make schools account for pupil assessment’, two inspectors claimed that it is important to monitor how schools deal with the available tests, which is reflected in the maximal score in the quantitative data (4.00) given by the inspectors with regard to this purpose. Also the stated intentions (both accountability- and development-oriented) regarding ‘pupil counselling’ and ‘education to pupils with specific needs’ were, on average, considered as relatively important aims of an inspection. Both the interview data and the quantitative data show that the intention to ‘make schools account for the education of pupils with specific needs’ was strongly supported by experts from the category ‘school’—and particularly by the parents’ representative.

There was a remarkable difference between the importance attached to the two stated intentions with regard to the ‘teaching methods of the teaching team’ on the one hand (considered to be very important), and the two intentions regarding the ‘teaching methods of individual teachers’ (considered less important). The experts thought, on average, that the inspection should have less impact on individual teachers, and more impact on the teaching team as a whole. This particularly reflects the opinions of experts from the categories ‘counselling services’ and ‘policy-makers’, but also reflects those of the teachers’ and parents’ representatives. These experts argue that the evaluation of individual staff members falls within the remit of the school’s principal. If the school has a well-functioning personnel policy, it does not need an inspection to assess individual teacher performance. Furthermore, according to these experts, allowing inspectors to make statements about individual teachers would serve as a threat to the establishment of a successful internal mechanism of teacher appraisal. Moreover, the idea of an inspector about the performance of an individual teacher is considered to be too fragmented to allow a reliable judgement:

I wonder whether an inspection, or an inspector really has to observe individual teachers: to what extent can he get a view on a teacher in about one hour? A teacher that is suffering from stress because he’s being watched or questioned, so to speak? The inspector will never see the full picture, I reckon. (Parents’ representative)

In contrast, three Flemish experts (both the principals and the member of a school board) thought that inspection should pursue an important purpose concerning accountability and development with regard to the performances of individual teachers. From the point of view of the development-oriented purpose, these three experts think that teachers should receive individual feedback from the inspectors about their teaching performance, and that this would result in a larger positive development effect. Moreover, teachers have few chances to receive feedback from external professionals with educational expertise, so these experts felt that one should not allow this opportunity to pass. Regarding the accountability-oriented purpose, it was argued by these experts that it should be possible for the inspectors to comment on the performances of teachers that are blatantly substandard. The experts from the Flemish inspectorate take an intermediate position in this debate. They argue that teacher appraisal would benefit from an external contribution, but they doubt whether the Inspectorate is the most appropriate body to do this:

A secondary school principal who has never been taught Economics, goes into the classroom of the teacher in Economics in the final grade, and he can judge from his experience on the teacher’s classroom management, and on how the teacher deals with the pupils and whether this is constructive, and whether he works in the school’s line of thought, but whether the content told by the teacher is right or not, cannot be assessed by that principal. (…) Whether it is the inspection that should do it, or another body… by the way in which we have received our remit and how we are operating now, we clearly cannot do this… but indeed, I think it would be a good thing if that gap is addressed. (Inspector A)

The quantitative data in Table 11.1 indicate that the experts from the category ‘inspection’, on average, attach lower importance particularly to the accountability-oriented purpose, but also to the development-oriented purpose with regard to the teaching of individual teachers. This may be explained by the strong conviction amongst inspectors that the inspection should not pursue judgements about the pedagogical qualities of the school.

A school consists of teachers, a school is made by teachers. I think we absolutely need to take care that we [inspectors] give teachers the feeling that we are not judging them as a person, that we are not drafting a report about ‘you as a person are doing well, or not so well’ (Inspector C)

Finally, the accountability-oriented, and particularly the development-oriented aims with regard to ‘parental involvement’ and ‘school and classroom climate’ were judged to be less important. Especially related to the stated intentions with regard to ‘parental involvement’ the opinions of different experts strongly diverged: according to several experts from the category ‘school’ (and, not surprisingly, pre-eminently the parents’ representative), both stated aims regarding this school area should be considered to be highly important, whereas inspectors attached lower importance to these aims.

Stated Aims with Focus on Personnel Matters

Inspections in the Flemish educational context pay a great deal of attention to the professionalization of the school team. Our data show that this practice is supported by the expert panel. The quantitative data indicate that inspectors attach even greater value to these stated aims than other experts do. The lowest score for this purpose was given by experts from the category ‘school’, which may be explained by the idea of these experts that the realization of these stated aims depends strongly on the stance taken by the school’s policy.

In an ideal world, there is an evaluation interview, during which the principal takes the inspection report, and he says [to the teacher], ‘you have a weak score on this and that area, and these may be opportunities for improvement. Let’s take a look, I’m taking my book of professionalization activities, what would be trajectories or courses that would be useful to have you join them?’ (Principal B)

Furthermore, experts attached particular importance to the stated aims ‘to make schools account for, and to foster development regarding the self-conception of the principal’s job’ and ‘to make schools account for their personnel policy’, as desirable aims of an inspection. The development-oriented stated aim regarding personnel policy was considered to be less important: “Remarks regarding the leadership and personnel policy are less effective as it mostly concerns tenured (statutory) teachers” (School counsellor C).

Also other stated aims that had been derived from the first research phase (‘make schools account for, and foster development regarding the education degrees of the staff’) were supported by the expert panel but only to a lesser extent.

Stated aims with focus on the school policy. Each of the school areas of the stated aims discussed in this section is related to the schools’ policy-making capacities. No consensus was reached in the discussion on whether or not inspections should pursue the goal to make schools account for their policy-making capacities or to contribute to development of their policy-making capacities as a result of an inspection. Two experts from the category ‘counselling services’ indicated in the first and second research phase that these are not valid aims for an inspection. Their opinion was based on schools’ autonomy to determine their policy, and on the (in their view) small consensus regarding what should be understood as ‘policy-making capacities’. Moreover, according to these experts there is a certain danger that policy-making capacities are quantified when they are judged by an inspection, which would inevitably bring an impoverishment to this complex concept. In contrast, the other experts thought that the policy-making capacities of the schools are such an essential component of the quality of the school, that an inspection should not overlook this school area. The third research phase showed that the two school counsellors who stood out in the first and second research phases, also valued the stated aims regarding policy-related school areas in spite of their earlier reticence.

The most important stated aims with regard to school policy, are that the inspection ‘makes schools account for, and contributes to development regarding the internal quality assurance within the school’. The legislative regulation that an inspection needs to control whether or not a school monitors its own quality, is therefore broadly supported: “Because I do really believe that whether or not a school is capable of shaping its process of internal quality assurance, that is the most important thing for a school” (School counsellor B).

Other broadly-supported stated aims concerned the accountability for, and development of the school’s educational policy: the orientation towards shared goals, the consistent approach towards school policy, the school team as a professional learning community and the innovative capacity of the school. Most of the experts also confirmed that the inspection should have a stated aim regarding accountability for, and development of the internal communication mechanisms, but this does not apply to the schools’ external communication. Particularly regarding the stated aim ‘to foster the development of external communication’ it is remarkable that experts from the category ‘inspection’ thought that this stated aim carries a lower importance for an inspection, compared to experts from the categories ‘counselling services’ or ‘school’. Furthermore, six other stated aims were found to be of medium importance, namely the accountability for and development of the school’s reflective capacity, shared leadership, and responsive capacity. In most cases very specific reasons underlay the slightly lower scores assigned in the third research phase for these stated aims. For instance, school counsellor A argued that in some cases, the best way for a principal to act is to take (temporarily) a very directive leadership style, which is not complementary to the stated aim ‘to make schools account for, and to foster development related to shared leadership’. Finally, the experts generally thought that it is not a desirable purpose for inspections to make schools account for or to foster development regarding the school team’s perception of the school’s policy.

Output-Related Stated Aims

Seven school areas discerned with regard to school output; this implies that fourteen stated aims were formulated. We distinguished between stated aims regarding the pupils’ learning output, and other output-related stated aims.

Stated aims regarding pupils’ learning outcomes. There was a strong consensus between the Flemish experts that an inspection should make a school account for, as well as foster development regarding the attainment of learning outcomes. The accountability-oriented purpose regarding these learning outcomes should be-along with the above-mentioned accountability-oriented aims regarding the school infrastructure—the most prominent stated aim of an inspection. “The remit of an inspection is the monitoring of the quality in terms of control of the pupils’ results and the compliance of legislative regulations” (School counsellor A).

As a result of the data obtained during the first research phase, the stated aims regarding pupils’ learning outcomes were divided over two different school areas, namely the pursuit/achievement of attainment targets and development goals, and the achievement of the goals determined by the curricula. This is related to the Flemish educational context, in which the government sets minimal learning outcomes (attainment targets and development goals). Due to their pedagogical autonomy, schools may broaden these minimal learning outcomes in the curricula they use. The data obtained in the second and third research phases reveal that the stated aims related to achieving the attainment targets and development goals are, on average, considered to be slightly more important. This is explained by democratic support for these attainment targets and development goals, but also by the idea that each school should be judged by the same standards. Nevertheless, different arguments were also raised to support the importance of the stated aims regarding achieving the goals set by the curriculum:

Because in the end, well (…) also these parents choose that particular school (…) because of the curriculum and the pedagogical ideas, so in that sense, I think as an authority and as Inspectorate, you are assigned to check whether these goals set by the curriculum are achieved. (Policy-maker B)

Notably, there was only a consensus in the category of experts from the ‘inspection’ (who were in favour of ‘accountability and development of achieving the attainment targets and development goals’), whereas in each of the other categories there were proponents of both viewpoints. However, several experts indicated that the difference between both output measures is rather small: “In my opinion, this is a merely hypothetical discussion. I do not understand this discussion” (Policy-maker A).

Other output-related stated aims. The four main stated aims of inspection considered important by the expert panel with regard to other output-related school areas, were ‘to make schools account for, and foster development regarding pupil well-being, and regarding the educational progress of pupils (limitation of grade retention and alignment with further education/labour market)’. The quantitative data additionally indicate that the stated aims regarding ‘accountability for and development in staff satisfaction and principal satisfaction’ were considered to be slightly less important, while the stated aims ‘making the school account for, and foster the development of the school image’ were, on average, not considered to be important.

Conclusion and Discussion

In the past two decades, several descriptive studies have been conducted to examine the impact of inspections. However, only little is known about the aims inspections should pursue. The merit of the present study lies in the fact that this question has been addressed for the first time. With an explorative approach, this Delphi study has resulted in a comprehensive overview of general and stated aims that inspections in the Flemish educational context should pursue. In so doing we departed from the idea that these aims are determined by the shared expectations of different stakeholders in the field of education (Faubert 2009; Harrington and Harrington 1994).

We conclude that, according to expert stakeholders, the Flemish inspection system generally needs to address three aims, which are in line with the international literature on this subject: an accountability-oriented purpose, a development-oriented purpose, and a policy-informing purpose. However, our results bring some nuances to this general idea: (1) the accountability-oriented purpose reflects mainly “contractual accountability” and “public accountability” (towards authorities and parents/pupils, respectively) and only to a lesser extent “professional accountability” (towards internal and other stakeholders) (IIEP 2011); (2) the development-oriented purpose only minimally includes that the inspection stimulates staff members’ reflection about their school, and that a stronger insight into the quality of their functioning results from the inspection (a ‘conceptual’ operationalisation of the development-oriented purpose). No consensus could be reached about an instrumental operationalisation of the development-oriented purpose, which would imply that inspections are explicitly challenged to foster schools’ engagement in improvement activities. Underlying the lack of consensus with regard to this issue, is—amongst other reasons—the idea that an instrumental operationalisation would imply that inspectors would provide concrete recommendations for improvement. Due to the strict distinction in Flanders between the role of school inspectors on the one hand, and school counsellors on the other hand, this is currently not the case. Moreover, it was considered to be undesirable by several experts on our panel.

The accountability-oriented and development-oriented general aims are further operationalised by the different school areas to which these aims apply. The small number of respondents impels us to treat the quantitative data cautiously, but nevertheless it may be concluded that the following three stated aims are considered to be the most important aims of an inspection: (1) to make the school account for the hygiene, safety and habitation conditions of the infrastructure; (2) to make schools account for the pursuit/achievement of the attainment targets and development goals; and (3) to contribute to development with regard to the pursuit/achievement of these attainment targets and development goals. Other major stated aims should be that the inspection makes schools account for and fosters development regarding the teaching of teaching team, pupil assessment, professionalisation of the school team and internal quality assurance.

Not every stated aim was considered to be important, and we could not reach consensus about the importance of each one. Due to the explorative nature of this study, we did not intend to provide a fully-completed set of aims supported by every expert; rather, we aimed to explore the field, to identify different aims and to assess the importance attached to them. Some notable differences deserve some attention here. It is evident from our data that the experts from the category, ‘school counsellors’, are more inclined to advocate points of view that deviate from the points of view of others. This concerns the abovementioned operationalisation of the inspections’ development-oriented aim, and the question of whether inspections explicitly need to assess schools’ policy-making capacities. It is probably not coincidental that each of these issues is closely related to the roles assigned to inspectors on the one hand, and to school counsellors on the other (the distinction between both is more carefully guarded by the school counsellors compared to other stakeholders), and to the interpretation of the constitutional principle of ‘freedom of education’. Furthermore it is notable that the experts mainly advocating a more guiding and advisory role for inspections were those closely acquainted with day-to-day practice. Finally, experts related to school management (principals and school board) deviated in their opinions on the desirable function of inspections with regard to teacher evaluation. It may be that these experts feel that the current mechanisms for teacher appraisal insufficiently guarantee that an effective staff policy can be conducted. In those cases where no consensus could be reached, the present study sheds light on the various arguments used by the different experts.

The results of the present study strongly correspond with current policy and practice regarding inspections in Flanders. Indeed, inspection strongly emphasizes the control of compliance with regulations regarding the hygiene, safety and habitation conditions of the infrastructure. Additionally, the control of the schools’ mechanisms for internal quality assurance is explicitly mentioned by the Decree regarding the Quality of Education as being one of the inspection’s tasks. Moreover, some of the school areas mentioned by the experts closely align with the CIPO-framework used by the inspection, and school areas such as the professionalization of the school team and pupil assessment are very often selected by inspectors as part of the inspection focus. This close alignment could be interpreted as a strong correspondence between current policy and practice on the one hand, and the expectations formulated by different experts regarding the desirable aims of inspections on the other. However, another possible explanation is that these expectations are (partly and implicitly) influenced by the current policy and practice of inspections. Particularly with regard to the expectations formulated by inspectors (who work within this current context) and by policy-makers (who share responsibility for making the current regulatory framework), but also with regard to other experts, such an influence of the current policy and practice on their expectations is not inconceivable.

Since Flanders (in 1991) was granted the authority to establish an independent education system, taking into account the constitutional principle of ‘Freedom of education’, this principle have always been interpreted by policy-makers and other stakeholders in a very strict manner. Schools are therefore entitled to set up their own curriculum, on condition that it is aligned with the attainment targets, set by the Flemish government. Because of the strict interpretation of the constitution by Flemish policy-makers the ‘authorities’—represented by the Inspectorate—were traditionally entitled only to make a judgement on whether the school can show to a sufficient degree that its students are achieving these attainment targets. However, due to the absence of standardized tests or exit exams—another policy choice that is embedded in the strict interpretation of ‘Freedom of education’—this entails by definition a task that depends to a large degree on inspector discretion. Inspectors also need to refrain from giving constructive advice to schools.

In recent days, we see that some of these policy choices have become point of discussion. Since 2009, schools are obliged to have some kind of internal quality assurance in place, and inspectors can control whether school comply with that obligation. This is a first step towards accountability for the processes in the school, rather than merely the outcomes—this study shows that this idea is supported by different stakeholders. Also the implementation of standardized tests is currently debated on, amongst others influenced by OECD recommendations (Shewbridge et al. 2011). However, granting inspectors a role that allows them to take a more advising or counselling role to contribute stronger to school development, still seems ‘a bridge too far’ for policy-makers (and particularly for the umbrella organisations of education providers), in spite of empirical evidence that shows that it would strengthen the positive impact of inspections on Flemish schools (e.g. Penninckx et al. 2014).

In order to be able to realize the aims of an inspection, further research is required to explore the extent to which the accountability-oriented aim can go together with the development-oriented one, as well as the conditions under which the co-existence of both may be feasible. Although it is evident from this Delphi study that the experts perceive only minor difficulties with reconciling the accountability-oriented and development-oriented inspection aims, further research into the combination of both functions seems to be warranted.

As elaborated in the conceptual framework, this co-existence has often been problematised, because (amongst other reasons) it is assumed that schools will not completely open up for an external evaluation, particularly if the inspector uses this power to take high-stakes decisions about the schools with potentially far-reaching consequences for the school image, pupil input and financing. In the words of Swaffield and Macbeath (2005) “external inspection and honest disclosure by schools are unlikely bedfellows” (p. 242). Further research could, for example, examine how a structural and directed cooperation between inspectors and school counsellors could be established in order to maximize the extent to which both the accountability- and development-oriented aims are addressed, without any of these partners losing their individuality. Without a doubt, an important issue in this study would be the operationalisation of the autonomy granted to schools. It is evident from the present study that this operationalisation is crucial for how one defines the role and the aims of an inspection.

Notwithstanding the embeddedness of these arguments in the Flemish educational context, the arguments mentioned in this chapter may also apply in several other educational contexts. The present study has both an academic, as well as a policy-oriented benefit. Due to the application of the Delphi technique this study contributes to the identification of the desirable aims of inspections, whilst also clarifying different the arguments of different experts which support these aims. The findings from this study create a sound basis for future descriptive and explanative research into the effects of inspections. At policy level, the results from this study offer guidelines to the Inspectorate and to policy-makers to make the aims of inspections explicit and to communicate openly about them. This study could also contribute to the robustness of the preparations made by schools and by school counsellors when given notification of a planned inspection.

Given the explorative nature of this study, the results obtained should be subject to further validation and refinement, because expectations and opinions are susceptible to changes, depending on new developments within and without the field of education. It should also be noted that these results are embedded in the Flemish educational context and culture. A critical stance towards these results should be taken when applying them to other educational contexts and cultures. In our opinion, the quest for the aims of inspection should be fundamentally addressed in every educational context, and this Delphi study may provide both methodological as well as substantive inspiration for this purpose.