1 Introduction

The research explained in this paper is about a Western Australian (WA) case study of a school inspection unit working inside the WA state government’s Department of Education (DoE). The Expert Review Group (ERG) is the official name for this small unit, and it is staffed with school inspectors. Since its establishment in 2008, ERG inspectors have conducted investigations and site inspections on more than 200 specially targeted pre-K to year 12 government schools in WA.

The powers of the ERG are comprehensive, as ERG inspectors have legislated powers to inspect all aspects of government schools’ operations. These aspects include:

  • Student academic achievement

  • School culture and community well-being

  • Organisational management and administration

It is timely to research the ERG. This is because, since its establishment a decade ago, the ERG has not been subject to public account. No performance indicators, reports or audits of ERG operations have been released from the DoE into the public domain, and there have been few mentions of the ERG at state parliamentary level. Also, the ERG has not been much discussed or debated in trade or academic journals.

This lack of information about the ERG is a problem because we do not know if the inspection processes the ERG uses are fit for purpose and achieving its important mission. This mission is to improve and hold to account the government schools it has inspected and judged (DoE 2008). So, in an attempt to begin to bridge this knowledge gap, my preliminary research questions sought to find out:

  1. 1.

    What is the ERG, and what is its history?

  2. 2.

    How does the ERG carry out government school inspections in WA?

Once I had found the data to answer research questions 1 and 2, I then analysed that data to try to answer the final, and most important, research question for this case study:

  1. 3.

    Do ERG inspection processes lead to schools improving?

In the next section of this paper, I start by offering context for the case study by comparing school inspection policies and processes from around the globe. I then justify my methodological approach and outline the research processes I used to collect data on the ERG. After that, I present my findings that uncovered information about:

  • The roots and development of the ERG

  • The quantity and type of ERG inspections from 2008 to 2016

  • How schools are targeted for inspection

  • ERG inspection processes.

In the final section of the paper, I consider whether the ERG’s school inspection processes have helped WA government schools improve academically, administratively and socially. The paper ends without a firm yes or no answer to research question 3, but it does offer some qualified remarks and a number of suggestions for further research that may help keep the ERG accountable.

2 School inspection policies

The rationale for what type of school inspection policy an education system creates can be conceptualised by understanding what assumptions held political sway at the time the policy and framework documents were written and then implemented. These assumptions are colourfully explained in a “not too far-fetched metaphor” by Gilbert (2012). It regards the UK’s education system, but may hold true for WA:

On the one hand you have a gung-ho approach driven by the fact that if you scratch the surface of any teacher you will find a complacent slacker who needs his (sic) arse kicked in order to do a half decent job for those poor children not lucky enough to get an independent education. On the other hand you have people … who knows (sic) that education is not about (although does involve) grades and league tables. Instead improving education is about helping teachers plan lessons that genuinely engage learners in the process of learning as well as developing skills and competencies that will help them thrive in the 21st-century workplace. (p. Foreword)

If Gilbert’s first approach holds sway, then inspection policies may mandate frameworks designed to be administered by inspectors who are external from the school that is being inspected. They are also likely to be compliance based and perhaps even punitive (Hutchings 2015). These frameworks have a ‘one-size-fits-most’ philosophy and judge schools from a top-down perspective, requiring conformity to a shared vision and purpose of education.

If the second approach is adopted, then something like what Hopkins (2007) calls ‘intelligent accountability’ may underpin the framework design. This is where inspection policies are written so inspections are still externally administered but that external judgement is counter-balanced by using internal-to-the-school review processes too. Inspection and accountability processes under this philosophical regime are supposed to enable trust in school staff, balance the judgement of external reports with internal school evaluations, and allow for the use of testing measures designed to be broad, varied and non-standard enough to encourage the development of every student (Cowie et al. 2007).

If neither approach is adopted, then it is most likely the school system is one of international rarity. This is because there are very few documented cases of school systems anywhere in the world which are either not subjected to external inspection policies at all, or where schools are trusted to conduct internal inspection processes and self-evaluations only (Webb et al. 1998).

Externally administered school inspection policies are found around the world in most education systems. The arguably most researched inspection system in academia today is the UK’s externally administered school inspection policies developed by the Office for Standards in Education, Children’s Services and Skills—better known by its acronym Ofsted (Baxter and Clarke 2013). Ofsted inspects and regulates all educational “services that care for children and young people, and services providing education and skills for learners of all ages” (https://www.gov.uk/government/organisations/ofsted/about). The early work of Ehren and Visscher (2006, 2008) suggested positive results for external school inspection frameworks. The proviso was that if inspected schools had “provision of feedback about weaknesses, the assessment of these weak points as unsatisfactory, and the agreements between an inspector and the school regarding improvement activities” then these types of inspection in the UK “do appear to make a difference in promoting school improvement” (2008, p. 205).

There are externally administered policies of school inspection in many other nations too, and they show a varied nature. In China, school inspection units recently acquired punitive powers so they could fine school leaders for inadequate performance (Du and Zhang 2013). In Sweden, lawyers are used as education inspectors (Baxter and Clarke 2013). In Spain, school inspectors have their own union, perhaps because the inspectors can be punished for irregular administrative evaluations (Monje Herrero and Onega Onega 2006, p. 21). Scotland’s school review model inspired several states in Australia enough that they copied the Scottish model for their inspection structures (Cuttance et al. 1998; Hoffman 1994).

Externally administered policies of school inspection are not the only way to hold schools to account for the way they spend public funding and how much their children learn. As is typical in many other areas of educational debate and practice, Finland is an outlier that does not adopt external or hierarchical school inspection protocols. Finland uses internal school reviews, which are conducted by stakeholders and are only for stakeholders’ approval in each school (Exley 2014; Kimber and Ehrich 2011). Finnish educator, author and scholar, Pasi Sahlberg, views external inspection systems as being a part of the Global Education Reform Movement (GERM), which he disparages as a global orthodoxy of standardisation, narrow curricula, low-risk pedagogies, managerialism and test-based accountability—all of which stifle creativity, critical thinking skills and school community well-being (Sahlberg 2012). Whilst an international outlier on inspection philosophy, it is worth noting that Finland’s education sector remains in the highest of rankings in international league tables that measure comparative education quality and standards.

Very recently, Ehren and Shackleton (2016) described a trend where school systems are turning to what they term ‘risk-based school inspections’. This is where inspectors go to “high-risk schools for visits, while schools with satisfactory student attainment levels are excluded from inspections” (p. 299). They have drawn a link between the onset of risk-based school inspection models and the economic austerity period in Europe, and elsewhere, during the second decade of the 21st century. Risk-based inspection models are still external in nature but can be quantified as more efficient, and thus more cost-effective, than models where all schools are inspected. Risk-based school inspection has also attracted criticism. De Wolf and Janssens (2007) warned that, when school inspections come from a starting point of specific risks, this may bias the judgement of inspectors as they come to a school with preconceived notions instead of going to observe a school’s performance with an objective and open mind-set.

During the research process, I learned that the ERG was one of the first inspection units designed using this newer proportional, risk-based inspection model. While the ERG can inspect any government school in WA, it does not inspect all of them and only inspects schools they consider extraordinary. Research question 2 considered how the ERG chooses whether a school is or is not extraordinary, as such an important decision deserves to be unpacked and critiqued.

2.1 A critical education policy methodology

The methodology chosen for this research was critical education policy analysis because, as Diem and Young (2015) explain, it enables a researcher to “engage in critique, interrogate the policy process, and the epistemological roots of policy work, examine the players in the policy process, and reveal policy constructions” (p. 841).

Policy is made due to human intentions (Prunty 1984), and this research started from the supposition that the dominant groups’ intention at the time of the policy’s creation is key to understanding the ERG’s axiological underpinnings.

In the large, government controlled, bureaucratically organised social apparatus of public education, policy is the expression of the prevailing political groupings and political ideas of what should be the scope, form and nature of education. Policies are explicit and implicit statements of value. In defining directions, allocating resources, including and excluding segments of society, ordering priorities, and setting goals, policies encapsulate a notion of what ought to be, according to the particular values of the formulating body. (Grimley 1986, p. 19)

School inspection policies have both an explicit and implicit set of values. By examining the ERG through a critical policy framework, I align with a

growing community within the educational leadership and educational policy fields, who are … seeking to … critique … the contextual nuances and complexities of the policy processes” alongside “scholars [who] are questioning the players involved in the policy process, how policies are constructed, and the role of power and voice. (Diem and Young 2015, p. 846)

Rata suggests that a critical education policy methodology can occur in three stages:

These are: firstly, employing a political economy theoretical framework that identifies causal links between global forces and local developments; secondly, analysing educational policy within that theoretically conceptualised context; and thirdly, undertaking empirical studies to examine educational phenomena located within the larger context of political and economic forces and policies. (2014, p. 347)

This paper is set out in a manner that is influenced by Rata’s three stages. The previous section contextualised the ERG within the international research literature regarding school inspection policies. It did not look for causal links, but sought to contextualise the ERG’s local development within the global forces affecting school inspection theorising. In the “Findings” section, I present an analysis of the educational policies that created and sustain the ERG. And in the next section, I outline my empirical research process for examining the ERG phenomenon.

3 The research process

As justified and explained in the above sections, my research is underpinned by a critical education policy methodology and seeks to better understand school inspection processes in Western Australia. My research questions were open ended and preliminary:

  • What is the Expert Review Group (ERG), and what is its history?

  • How does the ERG carry out government school reviews in WA?

  • Do ERG processes lead to schools improving?

After receiving my university’s ethics approval,Footnote 1 research commenced using an iterative process. I used iteration because:

The role of iteration, not as a repetitive mechanical task but as a deeply reflexive process, is key to sparking insight and developing meaning. Reflexive iteration is at the heart of visiting and revisiting the data and connecting them with emerging insights, progressively leading to refined focus and understandings. (Srivastava and Hopwood 2009, p. 77)

The starting point of my critical education policy analysis was a document search. I searched for any official document that might answer as many of the research questions as possible. In the end, the reviewed documents included: government legislation and acts, DoE reports, media articles, academic papers and ERG reports. The search was Australian focussed, and the keywords used included: Western Australia, education, accountability, inspection, Expert Review Group, school review and Department of Education.Footnote 2

An extensive literature review found very little academic research mentioning the ERG and none which focussed on its activities. Of note was a PhD thesis on school accountability in the WA government school sector which briefly mentioned the ERG and provided good context reading (Duggan 2009). The search did find a number of historical reports detailing quite contentious political debates that led up to the establishment of the ERG (e.g. Barcan 2011; Cuttance et al. 1998; Misich 1996).

The key quantitative data on the ERG were found in DoE annual reports (e.g. O’Neill 2016). Buried deep within the pages of each year’s account to the parliament by the Director General of Education about the state of affairs in her department, and under sub-headings slightly different in each of the yearly data sets, are paragraphs that declare the number and types of ERG school review processes that had been undertaken in the previous financial year. These data are displayed in Table 1.

Table 1 Quantity and type of ERG school reviews from 2009 to 2016
Full size table

A treasure of key qualitative data was found within ERG inspectors’ school reports from 2009 to 2017. The executive summaries (from schools of concern) and full reports (from exemplary schools) provided vivid, descriptive vignettes that were crucial to the viability of this research project. Through methodical searching and cross-checking, information was eventually found on 40 of the schools which have been reviewed between 2009 and 2017 by the ERG. I would argue that collectively the 40 ERG reports represented the most vivid and informative data set for this research project, and I was able to access most of the reports from the publicly accessible DoE website Schools Onlinehttps://www.det.wa.edu.au/schoolsonline/home.do.

Extra documents were also sourced from traditional and internet-based media stories, and some information was also garnered from Google searching using the keywords ‘ERG’ and ‘WA’. The results found a few sites from parent and community groups, plus parent and/or teacher blogs that mentioned the ERG in their postings, and this searching helped me to find quite a few of the 40 schools that I have evidence of being subject to ERG inspection in the last decade.

When each document was acquired, all of its data were mined, analysed and coded into categories which aligned to my research questions. The duration of this iterative analysis process continued and deepened as time went on—even up to the point now when I am writing this final draft after this publication’s reviewers have offered me feedback on earlier drafts. The time taken to visit and revisit the data on multiple occasions has allowed for me to engage in continuous meaning-making, to be reflexive and progressively focussed on emerging data, and eventually, this has led to my refined understandings of the Expert Review Group, which I am presenting and outlining on these pages.

4 Findings

In this section, I set out the results of my research investigation in relation to the first two of my research questions. To answer question 1, I explain the roots and development of the ERG and then detail the quantity and types of inspections they have carried out from 2008 to 2017. Question 2’s findings reveal how the ERG decides which schools are targeted for extraordinary inspection and then outlines the entire inspection process and protocols.

4.1 Roots and development of the Expert Review Group

In the late 20th century in WA, district superintendents (ex-principals promoted up the bureaucratic hierarchy) were in charge of inspecting the schools in each of their respective districts. District superintendents, later renamed district directors, were line managers to the principals in their district, and school inspection processes were tied to principals’ performance management. Thus, inspectorate power was held at the middle level of the regularly renamed education agency (e.g. Central Board, Ministry, Department), and metrics gathered to provide evidence of school effectiveness were localised, contextualised and variable in quality.

The beginning of the 21st century saw an entrenchment of “a neoliberal era premised on markets and new managerialism” (Blackmore 2011, p. 443). In WA, a new school inspection policy was being created against a backdrop of increasing market competition, measurement and standardisation in schools (Apple 2006; Biesta 2010). The new school inspection policy was developed at a time when the WA School Education Act (1999) was updated:

S.176 (1) The Minister may authorise persons to inspect registered schools (p. 126)…

S.178 Any person authorised under subsection (1) may, on 7 days’ notice enter and inspect any premises of a registered school; and require the production of records (including student records) and documents relating to the school and inspect or take copies of them; and require any person to give assistance reasonably necessary for the exercise of the powers conferred by this section. (p. 127)

The intent of the legislation is to ensure that government schools are well run and that this is confirmed through an external quality inspection process. As Grimley (1986) reminds us, policy made from legislation can be a statement of implicit assumptions, and in WA there has been a long-standing assumption that schools cannot be left alone to judge whether they are functioning well without external accountability mechanisms in place to ensure compliance and quality delivery of school services.

By 2007, the new policy came into operation. Regional educational directors (REDs) superseded district directors. Crucially, these REDs lost inspectorial responsibilities, and this role was handed over to the newly established Expert Review Group—the ERG of central focus to this study.

The ERG is first mentioned in the official DoE policy document, the School Improvement and Accountability Framework (SIAF). The SIAF policy mandated that,

Every school is required to seek continual improvement and account for its performance … Principals, in collaboration with school staff, are required to participate in and actively respond to, as required, reviews conducted by the Expert Review Group. (DoE 2008, p. 2)

The rationale for the then-new ERG policy is evident in the DoE statement still quoted on its public website page today (http://www.det.wa.edu.au/schoolsonline/expert_review_report.do?schoolID=4185&pageID=AD19):

The Expert Review Group has been established to strengthen the Department of Education’s accountability processes. As an independent body, the Expert Review Group provides authoritative interpretations on school performance. Any work commissioned for the Expert Review Group by the Director General, Department of Education, results in a report which is presented to the school.

Despite the above quote claiming that the ERG is ‘an independent body’, this is disingenuous. Granted, the ERG team are staff who are external to the school being reviewed, but they are internal staff of the DoE. Organisationally, the ERG sits within the DoE’s central organisation and is staffed by a small group of inspectors, who are line managed by the Executive Director, Innovation, Performance and Research, who answers directly to the Director General. The small team of staff in the ERG are bureaucrats usually promoted to the position because they had previous roles as highly experienced DoE principals. When necessary, other expert DoE principals are utilised using short secondment periods to supplement this core team in the conduct of school inspections. This means that the ERG is not a truly independent body, as the staff who are responsible for judging a school system get paid by that very school system.

4.2 Quantity and type of ERG inspections

The reports found within the DoE’s Schools Online portal laid bare the processes the ERG inspectors used to review each school and assess its performance. It also revealed that the ERG inspects schools for two basic reasons:

  1. 1.

    Because a school is at risk and shows evidence of academic achievement measurement, or other indicators, performing badly over a prolonged period

  2. 2.

    Because a school is showing evidence of achieving well beyond expectations, in academic achievement and other indicators, and has potential for role modelling extraordinary improvement to other schools

The data outlined in Table 1 were found inside a decade’s worth of DoE annual reports. It reveals details about the quantity and various types of ERG inspections that have been visited upon government schools in WA targeted as being ‘of concern’ or ‘exemplary’.

The information articulated in Table 1 shows that far more schools have been reviewed by the ERG because of concerns raised about schools’ performance (196 schools) as compared with the relatively few reviews conducted by the ERG which case study exemplary schools which have reached extraordinarily high achievement targets (28 schools).

Table 1 also outlines that ERG inspectors made 242 follow-up visits to all schools of concern to check on each school’s progress with implementing their mandated recommendations.

Table 1 also suggests that the work of the ERG has expanded over time. In its first financial year, 2008–2009, the ERG inspectors undertook 17 performance reviews on schools of concern and got through following up 7 of those schools 6 months later. By 2016, ERG inspection types included three types of inspections for schools of concern:

  1. 1.

    Performance reviews a whole school inspection which evaluates all aspects of the school community, including its academic performance, organisational effectiveness, financial health and community well-being

  2. 2.

    Incident enquiries a very specifically focussed inspection that targets an extraordinary incident that needs urgent evaluation

  3. 3.

    Performance enquiries a smaller inspection than the performance review, because this format only evaluates one aspect of school operations, which is usually the academic performance criteria

There are also three types of inspections for schools considered to possibly be exemplary:

  1. 4.

    Performance reviews a whole school inspection which evaluates all aspects of the school community to see if and how the school is exemplary

  2. 5.

    Performance enquiry a very specifically focussed inspection that targets one exemplary aspect in a school

  3. 6.

    Performance studies of IPS the newest type of inspection, which only targets exemplary independent public schools.

The complexity of types of inspections makes it difficult and more complex to analyse the data, but it is reflective of WA’s complex and somewhat convoluted government schools categorisations. The newer ERG inspection types have, in part, come about due to a structural change, begun in 2009, when the government sector of education in WA became two tiered (Franklyn 2016). There are now two types of WA government schools. Government schools in the newest category are, confusingly, termed independent public schools (IPS). IPS are the ‘elite’ schools in the government sector which competed to win the title and

assume the authority to make decisions related to curriculum, student support, human resources, recruitment and selection, payroll, financial management and building and facilities, while centralisation largely remains in such areas as policy and strategic direction, performance monitoring and measurement, and curriculum. (Gobby 2013, p. 19)

Then, there are all the other government schools that have been left to function in the ‘old way’, where DoE’s Central Office maintains all those authorities. All IPS schools, from their inception, were regulated to undergo external inspection once a year by another review group staffed outside of the DoE in what, until late 2017, was named the Department of Educational Services. The comprehensive and regular nature of those reviews may have seemed to negate the need for the ERG to conduct any extraordinary inspections on IPS. From 2012, however, the ERG started to inspect these schools in the exemplary schools category, and for the first time in 2015, the ERG’s data led them to inspect an IPS targeted as showing extraordinarily concerning performance. In other words, some higher-tier ‘successful’ schools, which gained IPS status, have since proven not to be, or have deteriorated away from still being considered, successful.

On a final note, an interesting change in the reporting of ERG inspections occurred in DoE’s 2016–2017 annual report (which was published too late to be included as Table 1 data). Fewer details were included, with only this one paragraph referring to any ERG statistics:

In 2016–17, our Expert Review Group conducted reviews of 12 schools where performance was of concern. School leaders were supported to implement improvement strategies that focused on whole-school approaches and unified teaching practice delivery. In addition, three studies were conducted where exemplary practice was exhibited, with findings available to all school leaders. (O’Neill 2017, p. 35)

These numbers represent a drop in schools-of-concern inspections, but there is no mention of what type of reviews were undertaken. Also, no mention is made of any follow-up visits during the 2016–2017 financial year, and no information is given regarding whether any of the schools-of-concern were in the IPS category.

4.3 How the ERG decides which schools to target for extraordinary review

WA’s government school system now has standardised annual academic test results for each school, and these data provide the first warning trigger for the ERG to consider investigating a school.

From at least 2008 until 2016, DoE’s student test results and other school-wide performance data were evaluated by the ERG via a three-range colour-coded formula, affectionately known as the ‘traffic light schema’. Yellow schools were seen as performing to the expected level of achievement if they were situated within 1.0 standard deviation below or above the expected academic standard. Schools 1.0 below that standard deviation were coloured red and labelled below expectations, while schools 1.0 above the standard deviation were coloured green and labelled as above expectations. Table 2 shows the standards that were applied and also adds the percentage of schools located in each of the three standards in the schema until late 2017.

Table 2 Original measurement tool of WA schools’ performance
Full size table

In 2017, the DoE devised a more refined schema by adding two more levels of performance and compliance standards—well above and well below expectations. This effectively increased by half the required standard deviation expectations for schools from a 1.0 standard deviation range to now having to stay within 0.5 of the standard deviation if a school is to remain within the minimum expected level (yellow). Table 3 illustrates the new schema and also adds the percentage of schools the DoE projects will now be likely located within the five-standards schema.

Table 3 New measurement tool of WA schools’ performance
Full size table

This up-to-date schema does not alter the data sets which the ERG uses to assess schools, but it does refine the way the ERG can pinpoint which schools to inspect. From now on, it is likely that the ERG will only inspect schools labelled as well below (bright red) and well above (dark green) expected levels of performance and achievement.

As senior bureaucrats in the DoE hierarchy, ERG staff have access to all of the data sets articulated in Table 4. After the ERG has been alerted to a school that is performing extraordinarily, according to the new schema, they can then examine other standardised data about the school and decide if an inspection process needs to be initiated.

Table 4 Sources of information available to ERG inspectors when evaluating which schools to target
Full size table

As Table 4 indicates, there is an extensive amount of data that ERG staff use when they decide which government school to target for inspection. The ERG was designed as a proportional, risk-based inspection unit (Ehren and Shackleton 2016), and given that this type of inspection model is supposed to be cost effective, the capacity of the ERG to conduct a desktop audit of any government school prior to initiating an inspection process may be a very good thing. This is because, as the following subsection will show, when a school does go through an ERG inspection, the process takes a lot of time and resources.

5 ERG inspections

ERG inspection processes follow six phases (SSTUWA 2012; R. Lowther and C. Cunningham, personal communications, 17 April 2015).

Phase one: Initiation The Director General (DG) determines which schools to inspect, on the advice of tenured ERG staff who have found concerning or promising measurements against criteria in the available DoE data on the targeted school.

Phase two: Preparation This involves schools being informed they will be inspected and seeking secondment for complementary DoE school leaders to be a part of a team of up to five ERG inspectors. Then there is a pre-inspection visit to the school by the ERG team leader to prepare the principal, who has to facilitate the next phase in the process.

Phase three: School engagement During the ERG inspection team’s on-site visitation of the targeted school, the team investigates various lines of enquiry via interviews, meetings and focus groups.

The school stakeholders who are invited to participate in the ERG inspection usually include the following:

  • The principal

  • The previous substantive principal (if very recent)

  • At least one deputy principal

  • Multiple teachers

  • Representatives of the teacher’s union

  • Education assistants (EAs) and/or Aboriginal and Islander educational officers (AIEOs)

  • Parents representing the Parents and Citizens Association (P&C) and School Council/Board

  • Student Council representatives

Phase four: Writing the report The ERG teams author and edit the report. A final draft is submitted to a deputy director general schools case conference for consideration and approval. The report then goes to the DG for ultimate endorsement.

The final school report follows a template formula in this order:

  • Executive summary

  • Context

  • Performance data

  • Findings

  • Narrative

  • Prescribed improvement strategies

  • Conclusion

The targeted school’s executive stakeholders are allowed to view the final draft prior to publication, but as long as the reported information is factual, they have no capacity to change its content, especially not its recommendations. The executive summary is uploaded for public access onto the Schools Online website at the end of this phase too.

Phase five: Recommendations implemented The school executive is tasked with implementing the recommendations set out in the ERG team of inspector’s written school report. The ERG team and the school’s regional education director (RED) are expected to support the school as it tries to implement each recommendation. In the executive summary reports I have read, no offers of extra funding or resourcing were mentioned for this phase of the process.

Phase six: Revisiting the school Follow-up visits to the school, after varying numbers of months, occur if the type of ERG inspection is for a school of extraordinary concern. Then progress is reported back to the DG. The ERG inspection process ultimately ends at the 24 month period. At that time, the school conducts a self-assessment against the ERG report’s original findings. This assessment is reviewed by the RED and ERG staff. Feedback is provided to the school, then the DG is informed and the process officially ends.

If the school has been inspected for exemplary performance, in this final phase, the DG visits the school personally and conducts a media event on-site to promote the school’s quality to the wider community in WA. Figure 1 offers an example of a typical, favourable community newspaper article and photo. This media event seems to unofficially end the inspection process for an exemplary school.

Fig. 1
figure 1

Favourable local paper story about an ERG inspection of an exemplary school. http://www.pressreader.com/australia/canning-times/20160216/282144995401306

Full size image

6 Inspection to improvement?

In the previous pages of findings, I articulated what the Expert Review Group is and what form of school inspection they utilise in their operations. Now, it is time to discuss what I have learned from asking the third research question, which asked if ERG inspection processes lead to schools improving?

The short answer is that this case study could not answer that question adequately. The data I gathered, analysed and extrapolated from do not find any direct links or demonstrations of whether an ERG inspection process leads directly to schools improving. This does not mean that there is a possibility that ERG inspections do lead to schools improving in the areas that an ERG report defines, but it does mean that improving is a nebulous, quite unquantifiable term and a difficult measure to benchmark. Nonetheless, the DoE uses improvement as official jargon to justify the work of the ERG, for example (https://www.education.wa.edu.au/documents/2548175/2664299/School+Accountability.pdf/b2b8e31a-03db-4ba7-b697-250b3301890c, with italics added):

The Expert Review Group will also prescribe strategies directed at improving school performance. Based on this feedback the principal will be responsible for developing and implementing an improvement plan.

However, because the final, internal school report that is created at the end of a 24-month ERG inspection process is not released publicly, we do not yet know if the ERG’s prescribed, directed strategies actually result in improved performance by the targeted, inspected school.

What I have ascertained through considering research question 3, and unpacking what I was trying to elicit from asking the question, is that it is important to know if the ERG’s work can be considered a success or not. In the DoE’s 2012–2015 strategic plan, its priority 4 asserts that the DoE should be “using resources wisely and making open and transparent decisions to build community confidence in public education” (O’Neill 2016, p. 44). So, in the final pages of this paper, I offer my thoughts on how I see the ERG has helped the government school system and what could be done to build communities’ confidence that the ERG is helping schools improve, as its mission intends.

6.1 Role modelling

The ERG model of school inspection is relatively unique because of its inspection of schools with exemplary performance in some or all aspects of its operations. My analysis of the exemplary ERG reports suggests that they have attained what the DoE asserted, “These reviews will help us learn about and spread excellent practice more widely throughout the public school system” (O’Neill 2016).

Over time, the ERG has compiled enough information from multiple school sites to now know what the most common features of leadership and pedagogic practices in a school that has reached exemplary school performance benchmarks are. These include:

  • A school-wide approach to maintaining consistently high standards and a common direction

  • The principal applies expert instructional leadership skills and strategies

  • A calm, orderly and safe learning environment

  • A welcoming and nurturing ambience, with a sense of connectedness among students, parents and staff

  • Regards families as genuine partners

  • Civic values are embedded in the fabric of daily school activity

  • Well-established processes for delivering targeted services to students at educational risk

  • Astute management of resources

  • Keeps pace with advances in technology that enrich the classroom learning environment

  • Highly structured collaborative teacher team meetings

Also, the ERG writes many inspection reports for schools deemed to be of concern. By examining that data set, it is also now possible to gain some insight into what school attributes the ERG have found in multiple underperforming schools, for example:

  • Performance in 18 of the 20 assessment areas was below the expected means

  • Systemic student performance data are not valued, understood or embedded in planning for teaching

  • The view that, because student performance lies generally within the expected range, there is no obligation for the school to perform at a higher level

  • Teaching practices do not deliver an authentic, timely, challenging and personalised curriculum

  • The absence of shared beliefs and understandings relating to teaching and learning

  • Leadership has little understanding of the fundamentals of change management, no capacity to prescribe actions and no will to ensure that expectations have been met

  • The absence of a strategic plan in recent years has led to a lack of defined curriculum direction

  • A cohesive and united school executive team with the collective responsibility and professional obligation of all its members does not exist

These attributes are stark and worrisome, but they do provide a lesson in reverse, about what can go wrong in a school and perhaps how to avoid those problems.

6.2 Accountability

Since the ERG has been in operation, all government schools in WA have had to be prepared for the possibility of facing an ERG inspection, despite only a relatively small group of schools having actually been inspected by an ERG team. This, theoretically, means that all school principals are ‘kept on their toes’ by the always overhanging threat of ‘being ERGed’. Whether this is a positive contribution that the ERG has delivered, or not, depends on one’s educational philosophy. Certainly, the DoE places great value in external accountability mechanisms, but critical education theorists and unionists are scathing of some of the intended, and unintended, results; For example, in 2016, the WA state schools’ teachers union, the SSTUWA, raised these concerns about the ERG:

… the actual process as it is currently conducted can actually weaken the school leader’s credibility and capacity to lead an improvement agenda post ERG. This is because a public report, which only contains the school’s failings, works to undermine the principal in his or her efforts to address the identified issues. The report is generally silent on the system support necessary to improve the school; almost formulaic in the prescribed improvements and produces no final public report to say the school is now considered to have met the prescribed improvement threshold. (Byrne 2016, n.d.)

6.3 Access and transparency

There are two reasons why it is important for the general public to know much more about the workings of the ERG. The first is because the DoE’s School Improvement and Accountability Framework (SIAF) mandates that “a summary of (each ERG) report will be made publicly available” (2008, p. 13). I can only assume that the decision to mandate such transparency was based on the democratic principles of open government. This leads to the second reason why access to information about how the ERG does its work is needed. Diem et al. (2014) argue, “We should be constantly asking questions: is this the way it has to be; what’s the value of doing it this way; how are people hurt by this; what are the alternatives?” (p. 1076). If we know how the ERG functions then we can better determine the value in the way the ERG inspects schools.

My experience during the 3 years since I started this case study suggests that the ERG is a unit extremely reticent to provide access and transparency to its data, despite its public announcements in support of such values. The following vignette provides but one example of DoE gatekeeping mechanisms used to discourage public access.

The SIAF mandated that ERG reports be made public but did not clarify for how long and in what form a summary report must be made publicly available. So ERG staff have been left to interpret this mandate. The ERG, and DoE executive staff more generally, seem to have interpreted the SIAF rule as narrowly as possible.

The first caveat they have imposed is that only recent reports are publicly available, because reports are usually uploaded onto the website for only 6 months duration. Access to ERG report PDFs is also difficult to find, because a person searching for a report must first know the name of the school which has had the ERG intervention. This is the ‘password’ into accessing the publicly available reports. The names of which schools have, or are having, an ERG inspection is closely guarded information kept within the DoE ERG staff—I know because I asked for the information on repeated occasions but was not granted access to their lists.

If a member of the public knows the name of the school and attempts to find the report within that 6-month time limit, the search is still quite a tricky process. To actually find an ERG executive summary report, a member of the public has to perform four mouse clicks onto various internal tabs found from within the front page of the DoE’s website Schools Onlinehttp://www.det.wa.edu.au/schoolsonline/home.do in order to access the PDF version of an actual executive summary.

Whether this provisional access to ERG reports is a good or bad thing, again, depends on one’s educational and ethical perspectives. On the one hand, access to information provides lessons to be learned; on the other hand, could access to these reports provide opportunities for shaming and blaming schools?

6.4 Repercussions

When a school is judged as of-concern or exemplary by the ERG, this is likely to influence a school’s perception of itself as a winner or loser within the policy framework.

When a school earns an exemplary performance review by an ERG team, it becomes a policy winner. The school receives a highly flattering report about the school’s outstanding examples of academic performance. Comments about excellent community well-being can be quoted in school marketing material and used in selection criteria when competing for grants, school awards and staff promotions. This is just one example of a quotable section of an exemplary school review from August 2015:

High expectations of staff and student performance accompanied by a strong work ethic have resulted in outstanding student achievement. Effective teaching is undertaken by deeply passionate, committed and caring teachers learning from, and supported by, their colleagues in an environment of healthy collaboration. (Hunt et al. 2015)

When a school is judged to be of concern by an ERG team, there are consequences for being a policy loser. Schools that have undergone ERG inspections for educational performance concerns have been targeted in the media on too frequent occasions. Always keen for a bad news headline, some of these schools have found themselves on the front page of state newspapers and national online websites fronting brutal headlines, for example:

  • Perth Online (Phillips 2012): The best and worst of WA schools

  • ABC News Online (Shine 2016): [Name of] school violence: Damning report finds ‘inadequate leadership’

Hypothetically, school policy losers could become policy winners if the school manages to accomplish all of their ERG-prescribed improvement strategies. However, it is hard to imagine how success could be reached in many of these schools’ circumstances without extra support, as many of the schools that have undergone a concerning ERG school inspection are in communities that are below (and often well below) average on the index of community socio-educational advantage scale.

6.5 Extra support

The schools of concern that the ERG have reported on over the last decade display entrenched, systematic disadvantage. If such schools are to improve in any way, they need extra support to achieve their improvement strategies. This support should include extra time, funding and human resourcing. In the 40 ERG reports that I read, none of the reports offered extra money, staffing or time commitments for any of the schools.

Recent data suggest a possibly hopeful, new form of support. In the DoE’s annual report for 2015–2016, a small paragraph notes:

A Quality Teaching Support Team was formed at the start of 2016 to support principals and school leaders to address teaching and processes related to substandard performance. The team had formal engagement with 24 schools. (p. 44)

Then, the 2016–2017 annual report mentions:

Our Quality Teaching Support Team, formed in 2016, continued to work in partnership with schools to assist school leaders to manage processes related to the unsatisfactory performance of teachers. During 2016–17, the team worked with 51 schools, supporting 114 school leaders. (p. 39)

Although the annual reports do not directly link the Quality Teaching Support Teams (QTST) with the ERG, it might be that these QTSTs have been deployed to assist ERG schools-of-concern after they receive an ERG inspection. Although I am not yet certain, the creation of the QTST may also be the reason the 2016–2017 annual report includes no follow-up visits to schools by ERG teams, and it may be that the QTST have replaced that ERG service.

7 Conclusion

This paper documents a research project which investigated the Expert Review Group (ERG). Since scant information about the ERG had been on the public record, this paper now gives the reader an understanding of how the ERG was created and how it functions today.

The ERG is functioning as it was designed, and there is value in the ERG holding government schools to a high standard and in helping to develop and nuance what those standards are. As a well-functioning and high-performing bureaucratic entity, the ERG has significantly altered the way WA government schools are held to account for their academic, behavioural and administrative performance. The ERG has helped to define what a highly functional school in WA is and has also uncovered what sort of problems can hold a school back from achieving its potential. The ERG identifies schools to use as role models for others, and this may allow for a flow of exemplary practice to spread across the school sector.

However, the ERG is not a perfect school inspection entity, and this paper also describes its controversies. ERG inspection processes have led to media shaming of schools. The ERG has normalised a culture of external, standardised measurement which emphasises academic performance accountability throughout WA’s government school sector. This may, in turn, have helped lead to a privileging of learning areas (literacy and numeracy) that are tested across Australia in mandated, standardised formats. As Sahlberg (2012) warned, this could lead to schools being confined and restrained by test-based accountability processes. In the future, the ERG may need to emphasise in its criteria for exemplary schooling strategies that ensure schools do not stifle creative pursuits, critical thinking skills and all those learning areas that foster the talents and skills of students that are so difficult to evaluate via a multiple choice, online test.

Finally, this paper concludes with a recommendation that further research be carried out on the ERG. It would be edifying to do research that seeks out the opinions of staff and other school community stakeholders who have been directly targeted for ERG inspection and to know about their lived experience of the process. It would also be enlightening if a researcher could gain access to the end of ERG inspection process reports and determine whether they reveal whether ERG inspections actually improve school performance.