Introduction

In an era where standardised assessment has been linked to accountability and reshaped teachers’ work (Pastore, 2023), this paper explores the practice of social moderation as an opportunity for teachers to exercise agency in building their own assessment literacy. Moderation, as a social process, involves teachers collectively reviewing awarded grades against predetermined criteria and standards to achieve scoring consistency (Maxwell, 2021; Smaill, 2020). Moderation, when it involves an interactive dialogic process, is claimed to build teacher assessment capabilities (Smaill, 2020), assessment literacy (Popham, 2011; Stiggins, 1995) and inform next-step teaching practices (Wyatt-Smith & Gunn, 2009).

This paper contributes to understandings of (1) how official and unofficial texts coordinate Australian school teacher assessment and moderation practices and (2) how teacher involvement in these practices can inform their ongoing assessment and teaching work. Informed by the theoretical work of critical feminist scholar Dorothy E Smith, the paper investigates ‘how things are actually put together’, in other words, ‘how it works’ (Smith, 2006, p. 1), with respect to teacher assessment and moderation practices. Smith’s sociological approach draws attention to the power of texts to organise people’s daily activities in modern society (Smith, 2005). In this approach, a text is defined as any instrument of communication (e.g. data on a spreadsheet, a form, a guideline, a poster, a policy document). During moderation meetings, teachers draw on multiple texts (e.g. curriculum, standards, student work, assessment policy, supporting frameworks) to make decisions about the quality of student work (Clark, 2015).

The paper first presents the background of the policy and broader research context. We explain how the theoretical contributions of Smith’s work have informed the analysis of data. The research methods are then presented followed by the findings and discussion. We illustrate the powerful influence of policy texts to shape, organise and maintain teachers’ moderation work as well as related aspects of their everyday teaching and assessment practices. We also show how activities from the broader research project, including the use of commentaries that explicated their judgement decisions (cognitive commentaries), reoriented teachers’ work for improvement and accountability purposes.

Background

Australian national and state policy and related resources outline expectations of teachers for assessment, moderation and reporting of student progress. The Australian Education Regulation (AER; Commonwealth of Australia, 2013) requires teachers to ‘give an accurate and objective assessment of the student’s progress and achievement… relative to the performance of the student’s peer group’ (s. 59 AER, 2013). Furthermore, teachers are required to report student learning progress using a 5-point scale or equivalent for each learning area ‘clearly defined against specific learning standards’ (s. 59 AER, 2013). The Australian Professional Standards for Teachers identify moderation activities to ensure teachers ‘make consistent and comparable judgements of student work’ (Standard 5.3; Australian Institute for Teaching & School Leadership, 2017).

In Queensland, the AER is enacted through legislative procedures, which include mention of the role of moderation ‘to ensure comparability across the State and at recognised schools, of the assessing teachers’ judgments in deciding results’ (Queensland Government, 2014, p. 62). The expectation of moderation is that regardless of school geographic location, there is consistency of teacher judgements when assessing student work and that these decisions are collaborative. The assessment policy of the Queensland Department of Education (2022b) states that teachers in state schools ‘use a whole school approach to moderation processes to align curriculum, pedagogy, assessment and reporting; and to ensure consistent judgments and accurate reporting against the achievement standards’ (p. 5). Supporting materials include suggested curriculum, assessments and scoring rubrics (e.g. Queensland Department of Education, 2022a: Curriculum to the classroom [C2C]); examples of graded student work; and guides for moderation practices.

Research context

Data for this paper are part of an Australian Research Council Linkage Project (2019–2023), which investigated the development and use of scaled exemplars and associated commentaries of teacher judgement decisions to support ongoing teacher judgements. The project focussed on the middle years of schooling, specifically Years 4, 6 and 8 (student ages 9 years to 13 years), in the discipline areas of English narrative writing, mathematics and science investigations and religious education. The study involved 187 teachers from two Australian states, Queensland and Western Australia. All project activities were accessed online. Of note is that the project occurred during the period of COVID lockdowns and the resultant intensification of teacher work as it shifted to online modes of delivery and heightened absenteeism of staff and students.

There were five project stages in which teachers (1) submitted classroom assessments, (2) judged student performances using a pairwise comparison process (see Humphry et al., 2023), (3) matched scaled samples/performances with A–E standard descriptors, (4) wrote cognitive commentaries, that is, descriptions of their thinking of how they made their judgement decisions and identified next-step teaching strategies (Wyatt-Smith & Bridges, 2008), and met online to discuss and negotiate final wording of the collated commentaries, and (5) trialled using the exemplars with the associated cognitive commentaries to moderate grading of their own classroom assessments.

This paper focusses on the transcribed dialogue from the Stage 4 online meetings that involved the Queensland teachers (n = 40). In these meetings, teachers discussed and negotiated the final wording of the English, mathematics and science cognitive commentaries that would go forward with the associated scaled student work samples (exemplars) to be trialled in the final stage of the project. A descriptive cognitive commentary captures a teacher’s thinking about the strengths and weaknesses in a performance, and decision-making about how these features came together in the final grade (Adie & Wyatt-Smith, 2021; Smith, C., 1989, 1995). In this study, the commentary also identified next-step teaching strategies to progress learning for the student. Teachers wrote individual cognitive commentaries for selected student work samples that had been scaled through the pairwise and standard setting prior stages (e.g. Year 4 science, B grade). The following questions guided construction of the commentaries:

  1. 1.

    What knowledge and skills are being demonstrated in the performance?

  2. 2.

    What are the areas for improvement in the performance?

  3. 3.

    Based on your assessment, what is the on-balance or overall grade you would award?

    1. a.

      What compensations have been applied in reaching the overall grade?

    2. b.

      How have the strengths and weaknesses in the performance been combined to reach an overall grade?

  4. 4.

    What would be the next-step teaching required to progress learning for this student?

    1. a.

      What resources could be drawn on to support this learning?

The individual commentaries were submitted to the research team and combined into composite documents for each discipline, year level and grade. In the online meetings, teachers discussed points of disagreement identified in collating the individual commentaries, the evidence of sequential progression within the A–E commentaries, the representation of the standard in the commentary, the discipline appropriate wording/terminology to describe a specific standard and the usefulness of the commentaries in their teaching and assessment practice. Teachers also discussed their local assessment and moderation practices.

Theoretical framework

The research presented in this paper is informed by Smith’s (1999) sociological approach, institutional ethnography, which begins from the ontological position that individual experiential accounts of people matter. Smith (2005) has argued that research inquiries should begin by talking to people at the frontline of institutions, in this case, teachers. Accordingly, we approached the research from the position that (1) people are experts in how their own lives are lived, (2) actions (happenings) are located in sites throughout society and (3) these actions occur in similar ways across different locations within the institution (Deveau, 2009; Smith, 1987). An institution, in this sociological approach, is understood as a cluster of arrangements that direct attention to ‘intersecting work processes’ (DeVault & McCoy, 2006, p. 17) that occur across multiple locations and specific functions, such as education. Work processes are maintained by ideological codes that ‘order and organise texts across discursive sites’ (Smith, 1999, p. 158), such that texts are reproduced or talked about in similar ways and not requiring explanation.

Research informed by an institutional ethnographic approach aims to discover how things work, often by researching how key institutional texts coordinate activity (Smith, 1997, 2005). Smith (2008) has argued that detailed forensic research begins with experiential accounts of people to reveal patterns which show how things happen (Smith, 1997, 2008). Smith’s (2005) own research highlighted that the textual organisation of happenings can be complex, and often hidden in the everyday. Understanding how peoples’ work is actioned in similar ways across locations can show how texts operate to coordinate work across different sites (Smith, 2005). An institutional ethnographic approach is apposite for this paper given that much of what happens within schools is connected to texts which maintain the broader institutional structures and organisation of education. For example, teachers are located in vastly different schools, yet typically implement the same texts related to assessment, such as curricula, policy, planning documents, procedures and guidelines. How these texts coordinate and organise teachers’ curriculum and assessment work is not always obvious.

A text, as a communication device, shapes and is shaped by a metaphoric institution (Smith, 2006). Often this occurs as a text-act-text sequence in which texts activate actions which then produce new texts. By this, Smith captures the idea that a text needs someone who reads it and then takes action if they are to influence what happens in institutions. For Smith (2005), ‘as a reader activates a text, she or he engages with its language and also respond[s] to it’ (p. 104). Often, this activation leads to the creation of subsequent texts. For example, a school principal reading a policy that has been sent by a departmental supervisor might respond by emailing teaching staff asking them to comply with policy expectations. This research approach has been used to investigate a range of educational initiatives and policy outcomes. For example, Comber (2012), whose research was also informed by Smith’s theoretical contributions, described the multitude of texts related to the National Assessment Program – Literacy and Numeracy (NAPLAN) as: ‘handbooks, test kits, markers’ rubrics and so on – [that unleash] complex chains of action and new economies’ (p. 15). Others have used an institutional ethnography approach to investigate (1) how data culture and national testing in schools organised teacher’s work (Spina, 2020); how literacy assessment and accountability requirements were textually authorised activities (Kerkham & Nixon, 2014); the impact of newly introduced regulations on early childhood educators’ work (Grant et al., 2017); and the impact on teachers’ work on newly introduced funding policy for students with disability (Gallagher, 2022). This paper uses the analytic tools of institutional ethnography to understand the textual organisation of teachers’ assessment and moderation work in similar ways across locations.

Method

The paper focusses on data collected during 14 online meetings with 40 primary and secondary school teachers which took place in the state of Queensland (Table 1) with ethics approval received from the Human Research Ethics committee of the Australian Catholic University (Ethics Approval Number: 2019-11H) and partner organisations. The purpose of these meetings was to examine and negotiate the wording of the collated A–E cognitive commentaries submitted for specific year levels and disciplines.

Table 1 Number of teachers and online meetings per year level and discipline
Full size table

Participants

Forty teachers participated in two online meetings, each being 150 min in duration. The aim of each meeting was to review the collated A–E commentaries for a specific year level and discipline (Table 1). A total of 14 meetings were held using the Microsoft Teams platform. All meetings were video recorded with the videos deleted after transcription of the meeting discussion, consistent with ethical requirements. Meetings were led by a member of the research team and teacher numbers were restricted to a maximum of five teachers to allow for all to contribute to the discussion. During the meetings, teachers were asked to (1) check that the intention of their comments in their submitted commentaries was represented in the collated version, (2) discuss points of difference in the collated commentaries and (3) ensure that the final commentary was representative of the associated year level and discipline standard.

Data collection

Data for this paper draw on the discussion transcripts from the 14 online meetings which comprised approximately 500 pages of meeting transcript. In these meetings, as teachers negotiated the wording for the final cognitive commentaries to go forward to the Stage 5 trial, they also discussed their school assessment and moderation processes. The teachers’ accounts provided insight into the documents they considered to be most useful in their review and assessment of student work, and how they were used to organise assessment and moderation work in schools across the state. From these accounts, a type of textual hierarchy was identified to show how texts mediate teacher moderation work.

Data analysis

The analytic aim of the data presented in this paper was to understand how teacher assessment and moderation work was textually organised. The data analysis drew on the qualitative analysis work of Braun and Clarke (2019) and three analytic tools commonly used in institutional ethnographies: indexing, mapping and writing accounts. Braun and Clarke (2019) describe qualitative thematic analysis as a ‘generic method’ that is ‘theoretically flexible’ and has the capacity to ‘reflect different assumptions about and orientations to, qualitative research’ (p. 592). A core premise of this type of qualitative analysis is that themes do not automatically emerge from data, but in fact these threads are an outcome of the researchers’ standpoints as they engage with the data to make sense and meaning. This means that as the researcher makes sense of the data, threads become visible, and relationships and connections between the threads become apparent.

The institutional ethnographic tools of indexing, mapping and writing accounts are used to investigate these relationships and understand how things happen beyond the official documentation, thereby revealing the workflow or sequences of action, the actualities of people’s everyday work. Individual accounts are collated for temporality to determine if activities are replicated over time and across locations, and to show how they spark further work and text-act-text sequences of action (Campbell & Gregor, 2004). These tools have been used by van Leent and Spina (2022) to explore teacher accounts of gender and sexuality representations in the primary school curriculum.

Indexing is ‘a way to organise data into linked practices and happenings to support an analytic view into the institution’ (Rankin, 2017, p. 5). Indexing the data shows how practices are linked across people and settings. In this study, indexing involved careful and consistent analysis of the descriptions of work actions, words or phrases and named texts to categorise the data. An example of indexing for this study is shown in Table 2.

Table 2 Indexing example—Year 6 mathematics
Full size table

Through indexing, relationships become visible, revealing how things happen, and where and by whom specific actions come to be activated (Koralesky et al., 2022). This process made teachers’ moderation work discoverable.

Indexing was followed by the analytical exercise of writing accounts to reveal institutional processes within people’s everyday experiences. The method of writing accounts is a way to engage with the transcript data and uncover analytic threads to understand how people’s work is socially organised (Rankin, 2017). The process of writing accounts involves selecting an initial quote from an interview, from which similar accounts from other participants are identified to build a detailed description of institutional processes. For example, beginning with one teacher’s account of limited experience with teaching and assessing science, similar accounts from teachers from different contexts (schools, year levels, discipline areas), who identified as early career, in remote locations or new to a discipline area revealed how their involvement in the project had supported their ongoing school-based assessment and moderation work. Quotes are identified in the Findings by the year level and the discipline area that teachers focussed on in the project, followed by sequential numbers to indicate different teachers within the same year level and discipline focus (e.g. Year 4 English teacher 1, Year 4 English teacher 2).

Mapping is an indexical tool that results in ‘analytic diagram(s)’ (Rankin, 2017, p. 5). The process of mapping is a synthesis of indexing and written accounts leading to a visual representation of complex (textual) systems (Turner, 2006) and intertextual connections. Through understanding how actions and texts are connected, the social relations of peoples’ everyday experiences are noticeable (Rankin, 2017; van Leent & Spina, 2022). These tools, when used together, identify sequences of action and the social processes.

Findings

As teachers shared accounts about their local moderation processes, similarities across school sites, year levels and disciplines were evident. Teachers identified key texts they used and how these texts informed their moderation work. They also talked about how the project activities had reoriented their work.

Three key findings were distilled from the teachers’ talk. First, the textual power of ‘hidden’ policy texts was notable in the coordination of teachers’ moderation work. These policy texts acted as powerful forces that coordinated teachers’ work across different schools, such that activities occurred in similar ways across multiple sites. The study shows that these types of texts had a standardising influence. Second, participation in the broader project activities reoriented the local work of teachers, as they applied their learning in various ways to address local contexts and their own pedagogical priorities. Finally, teachers noted the potential of the cognitive commentaries to enhance assessment literacy within their schools.

Policy texts coordinate moderation work

When discussing local moderation practices, teachers repeatedly featured the A–E reporting requirements. There was no mention of the AER which articulates this requirement; the AER was in the shadows, with no immediate visibility for teachers. Rather, this national requirement was embedded in state moderation resources and support materials. These included the C2C resources, mentioned earlier, the Australian Curriculum and the Queensland Curriculum and Assessment Authority (QCAA) marking guides. Teachers talked about how they used these resources to produce local marking guides to evaluate student work and give feedback.

Figure 1 shows the analytic mapping of this textual sequence. It commences with the rectangle shape containing the AER. The oval shapes present the texts teachers referred to when talking about moderation and how these were connected to each other. The final shape on the right maps how all the texts contributed to teacher curriculum and assessment planning.

Fig. 1
figure 1

Textual coordination of moderation

Full size image

The formal texts of the Australian Curriculum, C2C and QCAA resources contributed to the school-based marking guides, and these supported teacher planning. By mapping the text-act-text sequences, the textual power of the AER is apparent. While the map shows the connections between documents, importantly the sequence of actions commenced with the AER. The requirements of the AER were taken up and maintained through chains of texts without ever being named by the teachers. In other words, the AER had the textual power to coordinate teacher work from a distance. While not visible to teachers, the ideological codes of the AER (i.e. standards and reporting requirements) transcended school locations. This ideological code was maintained by teachers in their everyday work with a washback evident in their planning for a unit of work, including assessment.

The 5-point scale requirement of the AER was presented by all teachers as an A–E grading scale, despite not being mandated as such. When teachers were talking with colleagues from other schools, it was clear that the meaning of the A–E grading scale was consistent, it required no explanation. The A–E scale was incorporated into marking guides, and once embedded, teachers did not challenge its veracity. Instead, they shared details of their judicious application of the scale when evaluating student work and their reliance on the A–E descriptors when finalising decisions about the quality of student work. A teacher explained: ‘there is a really structured approach to creating, describing a range of performance on a marking guide and forming that A to E through a process that’s pretty rigid’ (Year 4 English teacher 1). The same teacher also shared the process for applying the A–E requirements: ‘I had the work sample and then I was highlighting on the marking guide with the A to E scale and then I was highlighting… evidence’. This structured approach to applying the A–E grading scale was replicated across all project year levels, disciplines and school sites. It was accepted without challenge and teachers expressed confidence in the integrity of the process.

Moderation preparation also followed a similar textual sequence across locations. The moderation sequence of actions was textually mediated by the A–E grading scale. Figure 2 shows the moderation sequence commencing with teachers planning the unit of work including the learning and teaching sequences and assessment activities in accord with the Australian Curriculum. Teachers annotated marking guides by discussing the meaning of the A–E standard descriptors and noting how this would appear in student work.

Fig. 2
figure 2

Moderation sequence

Full size image

To prepare for moderation, teachers were involved in a calibration activity in which they marked the same set of work samples and then met to discuss their grades. Through this process, teachers expected to develop a shared understanding of the A–E standards before commencing grading of their own class sets. It was anticipated that this would lead to minimal discrepancies in the moderation meeting.

Teachers selected A–E student work samples to present at the moderation meeting. This process continued the maintenance of the ideological code attached to the A–E grading scale. While moderation was a formal process for English and mathematics across all year levels, only secondary teachers spoke of formal moderation for science. When a formal moderation process was not in place for primary school science, teachers sought out colleagues to have informal moderation conversations about their grading decisions:

I would just go into him [teaching partner] and say, ‘Hey, not sure about this. This is what they’ve done. I’m kind of on a B/C, what do you think?’ So, it’s very informal… At our school, teaching partners are very open to a team aspect in helping you make those decisions. (Year 6 science teacher 1)

Even when formal moderation processes were in place, teachers sought individual advice from colleagues to validate grading decisions across year levels and learning disciplines: ‘We go through the marking guide together… those borderlines… We're currently doing that informal moderation’ (Year 4 mathematics teacher 1), and ‘the lady I always work with… “I’ve given this [grade to the] student… Can you just have a little look? What would you give them?”. Or sometimes I don’t even mark it down and go, “I’m really confused with this one”’ (Year 8 science teacher 1).

Teachers shared how working with colleagues to unpack the marking guide or to consult about grading decisions was an important collaborative activity. In this practice, the marking guide was a dominant text for teachers, serving to mediate moderation decisions, whether it was a formal or informal activity. For example, in the event of disagreement, teachers deferred to the marking guide to mediate the outcomes (Fig. 2). When there was no resolution among moderating colleagues, the student work sample passed on to a curriculum leader who also deferred to the marking guide and the Australian Curriculum to make a final decision. As shown in Fig. 2, the disagreement loop was essentially a replication of the process already undertaken but this time by a third party:

Every assessment in English, we have a moderation partner… [when] we can't come to an agreement, we would then go to [another teacher], who's still in our teaching team, and go, ‘What do you think?’. If he also can't agree with us, then we would take it to the HOD [Head of Department]. (Year 8 English teacher 1)

The text-act-text sequence of resolving moderation disagreements (Fig. 2), as it was explained, was considered by teachers to be a robust way of arriving at consensus. This sequence is again seen in the description by a Year 4 English teacher:

We have lively debate. We all come out pretty unscathed. We all know we’re looking at it from that really factual, no, this is what it says on the GTMJ [Guide to making judgements/marking guide], this is what the achievement standard is and when we’re looking really directly at that and we go into annotations, next to our GTMJs, we write next to each one where they’ve achieved it and how they’ve achieved it. There’s no arguing with what you can see. (Year 4 English teacher 2)

Not reaching agreement through the established moderation processes and disagreement loop appeared to be nonsensical to the teachers. When probed in the online meetings about what happens if they ‘ever come to the impasse, where you’ve just got to agree to disagree?’ (interviewer), one teacher eventually responded with, ‘But that’s where that pre-moderation is so important, isn’t it? To eliminate [disagreement]… right from the start’ (Year 4 English teacher 3). Disagreement was avoided because of the preparation work during planning meetings where teachers developed a shared understanding of expected A to E qualities within the targeted curriculum: ‘since we’ve really been digging into our before phase [planning]… after phases [teaching, grading, moderating and reporting] have just been going even smoother and a lot easier’ (Year 4 English teacher 2). The value of the preparation (before) phase was reiterated by another teacher: ‘If you invest the time in the before, after [is] dead easy… [we] used to have… the potential for conflict or robust conversation. Whereas putting in that before phase, you iron out all of those issues’ (Year 4 English teacher 3). The pre-moderation actions contributed to teachers’ perceptions of robustness of the process and the likelihood of agreement in decisions about quality when grading student work.

The teachers, across year levels and learning areas, presented as having pride in their collegial practices and how their moderation conversations occurred. The moderation processes described in each meeting did not elicit any further queries, alternative processes or challenge from other teachers participating in the meetings. This text-act-text sequence shows how policy and other official texts (the Australian Curriculum, the A–E standard descriptors, the annotated marking guides) have the capacity to organise teachers’ work (e.g. calibration process, grading, formal/information moderation, disagreement loop).

Teachers’ work is reoriented

Participation in the broader project changed how some teachers approached their local assessment and moderation work, and how these practices were shared with their colleagues to change school practices. The text-act-text sequence of creating cognitive commentaries for the project resulted in some teachers shifting their focus within their own planning to the design of their assessment tasks and how they allowed student performance at each level or standard to be demonstrated (Fig. 3).

Fig. 3
figure 3

Introducing the cognitive commentary

Full size image

Teachers shared that as part of their initial curriculum planning they began to be more evaluative of the assessment task and marking guide to ensure alignment between these texts. They were also checking to ensure students had authentic opportunities to demonstrate their learning. For example, did the marking guide articulate the expected knowledge against the achievement standards? As shown in Fig. 3, some teachers explained how they reworked the assessment task to ensure this alignment. The following talk segment was typical of responses:

We recently had a staff meeting where we were moderating… We looked at the actual task on paper and talked about the pros, the cons of the assessment piece. Not the student work, but the actual assessment piece, which was really enlightening to me… Now we’re going back and looking at our assessments. (Year 6 mathematics teacher 1)

This ‘front ending’ (Wyatt-Smith & Bridges, 2008, p. 55) of the assessment was a significant (and auxiliary) outcome of the broader project. Front ending assessment involves aligning curriculum, assessment, teaching and learning at the planning stage. Teachers described how, as part of their own curriculum planning, they reviewed the alignment between the assessment task and what was expected at each A–E grade point (Fig. 4). The project texts (e.g. the cognitive commentary questions) were immediately useful and practical in their everyday work. The processes of analysing student performances and writing a cognitive commentary of their decision-making to reach an on-balance grade triggered teachers to reconsider other assessment practices starting at the planning stage.

Fig. 4
figure 4

Front ending assessment

Full size image

Teachers commented that using the cognitive commentary questions in their planning meetings helped them differentiate between the standards:

We used it [the analysis process embedded in constructing a cognitive commentary] on our Term 1 maths assessment. It was wonderful. It helped us. It was the fact that we could actually identify the strengths of the tasks, was how we made a discerning judgement between the A and B, and we also used the strengths to support our As. (Year 6 mathematics teacher 1)

In a separate Year 6 mathematics online meeting, another teacher endorsed this use of the cognitive commentary questions:

We introduced it [the cognitive commentary] and we had a go at going through some pieces [student performances] and doing it [applying the analytic process] as a year level team. So, we did it across the school from prep to Grade 6… It covers your different ability levels. (Year 6 mathematics teacher 2)

Teacher professional learning extended to consideration of next-step teaching strategies and differentiated learning opportunities for students, reminding teachers of strategies to try, and prompting clearer feedback about learning. In the third Year 6 mathematics online meeting, the commentary questions prompted another school to refocus their parent–teacher interviews:

We're looking to that saying, ‘What are the strengths (mathematics, literacy, numeracy), areas of improvement and our next steps for teaching?’. So, conveying to the parents what we actually see next steps curriculum wise, not just, you know, social, emotional discussions. (Year 6 mathematics teacher 3)

The cognitive commentary questions provided a structure for conversations with parents that moved teachers away from generalised feedback about behaviour, to a more focussed conversation about learning.

In some cases, an overhaul of multiple aspects of curriculum planning was activated within whole school communities:

We took our assessment piece for this term and we had a grade, so [Years] 4, 5 and 6, and we looked at the different assessment pieces… in terms of accessibility… how they could access it and make it more manageable for the children to achieve. We looked at the marking criteria and aligning it with the curriculum… in our year level meetings… We’ve actually gone through every term… looking at… our units… and our assessments… ensuring that they are matching the curriculum. (Year 6 mathematics teacher 2)

While this school adopted and adapted project activities to improve their local assessment processes, the teacher also expressed concern about the time taken to complete this task:

I know from first-hand it’s onerous [the process of writing the cognitive commentary]. It does give you a lot of information… So, it’s a really good tool for assessment for learning, but I just worry about the time it takes. (Year 6 mathematics teacher 2)

However, other teachers noted that the extra time was well worth the effort:

It [the cognitive commentary process] is very straightforward… You can apply it and have it as a template… When you are working with another school... and they’ve done different tasks… we had nothing to compare because they did completely different things… I’ll definitely be stealing your template. (Year 6 science teacher 3)

Through the cognitive commentary process participants saw value in their assessment conversations in their local school sites. They shared their learning about assessments and insights into the quality features of student work with their school colleagues to then replicate the process in contextually significant ways. Actions were stimulated that were immediately productive and had value and utility in local contexts, strengthening the assessment community in the school. This had reach both within and across year levels.

Teacher assessment literacy

The textual power of the cognitive commentaries to organise teacher work was evident. Figure 5 maps how teachers used the collated commentaries and scaled exemplars from the project, along with the cognitive commentary guiding questions to mount inquiries into their school assessment and moderation activities. Working through the project activities (i.e. pairwise comparison of assessment tasks, matching tasks with standard descriptors, deep analysis of the scaled exemplars to produce cognitive commentaries and using the exemplars with their associated cognitive commentary to grade their own student work) developed teacher confidence in their grading decisions and understanding of the standards. As previously shown, when teacher learning from the project was shared with colleagues, teachers worked together to improve their local assessment processes and build assessment literacy within their schools. This was particularly evident for those teachers working out-of-field, in rural and remote locations, and teachers new to a year level, including graduate teachers.

Fig. 5
figure 5

Teacher application of the project

Full size image

As teachers described how they used the project tools, they also talked about how this built professional confidence: ‘I’m grateful for this opportunity so early in my career because I feel like I have the confidence… and also, the experience… I'm grateful for this process because I feel like I can actually stand on my own’ (Year 6 mathematics teacher 3). As an early career teacher, participating in the project had resulted in increased confidence about their competence with grading assessment tasks and justifying the awarded grades.

Teachers’ reflections on their project involvement and its application to their local assessment processes resulted in improved processes for planning assessment and moderation:

It was really beneficial to be able to talk these things out at the start of the term rather than coming to the end of the term and go… I forgot about this, or how am I supposed to achieve that if we haven't taught this. So, making sure that we've got all of our ducks in a row at the start. (Year 4 mathematics teacher 2)

Having ‘all of our ducks in a row at the start’ was considered ‘beneficial’ by this teacher to ensure they have both planned for and taught all the learning area content, and more importantly, they commenced teaching with shared understandings of expected standards.

The selected exemplars with cognitive commentaries were identified by some teachers as useful resources to build assessment literacy within their schools (e.g. teaching out-of-field):

We have so many new staff, I'm currently upskilling four staff in other fields other than English as well... I see this [the exemplars with cognitive commentaries] as being so valuable because it would take away so much of that additional upskilling time… Our new staff don't have the knowledge… so I'm seeing it as being super valuable. (Year 8 English teacher 2)

Additionally, the exemplars chosen to illustrate the standard were considered to be useful ‘if you are a new teacher to the year level or a new teacher to the school’ (Year 6 science teacher 2). For teachers in rural locations or the only teacher for a specific discipline and year level, having the online exemplars and commentaries built their confidence in their grading decisions as well as providing support to develop assessment literacy:

Our school network is over about a six-hour driving range… We don't do catchups with other schools or networking… As teacher in charge of Year 8 for English, I'm just finding this [access to the online exemplars and cognitive commentaries] really handy because it means that the judgements that the existing English teachers and I have been making, I feel like we are on track… I see this as being really valuable as a regional school in terms of if we're able to get this and have those exemplars there… Teachers can look at and go, okay, yes, we are on par with the state, because it is really hard for us to identify that. This is basically the only PD [professional development] I've done in three years because it's [online], and getting time off to do PD is difficult because it involves an overnight travel trip for most things. (Year 8 English teacher 2)

The project activities directly contributed to developing teachers’ assessment literacy. Being online meant that all teachers could access the wisdom of others. Teachers saw the deep analysis of performance and the associated activities as valuable as they adopted them to improve their local assessment processes and build assessment literacy across their teams.

Discussion

As teachers discussed the wording of the cognitive commentaries during the online project meetings, they described their local assessment processes and texts. The value of moderation processes was not questioned by teachers during the project; teachers demonstrated trust in the fidelity of the moderation processes being undertaken across schools to ensure consistency of teacher judgement. Previous research has demonstrated teachers’ trust in moderation processes for judgement reliability and how involvement can build their assessment knowledge and skills (Adie, 2013; Smaill, 2020; Wyatt-Smith & Bridges, 2008). The teachers’ talk illustrated how their engagement with cognitive commentaries supported them to meet the requirements in systemic policy documents while also supporting their own professional learning and practice. The chains of actions outlined in Fig. 6 provide a visual representation of moderation as presented by the participants in this project. The findings illustrate teacher maintenance of the ideological code of the AER which required the use of the 5-point scale for reporting student progress. While the AER was hidden from teachers, it mediated their formal and informal moderation practices.

Fig. 6
figure 6

Mapping moderation

Full size image

Also evident from this analysis was how teachers applied the project activities to their local contexts. In contrast to media representations of teachers’ work (Mockler, 2022), teachers in this project grasped the opportunity to enhance their local assessment practices. Project activities were adapted by teachers across locations, discipline areas and year levels to meet their local needs in practical and immediate ways. New knowledge was shared with colleagues to reorient teachers’ assessment and moderation work. The new practices, while embedded in policy, were driven by the teachers. This result was additional to the stated project outcomes. The application of project activities to local contexts is particularly noteworthy given that the project was conducted during the height of the COVID pandemic, which was a professionally challenging time for teachers.

The mapping provides insight into how assessment and moderation happens in teachers’ everyday work. By using teachers’ experiential accounts as the subject of analysis, we can illustrate the complex layering of teacher assessment work that is supported by a suite of integrated practices. Teachers described how project activities involving the deep analysis of student work caused them to think about and make more explicit connections to other aspects of their assessment practice, for example, the design of assessment tasks linked to teaching and learning sequences, the calibration activity and the focus on learning in parent–teacher interviews.

Conclusion

Moderation in this project, and indeed in these schools, was seen as an opportunity to extend teachers’ assessment learning and build shared understandings about expected characteristics of quality in student performance. The mapping of teacher assessment work provides an example for other systems and sites of education looking to engage in moderation, while also raising awareness of the ‘hidden’ ideological codes embedded within the assessment policy landscape. In addition, this work provides a basis for future research, to explore and extend moderation as a viable and valuable process for supporting teacher assessment literacy. Future work can investigate moderation, and similar teacher dialogue approaches, to further cultivate assessment cultures in schools that build shared understanding about quality and a common vocabulary to support student learning, and consistency in student grading. The type of analysis used in this paper could be further used to identify areas for improvement that is teacher driven.