1 Introduction

In this book we have probed some fundamental questions about the implementation of teaching performance assessments (TPAs) intended to leverage improvement in teaching quality. We have taken up the question: What does teacher education reform look like when evaluative expertise and issues of quality, evidence, and agency are placed centre stage? In the provocations we have heard the thinking of leading researchers and educators in response to a major question of our time: What are the social, digital and environmental counter narratives, the alternate responses, and the blind spots in education made apparent in the COVID-19 crisis that could be incorporated in thinking about and addressing issues for transforming teacher education?

While reforming teacher education has long been a contested topic, most would agree that the term quality is often used, though seldom defined. Its meaning remains elusive. In part, this can be explained through a sociocultural lens in which the act of assessing—determining quality—is an historical and contextual act, bounded by place and time, and interactions among people. For this reason, the quality of a teacher and teaching can and does vary significantly over time and place, reflecting different expectations and values. Quality is not a fixed measure.

A major challenge for us in contemporary times is to have a national conversation about teaching quality and the demonstration of professional competence at the point of completing a preparation program. Put simply, most people want to have confidence in the competence and capabilities of their child’s teacher. They want to know that their child’s teacher is setting their child up for success and that this is a realistic expectation. An expectation of professional competence is not particular to teaching: patients want to know they have a good doctor; litigants want to know they have the best lawyer. However, referring to Australia, until the introduction of TPAs, there was no approved exit or terminal assessment and no established standard intended as a common reference point or benchmark for gauging graduate readiness for classroom practice. Further, there was no quality assurance system for verifying a common standard of readiness across universities, or even across programs within universities.

Here we separate out standards applied to program preparation where regulatory authorities and universities can audit how standards have been addressed in teaching programs. While this can be a useful exercise to show points at which aspects of the standard are taught explicitly and even practised and assessed, it can be unhelpful when this is reduced to a technicist checklist. How standards are conceptualised—their nature and function—is of importance to avoid the reductive use of narrowly defined standards or aspects of practices that are assessed in isolation or even outside of actual practice.

Our focus is on the nature and function of the Graduate Teacher Performance Assessment (GTPA®)Footnote 1 and the accompanying standard to demonstrate professional competence. This marks a strengthened focus on standards as they are demonstrated both as inputs in program design and teaching, and standards evident in preservice teachers’ professional practice on completion of the program. The significance of the move to TPAs was that it opened the door to exploring authentic assessment of teaching competence in an actual classroom context at the point of completing a preparation program and that competence could be assessed against an established or agreed standard.

As we review the contributions of the chapters, we observe some rich portrayals of the experience of change as it shapes and reshapes ways of thinking about, speaking about, and doing teacher education. The chapters present insights into how the GTPA as a cultural disruptor has been taken up as a topic of inquiry through national collaboration in Australian teacher education. In some sense, the working through of change can be understood as a type of experiment in reform through collaboration and networks. The chapters lay out the intellectual, experiential and personal resources and related expertise brought to a new contested policy context in the form of TPAs. The accounts of teacher education renewal in the cameos of practice presented in the preceding chapters and our experience in the development and implementation of the GTPA provide contexts of practice that will be recognisable in many respects to international colleagues. We have used Australia as a case instance of the introduction of TPAs into initial teacher education and offer these accounts as of potential relevance beyond our site of origin.

The transformative potential of TPAs for discovery work as culture change is a recurring theme in this book. It has called forth new identities, new ways of working with and through technology, and new interactions within and across universities. Our critical enquiries into the culture and practice of teacher education have provided evidence of the power of collaborative thinking to confront historic practices and transform teacher education practice. We have incorporated learning from others outside teacher education to inform our understanding of a competence assessment and profession readiness. Thinking about how teacher educators can act with agency to give an account of their practice, inform policy, and respond to criticisms of the quality of teachers has led to the development of digital infrastructure. The latter has allowed the GTPA research to scale up, generating data that is useful for program review and renewal as well as the elements for ensuring sustainability of this change culture.

2 GTPA Longitudinal Studies in Initial Teacher Education

In 2021, seven years after the release of the TEMAG Report (Craven et al., 2014), we recognise that the accepted recommendation to introduce TPAs has the most potential for positive collaborative reform in teacher education. It is the best chance for the profession to reclaim accountability for the profession. This stance moves beyond the design of the assessment to include the collaborative work in producing large-scale data and the use of digital infrastructure to support this work. In the remainder of this chapter, we reflect on what we have learnt about sustainable culture change in teacher education, its complexities and challenges; and what we have learnt about the enabling conditions that may sustain culture change.

From the beginning, the GTPA was conceptualised as a cornerstone in a workforce study series, with two longitudinal studies underway. The first study, titled Establishing readiness in teacher education, has involved the design, validation, and standard-setting for the GTPA as a competence assessment (Adie & Wyatt-Smith, ; Wyatt-Smith & Adie, 2018; Wyatt-Smith et al., 2020) and related cross-institutional moderation. It is the focus of this book. The second study, titled Quality of initial teacher education through longitudinal analysis of linked datasets (Wyatt-Smith et al., 2021), examines characteristics of teacher education candidates and their trajectories through an education degree program from entry to exit as they progress through a series of hurdle assessments, including the GTPA. Also included are a national assessment of personal literacy and numeracy,Footnote 2 and school-based assessments of professional experience placements scheduled over the program of study (see Chap. 16 for further information on the longitudinal study).

For the study series, we recognised that going to scale (geographic spread and the number of participating universities) required the design and development of new digital architecture to examine the impact over time of the competence assessment on teacher education and its utility for determining graduate readiness. The term digital architecture (Fig. 17.1) is taken to refer to:

Fig. 17.1
figure 1

Digital architecture supporting judgement-making online and reporting in the workforce studies series

Full size image
  1. 1.

    The data infrastructure that was purpose-built. This includes a data warehouse for storing de-identified performance records against the common standard; an online scoring platform using the standard; and the GTPA application (app) customised for recording cohort performance at the criterion level.

  2. 2.

    The communication infrastructure that includes use of digital technologies to support online meetings and symposia for sharing learnings and reporting performance outcomes, and an online resource centre including samples that illustrate the required standard and materials used to calibrate judgement against a common standard.

Online cross-institutional moderation (CIM-Online™),Footnote 3 that incorporates the digital architecture, is an internationally distinctive feature of the work of the GTPA Collective (see Chap. 3). The teacher educators undertake calibration training and scoring, producing the data that form the basis for analyses and reporting on program quality and the application of the standard across universities. It is this data corpus that is used by the Institute for Learning Sciences and Teacher Education, Australian Catholic University, to prepare confidential, customised reports for each university. The reports present data showing performance scores at cohort level, drawing on demographic information about preservice teachers and program characteristics (for example, mode of delivery). The collection, collation and analysis of summative data reveals program quality both within and across programs. It also contributes to the policy intent of quality assuring graduates on entry to the workforce. Such quality assurance hinges on applying an established standard, recognised by the profession, and applied consistently across participating universities. For this reason, fidelity of implementation and maintaining the integrity of the assessment are essential pre-conditions for teacher educators, policy personnel, preservice teachers and regulatory authorities to have confidence in the fairness of assessment outcomes.

Beyond this policy intent, the use of digital architecture has offered the opportunity to break new ground. This involves using the performance data that was gathered initially for reporting the summative assessment result of profession readiness (Meets/Does not Meet), also for informing program review and improvement actions (formative purposes). The infrastructure is essential for teacher educators to access reports showing the effectiveness of programs in ways not previously available. They can see, for example, cohort and individual performance in planning, teaching and assessing as recognised core skills of professional practice. Using evidence, they also see the impact of programs and implementation approaches on preservice teacher competence. Data infrastructure and communication infrastructure promote assessment as collaborative inquiry through capacity building and partnerships in teacher education (Fig. 17.1).

The scale of the workforce studies, supported through the digital architecture, makes it possible to collect large-scale data and in turn ‘speak’ to policy using evidence. As suggested earlier, in Study One, the shift is away from TPAs as a compliance requirement, to the GTPA Collective of universities coming to work as a trusted group with expertise to use evidence and standards in new ways. In Study Two, the evidence produced makes it possible to identify the points of progression for particular candidates that present barriers to academic success. These have potential to allow more customised interventions at earlier points in time than was previously possible. The evidence also provides teacher educators with corroborating information to show the characteristics of candidates and the related probabilities of success in particular programs. The spotlight can be on mode of delivery, timing and combination of assessments, and program outcomes for cohorts of special interest, taking account of demographic variables. This has involved new ways of thinking, talking, doing and being for teacher educators, researchers and policy personnel who have worked as a multidisciplinary team.

3 Pre-conditions for Sustainable Culture Change in Teacher Education

In the following section, we identify a set of nine pre-conditions for establishing and sustaining an approach to culture change in teacher education. These are offered provisionally as an emergent conceptual framing to guide actions and decisions related to conceptualising, designing and implementing complex performance assessments. In presenting this we draw on our first-hand experiences as we have tried to show in this book (Fig. 17.2).

Fig. 17.2
figure 2

Pre-conditions for sustainable culture change in teacher education

Full size image

The pre-conditions include:

  1. 1.

    A critical inquiry approach to the design and implementation of TPAs. Data in and of itself is only useful once its meaning is interrogated and determined to be fit-for-purpose. A critical inquiry approach involves interrogation of empirical research literature to inform TPA design, validation processes and implementation decisions. It also involves ongoing quantitative and qualitative analyses of data collected from the scoring of TPAs for continual reflection on, and refinement of, TPA design and implementation processes. Critical inquiry takes into consideration site-specific variables of TPA implementation and intended uses of the data and related reports.

  2. 2.

    Rigorous processes for demonstrating validity and reliability of TPAs including the scoring rubric. Professional acceptance of a TPA hinges on the demonstrated alignment of the TPA and relevant professional and program standards. Also essential is agreement about conditions necessary for fidelity of implementation, protocols for resource and data collection, and procedures for effective communication. Validity is determined through close qualitative and quantitative analyses of the expected characteristics of competence that the assessment calls forth and also evidence from implementation in the field. Trials are an essential condition for TPAs to gain credibility and professional acceptance. Trialling should extend to the scoring rubric with data from the field needed to demonstrate its reliability.

  3. 3.

    A concerted focus on what is taken to be evidence of professional competence and its utility for the field of practice for both licensure (summative) and improvement (formative) purposes. Historically, teacher education has lacked an evidence base to address or counter criticisms of the quality of teacher education and graduate preparation. A distinctive feature of the approach to TPAs in this book is the deliberate move to use summative data for formative purposes to inform program design and improve teaching strategies. Only when both summative and formative purposes come into view as complementary can the teaching profession reclaim accountability for itself.

  4. 4.

    Working at scale. The establishment of an evidence base to show the quality of teacher education requires large-scale data, collected across diverse contexts. It also requires longitudinal data to establish trends, to show changes and to show consistency in the application of the common standard. For sustainable change, the active collaboration with universities in this endeavour is essential.

  5. 5.

    Recognition of the complexity of the teacher education policy and practice landscape and standards. Teaching is understood as a complex activity that is responsive to context and requires ongoing data gathering and decision-making to meet student needs and progress learning. Teacher education practice and policy must be able to accommodate this diversity, preparing teachers able to meet professional standards and adaptable to the diverse contexts they will encounter as teachers. This calls for moving beyond standards as regulatory tools to the use of standards as a means to act with agency and inquire deeply into teaching practice.

  6. 6.

    The value of collective action and decision-making. A TPA should have a necessary focus on measurable knowledge, skills and capabilities which are assembled in professional practice. One of the barriers to this has been that teacher education has occurred in silos with little communication across programs and institutions. Establishing a common standard can only be realised through the pooling of professional evaluative expertise with teacher educators at the table with policy personnel, school partners and other researchers. This requires dialogue—thinking, acting and making decisions together—informed by evidence and research in arriving at TPA design and implementation decisions.

  7. 7.

    Customised digital architecture (data and communication infrastructure). The implementation of TPAs across a wide range of geographic locations and the collection of large-scale data requires customised digital architecture. This includes processes to upload assessments and assessment data; undertake cross-institutional scoring and moderation of samples; analyse data and automate reporting of results using data visualisations. Infrastructure is also needed for data storage, transmission security and ethical and privacy protocols.

  8. 8.

    Principles and practices for cross-institutional moderation. Both intra- and cross-institutional moderation are essential to establish judgement reliability and comparability in the application of a common standard for graduation and licensure as a teacher. CIM-Online involves teacher educators scoring samples using an established rubric. Teacher educators’ experience and evaluative expertise are essential in this process, taking responsibility for applying the established standard.

  9. 9.

    The transformative potential of TPAs to stimulate professional learning. Professional learning associated with the development and implementation of TPAs and the use of generated data can become transformative when teacher educators use outcomes of TPA implementation in discussions about program design and its impact on preservice teacher learning.

We offer the framework of pre-conditions to others who are considering, or are in the process of implementing, a TPA or other complex assessment of competence into their programs. It provides a holistic approach to think about the change processes and examine impacts of change that are inevitable in introducing TPAs.

4 The Next Move

Yes, we have to get TPAs right. Throughout this book we recognise that a significant policy shift, such as occurred in the introduction of TPAs in Australia, requires radical rethinking that goes well beyond the design of a TPA. The transformative potential of TPAs in teacher education is a recurring theme in this book, as is the need to attend to the range of impacts that flow from such a shift. In our experience this involves collaborative action to carry forward discovery work by teacher educators motivated to improve the learning of preservice teachers and in turn, school students. Beyond this, it involves significant investment of human and material resources, noting that there has been a lack of government investment at national and state levels in Australia. A serious national commitment enabling this shift—culture change and the turn to standards and evidence—calls for coordinated strategies across policy, practice and research. We reflect here that Australia is facing a further review of teacher education in 2021 with fractures in implementation of the TEMAG recommendations now crystal clear.

It is essential that impacts of the shift are monitored as they effect teacher educators, preservice teachers, school personnel and those responsible for implementing policy in teacher education. Some of these impacts of reviews and change in teacher education are strikingly clear while others are more subtle. CIM-Online, as discussed in Chap. 3, is a significant change resulting from the Research and Development Program work including the development of digital infrastructure. Another change is that we now have evidence to show program quality as exemplified through GTPA samples. Other advances include the considerable strengthening of collaborations among teacher educators, policy personnel and researchers. Over the last six years, across the GTPA Collective, conversations have changed. We now talk in-person and online, in formal and informal meetings, about what constitutes evidence as quality, what are the characteristics of a quality ITE program and what counts as evidence of professional competence at completion of preparation. We talk about inferring meaning from data and using it as evidence of program effectiveness. These conversations open up opportunities to reflect on other topics such as evaluative expertise, the application of standards, and professional judgement. In our recent conversations we have concentrated on the application of the use of the data to inform program review and curriculum renewal in developing impact case studies of culture change in teacher education.

Against this backdrop, we recognise the pervasive calls for evidence by governments, industry and the public to assess quality and track changes over time. Teacher education is not exempt from these forces. As mentioned earlier in Chap. 2, engagement with the TEMAG reform agenda has been uneven across the country. A consequence of this, as mentioned above, is that in 2021, seven years after the most recent review of teacher education (see discussions of TEMAG) a further review has been heralded. The declared motivation for the review is to return Australia to the top group of education nations. This includes a laser-like focus on “quality teaching, particularly initial teacher education, curriculum and assessment” (Tudge, 2021, n.p.). The following excerpts from the minister’s speech make this clear:

… some teachers are still graduating from their courses insufficiently prepared to teach in a classroom either because there has been too much focus on theory at the expense of practice, or because evidence-based teaching methods are not taught. I hear this consistently from school principals and graduates alike…

The next evolution of reforms is needed, to build from the TEMAG reforms. I will soon be launching a review to help shape such reforms. This review will investigate where there is still further work to do to ensure that all ITE courses are high-quality and adequately prepare our teachers to be effective from day one. (Tudge, 2021, n.p.)

The challenge facing teacher education is how to generate evidence that standards are being met and that preparation programs are high quality. Teacher education has the opportunity to satisfy standards of evidence and to produce evidence of standards to show that quality graduates are entering classrooms. Getting TPAs right is central to professionalising teacher education.

5 What is Involved in Getting TPAs Right?

We hold the position that getting TPAs right involves rigorous data-informed processes supported through digital architecture and input from a diverse range of specialists across different fields. The TPA is not just another assessment. The purpose and consequences of a TPA are far more significant than other assessments completed in a teacher education program. Based on this stance, getting TPAs right has to involve all aspects of approving, designing, implementing, and scoring TPAs and using TPA-generated data for improvement purposes.

Evidence of demonstrated comparability in what counts as the passing standard in initial teacher education is a non-negotiable expectation in getting TPAs right. This is essential for the Australian public to have confidence in the quality of education systems and the quality of graduates entering classrooms. Without a focus on comparability and rigorous mechanisms for demonstrating that it is being achieved across teacher education providers, we could simply revert to each university having their own standard, offering no evidence of consistent expectations of quality across the country. This is not arguing for standardising programs; rather, we assert that programs must take account of demographic variables and candidates must be developed professionally with the knowledge, skills and dispositions to be responsive to diverse cohorts of students and contexts.

It is time for Australia to make transparent a common or agreed standard of graduate readiness that applies irrespective of location and mode of preparation. Cross-institutional moderation can be fundamental to achieving this goal where rigorous quality assurance systems and processes are built into how it is designed and implemented. We call for moving beyond social moderation to more sophisticated moderation processes that combine qualitative processes with statistical analyses. Social moderation can lapse into talk and interactions intended to achieve consensus about a grade. This is a wide berth away from an intent to demonstrate reliability of judgement against an established standard. This is not an argument for statistical moderation, rather it is an argument for the combination of social and statistical moderation, data analytics, digital architecture and the collaboration of cross-disciplinary experts. A weakly framed position on moderation is likely to undermine the prospect for delivering the promise of TEMAG to lead to reform in teacher education that includes the strengthening of quality assurance of graduate readiness.

Currently teacher education in Australia lacks an agreed methodology for benchmarking graduate quality on completion of a program. Also lacking is a methodology for establishing the comparative strengths of teacher education programs, both within and across universities. In the GTPA Collective, CIM-Online functions as a form of benchmarking. As discussed in Chap. 3, the output of GTPA CIM-Online includes (1) reports of evidence about the application of a common standard across participating universities, and (2) reports of program performance at the cohort level. These reports are encrypted and sent securely online going directly into the hands of Deans, Program Directors and other teacher educators across the country. They provide reliable evidence showing the application of the established standard in their programs and their relative strengths and possible areas for improvement. Teacher educators’ engagement with and use of the reports is dependent in large part on their expertise in reading evidence or performance data presented through a range of data visualisations.

The experience of the GTPA shows that getting TPAs right will involve building data literacy in the teacher education workforce, including developing expertise in how to read or interpret such evidence. It will also involve building capability in how to apply the inferred meaning of the data to inform curriculum review and program renewal. By extension, the role of the teacher educator could extend to program evaluation. We propose that this is a role that could be undertaken with optimal effect when it involves and engages colleagues in the schooling sector, and in particular, principals and mentor teachers.

To track the movement of the standard over time requires a strong position on cross-institutional moderation that includes the provision for anchor samples. We know that standards can rise and fall. Unless attention is paid to capturing this movement, the case could be made that it was easier to graduate from teacher education last year or the year before. This brings into play issues of fairness in tests of graduate readiness to enter teaching. Legal precedence for cases contesting grading decisions in the case of the Educative Teacher Performance Assessment (edTPA) and Performance Assessment for California Teachers (PACT) are instructive for Australia (see Wyatt-Smith et al., forthcoming). From our experiences in the research, policy and practice of moderation, including in several universities and the schooling sector, the focus should be sharply on judgement and analysis to demonstrate judgement reliability or comparability. This recognises the need to also attend to evidence of rater severity and lenience.

While a strong position on moderation and cross-institutional moderation was not a feature of the move to TPAs in Australia, our research has shown its importance as well as the benefits of investing in digital architecture for supporting universities to demonstrate reliability. A light touch interest in, and regard for, the potential of moderation could undermine current efforts to build public confidence in what graduate readiness looks like. Policy attention to rigorous cross-institutional moderation and use of the evidence it generates are linchpin to developing public confidence in teacher education.