Keywords

1 Introduction

Effective teaching only happens when teachers know their students and are responsive to their needs. Using extensive assessment data can have a demonstratable impact on a teacher’s understanding of learner needs. High-impact assessment data is collected daily within schools and at key points of cognitive and biological development using national standardised testing. Since the introduction of assessment-for-learning principles (Black & Wiliam, 1998, 2010) in Australian schools, assessment has been considered fundamental in the teaching and learning cycle (Hattie & Timperley, 2007). More recently, the emergence of international testing such as the Programme for International Student Assessment (PISA), the Trends in International Mathematics and Science Study (TIMSS), and the Progress in International Reading Literacy Study (PIRLS), and national testing in literacy and numeracy in the National Assessment Program—Literacy and Numeracy (NAPLAN) and science literacy with the Validation of Assessment for Learning and Individual Development (VALID), there is a plethora of high-quality assessment data for teachers to interpret and apply to their practice. Australian schools are awash with student learning data; to pre-service and early-career teachers this data can be overwhelming and opaque in its depth and volume.

Developing data awareness and analysis skills in pre-service teachers is an important deliverable in teacher education. The Australian Graduate Teaching Standards (Australian Institute of Teaching and School Leadership [AITSL], 2017) specify that at the graduate level, teachers know their students and how they learn so that the teachers can cater for diversity and differentiate the learning. To this a graduate teacher must be able to apply assessment practices, moderate, use consistent judgement in assessment, and engage in feedback and reporting processes. However, the school-based and national NAPLAN data and international PISA and TIMSS data that has emerged has not been immediately accessible to teacher-education programs, and is often hidden away behind the school gates; the restricted domain of the “real teacher”. To overcome these barriers, data-immersion professional learning within the school gates offers a unique, timely, and highly valuable experience for pre-service teachers. It is a necessary rite of passage for the pre-service teacher to gaze into the metrics, stare down the data, and come to comprehend the power of really understanding who the learner is and how teachers can move their learning further along the continuum.

2 Digging up the Data

Assessment can be defined as a fluid and continuous process that connects teaching to the learner. According to Moss et al. (2006), assessment is planned, enacted, and evaluated. The purpose of assessment is both formative and summative. Currently, policy discourse about closing the gap in student skill, achievement, or understanding has made data-gathering normative within schooling. Increasingly, global and national testing has concerned itself with benchmarking population cohorts in specific years of schooling in core skills such as literacy, numeracy, and, more recently, scientific literacy and problem-solving. Reflecting these international trends, national standardised testing has become ubiquitous within Australian schools. NAPLAN is an Australian test introduced in 2008 to document the literacy and numeracy skills of young Australian students in school years (grades) 3, 5, 7, and 9. It is a standards-based measure of a student’s skills in numeracy, reading, writing, and language conventions. The progression of each individual student is mapped and traced at these critical milestones of different school years and reported on a single standards framework. Comparison with aggregated data from the same school cohort or other schools close by, or even across the state and nation, is also provided to give an indication of system-wide and national performance. According to the Australian Curriculum and Assessment Reporting Authority (ACARA, 2017), NAPLAN provides information about individual and system performance. The performativity aspect of NAPLAN is frequently controversial and well-publicised in the community. However, the data that NAPLAN provides to schools is often not well understood, and pre-service teachers are generally on the periphery of this scene.

This case study illuminates the careful work of teachers and shares these practices with pre-service teachers. The purpose of the case study is not so much to engage in the controversy and policy debates about NAPLAN, but to understand what data on numeracy and literacy such measures provide for individual students, and how it is used in schools as part of the daily practice of teaching and learning. This case study shows how digging up the data is exhaustive, meticulous, and methodical work conducted by skilled teachers, and can be a rich practice shared with pre-service teachers.

3 Data Immersion

The data immersion occurred across three secondary schools over a two-year period. This comprised cohorts of 30 pre-service teachers participating in a data-immersion experience at a secondary school and receiving professional learning about how each school uses data provided by NAPLAN to create student and class profiles of individual literacy and numeracy needs. The data-immersion days used de-identified and aggregated class samples to enable pre-service teachers to apply skills in reading and interpreting graphs, standards, and learner profiles using metrics and descriptive statements. Samples of school- and NAPLAN-developed literacy and numeracy tasks were deconstructed and analysed based on constructs of numeracy and literacy. Constructs of numeracy comprised exploring the relationship between students’ demonstration of mathematical concepts across spatial, graphical, statistical, and algebraic domains and their responses to specific NAPLAN test items. Similarly, reading, writing, and language conventions were analysed as components of student literacy. Pre-service teachers interpreted students’ responses to all three components and were guided to consider how each student could improve at sub-skills within these components.

The immersion was led by practising teachers and school leaders as part of a professional-learning exchange between proficient, highly accomplished, and pre-service teachers. This dialogue was a powerful pedagogical exchange within the profession and promoted the agency and professional identity of all participants. This model of professional learning was informed by the work of Timperley and Parr (2007) in New Zealand schools, where professional-development inquiry cycles were used to affect student learning. Within this model, the following principles were applied:

  • The data shared was presented as useful and informative to the learning cycle;

  • Pre-service teachers were given opportunities to develop sufficient knowledge about the meaning of assessment and how to modify assessments;

  • School leaders shared professional conversations with pre-service teachers to unpack the data;

  • Pre-service teachers were required to reflect on the data and develop stronger pedagogical knowledge of literacy and numeracy strategies; and

  • The whole school was able to engage in evidence-informed cycles of inquiry to construct data knowledge (Timperley, 2008, p. 1).

This cycle of collaborative inquiry into evidence-based practices was a distinctive and successful feature of the immersion. Throughout the activities, pre-service teachers were guided to construct class learning profiles, observe literacy and numeracy activities in classrooms, debrief and reflect on these observations, and engage in the development of new or modified tasks to build specific literacy and numeracy skills in students. Other activities enabled pre-service teachers to engage in marking and moderation exercises using de-identified student samples, and to respond to assessment conversations with teachers and school leaders.

This case study collected qualitative comments about the perceived usefulness and professional relevance of the data-immersion experience from pre-service teachers. A cohort of 30 pre-service teachers were randomly assigned to one of the case-study schools for the immersion across key learning areas or teaching areas. Their feedback and comments about their perspectives of the experience were collected using open-ended survey questions and student feedback on a unit related to the data-immersion learning.

4 Results and Impacts

The data-immersion days offered pre-service teachers the opportunity to engage in rich pedagogical discussions and develop their perceived self-capacity to interpret aggregated data, apply constructs of numeracy and literacy to student responses, and analyse how differentiated learning can be developed further for specific learners. The hidden and nuanced skills of teachers and school leaders in using data about their students was revealed to pre-service teachers in ways that enabled conversation and promoted knowledge-sharing. The positioning of data, especially standardised and national sources, as useful and relevant to teaching practice disrupted discourses about performativity and anxiety with test taking. It provided an authentic context for exploring how assessment is used for teaching and learning. While the pre-service teachers could not master assessment and data analysis within the constraints of these experiences, they created spaces for the pre-service teachers to challenge their assumptions about assessment and standardised testing, and to understand how assessing student learning is part of daily practice within the classroom. Overall, it contributed to their perceptions about and stances on assessment and data practices in schools.

The following themes were identified in the case-study comments from pre-service teachers:

  1. 1.

    Increased understanding of NAPLAN tests and Smart Data, and how this data could be used at the school, faculty, classroom, and individual levels.

  2. 2.

    Improved confidence in analysing written responses and a stronger understanding of literacy.

  3. 3.

    Connections between classroom pedagogy and literacy and numeracy tasks.

  4. 4.

    Improved understanding of differentiation based on individual student data (Table 12.1).

Table 12.1 Comments from pre-service teachers based on perceptions and experiences of the immersion
Full size table

Critical to these immersion experiences was the role of school leaders and experienced teachers in promoting the conditions where teachers talk data. This data talk is informed and guided by the need to know more, and understand better, how a student can improve. These conditions are powerful in making teachers willing to change their practice to achieve better outcomes. The participatory voice of teachers and school leaders is significant for improving the perceptions of pre-service teachers about data and assessment practices.

5 Using Evidence-Based Teaching Practice Within Pre-Service Teacher Education

Evidence-based practices that include teachers in the dialogue and the direction of student learning are noted to improve school success (Wandersman, 2014). This approach has been aptly described as “empowerment evaluation”. This form of data-driven teaching is effective, as it makes teachers think critically about the learner and inquire into their own practice. This model has been used to inform the model of immersion in schools as it relates to data understanding and interpretation. This has been situated as an inquiry into practice and perceptions of this experience, captured to consider how pre-service teachers can develop their pedagogic choices in applying data to their teaching. The pre-service teachers observed how data was extracted using school systems, applied to individual student learning progression and shaped classroom practices and interventions. Pre-service teachers were exposed to the inquiry-led practices of accomplished teachers and school leaders in this case study.

Inquiry is a participatory discourse that enables teachers to make pedagogic choices and use both judgement and evidence (Timperley, 2011). Inquiry happens when teachers critically reflect on their practice, shared beliefs and theories, student understanding, and curricula, and intervene to change, redirect, or build greater coherence between all of these aspects of teaching. Inquiry brings disparate ideas together and situates them within the context and reality of a teacher’s school. Inquiry is a form of participatory, noisy action that gives voice to teacher thinking and pedagogic choice. It allows for conceptually thick ideas and practices to be critiqued.

Pedagogic inquiry comprises open-ended, generative questions, rich tasks, collaboration, analysis and creation, and communication of and reflection on new ideas or understanding that solves problems or responds to substantive and significant questions. Inquiry is complex and situated in relevant, real-life contexts and problems. It involves making predictions, using evidence-based practices to discern and distil information, testing and evaluating conclusions, and communicating findings. It is cyclical and intellectually robust, and allows for a degree of autonomy, questioning, and reassembling of concepts (Maab & Artigue, 2013). It enables voice, develops critical thinking, and follows evidential practices for all teachers.

Successful inquiry within evidence-based teaching includes the following features:

  • Collaboration and opportunities for discussions amongst teachers

  • A consistent focus on teaching quality

  • Use of assessment and student data to guide the learning

  • Opportunities for creative problem adaptation (Giles & Hargreaves, 2006).

Additionally, when professional learning solves a local or contextual issue for schools and teachers, the solution and learning are more likely to be transferred and sustained. Mentoring and engaging with pre-service teachers in data-analysis conversations and scenarios was an important learning from this case study. The opportunity to build a community of practice in this case study was an important feature of successful professional learning that will inform pre-service teachers’ identity as graduate teachers. The relational nature of professional learning and mentoring supports pre-service teacher development and allows for teachers to share their context, experiences, successes, and failures. The inquiry stance used at the data immersion facilitated pre-service teachers’ perceptions of data analysis, and was observed to increase their confidence in applying literacy and numeracy strategies. Additionally, pre-service teachers in the case study were observed to have a stronger understanding and awareness of how to differentiate for learner needs. Central to this model of professional learning was the concept that the pre-service teacher is both a learner and an actor within a school setting.

6 Lessons for Future Practice

Immersing pre-service teachers in data analysis and evidence-based practices within schools is an innovative opportunity for praxis in teacher education. It provides pre-service teachers with a lens to see student, class, and school learning needs and adapt, differentiate, and modify their practice. Pre-service teachers can imbibe the data, lean in without risk of failure, and understand how effective teaching is predicated on knowing their students and how they learn. It provides an authentic context and purposeful inquiry into student learning. Using high-impact assessment data challenges pre-service teachers’ prior assumptions and policy discourse of performativity in standardised testing, as it requires critical and careful deconstruction of the evidence of learning. The immersion within the professional spaces that teachers and school leaders inhabit inducts the new teacher into the profession and creates opportunities for their professional participation. Pre-service teachers in these spaces have a participatory voice and learn to grapple with live, messy, and complex data.

Despite the density and uniqueness of the data presented in NAPLAN, pre-service teachers in this case study were observed to have improved confidence about starting data conversations. These pre-service teachers through their comments demonstrated evidence of reflection about the value and usefulness of student-assessment data. These perceptions of data practice were built on an inquiry stance practised by teachers in an authentic context. The lessons from this case study are real and build pre-service teachers’ openness to using an inquiry stance to gather data and apply it within the future. Thus, using inquiry-based approaches grounded within the school gates that encourage dialogue and pedagogic unpacking between school leaders and pre-service teachers could inform future models of teacher education.