Data, Knowledge, Action: The Uses of Data Systems as Auxiliary Tools to Aid Teachers’ Thinking About Children’s Curriculum Experiences and Teacher Practices

Abstract

The Data, Knowledge, Action (DKA) programme of research begins from the premise that access to and use of quality data can enhance early childhood teachers’ practices in multiple ways, including assessment for children’s learning, pedagogy and relationships with children’s families and evaluation of teacher practice. In our work to date in the DKA research programme, we have worked with teachers in seven New Zealand kindergartens across three projects to explore the use of different data systems and tools intended to help teachers gather information to broaden and deepen their knowledge about their pedagogical practices and children’s curriculum experiences and learning. Each project has collected data related to specific aspects of practice and children’s learning that teachers inquired into. As part of our work to support teachers’ inquiries, we also explored and collected data on teachers’ experiences, perceptions and shifts in thinking and practice resulting from their engagement with the data systems and ensuing information. In this chapter, we provide an overview of the DKA research programme and component projects and describe the key tools and systems used to date. The impact on teachers’ thinking about both children’s curriculum experiences and their own pedagogical practice through use of these tools is examined.

1 Introduction

The Data, Knowledge, Action (DKA) programme of research is built on the premise that access to and the use of quality data supports early childhood (EC) teachers’ practice across multiple aspects of their work, including assessment for children’s learning, intentional pedagogy, relationships with children’s families and formative evaluation of their practice. In this chapter, we provide an overview of the DKA research and the data tools used in three projects completed to date. The impacts that using these tools has had on the participating teachers in three key areas are then examined: their confidence in using data tools and working with data; their strengthened understandings of children’s curriculum experiences; and their reflections on and shifts in their pedagogical practices. The chapter concludes by examining the implications of findings from across these projects, particularly in terms of the policy and professional support required to assist EC teachers to effectively gather and use data to enhance their pedagogy and strengthen children’s learning outcomes.

2 The New Zealand Early Childhood Education Context

Early childhood education (ECE) in New Zealand caters for children from birth through five years of age. While not required to attend school until age six, almost every New Zealand child begins school on—or shortly after—their fifth birthday. Diversity of provision has been a hallmark of the sector with services licensed under three key regulatory umbrellas: teacher-led; parent-led; and whānau (extended family)-led¹ services. Brief details of the main service types are outlined in Table 13.1.

Table 13.1 Main ECE service types in New Zealand

All EC services are required to implement Te Whāriki (Ministry of Education, 2017), the national EC curriculum. Built around a framework of five key strands and four overarching principles, Te Whāriki contains 17 goals, focused on characteristics of learning environments and pedagogies to guide practice and 20 learning outcomes encompassing the ‘valued knowledge, skills, attitudes and dispositions that children develop over time’ (Ministry of Education, 2017, p. 22). A whāriki is a woven mat and its use as a metaphor represents the notion that teachers and educators are expected to weave curriculum experiences that are reflective of the local community and responsive to parents’ aspirations for their children’s learning. The curriculum emphasises a strengths-based approach, focused on children as competent and confident learners and highlighting the importance of formative assessment or assessment for learning. Assessment for learning is primarily undertaken using Learning Stories, a narrative assessment approach developed in New Zealand (Carr, 2001). Initially designed to assess children’s developing learning dispositions aligned to the five strands of Te Whāriki, Learning Stories aim to capture children’s experiences and learning interests and may include contributions from children and their families alongside teachers’ assessments (Carr, 2001; Carr et al., 2002).

The endemic use of narrative assessment approaches, primarily Learning Stories, since their introduction in 1999 (Carr, 2001), has seen a reduction over time in the use of other observational techniques and data collection approaches to support assessment for learning and teacher evaluation of their own practice (Cameron, 2018; Mitchell, 2008). The New Zealand Education Review Office (ERO⁴) has also noted the need for services to ‘improve processes for the gathering, analysis and use of information in self review’ (ERO 2009, p. 19), suggesting that there is a need for a stronger focus on the collection and use of data within local service contexts to support assessment, planning and evaluation.

3 Data-Informed Teaching

Research into the use of data and data-informed teaching is more extensively located within the schooling sector, where it has been positioned both within a data-driven decision-making accountability framing (Gullo, 2013) and as a formative and iterative process that aims to strengthen classroom practice (Hoogland et al., 2016); it is this latter approach that underpins the research reported in this chapter. While multiple definitions of data inquiry and data literacy exist (e.g. Bocala & Boudett, 2015; Gummer & Mandinach, 2015; Jimerson & Wayman, 2015), Schacter and Piasta (2021) noted that early childhood ‘teachers defined data in a practical way as information about children’ (p. 9), including information about their learning interests. Three teacher profiles with regard to data use emerged from their research: teachers who gathered data but did not use it to inform their practice; teachers who primarily gathered and used data informally to inform their ‘in the moment’ (p. 13) interactions; and teachers who integrated multiple data sources into their planning and adaptation of learning experiences to meet children’s individual needs. This variability in teacher practice was also evident in Trawick-Smith et al.’s (2016) study of data-based meetings focused on teachers’ use of maths talk in early childhood settings and in Datnow and Hubbard’s (2015) review of research into school-teachers’ use of data.

The importance of supporting teachers to be able to gather, make sense of and use data to support their planning and pedagogical practices is well recognised (e.g. Bocala & Boudett, 2015; Dam et al., 2018; Hoogland et al., 2016; Skov Hansen, 2018; Trawick-Smith et al., 2016). Such supports include the use of external facilitators and coaches (Hoogland et al., 2016; Marsh et al., 2015), professional learning communities (Marsh et al., 2015; Skov Hansen, 2018) and opportunities for professional learning (Datnow & Hubbard, 2015; Jimerson & Wayman, 2015; Schacter & Piasta, 2021; Trawick-Smith et al., 2016) that enable teachers to build confidence in using data and develop a data culture (Datnow & Hubbard, 2015; Hoogland et al., 2016). Along with being able to identify and use relevant data, Earl and Timperley (2008) note that having an ‘inquiry habit of mind’ and engaging in ‘relationships of respect and challenge’ are critical qualities necessary for teachers to have ‘evidence informed conversations’ (p. 3) that strengthen teacher practice and outcomes for learners.

Beyond these supports, Schildkamp and Poortman (2015) have highlighted characteristics in relation to data (e.g. quality, multiple sources, availability of tools and information management systems), school organisation (such as leadership, training and support and having shared goals) and individuals and teams (knowledge and skills, attitudes and beliefs, collaborating on data use) that influence, either as barriers or enablers, teachers’ use of data. Collaborative and dialogic approaches have emerged as foundational to effective data inquiry and practice (Bocala & Boudett, 2015; Hoogland et al., 2016), with Marsh et al. (2015) noting the importance of horizontal expertise or ‘knowledge that is created through interactions and movement across contexts’ (p. 8) and which occurs through ongoing dialogue. Taken together, data competence and confidence for teachers require a range of professional learning supports and enabling factors and should be embedded within meaningful contexts and aligned with curriculum and assessment values.

3.1 The Data, Knowledge, Action Programme of Research

In this section, the Data, Knowledge, Action (DKA) programme of research is presented. This work, as noted above, is predicated on the belief that teachers’ practices across multiple spheres of their work can be strengthened when they are supported to collect and analyse data that provides them with new knowledge about their own practice and about children’s curriculum experiences and learning. In previous writing, we have cautioned against a narrowing of assessment and evaluation approaches (McLaughlin et al., 2020) with one of us (McLachlan, 2018) arguing for a broader range of approaches to be used in order to make valued learning visible. In addition, we have advocated for a stronger focus on intentional teaching (e.g., Cherrington, 2016; McLaughlin & Cherrington, 2018) situated within play-based approaches and where both children and adults might initiate and extend the play (Edwards, 2017). Through its focus on data-informed teaching, the DKA research programme addresses this need to keep a broader view of possible assessment and evaluation tools while supporting a focus on intentional teaching in play-based ECE services.

The DKA programme of research has been led by principal investigator Tara McLaughlin and co-investigators Sue Cherrington, Claire McLachlan and Karyn Aspden. We have partnered with Ruahine Kindergarten Association and worked closely with Lynda Hunt as the lead teacher-researcher across projects. To date, the DKA research programme has completed three projects across five years. Table 13.2 provides an overview of these projects. The first project piloted a number of data tools described below. The fourth data tool, using pedometers to measure children’s levels of physical activity was discontinued in later projects and thus is not discussed here. Adaptations to tools and additional tools have been developed in the latter projects over time.

Table 13.2 Data, knowledge, action projects

Both the TLIF (Ruahine Kindergarten Association, n.d.) and the TLRI (McLaughlin et al., 2022) projects utilised an inquiry approach where teaching teams worked, with our support, to identify inquiry foci and questions related to teaching and learning in their kindergartens. The tools used in each project enabled data to be collected to support teachers as they undertook these inquiries. Three of these tools were project developed: The Child Experience Observation System (CEOS; McLaughlin et al., 2018a, 2019), the Play and Learning Analysis System (PLAS; McLaughlin et al., 2018b) and the Child Profiles (McLaughlin et al., 2018c).

The CEOS tool is a structured live observation based on pre-determined frequency and duration codes focused on children’s engagement within the programme, who they interact with and the nature of those interactions. Observations are undertaken by a trained observer (i.e., teacher-researcher—see Chap. 14) and are recorded on a tablet using observational software. Depending on the inquiry focus, additional frequency codes may be included. Such codes have included types of social play children have engaged in, the nature of teacher–child learning interactions and children’s activities during regular visits to a local nature reserve. Individual children are typically observed for a 2-h period and the resulting data collection file is run through the observational software base programme to generate a summary of the data. Data are then entered into a project-developed Excel™ template to produce a graphed data report for each observation period. Contextual notes can be added to the report to provide additional information as needed.

The PLAS involves either a child or teacher wearing a small GoPro video recorder slotted into a light-weight chest harness worn over their clothing for a 2-h period, capturing video footage from the wearer’s perspective and audio from the wearer and those around them. PLAS recordings have been used in two ways with teachers: firstly, viewing short clips that provide opportunities to gain insights into children’s curriculum experiences that occur across the programme, whether in the company of teachers and other children or on their own. Given the free-flowing nature of most New Zealand ECE programmes, where children have considerable freedom to make decisions about where they will play and with whom, such clips have enabled teachers to see and understand children’s perspectives, interactions and behaviours in previously unseen situations. Video clips taken from the PLAS recordings have also been used to undertake micro-level analyses of interactions between teachers and children, where the moment-by-moment shifts that occur across an interaction are identified and explored by teachers and researchers.

The Child Profile is a teacher-completed tool that prompts teachers to consider children’s curriculum experiences, learning and development through a variety of questions and focus areas. Teams can elect to use the full profile or select sections aligned with their inquiry focus. Teachers either complete the profile individually and then discuss within the team or complete it as a collective activity; the key to its effective use is the discussion among teachers about what they know—or don’t know – about individual children and teachers’ different relationships with and perspectives about a child.

Alongside these three project-developed tools, the third project outlined in Table 13.2 has drawn on externally developed tools: the Early Childhood Environment Rating Scale Curricular Extension to ECERS-R (ECERS-E; Sylva et al., 2010), the Sustained Shared Thinking and Emotional Wellbeing Scale for 2–5-year-olds provision (SSTEW; Siraj et al., 2015) and the Language Environment Analysis (LENA; LENA Building Brains through Early Talk, n.d.) recording and analysis software. Both the SSTEW and the ECERS-E are internationally known rating scales that assess curricula provision, pedagogy and environmental resources within EC settings. In the context of our project, SSTEW and ECERS-E data were gathered by two trained observers and scores and observation summary notes were provided to teachers in graphed and written reports for each observation scale. Observation summary notes highlighted practices observed and areas for growth.

In contrast to education-based tools, LENA has most typically been used to capture the extent of language interactions between children and adults within home settings as part of programmes designed to support children’s oral language development in the early years. The LENA system uses an audio recorder worn by the child for up to 12 h (although the maximum period was up to four hours in our project). The recording is then uploaded to software designed to quantify the number of adult words spoken to the child, the number of child vocalisations and the number of adult–child conversational turns throughout the collection period. Data reports showing trends in these variables along with the quality of the audio environment can be created.

Full ethics approval was gained for each of these projects through the second and third authors’ university. Informed consent was gained from teachers and from parents for their children to participate as ‘focus children’ and for video recording. Videos that inadvertently captured footage of children where informed consent had not been given were deleted immediately following the recording. All focus children were asked to assent to wearing the GoPro and LENA devices and could remove these at any point during the observation period.

The CEOS and PLAS data were given to the teachers within 24 h of the observations and video recordings being completed. Regular (one-two times per term) data review meetings were held during which all collected data were shared and discussed by the researchers and teachers. Three members of the research team worked with teachers to unpack and make sense of the data, in relation to individual children or across the group and across the teaching team. The focus of data support meetings changed over time. Initial meetings focused on accurate interpretation of each data source available; creating a space of trust, comfort and respect for data review; and discussing emerging insights and ideas. Later sessions focused on integrating information across data sources, deeper discussions of data and challenging assumptions; or creating data-informed action plans for teacher practice or child learning.

The structure of the two main projects differed in terms of data collection for the teachers’ inquiries. In the TLRI project, the first two authors undertook three cycles of STTEW and ECERS-E observations while the fourth author undertook regular data collection using the CEOS, PLAS and LENA tools in each kindergarten and prepared the resulting data for review by the teams and researchers. In contrast, the TLIF project involved one teacher from each of the four kindergartens acting as a teacher-researcher for another team (see Chap. 14 of this volume). During each kindergarten’s data collection period, their teacher-researcher would undertake the CEOS observations and support the focus child to wear the GoPro. Following the observations, the teacher-researcher would graph the CEOS data and edit the GoPro footage into short clips for later discussion by the teaching team. Ongoing support was provided to the four teacher-researchers throughout the project as described in Chap. 14. This included training on the CEOS codes, graphing data and editing video footage along with support for the leadership roles that they each undertook within their own kindergartens as a result of their deeper data collection knowledge.

4 The Impact of Tool Use on Teachers’ Thinking and Practice

Across the three projects, three key themes emerged in relation to teachers’ thinking and practice: teachers’ increased confidence in using data tools and working with data; their strengthened understanding of children’s curriculum experiences; and their reflection on and enhanced pedagogical practices resulting from engagement with data.

4.1 Increased Confidence in Using Data Tools and Working with Data

Investigation of teachers’ confidence and skill in using data tools and working with data occurred primarily in the TLIF and TLRI projects. In these projects, we used the same pre- and post-project questionnaire to get teachers to rate their level of confidence in working with data and data tools. For this chapter, data from the two projects are combined, rather than reported separately. Teachers were asked to rate how confident they were with six aspects of undertaking inquiries. Data presented in Fig. 13.1 below indicates that at the start of the projects, teachers were less confident with the four steps of Collect Data, Analyse and Summarise Data, Make Sense of Data and Reflection on Practice. While teachers’ collective levels of confidence went up for all six aspects in the post-project survey, the greatest gains were in those aspects most explicitly focused on working with data.

Fig. 13.1

Pre- and post-project surveys: confidence in inquiry processes

Qualitative data collected throughout both projects also revealed teachers’ growing confidence with and appreciation of the data that were being collected about their children and their practice. In both projects, most teachers expressed an initial degree of trepidation about their ability to collect and work with data, as the following quote suggests:

We think of data and, oh, making it measurable and you think of those words and statistics and comparing but, yeah, I think like [colleague] said, it was a bit scary at the beginning and thinking, oh are we really going to be going down that track comparing numbers sort of thing. (TLRI kindergarten focus group)

Not unexpectedly, given the greater demands on them, this initial nervousness was more frequently expressed by the teacher-researchers: ‘For me it was all self-doubt because I can’t do this. It was that lack of confidence and lack of self-belief that actually gave me two steps forward and then I would take one or two back’ (TLIF teacher-researcher focus group). There were, however, some teachers who were ‘excited about what the data will bring’, with one saying, ‘I do love a good graph’ because ‘they focus your thinking’. (TLRI kindergarten pre-project focus group)

Teachers consistently identified that the tools were providing them with new information about children. Typically, this occurred first with the Child Profiles as teachers began working with these early in the projects and were able to complete them without support from a teacher-researcher or the research team. Teachers’ discussions revealed differences in what individual teachers knew about children and where there were gaps in their collective knowledge:

[W]e did the profiles on children individually and then shared as a group and when you actually have to answer questions about that particular child, it made you realise that you actually don’t know as much as what you thought you did about a child.’ (TLIF kindergarten focus group)

These insights were not limited to the Child Profiles. Rather, teachers reported gaining new—and at times, unexpected—knowledge from across all the data sources:

You have a perception of how it is ticking along and what children learn and how they engage and what happens and everyone has that perception about it and you think you know it all … And then [the] data reveals something and you are like, ‘I didn’t even know that was happening and I couldn’t see that happening’ or ‘that isn’t something I have even considered’. So, that has been really revealing with the whole team. (TLIF teacher-researcher focus group)

Initially, many teachers were unsure how to make sense of the data that they were looking at. As one teacher expressed, ‘I thought I knew what I was looking at but hearing somebody who’s got a lot of experience with interpreting that stuff made me understand it a lot better’ (TLIF kindergarten team meeting minutes). However, as teachers became more familiar with the data tools and with engaging with the data presented to them, meeting notes recorded teachers reporting ‘having a great time with looking at the data. Impacting on every aspect of their practice, including planning and teacher reflections. Having lots of teacher conversations … Teachers becoming more and more enthusiastic about collecting and using data’ (TLIF Kindergarten, teacher-researcher meeting minutes). Teams experienced a-ha moments as they started to make sense of data about their children: ‘All of a sudden, we are like—ding, ding, ding. We could be doing this with him and we need to be doing that with him’ (TLIF kindergarten meeting minutes). Teachers also recognised the value of different data tools and how information gained from each supported them to have a broader understanding of their children:

I think what adds strength to what we see is that we have like the three different tools so we’re using the profiles, the graphs and the GoPro. I think if either of them was sitting alone they wouldn’t be as robust, you wouldn’t see such a holistic view of the child. (TLIF kindergarten team meeting notes)

Data review meetings were held with each teaching team to support them to engage with and understand the data being collected about their children and their own practices. As noted above, during early review meetings members of the research team would scaffold teachers to analyse and make sense of the data and to consider the implications for children’s curriculum experiences and learning and for their own teaching. One particular process introduced by the second author was the ‘data walk’. Data walks involved laying out graphed data for each focus child in their kindergarten and looking for patterns across these data. Initial data walks were heavily supported by the research team pointing out key features of the data we could see across children and asking probing questions. In later data walks, teachers were independently identifying trends they saw across children’s data:

And the fact that we could actually understand what the data was telling us by that point, whereas at the beginning with our first data walk we were looking at Tara like what, what’s she talking about... It’s like an alien language, wasn’t it at first. (TLRI kindergarten team focus group)

By the end of these projects, teachers were confidently working with data and several teams had identified which tools they planned to embed into their future work to support planning and internal evaluations:

… it was quite funny at the end of this research how we sort of picked the data tools that we were going to use to go with our focussing. That was quite fun. Like actually knowing what we think would work with what we were focussing on. (TLRI kindergarten team focus group)

Other teachers were encouraging their pre-service student teachers to explore using a wider range of observation approaches. One teacher suggested that her student teacher use interval and event recordings, rather than running records as she could get data better targeted to what she wanted to look at and graph the results.

Teachers valued the data collected as part of their inquiries, identifying that it had supported their planning for individuals and groups of children as well as helping them to look more closely at their own practices and how effective these were in supporting children’s learning. They recognised that their data may not always give them the answers but may raise more questions that could ‘lead to conversations about what might be happening for a child, whether teachers need more data or information, how it might link to planning that had been in place for a child’ (TLRI kindergarten data review meeting minutes). Working with the data required that teachers step out of their comfort zone and open their practice up to greater scrutiny than they had previously experienced. However, the benefits were clearly articulated by a teacher who commented at the end of our TLRI project:

We could see over the time … when we look at those three comparison observations, the shift in practice. It was really like, ‘yeah, we have really taken so much onboard and applied it and we, yeah, really stepped up’ so that was cool to see those comparisons. (TLRI kindergarten post-project interview)

4.2 Strengthening Understanding of Children’s Curriculum Experiences

Alongside building their confidence in using different data tools and working with data, teachers across both projects reported heightened understandings of children’s experiences within their kindergartens. The GoPro video recordings were the most powerful data source for this aspect, enabling teachers to gain deep insights into children, particularly their conversations and play with others when they were not in close proximity to teachers, as well as those who needed additional support.

Multiple instances were described where the perceptions that teachers had about individual children were challenged by what they saw in the video-recorded episodes. For example, teachers reported seeing some children take on leadership roles when they had previously viewed them more as followers and were at times surprised by children’s confidence and assertiveness when interacting with others. Insights were gained into how children approached new experiences. For instance, one team appreciated how much time and care a child took to observe others during an excursion to a nature reserve before he chose to join in. Having watched this episode, they recognised that the child joined in confidently after carefully observing and they were more alert to him using this strategy back at the kindergarten.

Several episodes were recorded where teachers initially saw children’s behaviour as inappropriate; however, as these situations unfolded, teachers’ views were recast. In one instance a child tried to rescue a box of cereal his friend had taken from the cooking table and thrown in the rubbish bin. His teachers realised that:

Had we seen the situation without the GoPro, we would have jumped and accused that child of doing what we thought he had been doing. But that GoPro showed us that he does the opposite and tries to prevent it, but he is often the one caught with the … tipping the Weetbix into the rubbish bin. We would have gone, ‘why did you tip that in?’ when he was actually trying to pull it out to save them. That really made us think how many times as a teacher, are we jumping to conclusions when we don’t actually know? (TLIF teacher-researcher focus group)

Teachers gained greater understanding about individual children’s language interactions, particularly those whose language development they had been actively supporting. The video recordings enabled them to hear how much language children were using with their peers and to see the positive impact this was having on their interactions with others. In one episode, a teacher described how ‘we have actually realised what complex sentences he is saying and how far he has come’ (TLIF teacher-researcher focus group) when talking about a child for whom English was his second language.

Children’s use of self-talk or private speech (Vygotsky, 1986) was captured through the GoPro recordings, providing further insights into their thinking and how they managed situations. In the TLIF project, teachers in one kindergarten team described how children used self-talk when on the nature reserve excursions to help them cope with physical challenges: ‘oh, I’m scared but I’m being brave’ (TLIF kindergarten meeting notes). Another TLIF team described how one child used self-talk as he prepared to enter play situations. In a later discussion, his father confirmed that would talk with his son about playing with other children each morning on the way to kindergarten.

Finally, the video footage also helped teachers gain insights into those children who typically ‘go under the radar’ (TLIF teacher-researcher focus group) and children who teachers often did not spend a lot of time with during the session. As one teacher commented:

To film a child for [up to] two hours, oh my goodness. That gives you …it’s amazing what that gives you about a child. You work with these children for a year or two and you think you know them, but you have a two-hour footage and it actually gives you … when you hear what is being said and how they interact and … wow, it can really enlighten you. (TLIF teacher-researcher focus group)

4.3 Teachers’ Reflections on and Enhanced Pedagogical Practices

As outlined in the introduction, each of the projects reported on in this chapter has used different data tools to support a different focus, beginning with developing and piloting several tools, supporting teacher-led inquiries and exploring sustained shared thinking to deepen young children’s learning. Despite these different foci, across each project teachers have used the data to reflect on and inform their pedagogies, with different tools offering different prompts and opportunities for reflection. While most reflection was team-based, teachers also found the GoPro video recordings useful for individually reflecting on their own practices:

… the tools have made me think about the ways that I teach and how I can improve on my practice. Because sometimes, yeah, like that’s really important as well to just take a look at how am I doing this, what could I do differently or how can I extend myself as a teacher to support children’s learning. (TLRI post-project focus group)

One of the TLIF teaching teams had focused their inquiry on their programme of weekly visits to a local nature reserve by six children with one of the teachers and a parent. The nature of these visits meant that there were excellent teacher–child ratios and teachers featured frequently in the videos captured by the GoPro worn by an individual child. At the final focus group interview for this kindergarten team, one teacher commented:

I think we all wanted to watch our own full video… We didn’t have to, but it’s fascinating and you never get that opportunity to have nearly a two-hour slot of you and how you interact with one particular child over a length of time. You never get that, that’s valuable, that’s precious. (TLIF kindergarten focus group)

Team discussions and reflections on the data traversed several areas of practice, including the importance of creating space to have team conversations and being more specific about teaching strategies they might use to support children. Analysing their data helped to create ‘shared understandings amongst teachers … [and] consistency across the programme’ and teachers were better able to ‘understand what children are talking about when they share experiences undertaken with other teachers’ (TLRI data review meeting minutes). Such consistency of practice was evident in data from a TLIF team’s inquiry which helped them evaluate the effectiveness of a programme they had developed and implemented over several years to support children’s social interactions with others:

We actually saw that on the GoPro that a child had been hurt by another child and there were about four teachers that talked to the child and we all pretty much said exactly the same thing, like the consistency was just incredible. (TLIF kindergarten focus group)

Some GoPro footage was quite confronting for teachers as evidenced in this reflection:

And it also brought out—when you rewatched it—how much you miss when you are one Kaiako⁵ with eight tamariki⁶ and that focused child might ask you a question, but you are busy engaging with someone else and you miss it. How many opportunities you miss, how much you follow through your length of engagement with that child. I found that quite confronting because I thought I was quite attentive and aware of just eight children, but it was quite confronting to see that there were opportunities there and you can’t attend to every child all the time. (TLIF kindergarten focus group)

In addition to developing greater consistency of practice within teams as a result of building shared understandings, clear shifts in practice were evident across the projects. Two key areas were an increased focus on intentional teaching and strengthening teams’ existing planning and assessment processes. Our data reveal teachers describing how their analysis and discussion of their inquiry data resulted in them being more intentional in both their planned and ‘in the moment’ interactions. Data review meeting minutes noted that teachers ‘feel like they are more conscious of practice as a result of their work this term’ and that ‘they have been more intentional and have made greater use of teachable moments in a consistent way’ (TLRI data review meeting minutes) while teachers also reported in one of the final TLIF kindergarten focus group interviews that ‘having all this data and information has helped us with our intentional teaching strategies’.

Teachers also described being more intentional with their planning for individuals and the wider group:

Our teaching towards those children is a little bit more intentional, you know. Some of them are struggling with friendships and so we are working on that more, aren’t we and identifying those things in our planning for those children. We’ve become more purposeful. (TLIF kindergarten final focus group)

Several teams reported reviewing their existing assessment and planning approaches in order to incorporate ongoing use of some tools. For example, one teacher-researcher described how in her kindergarten, ‘the big ‘wow’ [from the project] has been around planning’ (TLIF teacher-researcher final focus group). They had been trying for a while to improve their planning system and the project had been a catalyst for developing strong shared understandings for planning along with new planning forms and structures. Such developments were often a ‘work-in-progress’ during the project timeframes, as evident in this extract:

The team has just spent the morning thinking about how to combine everything—planning, child profile, Kaiako goals, work with whānau.⁷ Have lots of great ideas but not connecting well yet—spent the morning talking about how to make things manageable as all valuable but current systems are not integrated. Want to continue to use the child profile in future. (TLRI kindergarten meeting minutes)

Teachers also made shifts in their assessment practices, particularly in how they framed their Learning Stories in order to capture more about children’s learning journeys, even where children may not have had initial success:

And writing … that interest story so, say if they’re trying to do something for the first time, capturing that is actually more important than we used to think … Because you can say, ‘today I noticed that you da and it’s okay to feel frustrated when …’ and ‘sometimes learning new things can take a long time and we know ya da ya da’. And then, hopefully in a little bit of time you capture the next story where they’ve done it and their sense of pride in themself is so rich and it’s not just about that they’ve conquered it, it’s about the whole process that it’s taken. (TLIF kindergarten final focus group)

The impact of such shifts in practice on children’s learning were described by one team who reported that, in addition to planning for and intentionally using language around learning dispositions, ‘children are starting to use dispositional language in their interactions, e.g. ‘If you keep trying, you can do it’’ (TLRI team meeting minutes).

Beyond these two key areas of intentional teaching and adapting assessment and planning approaches, teaching teams described shifts in the kind of information that they shared with their primary teaching colleagues when children transitioned to school and using the data collected to evaluate long-standing practices and programmes. Teams also described how the use of the Child Profile sections, such as the section on children’s social-emotional learning, prompted them to re-think what they asked parents about in order to have a better understanding of new children as they began at kindergarten.

5 Discussion and Implications

Across all our projects completed to date—including the pilot where our emphasis was on developing and trialling tools—we have seen consistent evidence of teachers having deeper and broader understandings of children’s curriculum experiences and learning and of shifts in their own teaching practices as a result of using a range of data tools beyond their usual use of informal observations and learning stories. The nature of the data collected through the various tools is predominantly based on authentic observations of teaching and learning in context using different lenses or foci. The use of teacher discussion and analysis is critical to making sense of the information available. In many ways, the expanded set of tools has strengthened teacher knowledge of children and supported them to write more detailed or nuanced stories about children’s learning.

What teachers learnt about children through the CEOS graphed data often built on existing knowledge but did not always surprise them: teachers could often identify which of their focus children’s anonymised graphs they were looking at when first presented at a data review meeting. These graphs did, however, open teachers’ eyes to the variability of curriculum experiences across their group of focus children, especially the amount and type of interactions with teachers that children experienced. The CEOS data also provided teachers with clear information on shifts and progressions in children’s learning and interactions with others, both children and teachers, within their kindergartens that was less likely to be evident through their existing assessment practices. Both the PLAS and the Child Profile tools were notable for the discrepant data that emerged, with the former providing insights into children’s previously unobserved experiences and actions, while the latter highlighted gaps in what teachers knew about their focus children, either collectively or individually. Such discrepant data have been noted for their power in shifting teacher thinking (e.g. Earl & Timperley, 2008; Mitchell & Cubey, 2003) about learning and teaching and this was evident on multiple occasions with our participating teachers. Similarly, data from the ECERS-E and SSTEW observations—particularly the initial observations which provided data about aspects of practice that teachers may not have previously paid much attention to—created opportunities for teachers in our TLRI project to reflect on and reconsider existing practice. Teachers became more intentional in their practice while remaining deeply child-centred in how they planned and enacted curriculum (Edwards, 2017; McLaughlin & Cherrington, 2018).

Our intention with these projects was to explore the use of data tools that would be able to support and strengthen the approaches to assessment, curriculum planning and evaluation used by teachers in New Zealand, rather than to reject and replace existing approaches. The prevalence and utilisation of the learning story approach have been critiqued (e.g., Cameron, 2014, 2018; Wanoa & Johnstone, 2019) with many learning stories written in response to spontaneous events or episodes, rather than drawing on intentionally undertaken observations or being related to previous assessment data. Our findings revealed how teachers drew on the new (to them) data tools and their enhanced understandings to strengthen their learning story assessments: they wrote more detailed narratives and paid greater attention to children’s learning progress within their zone of proximal development while retaining a strong child-centred stance. Similarly, existing approaches to planning and evaluation were adapted by teaching teams to integrate the ongoing use of one or more of the tools or inquiry processes we had offered. Interestingly, different data tools ‘spoke’ to and were integrated into existing processes by different teams. Which tools were chosen reflected the impact of the often discrepant data on teachers’ perceptions and thinking, as well as how teams perceived they could continue their use of the tools without the external support provided by the research. Thus, teachers in our study saw value in tools that disrupted their thinking and provoked reflection but could also be readily integrated into practice.

While our CEOS tool utilised tablets to record observation data and software to support the graphing and presentation of these data, the frequency and duration observation techniques used can be simply gathered by teachers using more traditional pen and paper methods. While such straightforward observations have fallen out of use within the New Zealand ECE context, we note—and concur with—Trawick et al. (2016) finding that data collected in situ using such simple observational approaches and shared with teachers can have a positive impact, both on their practice and on children’s learning.

In parallel with the wider literature on effective data use by teachers (e.g., Schildkamp & Poortman, 2015), we identified a number of supports that contributed to teachers being able to develop confidence and capability in collecting, making sense of and using data to support their assessments, planning and evaluation. The first two authors acted as critical friends facilitating the projects and, with the fourth author, supported the teaching teams and teacher-researchers throughout their inquiries. Such external facilitation has also been found to be important in similar data-use projects in both early childhood (Skov Hansen, 2018) and schooling (Marsh et al., 2015) contexts. Providing support with the collection and preparation of data for teaching teams was a key support undertaken by the fourth author across each project, along with the teacher-researchers in the TLIF project. This support reduced the time and cognitive demands on the teaching teams, freeing them up to focus on making sense of and using the data in their inquiry projects and in their assessments and planning. As teams developed confidence with the different data tools used, they were able to consider how they might use these independent of external supports (other than their teacher-researchers) beyond the projects.

Collectively across the projects, an important support was the financial resourcing that created time and space for teachers to meet with members of our research team to plan their inquiries, learn about the tools, engage with and make sense of data and implement their inquiry and curriculum plans. We were able to scaffold teachers’ introduction to the tools and to making sense of data through regular planning and data review meetings for each team. In addition, our teacher-researchers in the TLIF project were supported through a more intensive programme (see Chap. 15) that included a mix of planned and ‘just-in-time’ knowledge to support them to use the CEOS codes, graph data and edit the video footage for their partner kindergarten team.

Leadership has been identified as having the potential to act as either an enabler or a barrier to teachers developing confidence and skill in using data (Datnow & Hubbard, 2015; Hoogland et al., 2016; Marsh et al., 2015; Schildkamp & Poortman, 2015). In our projects, supportive leadership within each kindergarten team and from the overarching kindergarten association facilitated teachers’ sustained engagement and success with their inquiry projects. The teacher-researchers also took on an important pedagogical leadership role within their own kindergartens, using their deeper knowledge of the data tools to support their teammates, particularly around making sense of the data that each tool could produce.

Our findings across the three projects undertaken so far within the DKA programme of research highlight a number of implications for both policymakers and the ECE sector if teachers are to be empowered to use a range of different data systems to effectively collect, analyse and use data that supports children’s learning and strengthens teachers’ pedagogical practice. Foremost among these implications is the need for initial teacher education programmes and ongoing professional learning opportunities to support teachers’ knowledge of and confidence with using data to support their formative assessment, planning and evaluation practices. The availability of external support people able to support teachers in using a range of tools and approaches and in developing systems that work in their local settings is a key element in this professional learning.

In this respect, the DKA research programme focuses on the collection of data and the use of data systems that support improvements in teaching and learning in the local context (Datnow & Hubbard, 2015; McLaughlin et al., 2020) rather than for teachers to gather data that will be aggregated and used to make summative judgements about the quality of learning and teaching at the service, organisational or national level (Gullo, 2013). The effective use of data to inform teaching and learning at a local, service-based level requires that teachers have sufficient time and space to collect, make sense of and discuss data. The collaborative and dialogic nature of effective data use (Bocala & Boudett, 2015; Hoogland et al., 2016; Marsh et al., 2015) suggests that creating non-contact space for teaching teams to regularly meet together is an important component. Currently, New Zealand ECE services vary considerably in the amount of non-contact time available to staff, either as individuals to collect and analyse data or for team meetings to discuss collective findings. This variability creates potential for inequitable outcomes for children, dependent on the time their teachers have available for data gathering and evidence-informed conversations (Earl & Timperley, 2008).

Finally, we note the importance of leadership—at the organisational, service and pedagogical level—as a characteristic that can either support and enhance teachers’ engagement with and use of data or act as a barrier against such engagement. Leaders play a key role in the development of a culture that values using data appropriately and in the creation of time and space that enables such a culture to flourish. However, strong and effective leadership is also supported and enabled by effective government leadership policies that support effective leadership, including the provision of professional learning opportunities for existing and potential leaders.

In summary, the Data, Knowledge, Action programme of research has worked with teachers to explore the premise that access to and use of authentic data can enhance early childhood teachers’ practices in multiple ways. We have paired this focus on access to and use of data with the professional support needed for effective data use, including supporting teachers to develop data literacy skills and modify data systems to work within their local contexts. Our findings suggest the use of new data tools paired with data supports can be transformative for teachers. We acknowledge the range of supports and resources made possible by funded projects, such as our TLIF and TLRI and encourage further research to explore the multiple pathways to building a culture of data use in a range of early learning service types.