Introduction

Policymakers and educators across the globe strive to provide equitable opportunities and outcomes for all students. Although equity has been an abiding concern in various educational reform efforts, accountability policies have intensified the use of data to illuminate and address differences in achievement across racial, linguistic, and income groups. Data on student achievement are now ubiquitous in many schools and school systems. Large-scale assessment data are widely available in a large number of countries, and many schools and systems are supporting teachers to engage in formative assessment as well. Online tools have facilitated this collection and analysis of data. There is a heightened expectation that data will inform decision-making in all aspects of schooling.

A growing body of research has been published examining the supports and constraints for data use, the role of teachers and leaders in the data use process, professional development for data use, and the successes and challenges of engaging in data-driven decision making (e.g., Coburn and Turner 2011; Lai and Schildkamp 2016; Mandinach and Honey 2008; Marsh 2012; Means et al. 2009; Schildkamp and Poortman 2015). The research on data use tends to be dominated by conceptual frameworks rooted in organizational learning (Christman et al. 2009). In an organizational learning framework, the focus is on identifying problems, creating action plans to solve them, and developing a collaborative culture of inquiry (Christman et al. 2009). Data use is conceptualized as part of a cycle of instructional improvement (Goertz et al. 2010) in which individuals engage in a process of defining a problem or setting goals, gathering data, analyzing data, followed by action planning and evaluating outcomes (Coburn and Turner 2011; Schildkamp and Poortman 2015).

The cycle of instructional improvement has its roots in the plan-do-study-act cycle of Total Quality Management, an approach that was prominent in the corporate sector in the 1980s (Goertz et al. 2010). These frameworks are not only common in data use research, they are also widely prevalent in school, district, and state data use practices (Goertz et al. 2010). Meanwhile, researchers have explained that the process of data use is more complex, contextual, and less rational than this linear model might suggest (e.g., Coburn and Turner 2011; Schildkamp and Poortman 2015; Vanlommel et al. 2017).

Power issues and other social justice concerns are features of the data use process, although many studies of data use have not focused on these elements. Power comes into play as individuals in higher level positions use data use processes as a form of managerial control, influence which data are prioritized, or influence interpretations of data (Coburn and Turner 2011). Some problematic data use practices further marginalize low performing students, such as publicly displaying test scores (Neuman 2016) or engaging in “educational triage” by focusing remediation efforts on students on the cusp of proficiency (Booher-Jennings 2005; Gillborn and Youdell 1999). Equity concerns related more broadly to high stakes accountability systems, such as a narrowing of the curriculum in schools serving low-income students of color, have also been raised and associated with data use (Diamond and Cooper 2007; Lipman 2004).

Large-scale accountability policies, while drawing attention to systemic inequities, are often narrowly focused on highlighting student achievement gaps at the expense of understanding and mitigating the effects of unequal educational conditions and processes. Reform efforts have consequently focused on ways to close these gaps. We concur with Ladson-Billings (2006) who argues that the term “achievement gap” unfairly constructs low-income and minority students as defective and lacking and places the burden on students to catch up. Instead, using the term “education debt,” moves us to a discourse that holds educators and policymakers accountable for providing students access to the educational resources that will allow them to achieve at high levels. She further advocates for deploying our best research knowledge, skills, and expertise to alleviate the poor educational conditions and learning experiences that students receive in many schools. Ladson-Billings’ arguments dovetail with those of Valli et al. (1997) who explain that, “Equity has replaced the older concept of equality of opportunity…equity places more emphasis on notions of fairness and justice, even if that requires an unequal distribution of goods and services” (p. 254) for equitable outcomes to be achieved (cited in Gillborn and Youdell 1999, p. 3).

Data use could be an important lever for achieving equity and ameliorating the educational debt, but how and under what circumstances this may occur has not been well documented. With a decade of data use policies and practices behind us, it is important to reflect upon the question, how do data use practices open or close doors for students? The aforementioned studies provide insight into some of the equity issues that may arise with data use, but a more thorough investigation is in order. In this article, we draw on prior research and data from our own qualitative studies to discuss the tensions that educators face in using data and the consequences for equity. We argue that an equity agenda needs to be at the forefront of the field’s understanding and study of data use in schools.

Methods

Over the past decade, we have conducted several qualitative research studies on data use, examining practices and beliefs. Our first study was conducted in 2006 when data use was becoming a prominent feature on the educational reform landscape. In this study, we focused on how school systems support schools to use data effectively (Datnow et al. 2007). We used case study methodology to examine two schools in each of four school systems across the US. The study included six elementary schools, one middle school, and one high school serving ninth graders only. Our site visits to the school systems and schools took place in 2006. We interviewed 3–4 central office administrators, six school principals, and teachers across grade levels at each school. We conducted approximately 70 interviews across the four school systems and schools. At each school, we also conducted informal observations of the school and classrooms and relevant meetings and gathered a plethora of documents related to data use.

The second study was conducted in 2007 and involved a case study of high schools that were engaged in data use, as most of the prior research in the field had been conducted in elementary schools at that time (Datnow et al. 2008). This study involved four high schools across the US. We interviewed site and district administrators, department chairs, and teachers from a variety of departments. Overall, we conducted approximately 50 individual semi-structured interviews and six focus groups, which allowed us to compose case studies using input from over 90 teachers and administrators across the four sites. We also observed data discussions in the schools during site visits and gathered relevant documents.

In both of the aforementioned studies, we purposively chose schools and systems that were considered to be early leaders in data use and had records of improving student achievement over time. Some of the systems were typical school districts and some were charter management organizations, as some of these organizations were early adopters of data use. It is important to note that these schools and systems were implementing data use practices in concert with other organizational, curricular, and instructional reforms, and that while they were making strides, by no means did they consider themselves as having “arrived” at best practices in data use. These schools and systems acknowledged that data use was part of an ongoing, iterative process of continuous improvement. Our findings from these studies are elaborated in numerous publications, along with a more detailed discussion of the methods (Datnow et al. 2007; Datnow et al. 2008; Datnow and Park 2009; Datnow and Park 2014; Datnow et al. 2012; Datnow et al. 2013; Kennedy and Datnow 2011; Park and Datnow 2009; Park et al. 2013).

In the course of these earlier research projects, questions around equity arose, especially as educators disaggregated data by student subgroups and made decisions about which students to focus their energy on or how to focus the curriculum. In 2014–2016, we conducted a much more intensive study that took a deep dive into teachers’ work with data in order to expand the existing research base on equity and data use (Datnow et al. 2018; Park 2018; Park and Datnow 2017; Park et al. 2017). Few studies in this area “zoom in” on teacher practice (Little 2012), and thus we were motivated to find out more about how teachers actually used data, what types of data they used, and how their instruction was affected. We approached this work from a social constructivist framework, acknowledging that teachers’ conceptions of data use and of their students’ abilities are produced in the course of their interactions with other teachers, administrators, and students. In this study, we were particularly interested in how teachers used data for instructional differentiation and how their conceptions of ability were influenced by the use of data.

We studied 4th and 5th grade teacher teams in four elementary schools located in four different districts. Over a period of 2 years, we conducted extensive observations of teacher team meetings and classroom and in-depth interviews, typically making biweekly visits to each school. We felt this in-depth work was necessary in order to answer these important questions about teachers’ use of data and examine how such efforts are impacting students’ opportunities to learn. In total, we conducted 82 teacher interviews and 17 administrator interviews, 180 h of teacher team meeting observations, 117 h of classroom observations, and a review of documents related to data use.

Teachers and administrators were interviewed 3–4 times over the course of the study. During these interviews, we gained background knowledge of the participant and school contexts, the existing data use culture in the school and teacher teams, and the uses of data for instruction. All interviews were taped and transcribed verbatim. During observations of meetings, we took detailed field notes on how teachers talked about data, students, and instruction. We also gathered relevant documents, such as meeting agendas, data discussion protocols, and rubrics. When observing in classrooms, we took running notes of classroom activities and also made note on how data were used to inform instruction and how students were grouped for instruction.

All data were coded using MAXQDA qualitative coding software. We began by analyzing the observational and interview data with a set of a priori codes that we developed from our reading of the literature. We coded data on a variety of constructs, including but not limited to teachers’ beliefs about data and data use; fixed, non-fixed, and mixed conceptions of student ability and their relationship to data and student characteristics; the role of leadership in teacher collaboration for data use; the types of assessment data used, when, and by whom; school or district policies and structures that shaped data use; class placement processes; and the logics of instructional differentiation.

This article draws on qualitative data gathered in the aforementioned studies, primarily the most recent study. It is this research base, interwoven with supporting examples from relevant literature, that we use to inform the analyses in this paper. The literature discussed in this paper was found using searches for research on data use and equity in schools, broadly speaking. We focused on empirical research, rather than “how to” guides that are common in the data use literature. Reviewing relevant sources also led us to other sources that had not come up in our original search. Colleagues and reviewers also recommended sources. We found that most studies that addressed the issues of equity issues in data use were part of broader studies, not those specifically focused on equity. This article is intended to provide a set of empirical arguments about data use and equity along several key dimensions.

Equity issues that arise with data use

Data-informed decision making can help schools monitor their progress towards equity goals (Skrla et al. 2004). But we cannot simply promote data use and expect good things to happen. Educators play a critical role in shaping how and why data are used, what counts as data, and what people are aiming for when they push the use of data in schools (Coburn and Turner 2011). Data do not drive decisions by themselves (Dowd 2005). Data-informed decision making is thus a more appropriate term for educational practice (Knapp et al. 2007). Although the term data-driven decision making is commonly used in the field, from here onward we will refer to the practice as data-informed decision making to signal this important shift in thinking about data use.

Our interest in this article is uncovering the ways in which data use may serve to open or close doors for students. There are a variety of ways in which data can be used in the service of expanding students’ opportunities to learn. Data can be also used, sometimes inadvertently, to limit students’ opportunities. In this article, we bring both extant literature and our own research findings to bear on a set of data use practices in which some of these core tensions are present. These include tensions between:

  • Accountability-driven data use and data use for continuous improvement

    • Data use meetings focused on instructional improvement and administrative compliance

    • Focusing on large scale assessment data and multiple types of data

    • Focusing on narrow bands of students and carefully examining data on all students

  • Using data to confirm assumptions and using data to challenge beliefs

  • Using data for tracking and flexible grouping to promote student growth

In exploring these tensions, we are not suggesting that these are dichotomous practices, with each existing on one end of a spectrum. In reality, there are a range of practices undertaken within each realm, some of them existing simultaneously within the same school. For example, it is quite common to find schools in which districts focus primarily on large scale assessment data, whereas teachers are much more inclined to look at students’ achievement holistically (Coburn and Talbert 2006). Within the same school, we can find educators who use data to confirm assumptions about students, as well as those who try to use data to challenge beliefs, and educators who do both, depending on the context. So too, even in schools that use data to track students by ability, we find educators using data in the service of flexible grouping. Our aim is to explore some of the equity issues that arise along these dimensions, while elucidating the complexity and variation in data use processes.

Accountability-driven data use and data use for continuous improvement

Accountability demands have figured strongly into data use in many countries including Australia, Belgium, the Netherlands, the United Kingdom, and the United States, among many others (Pierce and Chick 2011; Mandinach and Honey 2008; Schildkamp et al. 2014). What is clear in the literature is the delineation between high-stakes, accountability-driven data use, which emphasizes complying with external pressures and bureaucratic demands, and data use for continuous school improvement and organizational learning. Across the globe, educators often have to balance these two competing agendas simultaneously (Lai and Schildkamp 2016). As educators balance various demands, key decisions are made with respect to data use and equity.

Firestone and González (2007) explain that an accountability-driven data use culture focuses primarily on raising student test scores and tends to have a short-term time frame. Data are used mainly to identify problems and monitor compliance. When the stakes are high, “data can slant the system towards trickery and treachery” (Hargreaves and Shirley 2012, p. 39). These misuses of data and the overemphasis on test score data have led policymakers in some places, such as Wales and Alberta, to abolish assessments at various grade levels (Hargreaves and Shirley 2012). Other countries, such as Finland and Germany, test only a subsample of students rather than engaging in high-stakes testing of all students and, as Shirley (2017) explains, “This has sheltered German educators from the curriculum narrowing that occurred in US schools” as a result of No Child Left Behind (NCLB) (p. 61).

In contrast to accountability-driven data use, Firestone and González (2007) note that data use for continuous improvement focuses on student and organizational learning and instructional improvement with a long-term scope. In this model, data are used for diagnostic purposes and with integrity (Shirley 2017). Data are used as part of a reflective process aimed at enlightenment and guiding and mobilizing action to improve the school over time (Firestone and González 2007). While continuous improvement can be a useful frame for data use, equity issues may still go unexamined, unless problems are framed as such, as we will explain. However, a continuous improvement focus is more likely to cohere with equity than an accountability focus.

It is also important to note that even when data use is framed in terms of continuous improvement, educators still may experience data use as a form of accountability. In Goertz et al.’s (2010) study, while the Philadelphia district’s implementation of benchmark assessment was ostensibly aimed at continuous improvement in mathematics, teachers felt pressured by principals. “If you don’t do well, you get talked to,” explained a teacher (Goertz et al. 2010, p. 76). School context also matters. As Diamond and Cooper’s (2007) study of Chicago showed, higher performing schools could meet the vision of using data for instructional improvement, whereas low-performing schools who were under sanctions looked for quick-fixes to get out of probationary status. Many of these probationary schools served large numbers of low-income students of color and were characterized by a narrow focus on particular forms of data, as well as targeting of students, grade levels, and subject areas.

Data use meetings focused on instruction and administrative compliance

Leaders help set the tone for data use among teachers, focusing them away from or towards accountability, continuous improvement, and equity concerns. Correspondingly, the tension between accountability and continuous improvement is reflected in whether teacher collaboration meetings are seen as a place for genuine inquiry around student learning, or whether they are administratively regulated around compliance demands related to data use. These different foci have other important implications for equity, particularly when we consider the types of schools in which these different patterns of data use take shape. The focus on data use solely for accountability purposes (or not) shapes the decisions about what data are used, as well as how discussions among teachers unfold. Because professional learning communities are often the primary vehicle by which teachers learn to use data, the character of them matters a great deal (Daly 2012; Jimerson and Wayman 2015; Marsh and Farrell 2014). As Huguet et al. (2017) argue, if teachers have unequal opportunities to learn, so will their students.

Horn et al. (2015) found that teacher workgroups used different logics when interpreting the same district math assessment data. One teacher work group was focused on instructional management, spending their time categorizing students according to achievement levels. This teacher work group’s instructional management logic aligned with the principal’s focus on accountability. They adopted his frame of data use as a monitoring activity, rather than as a vehicle for examining students’ mathematical understanding. In contrast, another teacher work group’s discussions focused on instructional improvement, including deeply investigating the source of student mistakes. This pattern has been documented in other studies as well. Gannon-Slater et al. (2017) found that an accountability focused data culture at the school they studied led teachers to focus on the technical dimensions of teaching and “what works,” displacing equity aims. Huguet et al. (2017) explain, “In some cases, the work in a PLC (Professional Learning Community) can lead to evidence-based conversations that lead to a re-thinking of instruction, while in other cases, teachers can experience PLCs where time is invested in more procedural matters rather than reflection and learning tied to data” (p. 378).

We have also seen different orientations in the teacher team meetings we have observed. In our earlier studies, numerous schools utilized data analysis protocols and tools in teacher team meetings. Many teachers described productive work related to these data discussion protocols (Datnow et al. 2012). However, in other cases, the administrative regulation that accompanied the protocols led some groups of teachers to focus on the tasks (e.g., completing a form describing the outcome of their discussions of data), rather than on meaningful discussions around data and instructional improvement. For example, in one data team meeting we observed, the overall point of the meeting appeared to be making sure that all parts of the protocol were addressed. With the focus on form completion, teachers spent less time engaging in a deep discussion of instruction. At the culmination of the meeting one teacher filled in the last part of the protocol and remarked, “Yay, we’re done! What time is it…?” In contrast, at another school, a teacher described the stages the teachers went through when working with data for instructional improvement:

The first time we did it we just kind of shared results informally. The second time we did it we brought student work samples, and we got in vertical teams to … analyze what papers looked [like] at different grade levels, to kind of get that perspective, and really focusing on the positives of what sort of math knowledge is the student demonstrating through their work.

We have also found that teacher team meetings can have competing purposes, defined by teachers in terms of instructional improvement but in terms of accountability by the principal. In one school we studied, fourth grade teachers were eagerly engaged in collaborative planning on a writing assignment. One teacher explained that she planned to have students pull quotes from Cesar Chavez and write about how the quotes relate to their lives. At this moment, the principal interrupted their meeting time to discuss procedures for administering and building students’ stamina on the upcoming state test. By the time the principal was finished, the teachers had 10 min left in their meeting, which left them feeling rushed. Although the principal told the teachers that she wanted to protect collaboration, her interruption sent a strong message that accountability demands were top priority in this low performing school.

In contrast to the accountability focus of this principal, Park et al. (2013) reported on schools and districts in which leaders strategically framed the use of data in an effort to produce more equitable learning opportunities and outcomes for all their students. These leaders constructed sensemaking frames that centered on the need to confront disparities in opportunities-to-learn, school improvement as a shared collective responsibility, and making incremental structural and cultural changes to ensure sustainability of reform goals. The school principal and a coach in a school we studied redirected dialogue towards students’ strengths rather than weaknesses and oriented the conversation around improving practice (Park 2018). In both of these cases, we see a deliberate commitment on the part of leaders to frame data use for instructional improvement as an equity-driven activity.

What counts as data? Focusing on standardized test data and using multiple types of data

Educators’ and policymakers’ decisions about what counts as data play an important role as well. Standardized tests themselves raise questions from an equity standpoint as they have long been criticized for their orientation towards forms of knowledge that privilege white, middle-class students (Garner et al. 2017; McNeil 2002; Santelices and Wilson 2010). As such, they put students from marginalized communities at an unfair disadvantage. High stakes testing also “creates intense, unfair pressure for teachers and administrators to raise scores in a system that does not support success for students of color, emergent bilinguals, and students from low-income families” (Garner et al. 2017, p. 422). These are concerns with standardized tests not with data use per se, but it is important to note that these issues arise when data use is narrowly defined to large-scale accountability measures.

With these important pitfalls of data use in mind, it is essential to address the relationship between federal and state accountability policies and data-informed decision making. First, the emphasis on using evidence (broadly speaking) to inform educational decision making is not new and can be found in many earlier reforms in and outside education. Thus, we should not confuse data use and accountability, nor should we assume that the two must be linked. The problem is that the type of data use that is connected to accountability policies is one that is narrowly construed, both in terms of what data is used and how it is used. Accountability policies have prompted many districts and schools to be more focused on data use (Marsh et al. 2006). These accountability pressures also motivated many districts to adopt benchmark assessments aligned to state tests in order to examine on an interim basis how students were achieving relative to state standards (Braaten et al. 2017; Jimerson and Childs 2017; Snodgrass Rangel et al. 2017). In some cases, examining data on benchmark assessments has motivated teachers to look at a range of other information that can inform them about student learning and guide instructional changes (Datnow and Park 2014). In other instances, data use is organized only around these benchmark assessments and has led to problematic consequences, such as a district’s strong accountability orientation inadvertently compromising teachers’ attempts at meaningful instructional improvement (Braaten et al. 2017).

Educators focused on continuous improvement actively seek out a wide range of data and do not limit themselves to data linked to accountability mechanisms. As one teacher shared: “I look at [the benchmark assessment] as a snapshot on that day, but what I need to use is a range of data and see where the kids are falling.” The limitations of benchmark assessment data and teachers’ interest in creating a more complete portrait of student achievement leads teachers to draw on a wide range of data to inform instructional decisions. When we interviewed teachers, we asked them what information they drew upon to make instructional decisions and this helped illuminate the wide range of “data” they rely on. Without providing this clarification, teachers often conceptualized data only in terms of numeric representations derived from large-scale assessment tests, including district benchmarks. Meanwhile, the sources of data that teachers actually relied upon went beyond these measures and included teacher created assessments, curriculum embedded assessments, writing portfolios, results from student work with online instructional tools, and of course their own observations of student learning. Drawing on this wide range of data allows for a much fuller portrait of student learning, allowing for a greater possibility that the strengths of students will be evident.

Some teachers we observed had developed their own methods for gathering observational data on student learning to inform instructional planning. One teacher developed a unique form to document students’ mathematical understanding. As students worked in groups to solve a complex math problem, she walked around the room and noted (on a form she created) their problem-solving strategies, a compliment she could later give them about their work, a teaching point for small group, and future teaching notes for the class as a whole. This tool helped her more systematically document student learning in class (not limiting her assessment of learning to test score data) and linked her observations to instructional decisions.

In one district we studied early on, administrators mentored failing students, which involved shadowing students during the school day in order to better connect students to educational resources in the schools. The mentoring began as an effort to connect more with students; however, a secondary benefit was that it gave district administrators data on how their schools’ operated from a student perspective. While mentoring an English learner, the superintendent found that the student was receiving all Fs and nobody was doing anything about it. She also observed, “Probably not once in her instructional day… does she talk unless she’s made to.” This led the superintendent to organize professional development for teachers that was focused on student engagement and providing opportunities for students to frequently use language in the classroom. These examples show that numeric data do not yield all of the answers. In this district, leaders were motivated to look beyond numbers (which showed the majority of students meeting standards) to try to uncover why some students were receiving failing grades.

Examining discrepancies across multiple forms of data can help illuminate equity concerns and allow for a more complete picture of student learning. However, the data use process must not stop at the documentation of patterns. Rather, the examination of data must be connected to instructional improvement. As one administrator explained, “Even if they have 50 assignments and they’ve collected data, well that’s just data. That’s all well and good; really the big push then is now that you have it what do you do? How does this inform your instruction? What are you doing with the information is a big deal.”

A teacher at one school we recently studied talked about how analyzing multiple forms of data informed her instruction. She focused on gathering data about students’ progress towards instructional objectives chosen by her department:

It makes for better instruction when you know where the kids are at because otherwise you’re not really sure what you’re teaching…. And then also for us to be able to tell at the end of the semester what have the kids actually learned, and have they actually grown. And if you don’t have any data to support that, then you can have as much as anecdotal evidence as you want, but there’s no guarantee that they’re going to go to college more prepared or they’re writers.

An administrator explained that focusing on instructional objectives made teacher collaboration time more fruitful: “We’ve spent time in our department talking about which objectives we want to track, how we track them, how are the kids doing in those objectives, who’s doing what right now with those, and so it’s made our conversations, I think, more focused and more productive.”

In sum, using multiple forms of data to inform instructional decision making can help provide a more holistic picture of student achievement and provide more ways to see evidence of student growth. This allows for a greater possibility that the strengths of students who have historically been disadvantaged by standardized measures will be evident. Drawing on a wider range of data may also lead to more informed, differentiated instruction. Observational data and other forms of non-numerical data may also provide perspectives that could inform more equitable policies and practices, as was the case in the district where administrators shadowed failing students and subsequently planned professional development.

Conversely, focusing on limited data, especially data from high-stakes assessments that are themselves biased against low-income students of color, can work against equity goals, contributing to narrow instructional goals and higher dropout rates (McNeil 2005; Valenzuela 2005). As Valenzuela (2005) argue, “When the test is the sole or primary arbiter in decisions with such long-lasting consequences for children, we insist that students have the right to be assessed in a complete and fair manner, using as many criteria as may reasonably indicate children’s cognitive abilities and potential” (p. 2). At the same time, while using multiple forms of data is necessary, it is not sufficient, as we will explain.

Focusing on narrow bands of students and carefully examining data on all students

Under increased public scrutiny, NCLB and related policies in the US provided rewards and sanctions for schools based on the number of students who are performing at or above grade level. As one unintended consequence of NCLB, educators in Texas engaged in “educational rationing and triage” by examining state assessment data to determine which students are on the “bubble” or cusp of grade level proficiency and then investing resources to push these students over the threshold of the proficiency marker (Booher-Jennings 2005). The same practice was used by low performing schools in Chicago (Diamond and Cooper 2007). These practices were not limited to the US, however, as Hargreaves and Shirley (2012) documented a school in Ontario that focused test preparation on the 20% of students just below the passing mark. We found a similar pattern in our earlier research. As one teacher explained to us, “I’m always teaching to the middle—not even the middle, but just that passing mark and bringing kids above that passing mark.” Students who are very low performing were considered “lost causes,” and those performing above grade level were not focused on either. This “educational triage” approach leaves many students behind. By focusing on the use of data to avoid sanctions, educators, intentionally or unintentionally, subvert the intended goal of these policies.

This focus on students on the bubble shows up not just in individual teacher planning, but in teacher conversations as well. Halverson et al. (2007) also found that faculty data discussions overwhelmingly centered on helping students below proficiency levels with few discussions focused on raising students from proficient to advanced levels. While such practices may result in greater numbers of students achieving ‘above the mark,’ they inadvertently compromise the overarching purpose of data driven practice, which is to expand opportunities for all students.

Achieving goals of equity requires carefully examining data on each and every student. Two of the elementary schools we studied developed elaborate processes. One school has long used a process by which teams of teachers sit down with the principal, a counselor, and 2–3 intervention teachers three times a year (fall, winter, and spring) to discuss data and plan instructional interventions for every student. At the spring meeting, the group also discussed classroom placements for the following year.

To prepare for the meetings, the school’s data clerk (a position the principal chose to fund using Title I money) organized student data by student which enabled the team of educators to discuss each student and their performance on various assessments. These meetings took about 2 h and teachers were provided with substitute release time to attend. The team examined a wide variety of data on student achievement including a range of assessments that were school, district, and teacher administered, interventions or other designations (e.g., English learner), all of which appeared on a single sheet and was listed by student. They paid particular attention to increases and decreases in student performance because, as the principal explained, “We want to be sure that if they scored high or low in one particular area and it seems to be a fluke, we have other assessments that we can kind of tease it out to see, ‘okay, was this just a bad day or is…is this where they really are’?”

Pulling together an array of data on each student in a single location is critical to this process. As a coach at another elementary school explained the rationale behind this:

It’s easy to make assumptions on certain things, and when you have information spread in different places it’s easier to overlook things. So compiling the same information on all of the kids and having it just in one document to me was a lot easier. It was an easier reference tool for me, and then I kind of in there kept all of the resources and intervention supports that they had and what not.

A notable feature of these meetings, however, was that while meeting participants all had numeric data on student achievement in front of them, their discussion was not restricted to numbers. Most discussion focused on providing a full picture of the student beyond academic achievement and involved discussions of students’ home lives, social adjustment, classroom behaviors, work habits, completion of homework and so forth, none of which were reflected on the data sheets. Wardrip and Herman (2017) also found that “Teachers framed students’ instructional needs with information that fell outside of the data the school collected” (p. 14). In their study, knowing the students’ “backstories” (e.g., their friends, aspirations, interests) provided a context by which educators reasoned about student data.

Examining data on all students also promotes shared responsibility, which is a key component of data use for equity. District and school leaders play an important role in cultivating a belief that teachers must share responsibility for all of the students at the school, not just those they have contact with in their classroom. One district we studied attributed their success in improving high school student achievement across racial and ethnic groups to their beliefs about their students. She stated: “There’s ownership for every student. We take them where they are, regardless, and we move them as far as and as quickly as we can. When they walk in the door, they belong to us.” She believed that children learn when there are strong relationships, and this was a common mantra among many of the leaders and teachers.

A shared sense of responsibility was also a key element in an elementary school we studied; this is the same school we described above that examines data on every student. A teacher explained: “We’re here to serve the whole community, and not just the individual students that, as a grade level, we’re trying to help all students be successful. I think when you have that common purpose, it really helps.” When educators at this school collaborated to examine student data, they held each other accountable for ensuring student success. Teachers made it their business to know each other’s students and students’ families. When one teacher shared that he or she wasn’t having success at meeting a particular student’s needs, others shared information that may be useful. Underlying the educators’ shared sense of responsibility were common beliefs about high expectations for students and for themselves as professionals, as well as a high sense of trust. A teacher explained:

It’s just that mentality of, you’re responsible for every kid, you’re not responsible just for the kids in your classroom, and that goes back to really trusting your partnership, your team. When I hand them off to Amy or Carla, or I have to know that they’re going to do their job, and that they’re going to do the best that they can for that child. And vice versa…

Another teacher concurred and explained that building this culture took time. A longtime teacher at the school explained that they started with small goals when it came to collaborative work, sharing students with each other, and sharing data. Initially, a teacher team analyzed data on a benchmark assessment in language arts to uncover the concepts where students were struggling the most. They then agreed to regroup students across classrooms for 1 h a week to target those skills. She explained, “That limited amount I think was really beneficial because it gave us that area that we could have the conversations about and do some planning and kind of add in those elements of collaboration.” Relationships developed as trust grew.

In sum, equity cannot be achieved unless data use efforts focus on all students. When data use is narrowly focused on students “on the bubble,” many students get left behind. A focus on all students allows data use efforts to address students’ individual needs and for instructional changes to focus on lifting the class as a whole, rather than small groups of students. It also helps to promote a shared sense of responsibility for all students.

Using data to confirm assumptions and using data to challenge beliefs

A goal of data driven decision making is to bring evidence to light that will help educators think about student achievement in a new way. However, examining data does not always lead to new interpretations. In a study of data use in Belgium, Vanlommel et al. (2017) found that “teachers seek for data that confirms what they believe to be true and avoid data that questions their beliefs” (p. 81). Data can be used to validate existing understandings of students’ learning profiles (Oláh et al. 2010). As Oláh et al. (2010) explain, teachers interpret data in the context of their expectations for individual students or for the class as a whole and rely on their background knowledge about students when interpreting data. When data do not fit with teachers’ expectations of students this can send up a “red flag” as one teacher in this study noted, which may turn out to be useful in addressing students’ learning needs.

However, when educators use student characteristics as explanations for results, they can reinforce a culture of low expectations and stereotypes (Bertrand and Marsh 2015). For example, a teacher in Bertrand and Marsh’s study discussed data on her students such that “the connection between ‘resource kids’ and the lack of proficiency was self-evident, suggesting an assumption that students in special education score poorly on tests by nature” (p. 19). Moreover, this teacher attributed the assessment results to her students, not to her teaching. The attribution of results to what were thought to be stable student characteristics was a pattern in this study, and English learners, special education students, and other struggling students were “blamed” for outcomes.

When educators used data to confirm assumptions, this often occurred hand-in-hand with comments about students’ home lives being the primary explanation for student achievement. For example, one teacher we interviewed stated, “Go into the [district] website … and go take a look at the scores. You’re going to find out that Asian kids score higher than white kids and Latino kids, and my experience is because of the discipline at home.” When such assumptions are made, it is very difficult for educators to see data as a useful tool for improving their own practices, as they locate achievement patterns in the students’ backgrounds.

Paying close attention to teacher talk in the context of data collaboration meetings could help researchers examine how teachers think about student ability, as teachers often discuss data in relation to individual student achievement. In our own research (Datnow et al. 2018) and in Bertrand and Marsh’s (2015) study, the No Child Left Behind era accountability language has figured strongly into teachers’ talk. Terms used to describe students such as “low,” “high,” or “below basic” are reinforced by assessment data showing which students were below, at, or above grade level. These labels are imposed upon teachers by broader policies for accountability and have become part of teachers’ lexicon. This lexicon appears to reify a hierarchy of ability and can constrain the ways in which teachers talk about student achievement and learning, as well as their expectations of students.

At the same time, data use has been shown to be a powerful tool to push teachers to challenge existing assumptions about student learning and to reflect critically on instructional practices (Lachat and Smith 2005). In one district, addressing teachers’ low expectations for students who came from low-income families was a major hurdle. A principal had to explicitly persuade her staff that, “Yes, we have challenges, but our kids can do it. Now, look at other schools that are doing it, and comparing them, we can do it.” Interestingly, she noted that the shift towards viewing the relevancy of data did not come about until the school began to disaggregate data to the teacher level, rather than at the school level. She shared that teachers began to make the shift when the school started to examine individual teacher data for their students, teacher attendance, and so forth.

Also in this district, a school administrator explained that data have been powerful in changing teachers’ perceptions about special education students. Over a period of 2 years, data showed that inclusion was a success because many special education students could now perform at grade level. She recalled that staff members initially expressed doubts about its success. Data played a key role in helping teachers admit that instructional strategies, not children, were at the root of the problem of low student achievement. This belief shift was difficult. This administrator believed that the first step in changing attitudes was building trust, so that teachers felt secure enough to come to a meeting and admit that, “My kids are not learning and ask how can you help me?”

The process of closely examining data in the context of teacher team meetings can facilitate teachers’ focus on student growth, thereby shaping teachers’ beliefs about what they think their students are capable of. The team meeting can provide a space and a routine for teachers to point to various forms of “data” when making claims about student achievement. For example, in our own work (Park 2018), these opportunities for dialogue around data helped teachers move away from deficit framing of students and towards discussions of student growth and potential. Coaches or other instructional leaders played a critical role in reframing conversations around student assets. For example, when a teacher expressed concern or frustration about “low students” or attributed low academic performance to ability or motivation, leaders redirected the conversation to also highlight the specific learning skills that students did exhibit. This can be as simple as asking the questions, “What did the student do well?”

In the end, data can play a very powerful role in challenging stereotypes and providing an opportunity for educators to examine the relationship between instructional practices and achievement. But simply deciding to build professional learning communities is not sufficient to bring about change. As Pollock (2017) argued, school talk must debunk myths about intelligence as easily measurable and explicitly challenge common comments about young people or families that are harmful. Framing conversations carefully and providing the opportunity for educators to bring multiple sources to bear on conversations about student achievement are critical.

Using data for tracking and flexible grouping

Educators’ conceptions of student ability inform their logics about how to organize instruction for students. In a study of ten detracking schools, Oakes et al. (1997) found that some teachers held a conventional view of intelligence in which ability was seen as a uni-dimensional, fixed, and innate characteristic. This led them to believe that intelligence could be easily assessed and to support tracking and ability grouping. Beliefs that overlapped race with ability were also prevalent among this group. In contrast, another group of teachers believed intelligence was multi-dimensional and plastic, leading them to support mixed ability grouped classes (Oakes et al. 1997, p. 494). Data have long been an important part of tracking practices in schools. The use of data starts early in the educational process as students are often assessed for school readiness in kindergarten and then move through the system at different speeds (Oakes 2005). By high school, disproportionate percentages of African American and Latino students are in courses that include less rigorous instruction, less qualified teachers, fewer resources, and lower expectations than their white or Asian counterparts in higher track classes (Oakes et al. 1992). Thus, tracking has significant consequences for students’ opportunity to learn.

Just as data have been used in the past to justify tracking decisions, in recent years, educators have turned to benchmark assessments as a placement tool, which is not their intended purpose. Instead, these assessments are designed to provide educators with interim feedback on student progress relative to curriculum standards. This has been documented in numerous studies (Davidson and Frohbieter 2011; Heppen et al. 2012; Shepard et al. 2011). In addition to misusing the assessments for unintended purposes, the sole use of benchmark assessments places too much emphasis on one form of data to make such a high-stakes decision. One elementary teacher we interviewed lamented that benchmark assessments were the sole criteria used in her district to place students into different tracks of sixth grade math. She inquired at the district office and was told that her judgment and other forms of data, such as student grades, would not be considered for placement purposes. Although she was primarily concerned about students landing in the wrong courses, an obvious equity concern is the practice of tracking in and of itself.

In one school we visited, the number of spaces in advanced math classes in the upper elementary grades was determined before the teachers had assessed the students. Teachers then administered three assessments to the students. They equally weighted these assessments along with the state test and arbitrarily set a cut score of 90% or above for qualification for the advanced class. If a student received a grade below 90% on any one assessment, he or she would not qualify. Student placements were subsequently publicly announced. Needless to say, some parents and students alike were disappointed, but most did not question the problematic process that led to the placements. These uncritical and high-stakes uses of data can have long-term consequences for students, whose confidence and access to rigorous instruction in math is often tied to the labels that are applied to them. Doors to calculus in high school are also often closed for students who are not placed into advanced math classes early on. There are also long-term consequences for society, given the resulting narrow pool of students who are prepared to pursue careers in science, technology, engineering, and mathematics.

Whereas the use of data for tracking purposes limits student opportunities, the use of data for flexible grouping of students can help to expand opportunities. As part of their comprehensive, three-times a year data analysis process, educators at an elementary school we described earlier created language arts and spelling groups that shifted often. A teacher explained: “These groups are fairly fluid. I mean, not constantly, but several times during the year we’ll kind of readjust our groups as we see kids grow, or if one kid needs a different kind of learning style or teaching style.” The teachers then checked students’ fall benchmark assessment scores to see if the initial groupings made sense or whether changes were needed. As the year went on, homeroom placements remained, but groupings for particular subjects were dynamic as teachers reviewed student progress and made changes accordingly.

In another elementary school, teachers used formative assessment data daily to differentiate instruction and to place students in flexible learning groups to address particular skill areas. One teacher explained her grouping strategy in a reading lesson. After reading a class novel, one group of students worked on figurative language, another group worked on themes, and a third group worked determining good questions. These groups were formed around the concepts particular students needed more practice with: “I group them strategically so that I know that they’re thinking about things they need work on.” Closely examining student work or assessments led some teachers to move beyond categorizations of generalized ability and consequent instructional strategies to focusing on targeting students’ skill levels in particular areas such as fluency, comprehension, or mathematical reasoning. This allowed for a more expansive, nuanced view of what students knew and were able to do.

Although such a strategy can promote student growth and address individual needs, caution is required with flexible grouping as well. For example, in one school, a teacher explained that after students take a common assessment, the team will look closely at the data and see “who is ready to fly” or who “needs small group instruction.” In this description, the small group instruction is premised on the assumption that the grouping is only for remediation instead of pushing learning growth for all students. In this case, only some students would have an opportunity for a close learning encounter with the teacher.

Along these lines, Oláh et al. (2010) study found that elementary school teachers used small, pullout group instruction during a designated re-teaching week. Based on results on a benchmark assessment, teachers determined the students of most need. As a teacher explained, “If it’s half the class… I’ll just reteach the whole thing. But if it’s a few children… then I would definitely pull them out and get some special homework for them to work on” (p. 240). High scoring students received less direct instruction during the re-teaching week and instead were given enrichment worksheets, games, or more time on the computer. This approach to flexible grouping based solely around remediation also has the unintended consequence of stigmatizing the students who are repeatedly “called to the back to table” to work with the teacher.

This is in contrast to other instances where data were used to individualize learning and monitor growth for all students. One school we studied engaged in a process by which language arts goals were set for each student each trimester based on the students’ benchmark assessment results. For example, one student may work on vocabulary development whereas another may work on comprehending informational text, depending on what the assessment data revealed was the area requiring the most growth. While students may work in groups to advance in their goals, there was also a host of individual learning activities they could choose as well. In another school, teachers scheduled individual math learning conferences with students whose math learning patterns inspired questions for them. These math interviews provided a form of “data” that helped inform instruction.

Overall, while we have observed instances in which data use has inspired flexible grouping geared around students’ individual needs and skills, there is certainly a strong cautionary tale to be told about the connection between data use and student grouping. Problematic practices of tracking and ability grouping with long-term consequences continue to abound in schools and are legitimated with data. In fact, tracking remains one of the most enduring practices in American high schools, in spite of a robust research base denouncing it.

Conclusion

This article drew upon extant studies and original qualitative research to examine the ways in which the data use practices in schools may serve to open or close doors for students. We explored tensions between accountability-driven data use and data use for continuous improvement, data use to confirm assumptions and data use to challenge beliefs, and the use of data for tracking and flexible grouping. These tensions were examined and discussed across the scholarship on data use. By synthesizing these tensions as broad sets of decisions, we aim to contribute to research on how data use practices can promote equity.

These sets of decisions are not necessarily dualisms with dichotomous sets of practices. Rather, along each of these dimensions, there are active decision makers, complex processes of data use at play, and a great deal of variation both within and across contexts. These decisions are also intertwined and may need to be considered in relationship to one another. For example, the decision to challenge deficit beliefs about student abilities using data will not likely occur without a broader definition of what counts as data, especially beyond standardized assessments. Conversely, the decision to use multiple forms of data alone will not lead to a goal of expanding students’ opportunities to learn, unless there is a belief in using data to challenge assumptions. Educators and policymakers are faced with a set of critical choices in their work with data that can profoundly affect students’ daily educational experiences and trajectories. Some of these choices are more likely to expand opportunities for students, whereas others are more likely to limit them. This especially true when there is a pattern of choices within a particular school.

When accountability frameworks dominate teacher conversations and instructional interventions are focused on students for whom schools will get the most “bang for the buck” on standardized measures, students are more likely to suffer. The curriculum in these schools is likely to be narrower, with student achievement primarily focused on a single measure, which often disadvantages low-income, minority students. The use of data for tracking, long-term ability grouping, and placement is of particular concern, as it can serve to reinforce hierarchies among students. In schools where data are more often used to actively challenge stereotypes, to examine student growth as well as weaknesses, and to differentiate instruction in dynamic ways, students likely have more opportunities for success. Students are also more likely to thrive when educators share responsibility for their success.

Just as equity needs to be an explicit goal in data use practices, research on data use also needs an equity lens. Educational improvement and policy lenses tend to prioritize organizational changes associated with reforms. Although many educational researchers list equity as an important goal of school reform, this concern may not be central to their work. Putting equity at the center of studies of data use—and indeed all educational reforms—may involve the use of different research questions, methodologies and/or theoretical frameworks. It requires explicit attention to questions and issues raised in this article and in others. In what ways do data use lead to inequitable outcomes for differing groups of students and what are the specific data use conditions and processes that contribute to unequal or equal opportunities to learn? How do educators’ assumptions, often tied to beliefs about students’ backgrounds, become challenged or reified by data use? What data use practices support shifts away from deficit thinking and fixed beliefs about intelligence or ability? How do data-informed leaders with an explicit equity-orientation build a school culture that expands opportunities to learn not just for students but also for teachers?

This equity-centered agenda on data use also may involve assembling diverse teams of researchers, a willingness to engage in difficult conversations, and carefully framing findings to do justice to the complexity of equity-driven change. With data use efforts occurring in a wide range of settings across the globe, researchers have an unprecedented opportunity to further document ways in which data use opens or closes doors for students. Studying data use through an equity lens provides unique affordances that can help to eliminate problematic practices and promote better schooling for all.