1 Introduction and rationale

Mathematics curriculum has long been considered an “agent of change” for improving learning when used by students. Advocates of mathematics education reform often modify the curriculum standards and written materials to improve student opportunities for learning, curriculum enactment/implementation, and ultimately student outcomes. While there is agreement about the importance of curriculum, there is little agreement on a definition for curriculum (Cai & Howson, 2013). In this paper, we define curriculum to be “a plan for the experiences that learners will encounter, as well as the actual experiences they do encounter, that are designed to help them reach specified mathematics objectives” (Remillard & Heck, 2014, p. 707). Our definition highlights both the role of student and teacher and their interactions in the operational curriculum or the teacher-intended curriculum, enacted curriculum, and student outcomes (Remillard & Heck, 2014).

The set of curriculum materials referred to and used in this study is the Connected Mathematics Project’s problem-based curriculum, Connected Mathematics (CMP) (Lappan et al., 2014). A growing body of research supports the theory that students make more sense of mathematics if the concepts and skills are embedded or encoded within a contextualized problem (Wirkala & Kuhn, 2011). A problem-based curriculum focuses on student thinking and differs from delivery mechanism curriculum materials, which uses a “transmission” or “direct instruction” model in which students memorize facts and practice demonstrated procedures (Choppin et al., 2015; Roth McDuffie et al., 2018).

The CMP authors’ view of mathematics curriculum materials as an agent of change differs from the historical view of mathematics curriculum projects in that it was designed for and used by both middle grades teachers and students (ages 11–14). As one CMP author states in Burkhardt and Schoenfeld (2020, p. 3), “[m]any times, decisions about what would go in the student book were based on teacher needs…We wrote extensive teacher support and developed professional development activities side by side with the student materials.” The CMP team’s focus on both teachers and students is reflected in their overarching goal, which is “to help students and teachers develop mathematical knowledge, understanding, and skill along with an awareness of and appreciation for the rich connections among mathematical strands and between mathematics and other disciplines” (Lappan et al., 2004, p. 1). The CMP materials reflect the understanding that teaching and learning are inextricably linked together and the view that mathematics curriculum materials are agents of change when used by students and teachers.

For over 30 years, CMP has engaged in curriculum development and design research (Barab, 2014) to improve the mathematics curriculum materials. In 2014, the CMP team set out to explore the potential of embedding CMP in a digital collaborative platform. Shortly after, a team of mathematics educators at Michigan State University, partnering with educational technology experts at Concord Consortium, received fundingFootnote 1 for a research project to develop a digital collaborative mathematics environment. In this paper, we report on our efforts to support teachers of mathematics in problem-based curriculum through the iterative design, development, testing, and refinement of a teacher dashboard. In this paper, we examine the following research question:

What are teacher dashboard features that support teacher enactment of problem-based mathematics curriculum embedded in a digital collaborative platform?

To address the research question, we ground the teacher dashboard features in a discussion of the design principles around effective problem-based mathematics teaching. We then contextualize our study and provide exemplifications of the design principles in three dashboard features. Finally, we draw on teacher interviews and observational data to describe teacher enactment of the teacher dashboard and how it was used to evaluate and refine the teacher dashboard.

2 Literature review, theoretical foundations, and design principles for teacher dashboards

2.1 Teacher dashboards and learning analytics

Teacher dashboards are visual displays of data that provide teachers with information about teaching and learning (Verbert et al., 2014). “In educational settings a dashboard may include summary information about learners’ attendance and attainment. It may also show an aggregated summary of information about a group of learners such as a class and provide a facility for its user to explore the individual scores that make up this aggregated summary” (Ferguson et al., 2016, p. 139). Teacher dashboards make use of learning analytics and data mining techniques in two primary ways to support students: (1) embedded analytics that can be used in real-time to guide engagement and (2) extracted analytics that can be used to support monitoring and reflection (Wise et al., 2014).

In mathematics education, the focus of learning analytics is predominately on student performance prediction models at the collegiate level, such as student achievement and course dropouts (Ramli et al., 2019). To understand and predict student learning outcomes (e.g., correct answers, achievements, grades), they utilize data mining, machine learning, data visualization, psychometrics, and other areas of statistics. Learning analytics draw on data generated from learning management systems (Chu et al., 2017), online course systems (Kim et al., 2018), and peer-tutoring software (Xing et al., 2015). The goal of these efforts is to develop fine-grained models that take into account individual student differences. Analytic techniques make use of various statistical models that consider many different variables and factors. These models require large amounts of logged data which are not typical or feasible in educational settings. In contrast, the teacher dashboard presented in this study provides teachers with real-time information about student learning processes.

Learning analytics are often a “black box” to the teacher. Hence their role is often reduced to interpret and make use of what she or he is provided and what has been analyzed by the “black box” system. Extending the work of student performance prediction models, some researchers are focusing on how teachers interpret and use learning analytics of student outcomes (Bertacchini et al., 2018), engagement (e.g.,, Kickmeier-Rust et al., 2014), and dispositions (Chu et al., 2017). Faber et al. (2017) report that teachers follow students’ progress on a dashboard to assess individual and whole class progress as well as to assign tasks and activities. The information from the analytics provided to teachers focuses on student knowledge and skills (Dani & Nasser, 2016), student difficulties (Gal et al., 2018), or technical issues in the digital environments. In contrast, the teacher dashboard presented in this study highlights the importance of the teacher and her/his knowledge. The teacher is provided with real-time evidence of student thinking in the digital environment with accompanying learning analytics and the opportunity to analyze, interpret, and use this information to improve teaching and learning of mathematics.

In summary, teacher dashboards that leverage learning analytics provide potentially new information to teachers about the teaching and learning of mathematics. The research underscores the importance of (a) what information is provided to teachers, (b) if and how that information is aggregated, (c) the assumptions that define progress on teaching or learning, and (d) how teachers interpret and use the analytics. More research is needed on the development and use of teacher dashboards that provide teachers with evidence of student thinking so they can access, interpret, and act on it in-the-moment.

2.2 Teacher dashboards and digital curriculum materials/resources

Teacher dashboards are increasingly found in digital curriculum materials/resources, and they differ depending on the nature of the curriculum materials. Choppin et al. (2014) provide a typology for analyzing digital mathematics curriculum materials in the United States. Their analysis on students’ interactions with the program, curriculum use and adaptation, and analysis of assessment systems reveal two distinct types of digital curriculum materials: digital individual learning programs and digitized versions of traditional textbooks. Across both types of digital curriculum materials analyzed by Choppin and colleagues is the particular focus on teacher dashboards with continuous real-time monitoring and reporting of student performance, particularly in the “form of adaptive assessments, data dashboards, and individualization” (Choppin & Borys, 2017, p. 671). These research results suggest that teacher dashboards are primarily embedded within delivery mechanism programs which are designed for novices from an expert performance perspective (Choppin et al., 2015; Roth McDuffie et al., 2018).

The authors of this study were not able to identify any teacher dashboards associated with thinking device curriculum programs (Choppin et al., 2015; Roth McDuffie et al., 2018). For example, we did not identify any programs that link directly with student platforms in meaningful ways to support monitoring of student thinking—individual or collaborative—in real time around curricular tasks. The existing teacher dashboards seem consistent with the delivery mechanism curriculum programs that contain mathematics tasks that are closed, such as multiple-choice items or fill in the blank items where the program can score tasks. Some teacher dashboards serve as classroom management resources where teachers focus more on learning objectives/standards, student grades, and attendance. We found that programs outside the U.S. also placed emphasis on features that support teachers using a “transition” or “direct instruction” model to (a) create and assign worksheets customized to the learning objective needs of their students, (b) monitor whether students have activated and/or completed assigned tasks, and (c) use summative data to improve the teaching and learning of mathematics.

2.3 Teacher dashboard: participatory relationship between teachers and digital curriculum materials

Our approach in this study emphasizes the importance of the relationship between teachers and technology devices. For example, Drijvers et al. (2010) highlight the complexities of how teachers integrate technology for particular purposes in the classroom. Their instrumental approach distinguishes between three critical elements for understanding teacher’s intentionality and use of technology: (1) the configuration of the technology in the classroom environment, (2) the decisions teachers make to leverage the configuration between teaching and learning, and (3) the ad hoc related decisions teachers make in the classroom. While teacher dashboards are new digital resources in mathematics classrooms, it is also important to consider the relationship between teachers and the curriculum, particularly when teacher dashboards are embedded with problem-based curriculum materials. Adler (2000) conceptualized resources/re-sources as both a noun and verb, where resource can refer to the material object (noun) or to mean source again or differently (verb) (p. 207). This underscores the importance of not only focusing on the resources, but also how they work with teachers of mathematics. Understanding the relationship between teachers and curriculum can provide insights into how teachers use curriculum materials effectively (Ball et al., 2008; Leshota & Adler, 2018).

In this study, the role of the teacher is essential in the design, development, and enactment processes of the teacher dashboard as a digital curricular resource. This includes the teacher’s intended experiences, what the instructional materials specify in terms of problems and activities, how the teacher presents the experiences to students, and what the teacher and students do in class (Remillard & Heck, 2014). We draw on Brown’s (2009) notion of pedagogical design capacity, or “the teacher’s ability to perceive and mobilize existing curricular resources” (p. 29). We also use Remillard’s (2005) definition of “use” as the participatory relationship between the teacher and their interpretation capacities. From this relational perspective, several questions are relevant to our study as teachers purposely make decisions: (a) Why is s/he designing instruction in this particular way? (b) Which aims and goals does s/he have in mind? (c) Who is his/her audience? (d) Which resource and tool is s/he using for instructional design? (e) How does s/he plan to evaluate the design of instruction? The interactions between the teacher and the teacher dashboard features guided the development process of the teacher dashboard. Thus, studying the teacher-curriculum relationship of enacting curriculum materials with teacher dashboards underscores the importance of examining the interactions between the affordances of the resource and the interpretative capacities of the teacher (Rezat et al., 2018).

2.4 Principles for teacher dashboards linked to a student digital collaborative environment with an embedded problem-based curriculum

Teaching problem-based mathematics is challenging as teachers negotiate a balance between curriculum design and teacher enactment tensions in problem-based mathematics classrooms (Edson et al., 2019). The features of the teacher dashboard with a problem-based curriculum are based on research-based design principles of effective mathematics teaching (e.g., Lappan et al., 2004; Stein et al., 2000). We organized the design principles around the Launch-Explore-Summarize instructional model used in problem-based classrooms. Thus, the design of the teacher dashboard, linked to a digital collaborative student environment with a problem-based curriculum, is guided by the following design principles.

During the Launch of the problem,

  • Design Principle 1: The teacher engages students in the mathematical challenge.

    Argument: In the Launch, the teacher engages the students in the challenge and helps position the problem within prior understandings during a whole-class discussion. This is different than “direct instruction” approaches. The whole-class discussion allows students to access problems from different perspectives (Boaler, 1998), provide comprehensive use of knowledge and skills (Capraro et al., 2007), and evoke meaningful mathematical discourse (Michaels et al., 2008). To this end, teachers need access to the problem and evidence of student prior knowledge and progress on curriculum, including individual student and collaborative progress on problems, prior concepts and experiences, mathematical representations and inscriptions, and student definitions of key terms. Examples of artifacts in this space include student predictions to questions, noticings and wonderings to photos, videos, and examples of student strategies and mathematical representations. Additionally, teachers need a whole-class space to present the challenge of the mathematics problem, write notes and expectations, record students’ predictions, and develop a common language about the problem setting, the mathematical context, and the problem challenge. As Jackson et al. (2012) observe, “teachers did not simply talk to students about the key features of tasks but instead solicited input from students” (p. 28). The teacher must be careful not to tell too much and consequently lower the challenge of the problem to something routine, or to be so directive that the rich array of strategies that may evolve from a more open launch of the mathematics problem is lost (Stein et al., 1996). In paper-and-pencil classrooms, teachers tend to use white boards and smart boards, so the teacher dashboard needs the option to be displayed to the entire class. During the Explore of the problem,

  • Design Principle 2: The teacher quickly scans and monitors individual and group progress and provides timely support.

    Argument: Developing deep understanding and problem-solving ability is more challenging. To provide for individual student needs, such as when students make no or little progress with a problem (Hiebert & Grouws, 2007), the teacher helps students by asking appropriate questions, providing confirmation and redirection where needed. Purposeful questions tend to elicit student solution methods, assess student thinking, support students’ conceptual understandings, and extend mathematical ideas (Chapin & O’Connor, 2007; Fraivillig, 2001). Teachers need to quickly scan and monitor individual and group progress on mathematics problems and provide timely support. The teacher moves about the classroom, observing and interacting with individual and small groups. To support teachers in continuing to circulate the classroom in the digital environment, a “quick summary” of student work in the teacher dashboard is needed. This requires real-time display of observational data or logged data generated by students on their laptop. Further, progress needs to be defined and measured in a way that provides non-inferential information to teachers so that it is easier for them to interpret and use during class. Teachers need a mechanism to send “just-in-time” supports to students. While we recognize that this is not necessarily done in-the-moment, teachers also use the supports to plan, reflect, and respond to student thinking as necessary outside of class. As the digital platform makes it possible to arrange supports so that specific prompts are revealed as needed, teachers need the affordances of scaffolding that are static and given beforehand (Miyazaki et al., 2015; Schukajlow et al., 2015) and scaffolding that are dynamic and used on-the-fly (Abdu et al., 2015) so that teachers could generate and/or release prompts to individuals, groups of students, and the entire class. This requires linking the teacher dashboard to the digital collaborative student platform used by students.

    During the Explore phase,

  • Design Principle 3: The teacher gathers evidence of students’ strategies and mathematical understandings and looks for students’ connections to prior understandings and strategies.

    Argument: To plan for the Summarize phase, the teacher considers evidence from the Explore phase that can be used to support student understanding in the whole-class discussion. This process requires the monitoring of student responses (Smith & Stein, 2011), recognizing mathematically significant opportunities (Leatham et al., 2015), and selecting student work that can be presented in the whole-class discussion (Smith & Stein, 2011). As described earlier, it is important that teachers have access to evidence of student strategies and mathematical understandings in the digital platform. Further, teachers need easy ways to navigate student work. Not only do teachers need access to work digitally but they need tools to act on it and use it in their instruction. In terms of looking for students’ connections to prior understandings and strategies, teachers need to be aware of how the mathematical storyline builds over connected sequences of problems. This draws on the research on learning progressions, such as the ones implicit in research-based curricula (Clements, 2007), that show how students’ prior knowledge and experiences build over time from informal knowledge into more sophisticated reasoning (Sztajn et al., 2012). Further, if students build and articulate connections among ideas, they develop a view of mathematics as a coherent and connected discipline (Fosnot & Jacobs, 2010). Allowing teachers to easily access and use student work from different problems in the digital platform provided teachers support for making explicit the student connections to prior understandings, strategies, and problems. This underscores the importance of designing a platform that is used over time and across connected sequences of problems, which is different than designing a teacher dashboard that is used for one or more isolated tasks.

    During the Summarize of the problem,

  • Design Principle 4: The teacher uses the information gathered, including student work, generated from the class to orchestrate a summary discussion.

    Argument: During a whole-class discussion, the teacher guides the students to reach the mathematical goals of the problem and to connect their new understandings to prior mathematical goals. This process begins when most students have gathered sufficient data or made sufficient progress toward solving the mathematics problem. Students present and discuss their solutions and strategies, discuss the embedded or encoded mathematics of the problem, and connect learning to prior and future knowledge. Research underscores the importance of establishing and maintaining productive mathematical discourse (Hufferd-Ackles et al., 2004; Silver & Smith, 1996). In the discussion, students pose conjectures, question each other, offer alternatives, provide reasons, refine their strategies and conjectures, and make connections. The Summarize discussion should result with students becoming more skillful at using the ideas and techniques that emerged from their experience with the mathematics problem. Teachers need space in the digital environment that can easily leverage the evidence of student thinking that is generated during the Explore phase and use these ideas in a space that can be marked up by the teacher and the students to produce new notes. Additionally, streamlining the experience so students take ownership of the whole-class space is needed. Further, this requires content to be generated in the teacher dashboard or student platform and be easily shared across the spaces.

    During the Summarize of the problem,

  • Design Principle 5: The teacher (and student) has easy access to the class discussion notes to use during and after class.

    Argument: In paper-and-pencil classrooms, teachers and students may use poster paper or projectors to display the work from the Explore phase. Other times, students may re-create or show their work on white boards or smart boards. In all these cases, students need time to write these ideas in their individual notebooks. This is particularly relevant when students are exposed to new conjectures, ideas, and strategies that they did not explore themselves. Further, students may want additional notes about the mathematical ideas (e.g. from other students’ work or discussions) as they engage in the whole-class discussion. In the digital environment where the teacher dashboard is linked to the student digital collaborative environment, any notes created digitally can be shared digitally to students so they can incorporate them in their individual notes or learning logs. This requires students and teachers to have their own digital accounts and passwords so that they can access the spaces after class.

2.5 Context of the study

The research reported in this 4-year study was conducted with seven experienced mathematics teachers from four school districts in the midwestern and northeastern United States. These mathematics teachers tested the digital resources in their seventh-grade mathematics classrooms (ages 12–13). Four teachers from two schools stopped their participation due to various school configuration changes. Each teacher taught 2–4 sections of seventh-grade mathematics and each class had approximately 25 students per class. Because teachers taught the units at different times throughout the year, the team was able to engage in multiple iterative cycles per year.

The student digital collaborative platform was designed for face-to-face instruction where each student has one-to-one access to laptops connected to the internet. As shown in Fig. 1, the upper left-hand workspace was the students’ individual workspace, while the other three workspaces shown in the four quadrants were the individual workspaces from their group teammates. As students worked together, they clicked and dragged work from other workspaces, incorporating copies of inscriptional work into their own workspace. The teacher typically had a computer in their classroom that was connected to a projection system, as well as a tablet that they could use to access the teacher dashboard. Internet access and an internet browser were required to access the digital materials; no software was needed to be installed. Work is automatically saved in the platform.

Fig. 1
figure 1

Example of student workspace linked to other student workspaces in the digital platform

Full size image

Informal teacher interviews were conducted at the beginning and end of each unit. The interviews focused on new digital features of the teacher dashboard, and planning and preparation for the implementation of the digital resources. Interview questions included (a) newly developed feature, (b) the extent to which teachers used these features, (c) how they designed and enacted their class, (d) how it supported or hindered student learning goals, and (e) their evidence that it worked (or not). In addition, classroom observations were videotaped for each class where students and teachers were using the digital platform. Observation notes captured information related to the enactment of the mathematics problem in the digital collaborative environment, including the role of the teacher and features of the digital platform.

Our retrospective, thematic analysis focused on transcriptions of the 44 interviews from each of the teachers across the 4 years. We coded segments of the transcript that focused on features of the teacher dashboard. Transcript segments were collated together around each teacher dashboard feature and each collated group was then coded again. Emerging themes were identified and recorded for how the features may or may not have supported the teacher enactment of problem-based mathematics curriculum. At this time, we examined the related classroom videos and classroom field notes that were highlighted by teachers in their interviews. Observation notes were summarized for main themes and used for research and development purposes. The researchers looked to see if there was sufficient evidence that supported (or refuted) the reflections teachers reported in the interviews.

3 Exemplifications of the design principles: dashboard features and teacher enactment

In this section, we provide exemplifications for three teacher dashboard features that were guided by the design principles: (1) monitoring evidence of student thinking in real-time or after class, (2) accessing workspace for whole-class discussions of the problem, and (3) creating and sending “just-in-time” supports. For each feature, we connect the development to the design principles. The team engaged in an agile development process of small chunks of the digital collaborative platform through rapid cycles of defining, building, testing, and refining for three high-level phases. Table 1 highlights the features of the teacher dashboard as it was co-developed with the digital platform for students. The pedagogical advantages and challenges teachers face throughout the iterative development process are also discussed.

Table 1 Major features of the digital collaborative platform by project year
Full size table

3.1 Teacher dashboard feature 1: monitoring evidence of student thinking in real-time or after class

Dashboard Feature 1 was guided by the Design Principles 2, 3, 4, and 5 (see Fig. 2). In Year 1, two teachers did not have digital access to evidence of student thinking in real-time. It required the teachers to monitor evidence of student thinking as they circulated around the room, examining student work in real-time during the Explore phase of the instructional model. While this typically occurs in paper-and-pencil classrooms, the digital environment was initially more difficult because: (1) every student used a laptop, which limited the viewing angle of what is displayed on the screen, and (2) students opted to show their groupmates work which made the viewing size of their individual contributions smaller on the laptop. No new affordances were evident for the digital environment for the teachers, beyond the affordances identified for the digital collaborative platform for students (e.g., student collaboration, student engagement).

Fig. 2
figure 2

Teacher design components of monitoring evidence of student thinking in real-time or after class

Full size image

In Year 2, six teachers could monitor evidence of student thinking in real-time by logging into the student platform as a student. This workaround allowed the teachers to join different groups and view individual student and group work in real-time on their own screens without interrupting student progress. As teachers identified this affordance as important, the limitations of this workaround were that the teachers had: (1) to look on their laptop screens during the Explore phase which limited their ability to circulate around the room, and (2) to join/leave different groups to monitor the entire class which was time consuming and took away from circulating the room and interacting with students. While every teacher used this workaround, this practice ultimately was not a viable long-term solution as teachers preferred to continue to circulate around the room during class. In Year 3, five teachers had access to a teacher dashboard that supported them in monitoring evidence of student thinking for the entire class without interrupting student progress (see Fig. 2). Teachers reported that in-the-moment decisions focused on (1) variations of student thinking that occurred, (2) new student insights or conjectures, and (3) potential issues that emerged in class that relate to student strategies, mathematical ideas, or the learning goal.

In Year 4, two teachers reported that they used the analytics of student progress to determine when to transition from the Explore to the Summarize phases of the lesson. In Fig. 2, on the right-hand side of the screenshot, 24 of 27 students have work started for the Initial Challenge (IC) section of the mathematics problem, 23 of 27 students have work started in the What If…? (WI) section of the mathematics problem, and 22 of 27 students have some work started in the Now What Do You Know? (NW) section of the mathematics problem. Since the measure reports on work started, teachers clicked on the different components to access more information on the quality of student progress of specific problems. Because teachers used tablets to monitor evidence of student thinking they did not need to take as many physical notes because the evidence of student thinking is always available. The teachers reported that it was particularly useful in preparation of the whole-class Summary discussions to switch between real-time student work and work published to the entire class.

Teachers also monitored evidence of student thinking after class. In Year 1, teachers relied on computer screenshots. At the end of each day, students emailed computer screenshots to their teachers for them to review. In Year 2, the six teachers could monitor evidence of student thinking after class by logging into the student platform as if they were a student. This no longer required student screenshots. Teachers could see the work for an entire group at once. This was important for teachers because it was easier for them to examine approximately 6 groups as opposed to 24 individual screenshots per class. In Years 3 and 4, all five teachers used the teacher dashboard and all its components to access all groups of student work after class. The following is a teacher reflection on this after-class feature.

For me as a teacher, I think was helpful to be able to look over your work sometimes in the evening and think about things I wanted to address the next morning. You know, if I felt like, sometimes Summaries do get rushed and that’s still one of my issues with CMP is there’s so much good work and such good meaty conversation that students be having…So being able to look at their work at night and say, okay, we covered this really well in the Summary, but we didn’t talk about such and such. So let me look through here and see, find a good example of that, and then the next morning, I could say, okay, I want to go back for a minute and look at so and so’s work from yesterday. So that was neat, too. I’d never be able to do that if they had notebooks. You know, that’s dragging home 30 notebooks is not going to happen, let alone 120. (School 4, Teacher 2, Year 4).

All five teachers in Years 3 and 4 reported that they engaged in monitoring evidence of student thinking after class to (a) aid in the planning of the next lesson, (b) to send supports to individual, group, or class, and (c) to reflect on student progress on the lesson learning goals.

Other refinements were made that were also useful for teachers to view student work. These refinements focused on (a) including the appearance of the student’s initials in the center of the four quadrants for each student in the group, (b) the ability for teachers to easily scroll through the work of individuals embedded within the work of the group, and (c) the ability for teachers to change the dimensions of the four quadrants and “zoom in” to view a subset of the four students in the group. Teachers navigated across the different groups in the class through the tabs in the upper-land hand section.

3.2 Teacher dashboard feature 2: accessing whole-class workspace for launch and summarize discussions

Dashboard Feature 2 of the teacher dashboard was guided by the Design Principles 1, 3, 4, and 5 (see Fig. 3). By the end of Year 2, all six teachers had their own accounts in the digital platform. With the teacher account, teachers had access to their own classroom workspace that was linked to the digital student collaborative environment.

Fig. 3
figure 3

Teacher design components of accessing whole-class workspace for Launch and Summarize discussions

Full size image

Teachers quickly scanned and selected digital copies of evidence of student thinking to support class discussions. One teacher commented that some students were more comfortable presenting their work digitally than standing up and talking in front of the room. Because student work was digital, the teachers and the students easily marked up, modified, and deleted student inscriptions. Some mathematical representations were linked together (e.g., tables and graphs). As their whole-class workspace was automatically saved, the work was archived for future class use. On occasion, four teachers embedded examples of student thinking from outside the class to support their discussions. For example, one teacher reflects on the use of the whole-class workspace:

And what that allows me to do is, it allows me to really focus on what they’re exploring during the Explore phase, where, when in the past, when I want to do a good Summary, not only am I supporting during the Explore and really looking, investigating with the kids, but I’m also trying to take copious notes of their strategy so I can recall it for later… The published work that I can go through and I can just quickly drag and drop it into my own workspace. I didn’t have to spend so much time with the note taking, and really concentrating on how I wanted to Summarize, when what I really want to be doing is, …, interviewing groups while they’re working and be with them as part of the Explore. I don’t know if that made sense. But… what I’m saying is, I really found it easier for the Summary to pick and choose what I wanted to use without having to put in all of the work that I would have had to do in the past to do that. (School 1, Teacher 2, Year 2)

Six teachers drew on published student work, thereby creating new inscriptional tables, graphs, drawings, and text during class discussions. Four teachers in Years 3 and 4 published the class workspace so students could embed a digital copy into their individual notes, annotate the inscriptions, and access it later. This allowed each student to incorporate the summary work into their individual workspaces. The teachers reported that the use of the classroom workspace allowed teachers to support student accountability and ownership of learning because it was easier and more efficient to incorporate student work into the discussions.

Two aspects of the teacher dashboard were also helpful to teachers. First, teachers could switch between student work displayed in real-time or student work published to the entire class. This was used by all teachers in Years 3 and 4. Published work differed from group work in that other groups and the teacher could also access the work. Students published when they had work to share to other groups or when the teacher asked students to publish their work for class discussions. Second, a digital tagging system was added to support five teachers in identifying and easily locating specific pieces of student work. In this digital tagging system, teachers used stars to tag student inscriptional work and accessed starred work in a repository during whole-class discussions.

3.3 Teacher dashboard feature 3: creating and sending “just-in-time” supports

Dashboard Feature 3 was guided by the Design Principles 2 and 5 (see Fig. 4). Creating and sending “just-in-time” supports emerged in the project as a way for three teachers to provide feedback to the evidence of student thinking found in the student digital collaborative platform. The teachers reported that the purpose of the supports was to connect the mathematics of the problem to the big mathematical idea of the unit, or to provide feedback that was “in-the-moment.” Examples included asking students to say more about an idea, to prepare for a whole-class discussion, or to provide encouragement on a mathematical idea. This was important for the teachers because they wanted to easily send individuals, groups, and classes comments, questions, and new inscriptions that were attached to existing student, group, or class work. The remaining four teachers did not request or use this feature.

Fig. 4
figure 4

Teacher dashboard feature of creating and sending “just-in-time” supports

Full size image

In Year 1, the two teachers requested a mechanism to customize the mathematics problems to meet individual student needs. They also requested access to comment on student thinking in real time. Examples included questions for students to consider during the lesson Launch, prompts or questions for students to assess their learning during the Explore phase, and ways to give feedback to the students between classes. By Year 2, the “just-in-time” teacher resources expanded beyond written text to the full set of inscriptional resources that students could access.

By Year 3, the participating teachers could use the teacher dashboard to send supports to the entire class, to an entire group, or to an individual student. Teachers used the “just-in-time” teacher support feature in a variety of ways. For example:

Yeah, it’s been great to be able to send notes to certain kids because I really wanted to be able to start to say to them, like, hey, you’re not doing your work here, I need to see X, Y, and Z from you, because that’s one of those things that I don’t, I’m not usually keyed in on, you know. In the classroom, I am looking for kids who aren’t doing anything, but I’m also looking for work that I want to highlight and talk to the whole group about I’m not going to stop and have a, you should be doing conversation with every other kid. I don’t like doing that. But so it would be a great opportunity to be able to just shoot them a sticky note that says, you know, hey, I noticed you didn’t do much here. Do you have questions? Do you want to meet with me at lunch? …So that’s a good, that’d just be a good teaching tool. You know, whether or not it’s just kind of a good behavior or management tool. (School 4, Teacher 2, Year 4)

Three teachers primarily planned supports before class to send before or during class, particularly for the Launch phase. Interestingly, and not surprisingly, the teachers occasionally sent “in-the-moment” supports during the Explore phase. Teachers also reported that this feature was used when monitoring evidence of student thinking after class. When students received supports, they were alerted in the digital platform with a notification that requires them to actively click on to receive the content of the support. These supports provide a written record of how the teachers customized classroom instruction to meet the needs of individual students. The record of supports could be accessed later by teachers.

4 Discussion, conclusions, and looking ahead

In this paper, we reported on design principles for teacher dashboards that are linked to a digital student collaborative platform with an embedded problem-based curriculum. Existing research indicated that curriculum materials in digital form focus students on mastery of skills and symbolic manipulation and/or do not take full advantage of digital technologies (Choppin & Borys, 2017; Choppin et al., 2014). Our study focused on the importance of teacher dashboards for thinking devices curriculum materials (Roth McDuffie et al., 2018), and how the design principles were embodied as teacher dashboard features that support problem-based mathematics teaching. We highlighted three teacher dashboard features and how they were enacted by teachers. Figure 5 shows a summary of the teacher dashboard design principle, teacher dashboard feature, and the related teacher affordances based on how teachers used them in their problem-based classrooms.

Fig. 5
figure 5

Teacher dashboard design principle, feature, and related teacher affordances

Full size image

Because the teacher dashboard is linked to a student digital collaborative environment with a problem-based curriculum, the dashboard provided teachers with new possibilities that are difficult to do in paper-and-pencil environments. Although paper-and-pencil classrooms contain calculators, computers, projection systems, paper notebooks, these technologies have no inherent integration as those reported earlier. Specifically, the teacher dashboard features provided teachers with new and timely information that supports problem-based mathematics teaching. For example, for the Dashboard Feature 1 to occur in paper-and-pencil classrooms, teachers would need to collect student notebooks on a regular basis. But it still does not allow teachers to look at all student thinking at the same time, in real time, or as needed to make “in-the-moment” class decisions. For Dashboard Feature 2 to occur in paper-and-pencil classrooms, teachers would need to produce physical copies of the class notes in-the-moment so that students could embed the class work into their individual notebooks. Alternately, students could take notes in class, but this would limit students in their contributions to the class discussion. For Dashboard Feature 3 to occur in paper-and-pencil classrooms during real time, teachers could write comments on paper slips and share them with students or verbally tell students the comments. But teachers could not easily attach the comments to specific aspects of student thinking, potentially disrupt students in their progress, or easily share the same comment with multiple students. Thus, our study reports on evidence of how teachers accessed, interpreted, and used evidence of student thinking in their classrooms with the teacher dashboard.

The results have provided insights into how the teacher dashboard provides teachers with an “in-the-moment” pedagogical perspective where teachers plan, implement, gather information, and make decisions in real-time. These insights are useful to teachers because they have more information to address enactment challenges than they previously did in paper-and-pencil classrooms. But new questions arise for further study: How and when do teachers quickly scan evidence of student thinking? What information are they looking for? Is there too much information? What supports and under what conditions do teacher provide them? When and why are paper-and-pencil environment more productive for solving a problem? Examining questions such as these not only underscores the connections within, between, and among the operational curriculum (Remillard & Heck, 2014), but they also provide further insights into how teachers can use teacher dashboards linked to problem-based curriculum materials as agents of change to support both teachers and students in mathematics classrooms.

The work reported in this paper provides design principles that guided the development of teacher dashboard features for enacting a digital problem-based mathematics curriculum. More work is still needed for teachers, particularly around digital features for teacher planning and reflection. While we are strengthening the connection between the teacher dashboard with the student digital collaborative environment, we are actively pursuing teacher support in a larger scale. In August 2020, a new project was funded to create a collaborative digital environment for middle school mathematics teachers to promote effective planning, implementation, assessment, and reflections. The project will learn about how teachers use resources, collaborate in the digital environment, and support each other through the network.