Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Across the country, a growing number of school districts have launched efforts to significantly reform their central offices to support districtwide teaching and learning improvement, and they are using various strands of research—here defined as formal social science studies of school district central office performance—to guide their reform designs and processes. Unlike other district reform efforts that focus on schools as the sole target, these reforms aim to shift central office policies and practices so that central offices operate as support systems for teaching and learning improvements in schools. Such efforts are theoretically promising, since a growing body of research and other evidence is beginning to suggest how central offices might operate differently to support districtwide improvement school goals (Hightower 2002; Honig et al. 2010). However, the research typically calls for fundamental shifts in central office administrators’ work—radical departures from the status quo in central offices—and such changes are notoriously difficult to implement. What happens when central office administrators engage with research that calls for fundamental shifts in central office work practices? Under what conditions, if any, do central office administrators buck predominant trends and actually use the research to engage in fundamental shifts in their own practice at the central office level?

We explored these questions with an in-depth qualitative analysis of six school districts engaged in fundamental central office change aimed at strengthening their central offices’ capacity to support districtwide improvements in teaching and learning. All the districts intended to use various forms of research to guide their process. For our conceptual framework, we drew on several strands of learning theory, based on the research-based hypothesis explored below that when practitioners such as central office administrators take up research-based ideas in ways that lead to fundamental changes in their practice, they engage in processes of professional learning (Collins et al. 2003; Brown et al. 1989; Lave 1998; Levitt and March 1998; Rogoff et al. 1995; Tharp and Gallimore 1991; Weick 1995; Wenger 1998). These strands of learning theory also emphasized that practitioners tend to learn in ways that result in fundamental changes in their practice when they have access to intensive assistance relationships in which assistance providers, knowledgeable about the new target practices, engage in high-leverage teaching moves that support learners’ depth of understanding of challenging ideas. We used these theories to frame our investigation involving 116 interviews, 499.25 h of observations, and reviews of approximately 300 documents. In this chapter we summarize some of the main findings that have emerged during our first phase of analysis.

In sum, we discuss that central office administrators used particular research-based ideas to shift district-level policies and their own practice, but to varying degrees. We found that particular conditions enabled research use such as administrators’ prior experiences with ideas consistent with the research and their engagement with intermediary organizations that engaged in teaching practices consistent with our conceptual framework. However, such conditions seemed necessary but not sufficient for research use at deep levels. More often, central office administrators tended not to take up research-based ideas at deep levels absent central office leaders who themselves engaged in high-leverage teaching moves to help their colleagues and staff integrate challenging research into their practice. These findings support the hypothesis that internal central office leadership is essential to fundamental central office change.

1 Policy Context

School district central office leaders across the country face unprecedented demands to help all students achieve at high levels and to use research to inform the process. For example, the No Child Left Behind Act of 2001 requires that district central offices provide professional development to schools that includes “instructional materials, programs, strategies, and approaches based on scientifically based reading research” (No Child Left Behind Act [NCLB] 2002). Educational research chronicles how districts have been turning to ideas from research, here defined as formal social science studies, to inform such decisions about school-level supports and change strategies (Coburn et al. 2009; Corcoran et al. 2001). But central office leaders are also looking to research to guide their own practice at the central office level (Honig and Ikemoto 2008; Marsh et al. 2004, 2006). For example, central office administrators partnered with the Institute for Learning (University of Pittsburgh), to learn how to observe for high-quality instruction using a research-based practice called Learning Walks in order to build administrators’ knowledge of instruction and to develop supports for principals and teachers (Honig and Ikemoto 2008).

Central office staff have new opportunities to use research to inform how they work, in part because a wave of relatively recent formal academic research has begun to take up the question of what forms of central office work might support school improvement (e.g., Augustine et al. 2009; Honig et al. 2010). For example, studies in New York City’s Community School District #2 have shown how the implementation of school reform efforts hinged in part on the continual efforts of central office administrators to support principals in building their capacity to help teachers improve the quality of instruction (Elmore and Burney 1997; Fink and Resnick 2001). In our own such research we have elaborated how central offices can significantly build schools’ capacity for improved teaching and learning through new partnership relationships with school principals and by reorganizing and reculturing central office functions to provide high-quality, relevant support services to schools (Honig 2012; Honig et al. 2010).

The use of research to reform how central offices work to support teaching and learning improvement in schools holds promise for realizing educational improvement goals. As research on school improvement has shown for decades, school-level efforts to strengthen teaching and learning lumber, plateau, or outright fail in part because central office administrators do not participate productively in their implementation (Bryk et al. 1998; Chubb and Moe 1990; Malen et al. 1990; Ravitch and Viteritti 1997). Educational research has begun to suggest that central office administrators’ participation involves remaking central office policies and work practices in particular ways (Elmore and Burney 1997; Honig 2008; Hubbard et al. 2006).

However, central office administrators likely face significant obstacles when shifting policies and practices in ways the actual application of the research would require. For example, the conclusions from a San Diego study mainly show that central office administrators struggled and largely failed to shift their work in ways that the research suggested (Hubbard et al. 2006). Across multiple studies, Spillane found that central office administrators routinely misinterpret or misappropriate ideas about what instructional improvement entails, viewing the ideas as reinforcing their current policies and practices even when such policies and practices depart starkly from what research recommends (Spillane et al. 2002; Spillane and Thompson 1997. See also, Coburn et al. 2009; Kennedy 1982; O’Day 2002).

Such results are not surprising. The kinds of changes that this research demands run counter to school district central offices as institutions. School district central offices were established at the beginning of the last century mainly to carry out limited business and regulatory functions, not to address or centrally support improvements in the quality of principal leadership and, in turn, teaching and learning in schools (Cuban 1984; Elmore 1993; Tyack and Cuban 1995).

Broader research on decision-making in central offices delineates various challenges with changing business as usual in central offices. For example, Hannaway (1989) showed that central office administrators approached their professional decisions with particular biases that led them to favor the status quo rather than consider or implement deeper changes in their policies and practices. Such patterns were especially true of mid-level central office staff members other than superintendents or other central office executives. Coburn and colleagues (2009) confirmed earlier findings by Kennedy (1982), by showing how central office administrators tend to interpret new ideas in ways that reinforce rather than disrupt their prior knowledge and decision frames. Across a series of studies, Honig (2004, 2009) found that even new central office employees fell back on long-standing central office routines that curbed their engagement with new challenging ideas.

Are policymakers expecting the impossible when they require that central offices engage with research fundamentally challenging their status quo in an effort to spark central office reforms for districtwide improvements in teaching and learning? Under what conditions, if any, might central office administrators buck predominant trends and make significant shifts in their work and capacity that the emerging research calls for?

2 Conceptual Framework

We explored these questions in a study of six districts that aimed to fundamentally reform their central offices to support districtwide teaching and learning with an emphasis on how they engaged with research to guide the process. Our conceptual framework came from several strands of literature on organizational and sociocultural learning. These theories elaborate that when people engage deeply with new ideas in ways that lead to fundamental changes in their work practices, they engage in a process of learning. As part of that learning process, practitioners such as central office administrators grapple with what new ideas mean and how to integrate these ideas into their ongoing practice. They compare the new ideas against their past experiences and their sense of what constitutes appropriate professional practice. Learners also may edit the new information, amplifying certain parts and downplaying others (Levitt and March 1998; March 1994; Weick 1995).

According to Argyris and Schon (1996), during single-loop learning processes, learners stick with a single conception of the underlying problem motivating their action and use feedback from their actions only to inform how they go about their action taking. During double-loop learning, learners use feedback to scrutinize their understanding of the problem they aim to address. Such double-loop processes lead to deeper understandings of new ideas and more profound changes in practice than single-loop learning which typically involves a learner engaging in different variations of the same general type of practice.

For example, a central office staff person might tackle the problem of a school principal not spending enough time working with teachers on the quality of their classroom instruction by conducting school visits themselves and writing reports directing the principal to visit classrooms more. Over time, the central office staff person might find that despite the reports, the principal still is not visiting classrooms frequently enough. In response, they increase the frequency of their own school visits and the consequences they associate with the principals’ failure to comply—marginal changes in their practice. However, another central office staff person might use the feedback that the principal’s practice remains unchanged to more fundamentally rethink their approach. They might hypothesize that the principal would like to spend more time in classrooms but they lack the capacity to shift their schedule to allow them the time to do so and they do not have the know-how to feel confident about the efficacy of their observations. That central office staff person might reframe the problem posed by that principal not as one of their lack of compliance but their weak capacity and, in turn, decide to fundamentally shift their own practice to put them in the school building more often working side-by-side with the principal to adjust how they allocate time and sharpen their skills at classroom observations.

Sociocultural learning theories further distinguish outcomes that represent first- vs. second-order changes (Grossman et al. 1999), and we used their distinctions to help us gauge central office staff’s shifts in practice. During first-order change processes, practitioners might not appropriate new ideas at all, continuing to engage in their work as they have always done. Practitioners might also “appropriate a label” or engage with the ideas at only a superficial level and not realize the deeper levels of understanding of the ideas that lead to significant shifts in practice. For instance, a central office leader might change the title of certain central office staff from “associate superintendent” to “instructional leadership director” to signal that those staff now focus on helping schools strengthen instruction. But this leader has only appropriated a label if she has not changed the actual work practices of these staff to support such results.

When practitioners “appropriate surface features” they intend to engage in the practices reflective of the new ideas more deeply than when they appropriate a label. But because they do so without grasping what the features mean or why they engage in them, they are likely to engage in these activities only temporarily or in certain contexts, not transferring the ideas to new contexts as is true of deeper change processes (Pea 1987).

Sociocultural learning theorists distinguish two dimensions of second-order change. Practitioners might engage with new ideas by “appropriating conceptual underpinnings.” When they do so, practitioners have developed a relatively deep, internalized understanding of the new ideas, they attempt to use them in ways consistent with the research-based ideas, and they are likely to apply them in new situations. Those who have “achieved mastery” are able to fully engage in practices consistent with the new ideas. They deeply understand what the new ideas involve and why engaging in them is important, they frequently demonstrate the new practice, and they are able to improvise, creating new extensions of the practice in ways that create new knowledge.

Various factors mediate such levels of appropriation. Among them, through particular kinds of learning assistance relationships, practitioners may receive support for disrupting their usual ways of thinking and acting and engage in fundamental change of either or both (Tharp and Gallimore 1991; Wenger 1998). Across a wide range of settings, “assistance” strategies have such effects when someone continuously (1) models or demonstrates modes of acting and thinking consistent with the new ideas; (2) develops and uses tools that help practitioners engage with the new ideas; (3) helps practitioners adopt the identity of people on a trajectory toward deepening their engagement with the ideas; (4) creates and sustains social opportunities—including challenging conversations—through which practitioners grapple meaningfully with what new ideas mean for their own work; and (5) bridges practitioners to and buffers practitioners from outside resources both to support practitioners’ sustained engagement in the new ideas (Honig 2008).

The so-called intermediary organizations may be particularly well suited to support such assistance relationships (Honig 2004). Intermediary organizations, because of their partial-outsider status, may be able to dedicate the time and other resources that assistance relationship requires; because of their partial-insider status, they may garner a trust with their partners also essential to such relationships (Coburn and Stein 2010).

3 Methods

We used our conceptual framework to help us strategically select six study districts based on their likelihood to engage in fundamental changes in their central offices consistent with emerging research-based ideas about how central offices might support improved teaching and learning in schools. We looked for districts that (1) aimed to reform their central office in service of improved teaching and learning districtwide, (2) understood that such reform would mean significant changes in their central office, (3) appeared to be drawing on various strands of social science research about central office performance to ground their approach, and (4) had access to intermediary organizations to support with the process who ostensibly understood their role as assisting administrators’ professional learning.

Our data sources include 116 in-depth interviews of central office staff and school principals, observations of meetings and coaching sessions involving central office staff totaling 499.25 h, and over 300 documents related to the central offices’ efforts to improve their performance in service of improved districtwide teaching and learning. We invited to participate in the study all central office staff who were engaged with the intermediary and who intended to use research to transform their central office, including the superintendents and central office executive staff members (i.e., Assistant Superintendent of Teaching and Learning, Director of Curriculum, etc.) and in two of the districts, those in the newly created instructional leadership director role.

We analyzed our data using NVIVO8 qualitative software in several phases. During our initial phase of analysis, we sorted our data into low-inference categories such as “outcomes” of central office change processes and topics of research that central office staff aimed to use (e.g., “superintendent role”). We also used our conceptual framework during this phase to code for various, broad potential influences on research use including the role of intermediary organizations. In the second stage of our analysis, we went back into our data by code and refined our analysis using higher-inference codes from our conceptual framework. For example, we recoded our “outcomes” data by degrees of appropriation. We also distinguished the work of intermediary organizations as more or less consistent with the assistance relationship practices highlighted in our conceptual framework.

4 Findings

In this section, we share emerging findings from our initial analysis. Specifically, we discuss that we could distinguish central office administrators’ use of research-based ideas by the degrees of appropriation highlighted in our conceptual framework. We demonstrate how we made these distinctions by discussing examples from administrators’ use of one out of the three main sets of research-based ideas with which our respondents engaged: research on how to develop and execute the role of principal supervisor or what the research calls “instructional leadership director.” The distribution of examples related to this strand of research reflects our overall data set which revealed instances mostly of first-order change or low levels of appropriation. Fewer examples indicated degrees of appropriation with conceptual understanding. By contrasting these examples and examining changes in levels of appropriation in some cases over time, we identified several conditions that seemed necessary for central office administrators’ engagement with ambitious research ideas at any level but not sufficient for second-order changes. These include individual’s prior knowledge and experiences and the nature of intermediaries’ assistance strategies. Those central offices and central office staff with little prior experience with or knowledge about the research-based ideas generally did not develop much if at all beyond their initial levels of understanding. While their intermediaries did engage in the kinds of practices our conceptual framework suggested would help central office staff significantly shift their own practice in ways consistent with the research, the intermediaries’ efforts seemed promising but not sufficient to realize such results given staff’s low levels of appropriation starting out. Those staff and systems whose work seemed to reflect progressively deeper practice shifts consistent with the research also worked with intermediaries, often the same ones as the other staff who showed more growth, but from the outset reported prior knowledge consistent with the research-based ideas. In addition, in those systems, leaders did not rely solely on intermediaries but themselves assumed teaching roles with other central office staff, assisting them with their engagement with the research. Our findings underscore other research on the importance of prior knowledge or experience, what leaders might operationalize as “readiness,” to engagement in fundamental practice changes. However, they suggest that intermediary organizations may be viable levers of change when prior knowledge and experience are relatively high, but that internal leadership, particularly that which proceeds from a teaching stance, may be essential to realizing deeper levels of appropriation, particularly in systems without substantial relevant prior knowledge and experience.

4.1 Appropriating Research on the Role of the Principal Supervisor

As one illustration of how our examples ranged by degrees of appropriation, we found that all six districts aimed to use research on a particular central office position referred to in the research as “instructional leadership director” (Honig et al. 2010). Those findings described the importance of districts eliminating their traditional principal supervisor position and replacing it with ILDs—executive-level staff charged with working as close to 100 % of the time as possible helping principals grow as instructional leaders, leaders who supported teachers in improving their instructional practice. ILDs worked with principals one-on-one as well as in principal professional learning communities which the ILDs convened. Research on ILDs called for second-order changes in central office practice in several respects. Among them, unlike some traditional principal supervisors such as area or assistant superintendents, ILDs did not also manage central office programs or carry out other functions; rather, they were dedicated to supporting principal growth, jobs typically left to retired principals or coaches deep in the central office hierarchy. The findings on ILDs thus flipped traditional arrangements for principal professional development on their heads, elevating support for principals’ growth as instructional leaders to a cabinet-level responsibility. The research also showed that ILDs who were successful in supporting principal growth did not lead in traditional central office fashion, with accountability or top-down directives. Instead, successful ILDs engaged in partnership relationships with school principals and aimed to teach rather than tell principals how to improve (Honig 2012).

While all the districts aimed to adopt the research findings about the ILD function, they varied in how they appropriated that idea across districts at two levels of analysis: (1) between districts, how central office leaders constructed the role, and (2) within districts, the extent to which individual ILDs appropriated ILD practices named in the research.

For example, in one of our mid-sized districts, in several interviews and conversations over time, those hired into the ILD positions reported that, in the words of one, “supporting instructional leadership” among principals was their primary charge. However, when we probed their understanding of what such support work entailed, they described their charge as providing general support to principals. In one ILD’s words,

Of all the experience I’ve had as a…school principal, there’s nothing that any one of these people is going to face that I haven’t faced two or three times before. So if I could help problem-solve something rather than have them working it through completely on their own, if I could help problem-solve it and get to solution quicker, then it enables them to have more time for the instructional piece.

Our interviews with and observations of the individual ILDs in this system suggested that none were engaged in the kinds of intensive teaching practices that the research emphasized. Instead, the ILDs typically worked with principals to help them understand and stay in compliance with various policies including those related to personnel. For example, one ILD described their work by saying,

So where they’ve [principals have] asked for advice and support has been in dealing with teachers struggling. So…if they’ve got a teacher that they’re concerned about, asking for me to review their observation summary before they send it…Another example would be something like a…kid wants to drop one of their seven courses because they’re a high-level tennis player…But there’s a district policy that says that a full-time student is seven classes. So working with the principal to try and meet the needs of the kids, but also make sure that we’re following district policy.

When we shadowed ILDs in this district, we most commonly saw them helping principals, assistant principals, and teacher leaders develop a shared understanding of the new teacher evaluation system, with an emphasis on following proper procedures when placing teachers on probationary status or helping teachers exit the system. While tasks such as these focused on instructional matters and related to principal needs, they did not involve the intensive focus on helping principals grow their instructional leadership practice as elaborated in the research.

In the other mid-sized district, we found adoption of the research on the ILD role at the level of appropriating surface features, a step deeper than in the other similarly sized district. At the level of individual ILDs, we found a broad range from adopting a label to adopting conceptual underpinnings.

To elaborate, a leader in this district initially set out to retrain her team of principal supervisors so they functioned as ILDs rather than as more traditional area superintendents. This leader reported that these staff would no longer operate as the principals’ single point of contact for various central office matters such as assisting with the redesign of school websites, helping principals determine of particular students met graduation requirements, and managing the installation of portable buildings on school campuses. Instead, they would be dedicated to supporting principals’ growth as instructional leaders. However, the first year of implementation reflected appropriation of a label because throughout the year these staff operated with a new title but essentially went about their work as they had under their previous title. The next year, that leader rewrote the job description to focus on principals’ growth as instructional leaders and turned over 80 % of those staff with people she believed came with a ready understanding of the new conception of the ILD role consistent with the research.

Through broad districtwide communications as well as in principal meetings and community forums, this leader frequently indicated that ILDs were to spend 75 % of their time in schools supporting principal instructional leadership. She created new processes within the central office so that any requests for ILD time from central office departments had to be run by her first so she could help teach others in the system about the new role. For example, in one meeting in which ILDs were discussing various requests for their time from central office staff that seemed inconsistent with their focus on principals instructional leadership, this leader responded, “When stuff like that comes along, forward it to me. You need to forward it to me. I don’t know who will do it. So forward it to me and I will figure it out.”

However, the research-based conception of the ILD role was not deeply adopted throughout the central office, reflecting a surface-level adoption of the role across the central office. For example, nearly 2 years after the creation of this role, a top cabinet official reported that there was still, in the words of one, a “misunderstanding” about the role and how ILDs supported principals differently than before. In one discussion, he/she reported that the head of the facilities department wanted ILDs to be involved in school openings, “They want more frequent interaction with you [ILDs] on what the community wants and the principals’ reaction to this. I’ve heard them say, ‘I wish I had more time with the [ILDs].’” In multiple settings throughout the duration of our study, we observed ILDs receiving requests from central office staff to engage in activities that did not align with the research-based redesign of their role, such as handling transportation mishaps at various schools or managing teacher displacement.

Among individual ILDs in this district, we found that levels of appropriation ranged significantly from appropriating with conceptual underpinnings to little or no appropriation over time. Similarly in our study’s other mid-sized school district, one of the ILDs appropriated the role with a label and spent time on instructionally related issues without a focus on principal learning. For example, this ILD spent significant time in schools but typically engaged in tasks that were the principals’ responsibility, rather than helping the principal engage in those tasks—a choice that was not only inconsistent with research findings about the ILDs’ instructional leadership focus but one that directly conflicted with a finding that ILDs lead through principals and not step in for principals or otherwise become integrally involved with school-level functions. For example, this ILD reported observing classrooms with one of his/her principals and identifying problems related to the quality of instruction such as teachers’ failure to use state standards. The ILD then described how he/she followed up on those issues by checking teachers’ lesson plans and otherwise working with the teachers himself/herself—rather than coaching the principal through next steps, as indicated by the research. Another ILD commented that this ILD acted as a “super principal,” handling responsibilities that the research suggested he/she should have supported the principals in leading.

This ILD presented his/her work with principals as “working intensively with [principals] so that they can elevate teacher practice,” but his/her interactions did not reflect the teaching approach consistent with research. For example, during one session where the ILD discussed his/her work with a particular principal, the facilitator from an intermediary organization directly asked him/her during a meeting of ILDs, “I hear you monitoring and setting some priorities but how are you teaching the principal how to do a better job?” The ILD responded to this question and similar probes by sharing numerous reasons why this particular principal did not follow his/her directions including the principal’s lack of experience and confidence and not being held accountable for following through in the past. He/she described various ways he/she was monitoring the extent to which this principal followed what he/she told them to do such as visiting the schools to check that the principal was providing the professional development as directed.

Two ILDs in this district who displayed surface level of appropriation reported in interviews that they understood their role as maximizing their time on principal instructional leadership. However, they still spent significant time responding to requests from other central office units that took their time away from their school principals and did not directly contribute to helping their principals grow as instructional leaders. One reported that he knew doing so ran counter to their charge, but he was worried about tasks formerly assigned to his role falling through the cracks.

The work of two other ILDs in this district reflected their adoption of research on the ILD role with conceptual underpinnings. In the case of one of these ILDs, we found through interviews, meeting observations, and reviews of e-mail exchanges, this ILD frequently declined requests for his/her involvement in tasks that took time away from her direct work with principals focused on their growth as instructional leaders. During our second year of data collection, this ILD reported that any time he/she received a request from a central office staff, he/she filtered it through the question, “What does this have to do with instructional leadership?” Also in the second year, he/she trained his/her secretary to screen phone calls so that noninstructional issues, such as school lunches, would be directed to someone else and restructured his/her one-on-one meetings with principals so the meetings started with instructional issues and not principals’ operational crises.

Over multiple meetings with this ILD’s principals, we observed this ILD engaging in the teaching practices highlighted as high leverage in the research, sometimes explicitly using terms from the research to name that she was engaged in modeling or brokering principals as learning resources for each other. For example, this ILD described that one of his/her principals did not seem to understand that their teachers were performing poorly in mathematics across grade levels. The ILD organized a meeting with that principal during which she brought a data coach from the central office to model for the principal how to analyze school-level data, in the ILD’s words, to help the principal learn to “generate questions from data” that would help the principal understand how to make inferences about the level of instruction using data. The ILD shared that such support for the principal extended beyond that one event to a series of meetings with the math coach to help with classroom observations to build the principal’s knowledge about high-quality math instruction. During a meeting of ILDs at which this ILD presented his/her work with this principal to colleagues for feedback, this ILD articulated a clear rationale for his/her approach, rooted in the research-based ideas about the importance of taking a teaching rather than directive or monitoring approach with principals:

I’m not using the conversation to turn up the heat. I’m trying to teach [the principal] because I’m still hoping that… [the principal will understand what the issues are in her school and take action.] I’ve told [principal] that…“I’m going to hold you accountable to the learning” but I am trying to teach [the principal] how to do it. “Let’s put the data in a bar graph so we can understand the numbers of the page.” I don’t think it is negative. I think the tone of the meetings is pretty positive because it’s not about the gotcha, it’s about, “We have to understand the data in order to move forward.” The data holds the secret. I’m trying to make [the principal] understand so we can make the achievement plan.

We also considered this ILD at the deep end of appropriating this research because he/she demonstrated an ability to apply the research to new situations not directly discussed in the research but consistent with the research. For example, in one meeting, the ILDs in this district were discussing whether or not they should all use the same form when providing feedback to principals after site visits. During the discussion this ILD asked, “Why does it make a difference?” probing the other ILDs to articulate the connection between using a similar format and their charge to operate as master teachers of principals instructional leaders. This ILD contributed that he/she believed that the manner in which they provided feedback to principals—and that they did so from a teaching rather than evaluative stance—mattered more to principals learning than what form they used.

In the smaller central offices in our sample (i.e., those with between 9 and 20 central office staff), central office leaders at first questioned the applicability of the research on ILDs to their systems. They generally argued that given their small size, the creation of a team of ILDs did not seem relevant to them. However, by the end of our study period, all districts had appropriated the research-based ideas, some at a surface level and others with conceptual underpinnings, by redesigning parts of the superintendents’ and other top-level positions.

For example, in one of these districts, the superintendent seemed to have begun to engage in ILD practices in ways often reflective of appropriation with conceptual underpinnings. The superintendent took on functions of the ILD role and reportedly dedicated approximately 40–50 % of his/her time to supporting principals’ growth as instructional leaders, compared to his/her previous interactions with schools that mainly focused on operational issues such as facilities repairs. Furthermore, when probed on the nature of his/her work with individual principals, the superintendent explained how he/she visited all the district’s schools every 3 weeks to conduct classroom visits with principals focused on observing specific aspects of the instructional framework. As he/she described,

A typical meeting [at a school] is I would go in--the principal needs to have a schedule prepared. The schedule will first have kind of an agenda around teaching and learning that addresses the initiatives that we’re currently working on and currently added professional development…We then go visit for an hour a number of classrooms. We come back and talk through it. We may pick a question…[for example] we saw this learning target in this classroom. Let’s try to interpret quickly what standard that teacher was teaching to.

The superintendent reported that through his/her one-on-one visits with principals, the superintendent aimed to deepen both his/her own and the principals’ collective understanding of instruction, as opposed to monitoring instruction and directing principals toward certain actions.

At the start of our data collection period, this superintendent reported in interviews that he/she struggled to observe classrooms himself/herself let alone in the role of teaching principals how to conduct them due to lack of experience with high-quality instruction:

I’ve used [my coach from an intermediary organization] pretty much exclusively this year kind of in a modeling role so I can learn from him. And I’ve partially done that with him because he has more experience…he’s been doing this coaching thing with principals for a long time. And so I can listen to how he asks questions, how he focuses in on information.

A year later, we observed this superintendent attempting to take a far more active role as the principals’ coach. For instance, during an hour-long discussion with one principal, the superintendent probed how the principal differentiated supports for teachers who varied in instructional quality. Consistent with taking a teaching stance, the superintendent framed his/her intent to approaching this conversation as an opportunity for the principal’s learning, not as an evaluative one, reflecting the superintendent’s conceptual understanding of how the ILD functions differed from a traditional principal supervisor position:

I’ll follow up with you and we’ll talk some more and we will set a time for me to come over and we can look at [what you’ve been working on] in more depth…We’re looking at this from a growth perspective, not from what can I find that you are doing wrong…what we’re trying to do is make an extraordinary difference.

4.2 Conditions that Mediate Appropriation of Research-Based Ideas

Our conceptual framework helped us identify conditions in each district that may help explain the different levels of appropriation between and within districts. While our methods do not allow us to claim a causal connection between these conditions and the levels of appropriation, they do suggest that these conditions theoretically correlate with the outcomes we observed.

4.2.1 Intermediary Assistance Strategies

Our conceptual framework highlighted that intermediary organizations might engage in assistance strategies that interrupt status quo practice in central office in ways consistent with the research-based ideas (Tharp and Gallimore 1991; Wenger 1998). Our findings suggested that particular kinds of support from intermediary organizations may be necessary but not sufficient for realizing such results.

To elaborate, in all our study districts, different intermediary organizations intended to support central office staff with using research to inform their own practice, with staff in ILD-type positions as key targets for their work. All the intermediaries engaged in the assistance practices emphasized in our conceptual framework at least to some degree. But in the mid-sized district with little appropriation, one intermediary mainly worked with the school, not central office staff. During interviews, central office staff reported that the intermediary offered some modest suggestions for their work with principals such as the importance of setting norms for meetings and a protocol for conducting classroom observations. However, despite extensive time on site, our team captured the intermediary interacting only with principals and not central office staff. For example, the intermediary ran meetings with principals focused on improving their leadership. In the meetings we observed, central office staff attended those meetings several times but did not participate in the meetings themselves. During interviews these staff reported that they did not have a specific role in those meetings and that they had minimal interactions with the intermediary other than quick phone conversations. The lack of assistance for central office staff in this district may at least partly account for its overall low levels of appropriation of any of the research we examined.

In support of such claims about an association between the lack of intermediary assistance and low levels of appropriation, the one ILD in the other mid-sized district whom we identified as not adopting the research-based ideas at any level of appropriation occasionally came late to or left early from professional development meetings with the intermediary organization focused on helping them use the research-based ideas; this ILD also seldom brought the assignments the intermediary requested that participants complete between the meetings to help them integrate the research into their regular practice.

Beyond these extreme cases, we found that the other intermediary staff consistently demonstrated the assistance practices featured in our conceptual framework and ILDs regularly engaged with their intermediary partners, but appropriation levels were quite mixed, as described above. As an example of staff of an intermediary organization engaged in the teaching practices consistent with our conceptual framework, we observed one facilitator run professional development sessions for ILDs at least once a month focused on helping the ILDs engage specifically with the teaching practices highlighted in the emerging research base on ILDs; during those sessions he/she consistently demonstrated such assistance relationship practices.

During one such session, the facilitator convened the ILDs at a school for an entire day and modeled for the ILDs how to teach principals how to conduct classroom observations as a strategy for helping their teachers improve the quality of their teaching. He/she began this session by engaging the ILDs in an intensive discussion, over more than an hour, about how to use an instructional framework to ground the observations. During this discussion, the facilitator guided the ILDs through a process of prioritizing which aspects of the framework to use on that particular visit, explaining the importance of prioritizing to adult learning. Then, he showed the ILDs how to have an extended conversation with their principals about what they would actually look for to know if they were seeing teaching reflective of the element of the instructional framework they prioritized.

The facilitator began the discussion of the so-called look fors by framing the activity with metacognitive comments that he was going to model how they can help principals generate look fors but that they, as facilitators, should also ask principals challenging questions about how the look fors actually reflect the given standards. He said that too often facilitators simply ask principals to brainstorm what they would look for as evidence of particular teaching standards but leave the suggestions unchecked. As a result, principals sometimes observe classrooms with look fors that are not well aligned with the standards.

During the conversation, the facilitator demonstrated how to have an extended conversation that pressed participants to deepen their understanding of how to observe for high-quality instruction—rather than simply asking principals what they will see in classrooms related to student engagement and charting their comments without checking their rationale for how the “look for” exhibits teachers’ instructional quality. For instance, at one point an ILD said teacher’s checks for students’ understanding by asking students to show thumbs up or thumbs down indicated the teacher’s ability to engage students at high levels, the element of the instructional framework on which the ILDs were focusing that session. The facilitator asked the ILD, “How do you relate to that to student engagement?” The ILD responded, “It gives all kids an easy way to say whether they are getting it or getting it but need more time.” The facilitator challenged the ILD to “calibrate” the look for more tightly according to how the instructional framework defined student engagement. The ILD then elaborated that checks for understanding would give the teacher information on whether all students were accessing the information or not. If there were no checks for understanding, students who did not understand would become disengaged as the teacher moved forward. The facilitator further probed the ILD to explain whether he/she was emphasizing a teacher’s checks for understanding or if students had the ability to tell teachers when they did not understand. This dialogue spurred a discussion among the ILDs on the locus of control in classrooms and generated another look for—the quantity and quality of students’ questions.

Our conceptual framework suggests that modeling and other assistance strategies are likely to help central office staff engage with research-based practices at deep levels of appropriation. Given the consistency with which the intermediary featured above as well as the facilitators working with the smaller districts engaged in assistance relationship practices, we might expect to see consistently deeper levels of appropriation in districts with such assistance. Those ILDs who frequently appeared in the few examples of appropriation with conceptual underpinnings credited their intermediary facilitators with helping them improve their practice with their principals. However, we did find substantial variation in levels of appropriation with the districts with such assistance, suggesting that intermediary assistance may be helpful but not necessarily sufficient for deepening central office administrators’ engagement in the new challenging work practices. What other conditions might help account for such within district differences?

4.2.2 Individual Prior Knowledge and Experiences

Our conceptual framework suggests that practitioners’ prior knowledge and experiences likely mediated the effects of the intermediaries’ assistance strategies. Specifically, practitioners fit new ideas into their prior knowledge, essentially, long-standing patterns of thinking and acting that they have developed from prior experiences (Kennedy 1982; Levitt and March 1998; March 1994; Weick 1995). In the process, absent disruptions to these patterns or frames, practitioners edit or otherwise simplify the new information so it resembles familiar practice. Such tendencies are particularly prominent in situations in which feedback on performance is unavailable or unclear. In our own applications of such ideas in previous studies, we found that central office staff with extensive experience within traditional school systems tend to aspire to traditional central office roles and careers and to be particularly averse to the risks involved in adopting new central office roles (Honig 2004, 2009). Under such circumstances, limited prior knowledge and/or certain prior experiences likely poses greater challenges for intermediaries in shifting practice.

Consistent with these ideas, we found that those central office staff who appropriated the research at deeper levels had prior knowledge that was consistent with the research and, in some cases, limited experience with traditional central office roles that ran contrary to the research-based ideas about ILDs. Those who appropriated these ideas at the level of surface features or below had prior knowledge not consistent with research and relatively long careers or experience with traditional central office roles.

For instance, one ILD in one of our mid-sized districts who appropriated the research-based ideas with conceptual underpinnings had limited experience as a principal (less than 5 years) and welcomed the contrast between the ILD role and his/her own prior knowledge of the principal supervisor position. In multiple interviews, he/she recalled that when he/she was a principal he/she wished he/she had ILD-type supports. This ILD often named the research report as a “playbook” for his/her practice as an ILD since his/her previous experiences provided examples only of how a principal supervisor should not work with principals. The other ILD in this district who appropriated ILD practices at a similar level of depth likewise had no experience as a central office administrator or with administrators in a traditional central office structure.

By contrast, one ILD who appropriated the role with surface features had spent at least 5 years working at a central office and approximately 10 years as a principal and explicitly aspired to move up the ladder in the traditional central office structure. One ILD in another district who demonstrated low levels of appropriation commented that he/she was “making it up,” meaning the job of being an ILD, as he went, a striking comment given that we observed him/her participate in a half day meeting at which he/she and other members of the district leadership team examined the research on ILDs and that the district invested in coaching for their ILDs from an intermediary (discussed further below). This ILD reported using his/her previous experiences as a principal in the district to help principals “problem solve” and to enable principals to have “more time for the instructional piece” and otherwise serve in general support roles for principals quite like the roles the new ILD positions were intended to replace.

However, in the smaller districts, central office staff consistently had limited to no prior knowledge of the research on ILDs, and all had come up through traditional educational administration pathways. Our observations suggested that our focal respondents generally had consistent access to intermediary staff who engaged in assistance relationship strategies. Yet, we still found mixed levels of appropriation at systems and individual levels. What might account for those differences?

4.2.3 System Leadership for Research Use

Our data suggest that internal leadership—specifically, district leaders who themselves led the teaching of the new research-based ideas—may be essential to central office administrators’ engagement with those ideas at deep levels of appropriation. We base this claim on both positive and negative examples of this leadership in our study districts which corresponded with the differences in degrees of appropriation between districts.

For instance, in the mid-sized district with little to no appropriation, we found scant evidence that executive-level staff were leading change processes within the central office, including those related to ILDs. The superintendent, for example, did create ILD positions but seemed to engage with those staff only minimally and rarely communicated in the system about the new roles or their importance. In the other mid-sized district, an intermediary facilitator talked extensively with the superintendent about leading the central office change process and doing so from a teaching stance. Our frequent observations in this district confirmed that this superintendent regularly mentioned the ILDs in various communications. However, such mentions were brief and typically involved terminology such as “instructional leadership” that, in interviews with other staff, we learned was ambiguous for most listeners. Even the head of the curriculum and instruction unit expressed significant confusion regarding what the superintendent intended the ILDs to do and how they were to relate to his/her staff which also focused on instructional matters. This superintendent basically turned over the weekly meetings of the ILDs to the intermediaries and on only one occasion facilitated one of those meetings.

By contrast, by the second half of our data collection, we saw the superintendents of the smaller system actively leading the central office change process, particularly around the ILD role. For example, during one meeting, typical of those we observed in the smaller districts during the second half of our data collection, we recorded how one superintendent led his/her executive team through the first of several strategy sessions in which he/she explained the process they would use to engage in significant reform of the central office to dramatically improve how it supports principals to realize improved teaching in every classroom. The superintendent started by explaining that he/she would be using a cycle-of-inquiry protocol to scaffold their central office change process. They elaborated that such a protocol prompted the executive team to pick a focus for their central office change effort and then work from evidence to clarify what problem with central office performance they aimed to address, develop a theory of action for addressing that problem, and continuously assess progress and adjust their plan. He/she said that you start with the question:

What’s the problem of student learning [that is prompting us to engage in central office reform]? And then you say “Okay, what do teachers have to do differently to address that?” And then we say, “What do we as leaders need to do differently to get teachers to do that thing to address it?” We really need to start with what we [in the central office] need to do. We can’t start with what teachers need to do. Because that next question is what do we need to do to enable teachers to do different work so students are learning… We have to start with what are we going to do differently that’s going to cause teachers to do something different…The next question [is], “What do we have to know to do that? What do we as leaders have to do differently or know that we don’t know now to enable teachers to change their practice?”

In this and related meetings, this superintendent demonstrated how he/she was taking the ideas about what central office reform should involve and how to facilitate it (i.e., from a teaching stance) and using them to inform his/her own hands-on leadership of the process. Such examples stood in sharp contrast to those in the larger systems where superintendents seemed to rely on the intermediary organization staff to lead key aspects of the central office change process themselves.

5 Summary and Conclusions

This chapter begins to elaborate the outcomes and conditions in six districts that aimed to use research that fundamentally challenged the status quo in their central offices. We argued that ideas from sociocultural learning theory can help researchers and practitioners move beyond relatively simple assessments of school systems as either using or not using research to distinguish different degrees of engagement with such ideas. We show that practitioners who attempt to use research to shift their practice may do so at a surface level or with deep conceptual understanding, or somewhere in between. Intermediary organizations can assist these practitioners in shifting their practice, especially when they engage in particular assistance strategies. However, such strategies may not be sufficient absent prior knowledge and experiences that may make practitioners more or less ready to benefit from the assistance from the intermediaries. Internal leadership, particularly that which proceeds from a teaching stance itself, seems essential to realizing deeper levels of appropriation with challenging research-based ideas.

This study suggests several important directions for future research and practice. First, our research demonstrates the value of studying research use processes as learning processes. We show how specific constructs from various theories of learning can help elaborate what such processes involve. In particular, the definitions of degrees of appropriation enabled us to distinguish key variations in how central office staff engaged with research-based ideas. These definitions move beyond binaries too common in discussions of research use that simply indicated whether or not practitioners used research to specify to what degrees practitioners integrated research-based ideas into their practice. Theories of learning also highlight different conditions that enable research use. As noted above, the extant literature mostly chronicles conditions that impede various forms of evidence use. Learning theory highlights supportive conditions.

Second, we would not have been able to capture different degrees of appropriation or the assistance relationship practices of intermediary organizations had we not conducted extensive, real-time observations as our main data collection strategy. For example, in our study, practitioners in the central office used research-based ideas and practices to describe their work. If we had only conducted interviews or had only conducted one or two observations, we might have concluded they were using research to a greater extent to which they were. But in fact, many had appropriated it at the level of talk, not deep practice changes. Observations in real time allowed us to not only make these distinctions in levels of appropriation but also see and understand the sometimes subtle moves of intermediary organizations reflective of their teaching approach.

Future research might also advance knowledge by exploring what degrees of appropriation practitioners might realize with the help from intermediaries over longer periods of time than our 18-month study. Our findings suggest that more time with research-based ideas, especially in the context of certain kinds of assistance relationships, might lead to greater degrees of appropriation. But does it?

This study also raises questions that practitioners might productively consider to advance their own central office change efforts. Among them, to what extent have we hired into the central office staff with the right prior knowledge and experiences to engage in the kinds of ambitious change the research highlights? How might we select the right intermediaries to assist us with our change process—those who take a teaching stance in their work? How can we ensure we do not turn leadership of the work over to intermediaries but rather make sure we build out capacity to lead the work ourselves?