Reconsidering replication: New perspectives on large-scale school improvement

Abstract

The purpose of this analysis is to reconsider organizational replication as a strategy for large-scale school improvement: a strategy that features a “hub” organization collaborating with “outlet” schools to enact school-wide designs for improvement. To do so, we synthesize a leading line of research on commercial replication to construct a “knowledge-based logic” focused on the production, use, improvement, and retention of effective practices in large numbers of schools. Drawing on findings from a longitudinal case study, we then use the knowledge-based logic to structure an interpretation of Success for All, a leading comprehensive school reform program. In contrast to common assumptions of organizational replication as a strategy that yields rapid results at the expense of local and professional control, we argue that organizational replication can be understood as a long-term enterprise in which program providers and schools collaborate to produce, use, improve, and retain practical knowledge. Capitalizing on this potential, however, is contingent on both proponents and critics re-examining common assumptions about organizational replication and recognizing value in replication enterprises that they would otherwise miss.

Introduction

Over the past 20 years, large-scale school improvement has emerged world-wide as a primary problem of education policy and reform. In the US, the replication of school-wide improvement models has emerged as one possible solution. Despite extensive support, questionable assumptions about the process and pace of organizational replication motivate criticism, hinder evaluations, and complicate investments.

The purpose of the following analysis is to reconsider replication, with the goal of supporting more informed consideration of organizational replication as a strategy for large-scale school improvement. We begin by reviewing the emergence of organizational replication in the educational and commercial sectors. We continue by synthesizing a leading line of research on commercial replication to develop a “knowledge-based logic” of organizational replication. Drawing on findings from a longitudinal case study (see “Appendix”), we then use that logic to structure an interpretation of Success for All, a leading replication enterprise with operations in the US and abroad. We conclude by discussing the potential value of replication initiatives as knowledge-producing enterprises supporting large-scale school improvement in the US.

In contrast to common assumptions of organizational replication as a strategy that yields rapid results at the expense of local and professional control, we argue that organizational replication can be understood as a long-term enterprise in which program providers and schools collaborate to produce, use, improve, and retain practical knowledge. Capitalizing on this potential, however, is contingent on both proponents and critics re-examining common assumptions in order to see value in replication enterprises that they would otherwise miss.

Review: Organizational replication

We begin with four tasks: defining organizational replication and examining its emergence; reviewing enthusiasm and concern for organizational replication; identifying and questioning common assumptions on which both support and criticism rest; and considering the resulting risks for educational replication enterprises.

Defining organizational replication

Akin to franchising, organizational replication is the creation and operation of a large number of similar organizations that produce goods or provide services, either by creating new organizations or by extensively modifying existing ones (Winter and Szulanski 2001). The goal of replication is to reproduce effectiveness. As such, the fundamental objects of replication are capabilities for production and service delivery.¹ Indeed, replicating organizational goals, structures, and culture without replicating capabilities risks replicating the broad form of an organization without replicating the coordinated, interdependent practices of its members.

Adapted to education, organizational replication features a central, “hub” organization that devises a school-wide improvement model that is enacted in “outlet” schools.² Hubs are typically nonprofit or for profit organizations. Outlets are new or existing schools. Together, hubs and outlets operate as school improvement networks, the members of which are linked by common designs for the work of learning, teaching, and leadership.

In the US, replication-based school improvement networks function as quasi-educational systems that operate distinct from (but in interaction with) the formal system of K-12 public education. These networks serve as one potential resource for schools and districts held increasingly accountable for improving student performance on state assessments. Examples of hub organizations using replication strategies to support school improvement networks include comprehensive school reform providers, charter management organizations, and education management organizations.

This conceptualization of organizational replication is responsive to current conversations about large-scale school improvement in the US. For example, it is responsive to widespread recognition that, especially in underperforming schools, improving practice and achievement requires systemic (rather than targeted) intervention. Further, it is responsive to calls to reconsider “scale up” as a policy goal (Coburn 2003; McLaughlin and Mitra 2001). Rather than focusing on increasing the installed base of schools, the call is to reconceptualize scale up as a process of effecting deep, broad, and sustained change in practice by supporting schools in fully incorporating (and taking ownership of) externally-developed programs.

This conceptualization also differentiates organizational replication from other approaches to large-scale school improvement in the US and abroad. For example, this conceptualization differentiates organizational replication from other educational networks in which teachers, leaders, and schools collaborate directly to share practices absent a common, school-wide design or a coordinating hub organization (Daly 2010). A leading example is International Networking for Educational Transformation, an initiative established in the UK in 2004 that has grown to include 5600 schools in 35 countries (iNet 2011).

This conceptualization also differentiates organizational replication from efforts to effect system-wide reform through the formal system of public education. In analyses of leading national and provincial education systems, these have been described as a “tri-level” or “Fourth Way” strategies in which schools, districts, and government agencies engage teachers and the public in building system-wide capabilities and coherence (Fullan 2009; Hargreaves and Shirley 2009; Levin 2008). Such strategies seek to establish political and organizational infrastructure supporting more plural and differentiated (and less centralized and standardized) means of developing and sharing practical knowledge.

The emergence of organizational replication

Since the late 1980s, steady streams of public and private funding have provided billions of dollars in support of organizational replication as a large-scale education reform strategy in the US. Key sources of public funding have included Title I of the federal Elementary and Secondary Education Act, the Obey-Porter Comprehensive School Reform Demonstration Act of 1997, and the No Child Left Behind Act of 2001 (which incorporated Comprehensive School Reform as a program). Key sources of private funding have included the New American Schools initiative, the New Schools Venture Fund, and large-scale grant programs by the Bill and Melinda Gates Foundation and the Walton Family Foundation (among others).³

Support appears likely to continue for the near future. For example, beginning in 2009, federally-supported options for chronically underperforming schools include “restarting” them under the control of a charter management organization or an education management organization (Duncan 2009; U.S. Department of Education 2009). In 2010, through its $650 million Investing in Innovation (i3) program, the U.S. Department of Education awarded two of the largest, $50 million, 5 year “scale up” grants to organizations operating school improvement networks, with 20% matching funds provided by private donors: the Success for All Foundation and the Knowledge is Power Program (U.S. Department of Education 2010a).⁴ Additional sources of continued support include a second round of i3 awards, the $50 million Charter School Grant Program competition (U.S. Department of Education 2010b), philanthropic support for district/charter collaboratives (Gates Foundation 2010), foundation support for network-based “design-educational engineering-development” (Bryk et al. 2010), and (with the purchase of America’s Choice by Pearson Publishing in 2010) private investment in comprehensive school reform.

The emergence of organizational replication in education parallels its emergence in the for-profit and non-profit sectors, with replication enterprises active in over 60 industries (Winter and Szulanski 2001). While most commonly associated with low-skill industries such as fast food, organizational replication is a strategy currently being used to create investor-owned hospitals and other medical facilities in the United States (Bazzoli et al. 1999), as well as pediatric AIDS clinics abroad (Baylor International Pediatric AIDS Initiative 2010).

Enthusiasm and concern

Enthusiasm and support for organizational replication derive, in part, from instances of rapid and remarkable growth, both in the commercial and education sectors. For example, in the commercial sector, Subway sandwich shops grew from 150 to 33,246 outlets between 1980 to 2010, for a 30 year growth rate of over 22,000%.⁵ In the education sector, Success for All grew from one to over 1600 elementary schools between 1987 and 2002 and, then, stabilized at 1200 schools from 2004 to 2008: for sake of comparison, more elementary schools than served by all but thirteen state education agencies (Peurach 2011), with more than 2 million students served (Slavin et al. 2008).

Yet rapid growth does not automatically translate into profits, sustainability, or effectiveness. For example, in the commercial sector, early research found that both the survival rate and profitability of independently-founded businesses actually exceeded those of franchised outlets (Bates 1994), with later research using improved measures showing an increased franchise failure rate over time (Holmberg and Morgan 2003). In education, effectiveness has been more the exception than the rule. For example, only a small number of comprehensive school reform programs have consistently demonstrated positive, significant effects on student achievement: for example, America’s Choice, Career Academies, Direct Instruction, School Development Program, Success for All, and Talent Development High Schools (Borman 2009; Rowan et al. 2009a, b).

In the US, concerns with technical effectiveness and sustainability have been matched with normative concerns. For example, some critics argue that franchising has had a negative impact on locally-owned businesses (Schlosser 2001) and has contributed to an undesirable homogenization of the organizational landscape (Ritzer 1993). Others argue that, especially in the non-profit sector, external control of core work processes usurps autonomy, agency, and motivation in outlets (Bradach 2003). In education, critics ranging from district officials to classroom teachers have cast organizational replication as a top-down, one-size-fits-all strategy at odds with a traditions of local control and professional autonomy (Hedrick 2000; Klugh and Borman 2006). Others have argued that hub-supported networks are part of a “new privatization” of school improvement that lacks transparency and oversight and that has potential to subordinate student welfare to market incentives (Burch 2009).

Questionable assumptions

In education, both support and criticism appear to rest on two common (but questionable) assumptions about the potential for organizational replication to effect rapid, large-scale school improvement.

The first assumption is that organizational replication adheres to a sequential model of innovation. Educational researchers have framed this model as an “RDDU” sequence: research, development, dissemination, and utilization (Rowan et al. 2004). Others have framed this model as a stage-wise innovation process: needs/problems definition; basic and applied research; development, piloting, and validation; commercialization; and diffusion and adoption (Rogers 1995). An early example of an RDDU-centered reform initiative was the construction of federally-supported research centers and regional laboratories in the US, the former charged with basic and applied research and the latter charged with devising and disseminating innovations for schools (Guthrie 1989). A later example was New American Schools, which drew directly from the sequential model of innovation to structure support for comprehensive school reform as a 6 year, four phase progression: competition and selection, development, demonstration, and scale up (Bodilly 1996).

The second assumption is that RDDU-like replication processes enable rapid, large-scale improvement by providing schools with “research-based” and “research-proven” programs that can be implemented quickly, effectively, and efficiently. For example, early evidence of positive program effects were instrumental in Success for All being identified as a “tried and true” program for improving underperforming schools (U.S. Department of Education 1997) that could enable rapid, large-scale school improvement “right out of the box” (U.S. Department of Education 1999). Yet this same “out-of-the-boxedness” also fueled criticism of comprehensive school reform as a top-down, one-size-fits-all strategy at odds with local control and professional autonomy.

Though common, research suggests that neither assumption holds in practice. Consider comprehensive school reform. In contrast to an RDDU-like sequence, Rand researchers found that the work to be performed in each phase of New American Schools was actually performed concurrently, in interaction, over time (Berends et al. 2002). In a complementary review of research, Rand researchers re-conceptualized replication not as a stage-wise process but, instead, as a set of simultaneous, interdependent tasks enacted jointly by hubs and schools over time: for example, developing the core model; recruiting and marketing; monitoring and supporting implementation; adapting to environmental contexts; obtaining financial support; and building hub capacity (Glennan et al. 2004). Further, in contrast to quick and effective implementation, Rand researchers reported that, after 6 years, both implementation and achievement outcomes were highly variable (Berends et al. 2002).

Comparable themes emerged in other research on comprehensive school reform. In a synthesis of findings from longitudinal studies of scaling up comprehensive school reform, researchers described organizational replication as a time-dependent process of sense-making and co-construction among schools, developers, and other educational agencies, in contrast to a rapid, sequential progression from concept to large-scale success (Datnow et al. 2002; Datnow and Park 2009). Researchers describe such collaborative, inter-organizational learning as requiring the management of a fundamental tension between exploiting available knowledge and exploring new directions (Hatch 2000), as fraught with challenges (Hatch and White 2002; McDonald et al. 2009), and as complicated by turbulence in educational and broader environments (Glazer 2009; Glazer and Peurach, in press). Again, in contrast to rapid results, a comprehensive meta-analysis of achievement outcomes in comprehensive school reform programs found that the strongest program effects came only after the fifth year of implementation (Borman et al. 2003).

Comparable themes are also emerging in research on charter management organizations. For example, in contrast to an RDDU-like process, researchers again describe the leadership of charter management organizations as requiring the management of a set of simultaneous, interdependent tasks paralleling those described by Glennan and colleagues (Farrell et al. 2009), with the work of achieving effectiveness, scale, and sustainability proceeding more slowly and requiring more resources than anticipated (Center for Research on Education Outcomes 2009; Education Sector 2009; Lake et al. 2010).⁶ Still others report a “tyranny of business plans” resulting from charter management organizations needing to project rationality and efficiency to secure funding, despite encountering complexity and uncertainty that require flexibility and adaptability in their work (National Charter School Research Project 2007).

The preceding research extends long-established findings that effective implementation of externally-developed educational programs depends on schools and external providers collaborating in reciprocal, mutually-adaptive relationships, with success typically the exception rather than the rule (Berman and McLaughlin 1975, 1978; Elmore 2004; Firestone and Corbett 1988). Moreover, the preceding findings are consistent with findings from broader research on innovation (Van de Ven et al. 1999). This research directly refutes the stage-wise model in order to frame the innovation process as a cycles of “divergent” and “convergent” learning co-enacted by developers and users over time. Consistent with exploration and exploitation as described by Hatch (2000), the former involves engaging in experimentation, discovery, and search to expand innovation alternatives, while the latter involves engaging in trial-and-error testing to design, learn to use, and refine the innovation.

Potential risks

Thus, research suggests that organizational replication does not function consistent with common assumptions of proponents and critics: either as a strategy for rapidly reproducing proven organizational models or as necessarily usurping local and professional control. Even so, the persistence of such assumptions risks undermining specific replication initiatives. For example, common-but-questionable assumptions about organizational replication interacted with exacerbating conditions to contribute to the rapid rise and fall of comprehensive school reform, despite billions of dollars of sunk costs in the development and evaluation of programs, hubs, and networks.

By the early 2000s, in part due to the perceived potential for rapid and large-scale improvement, more than one billion dollars had been invested in comprehensive school reform, with an estimated 6.5% of US public schools implementing a comprehensive school reform program (Murphy and Datnow 2003). In 2001, Comprehensive School Reform was formally incorporated into NCLB, with $300 million per year appropriated for program adoption. Enthusiasm for comprehensive school reform led some observers to describe it as “a fixture on the US education landscape” (Finn 2001, p. iv) and others to predict that “the CSR movement will continue to grow and thrive in the next few years” (Murphy and Datnow, p. 15).

Other observers were more cautious. In their summary analysis of New American Schools, Rand researchers described comprehensive school reform as promising-but-difficult (Berends et al. 2002). On the one hand, they reported that approximately 50% of schools implementing NAS designs demonstrated achievement gains relative to comparison groups. On the other hand, they predicted that expectations for quick success could interact with enduring problems of US educational environments to threaten continued support, including district turbulence, policy fragmentation, and a short issue-attention cycle.

Such concerns proved to be warranted. Between 2003 and 2006, comprehensive school reform suffered a rapid collapse (Glazer and Peurach, in press; Peurach 2011). In the early 2000s, the inability of all but a small number of programs to demonstrate positive effects on student achievement undermined arguments that comprehensive school reform programs could be implemented quickly and effectively “out of the box.” That, in turn, raised questions about the political influence of proponents and about returns on investment (Mirel 2001, 2002; Pogrow 2000). Such criticism interacted with other conditions to undermine policy support, including increased federal support for NCLB’s K-3 Reading First program, the planned termination of New American Schools, increased involvement of districts in school improvement, and increasing philanthropic support for charter schools.

In 2006, supplemental funding for NCLB’s Comprehensive School Reform program was eliminated from the federal budget. Schools that elected to continue were left to draw from Title I whole-school funding, special education, state poverty supplements, and other sources (Borman 2009). Providers that elected to continue were forced to make radical adaptations to their clients, programs, hub organizations, and financial strategies to sustain operations (Cohen et al., in press; Glazer and Peurach, in press; Peurach 2011). For those providers that elected not to continue, much of the value generated by way of new knowledge of large-scale school improvement was lost.

Amidst continuing (and formidable) investment in organizational replication in education, the possibility of such a rapid collapse in the support of specific replication initiatives should give pause for concern. Improving understanding of organizational replication in education cannot obviate the risk of abrupt shifts in policy priorities and support. Even so, weak understandings arguably heighten that risk.

Commercial replication: A knowledge-based logic

Toward further improving understanding of organizational replication, we turn to research on commercial replication. Though a focus of investigation in organizational studies, economics, and business over the past two decades, research on commercial replication has only begun to inform understanding of educational replication.⁷

Specifically, we focus on synthesizing a leading line research on commercial replication by Sidney Winter, Gabriel Szulanski, and colleagues.⁸ Consistent with research on educational replication, this research portrays commercial replication as a long-term process focused first and foremost on deep change in practice, increasing agency and ownership in outlets, and collaborative learning between hubs and outlets. Rather than describing an explicit strategy for replication, this research explicates and conceptualizes processes that appear to support the successful reproduction of capabilities within and between organizations.

In synthesizing this research, our objective is not to develop a set of “how to” prescriptions for educational replication. Rather, our objective is to develop an ideal type: an analytic framework for interpreting and comparing specific examples of organizational replication in education, with a primary focus on the reproduction and improvement of coordinated, interdependent practices across large numbers of schools.⁹

Problem: Common assumptions

In the commercial sector (as in the education sector), analysis begins with the argument that common assumptions of replication as a strategy for quickly reproducing established and effective organizational models simply do not hold. As with educational replication, commercial replication is commonly understood to have a hub organization developing and honing a formula or business model and, then, quickly diffusing the established model across large numbers of outlets (Winter and Szulanski 2001). The replication process, in turn, is commonly assumed to adhere to a sequential, long-linked research-and-development process in which knowledge supporting the core capabilities of outlets is developed, transferred, and eventually used, with new value added at each step. Needs/problem analysis informs basic and applied research, which informs innovation development, which is followed by commercialization, diffusion, adoption, and use.

One point of contention, however, is that common conceptions of commercial replication equate knowledge transfer with the straightforward communication of information from one step to the next (Szulanski et al. 2002; Winter and Szulanski 2001, 2002).¹⁰ Another is that common conceptions are based on assumptions that typically do not hold in practice: that the knowledge to be replicated is known perfectly by the replicator prior to large scale replication; that transfer is a seamless and instantaneous process; that effective use is transparent; and that problems of transfer and use are easily and quickly resolved through iterative communication between the replicator and the recipient. As explained by Winter and Szulanski (2001):

The formula or business model, far from being a quantum of information that is revealed in a flash, is typically a complex set of interdependent routines that is discovered, adjusted, and fine-tuned by “doing.” Growth by replicating such a “formula” requires the capability to recreate complex, imperfectly understood, and partly tacit productive processes in carefully-selected sites, with different human resources each time, facing in many cases resistance from proud, locally autonomous agents. For this reason, replication requires effort and naturally takes time. (p. 731)

Alternative: A “Knowledge-Based” logic

In contrast to common assumptions, Winter, Szulanski, and colleagues develop a “knowledge-based” logic of organizational replication. In this logic, the super-ordinate issue is the development, reproduction, use, and refinement of knowledge, on the grounds (again) that replicating goals, structures, and culture without replicating capabilities risks replicating organizational forms without replicating organizational effectiveness (Winter and Szulanski 2001).

Thus, the primary focus of the knowledge-based logic is understanding the generation and replication of the facts, information, and skills—that is, the knowledge—that enables effective practice in outlets.¹¹ The essential knowledge base supporting effective replication consists of three categories: knowledge of the practices to be replicated in outlets to ensure effectiveness; knowledge of organizational and environmental conditions favoring effective replication; and knowledge of how to replicate effective practices in different organizational and environmental contexts (Winter and Szulanski 2001).¹² This knowledge base can be understood as answering three questions: What, where, and how should the hub be trying to replicate?

Consistent with innovation as a cycle of divergent and convergent learning (Van de Ven et al. 1999), this knowledge base is generated, reproduced, used, and refined through multiple iterations of two interdependent learning processes co-enacted by hubs and outlets: exploration and exploitation (Winter and Szulanski 2001; see, also, Bradach 1998; Hatch 2000; March 1991/1996). Exploration involves the identification of new possibilities for what, where, and how to replicate through search, experimentation, and discovery. Exploitation involves selecting from among these possibilities, implementing them, and learning via use and experience.

The centrality of routines

A central tenet of the knowledge-based logic is that the practices to be replicated across outlets take the form of routines: repetitive, recognizable patterns of interdependent actions, potentially involving multiple actors (Feldman and Pentland 2003; Nelson and Winter 1982; Winter and Szulanski 2002). Routines are assumed to always include an explicit “information” component as codified in formal procedures and a tacit “know-how” component as retained in the minds and joint work of those who enact them.¹³

Routines are considered the foundational capabilities of organizations and the primary mechanism for supporting a level of coordinated, productive activity in outlets that would otherwise be difficult and costly to achieve. Routines take multiple forms: straightforward procedures; lower-order decision rules for selecting courses of action; higher-order procedures for structuring analyses and determining action under conditions of uncertainty; individual-level routines that combine to form collective routines; and routines that guide learning through analysis, reflection, and adaptation. Further, routines guide work at all levels of organization: operational, managerial, and executive.

Thus, within the knowledge-based logic, formal, codified routines are not understood as “coercive” devices commonly associated with bureaucracy, rigidity, and alienation (Adler and Borys 1996). Rather, routines are understood as “enabling” devices that codify guidance for the performance of technical or professional work in order to clarify and coordinate responsibilities, ease stress, and evoke efficacy and commitment. Adler and Borys explain:

Formal procedures do not have to be designed to make the work process foolproof. They can be designed to enable employees to deal more effectively with its inevitable contingencies. In what we call the enabling type of formalization, procedures provide organizational memory that captures lessons learned from experience (Levitt and March 1988; Walsh and Ungson 1991). Formalization codifies best-practice routines so as to stabilize and diffuse new organizational capabilities (Nelson and Winter 1982). The idea of an enabling type of formalization is consistent with Blau’s (1955) finding that “good” procedures are those seen as valuable resources that help professionals meet clients’ needs (p. 69).

Initiating the knowledge base: The template

Central to the knowledge-based logic is explication of the process by which hubs, outlets, and the knowledge base co-emerge and co-evolve over time, with particular emphasis on the emergence, formalization, and evolution of essential routines. The process begins with exploratory learning in a “template” organization that functions as working example (Winter 2003; Winter and Szulanski 2001). Especially important is examining practice in the template to identify, understand, and formalize essential routines, as well as to understand the interdependence of routines and their relationships to intended ends.

Optimally, analysis of the template would yield a perfect knowledge base: all of the knowledge needed to achieve the desired outcomes in each outlet, and none that didn’t. However, the knowledge base will always contain both partial and superfluous knowledge, as well as errant understandings (Winter and Szulanski 2001). Within the template, activities may combine to effect intended outcomes in non-obvious ways, relevant knowledge may be tacit, understandings of cause-and-effect relationships can be flawed, and apparently-important activities may be completely unrelated to outcomes. Further, the effectiveness of the template may be bound up with individuals, relationships, and local environments in ways not fully understood at the outset, and in ways that complicate their reproduction in new organizations and environments. Finally, market pressure to proceed quickly to large scale replication creates a hallmark urgency that precludes exhaustive exploratory learning, including pressure to realize returns on investments, to secure market share, and to increase profits.

Thus, hubs don’t optimize but, instead, satisfice (March and Simon 1958).¹⁴ They terminate exploratory learning having generated satisfactory knowledge (but not perfect knowledge), freeze the template (even though it is likely incomplete), formalize essential routines (even though results are likely problematic), and proceed to large-scale replication (despite risks arising from partial and problematic knowledge).

Recreating and using knowledge: Fidelity, then adaptation

The process of replication continues, then, with an initial, “broad scale” knowledge transfer from the hub to outlets sufficient in scope to create or to significantly modify their core capabilities and, possibly, their identities (Winter and Szulanski 2001). However, a premise of the knowledge-based logic is that complex knowledge of interdependent practices cannot be transmitted to outlets via straightforward processes of communication. Rather, this knowledge must be recreated in individual outlets.¹⁵

Recreating essential knowledge involves a developmental sequence that progresses from fidelity of implementation to adaptive, locally-responsive use (Szulanski et al. 2002; Winter 2010; Winter and Szulanski 2001). The former emphasizes exploitation as a core learning process: taking advantage of knowledge already generated by putting it to use in new outlets. The latter emphasizes exploration: adapting and extending this knowledge in response to outlet-specific exigencies and environments.

During early implementation, both hubs and outlets benefit from fidelity of implementation, especially through the precise enactment of formally-codified routines. Fidelity affords multiple advantages: mitigating against initially-weak capabilities in outlets; exploiting knowledge already generated and problems already solved; establishing conventional practices among outlets; developing understandings of routines and their interdependencies through repetition and reflection; and avoiding the creation of site-specific problems that outlets are initially unprepared to manage. While precise replication runs risks (e.g., a potential misfit between routines-as-codified and outlet-specific circumstances; the deferred benefits of favorable adaptations), the benefits are argued to outweigh the risks.

As they master faithful enactment and develop base-level understandings, outlets assume agency in the enactment of routines, engaging in exploratory learning through which they adapt core routines to resolve unexpected outcomes and to address local exigencies and environments. With repetition, reflection, and accumulating experience, successful adaptations of routines are typically retained tacitly in the core operations of individual outlets (rather than being formalized as written procedures). These adaptations, in turn, serve as a source of variation within the enterprise regarding understandings of “best practice.”

Refining knowledge: Knowledge evolution

The developmental progression from fidelity to adaptation supports a knowledge evolution cycle: an iterative process of expanding, refining, and further codifying the knowledge base over time (Zollo and Winter 2002; see, also, Feldman and Pentland 2003).¹⁶ After faithful implementation, and as outlets expand understandings of “best practice” through local problem solving, the hub engages in its own exploratory learning: for example, by monitoring variation across outlets to identify favorable adaptations; by monitoring the development of knowledge, technologies, and other resources and constraints in environments; and by experimenting and inventing on its own (Winter and Szulanski 2002). On-going, the hub evaluates possible adaptations to the core model and selects those likely to increase effectiveness across outlets.

Exploratory learning by the hub functions as a key means by which tacit knowledge is made explicit through such mechanisms as collective discussion of outlet-level variation, brainstorming sessions, and informal performance evaluations. It also functions as a key means by which tacit knowledge is initially codified, as in memos, performance analyses, white papers, and other working documents. Drawing on these analyses, program improvements are retained by codifying them as formal routines and/or by incorporating them into a template organization as a working example.

Changes to the model are fed back through the installed base of outlets via “narrow scope” knowledge transfer focused on incremental improvements in existing capabilities (Winter and Szulanski 2002). The core learning processes of exploitation and exploration then begin again, with outlets first enacting changes faithfully and, with gathering understanding, adaptively. The cycle continues, with successive iterations resulting in refined versions of best practice, more knowledge of best practice, increasingly codified knowledge, and a more complete template. That, in turn, increases both the initial, “broad scope” knowledge transfer to newly-adopting outlets, as well as the initial imperative to exploit the growing knowledge base through fidelity of implementation.

Such learning activity can yield what some might view as counter-intuitive results. One is that precise replication of codified routines functions as the foundation for producing new knowledge. Another is that formally codified routines support the enactment of “non-routine” tasks often thought to require the exercise of professional discretion and judgment. Zollo and Winter (2002) and Adler and Borys (1996) go so far as to argue that the advantages of formalization actually increase with technical and environmental uncertainty, with the hub exploiting its position at the center of the network to codify knowledge of what, for any given outlet, would be infrequent and exceptional activity.

All of the preceding depends on the “dynamic capabilities” of hubs and outlets: their capabilities to acquire, develop, and refine the essential knowledge base of replication (Winter 2003; Winter and Szulanski 2001; Zollo and Winter 2002). Dynamic capabilities are a sort of meta-capacity: a learned pattern of collective activity through which organizations systematically generate and modify operational routines in pursuit of improved effectiveness, continued legitimacy, and sustainability. Dynamic capabilities can be thought of as “routines to change routines,” themselves learned: for example, hubs and outlets co-enacting a Deming-like “plan-do-check-act” continuous improvement cycle (Dosi et al. 2001).

This is a matter of “more”, but not “all.” Neither the hub nor its outlets are (or ever will be) omniscient. The essential knowledge base is always partial, and key knowledge always remains undiscovered and/or tacit.

Summary: The knowledge-based logic

Thus, in contrast to replication as the sequential, point-in-time transfer of an optimal business model, the knowledge-based logic describes a set of interdependent learning processes by which the knowledge base supporting effective replication is developed, reproduced, used, and refined over time through collaborative learning among hubs and outlets. While the former assumes perfect knowledge prior to large-scale replication, the latter argues that better (but not perfect) knowledge can only result from large scale replication. Further, in contrast to usurping local control and agency, the knowledge-based logic depends on agency and adaptation in outlets as a key resource for network-wide learning and improvement.

We propose the knowledge-based logic as an ideal type: an analytic framework for interpreting organizational replication as a process of collaborative learning. It is not a description of how all replication enterprises operate. Indeed, it is entirely possible for replication enterprises to function in ways that inhibit collaborative learning. For example, hubs may simply seek to increase the installed base of outlets absent a commitment to effectiveness, or they might actually use codified routines to coerce activity in outlets. Further, even when hubs value adaptation, outlets may adopt a “bureaucratic” orientation: an interpretation of codified routines and initial fidelity as malevolent, thus evoking active resistance or reluctant compliance. Alternatively, outlets may adopt a “technocratic” orientation: an interpretation of codified routines as benevolent, from which follows enthusiastic compliance. As explained by Winter and Szulanski (2001), the reasoning is disarmingly simple: If the organizational model works, why change it?

Thus, our argument is not that all replication enterprises adhere to the knowledge-based logic. Rather, our argument is that research on commercial replication suggests that a core set of structures, practices, and understandings support organizational replication as knowledge-producing activity:

• A network structure in which hubs and outlets collaborate to enact an organizational model that goes beyond formal and social structures to a design for coordinated, interdependent practices.

• The design for practice as the product of collaborative, evolutionary learning between the hub and outlets via processes of exploration and exploitation.

• A knowledge base that addresses the questions of where, how, and what to replicate, with an emphasis on the formalization of knowledge using codified routines.

• A developmental progression from fidelity to adaptation as the means for recreating and improving knowledge in outlets, with attention to managing interpretations of the progression as enabling (and not coercive).

• Highly-developed dynamic capabilities in the hub organization that support continuous improvement (to increase effectiveness) and strategic management (to ensure continued viability in changing environments).

Educational replication: A knowledge-based interpretation of success for all

We continue by using the knowledge-based logic to frame an interpretation of Success for All. As cited above, Success for All is an original and leading comprehensive school reform program. Amidst turbulence in policy support for comprehensive school reform, Success for All has succeeded in expanding its scale of operations, demonstrating positive program effects on practice and achievement, and securing the resources needed to further sustain and expand the enterprise. As detailed in “Appendix”, our interpretation of Success for All derives from longitudinal case study that ran from 1996 to 2010, the purpose of which was to conceptualize the work and challenges of large-scale, externally-supported instructional improvement.

While proponents and critics have widely interpreted Success for All both as a quick fix and a top-down, one-size-fits-all intervention, we draw on the knowledge-based logic to interpret Success for All as replicating coordinated, interdependent practices by collaborating with schools in a knowledge-producing enterprise. As examined in more detail below, this approach has been central to Success for All since its inception. Even so, exigencies that arose during rapid scale up in the program’s first decade of operations led to an emphasis on fidelity over adaptation. Over its second decade of operations, key improvement initiatives centered on improving capabilities to support adaptation and collaborative learning.¹⁷

Our interpretation suggests the potential usefulness of the knowledge-based logic for analyzing instances of educational replication: enterprises with different knowledge and coordination demands over much longer periods of time than the low-skilled enterprises with which replication is commonly associated.¹⁸ In doing so, our interpretation corroborates (and extends) findings from other educational research about the process and pace of organizational replication, and it suggests potential value in school improvement networks as knowledge producing enterprises supporting large-scale school improvement.

A practice-focused network

The Success for All enterprise is structured as defined above: as a hub organization devising a school-wide improvement model that is enacted in outlet schools. The hub, the Success for All Foundation (SFAF), originated in 1987 as a project team in two leading research centers at Johns Hopkins University.¹⁹ In 1997, it was established as an independent, non-profit foundation, with capabilities for program development, training, research, and executive leadership. In 2008, SFAF reported employing approximately 250 people distributed evenly among its headquarters in Towson, MD and its regionally-distributed training organization, with funding from grants, materials sales, and fees for training services. Over its history, SFAF has also fielded initiatives in Canada, England, Mexico, Israel, and Australia, and its founders (Robert Slavin and Nancy Madden) have been instrumental in establishing the Institute for Effective Education at the University of York in the UK.

While SFAF supports an array of programs, its original and flagship program is Success for All, a comprehensive school reform program targeting low-performing, high-poverty schools eligible to use federal Title I funding for school-wide improvement (Slavin and Madden 2001; Slavin et al. 2008, 1996). The program features designs for scheduling instruction, assigning students and teachers to classes, and coordinating instructional and non-instructional services. It also features three “first principles” intended to serve as the cultural foundation for all participating schools: prevent academic and non-academic problems from arising in the first place; intervene early at the first sign of problems; and relentlessly evaluate and adapt student services until success is achieved.

As with the knowledge-based logic, Success for All includes complementary designs for coordinated, school-wide practice, with the goal of transforming what its developers consider to be the core capability of US elementary schools: reading instruction. Designs for practice evolved and expanded over time, working out from a cooperative learning model for students (Slavin 1983, 1990) to include designs for the teacher role in instruction (Stevens et al. 1987; Slavin et al. 1984), supplemental student services (Slavin et al. 1992; Stevens and Slavin 1995), school leadership (Livingston et al. 1996), and district leadership (Success for All Foundation 2008).

The Success for All network expanded in a pattern consistent with the knowledge-based logic. Developers drew from research and from existing programs to initiate Success for All in a single template school in 1987. From 1987 to 2001, a combination of conditions drove urgent growth to 1600 schools, including policy support for comprehensive school reform, increasing accountability for improving student achievement, and the mission of SFAF to expand its operations (Glazer and Peurach, in press; Peurach 2011). With subsequent declines in policy support for comprehensive school reform, the network contracted to 1200 schools by 2005. It remained at 1200 schools through 2008, with an average time-in-program of over 8 years.²⁰

Collaborative, evolutionary learning

Our initial analysis led to the operating assumption that Success for All was the product of a sequential process of research, development, diffusion, and utilization. For example, SFAF has cultivated a public image of the program as rooted in research-validated best practices (Slavin and Fashola 1998; Slavin et al. 1989). Further, from 1994 to 1997, Success for All was housed within the Center for Research on the Education of Students Placed At Risk, the mission of which was “to conduct the research, development, evaluation, and dissemination needed to transform schooling for students placed at risk” (Slavin and Madden 1996, p. ii). Still further, Success for All participated in New American Schools, the four-phase structure of which was drawn directly from the sequential model of innovation (Bodilly 1996).

However, our continuing analysis led to two new understandings. The first was that, rather than being completely and perfectly developed in advance of scaling up, Success for All has constantly evolved over its history. Evidence of evolution lies in changes in program materials over time, including a series of trade books detailing the rationale and design of the program as understood and pursued at distinct points in its history (Slavin and Madden 2001; Slavin et al. 2008, 1996). Evidence of the expectation for continued evolution lies in SFAF’s proposal for i3 funding, which details ambitions to expand the program to include district-managed networks of high-quality Success for All coaches (U.S. Department of Education 2011).

The second was that Success for All evolved not only (or even primarily) through the incorporation of new research findings but, also, through collaborative learning among SFAF and participating schools. For example, we identified evidence of collaborative learning in our historical reconstruction of the “practice focus” of Success for All, with expanding designs for practice largely a response to weaknesses in (and new understandings of) program implementation and outcomes. We observed such learning directly between 1999 and 2008, in the context of two development cycles in which SFAF staff collaborated with experienced teachers and school leaders to revise the program to improve effectiveness and to ensure sustainability (Peurach 2011). And we identified analyses of program implementation that provide corroborating accounts of collaborative learning in Success for All (Datnow et al. 2002; Park and Datnow 2008).

We also identified instances in which SFAF’s co-founders wrote openly about improving the program through collaborative learning with schools. For example, in an early report on the long-term maintenance of Success for All, co-founders Robert Slavin and Nancy Madden wrote that programs like Success for All “need to constantly be learning from schools themselves and from research, and then incorporating new ideas into new materials” to reward and sustain schools’ motivation and commitment (Slavin and Madden 1996). In a later chapter on the role of research in the development and scale-up of Success for All, they wrote:

Essentially, the continual development of SFA is a story of how developers, trainers, researchers, and practitioners work together. There is considerable formal research informing the program and its continual development. However, while there is a reliance on rigorous, quantitative research methods in informing model development, there is also a very strong commitment to learn from teacher practice. SFAF seeks a constant interplay between teachers’ practice and research. The knowledge of SFA trainers, many of whom were former SFA teachers, is also integral to the development of the model and its implementation strategies (Slavin et al. 2007:272).

Consistent both with earlier research on comprehensive school reform and with the knowledge-based logic, this collaborative learning involved both exploring possible program improvements and exploiting new knowledge as it emerged in the Success for All network and in broader environments. Exploration and exploitation were not enacted as distinct, cyclical tasks. Rather, consistent with the reconceptualization of the replication process by Glennan et al. (2004), exploration and exploitation were embedded in five interdependent functions co-enacted by SFAF and schools over our entire period of observation: designing organizational blueprints; devising supports for implementation; scaling up the network of schools and SFAF as an organization; continuously improving the program to increase effectiveness; and strategizing and adapting to sustain the enterprise in turbulent environments (Peurach 2011).

The formal knowledge base

The combination of exploration and exploitation functioned to move new understandings of Success for All from tacit to codified. New understandings typically emerged in the individual or shared experience of school or SFAF staff members via the exploratory processes described above. These served as a source of variation within the network. Potential program improvements were then selected for further development for different reasons, including evidence of effectiveness, consonance with research findings, relevance in changing environments, and available funding for further development. Program improvements were then retained through what SFAF staff members described as a process of “formalizing,” “writing,” or “embedding” program improvements in print, digital, and other resources. In a 2003 interview, one SFAF developer summarized the process:

What happens is, we get comments and feedback from both schools and trainers, or more research shows that another approach is more effective. It builds up until, one day, we determine that it’s time to rewrite a piece. A development team is identified. Who takes part depends on the component, but Nancy (Madden, SFAF co-founder and president) oversees all of the teams. She meets with the writing team to discuss what needs to be written. The writers begin to write and then they meet with her again and review what’s been written. Often they’ll ask other people to review the materials and provide comments, ask questions and the like. When our department gets involved, we share our opinions as well. That’s particularly helpful because we’ve all worked in Success for All schools in the field and know what is likely to work, and what’s not. And it just sort of morphs into this on-going process. Each item is revised and revised and revised based on internal feedback and feedback from talented trainers and even feedback from teachers.²¹

The result of this process is an ever-expanding formal knowledge base detailing what, where, and how to replicate. Regarding what to replicate, the knowledge base includes understandings of interdependent practices, technologies, structures, and norms supporting the improvement of K-6 reading achievement for at-risk students. Regarding where to replicate, the knowledge base includes detailed program adoption and contracting processes used to identify and secure relationships with committed schools and districts (including the in-principle requirement of a positive vote for program adoption by 80% of the instructional staff).²² Regarding how to replicate, the knowledge base combines the formalization of knowledge with coordinated professional learning opportunities that are, themselves, formalized: for example, conventional training sessions, practice-based learning among teachers and leaders, and site-based support from SFAF trainers.

Much of this knowledge is formalized in codified routines that are combined to support coordinated, interdependent practice among teachers, school leaders, and trainers.²³ These include “closed” routines that provide step-by-step directions for what, exactly, to do in particular situations. They also include “open” routines used by teachers, leaders, and trainers to devise courses of action responsive to local circumstances. Formalization goes further, to include information resources and supplemental guidance that support the selection, enactment, and coordination of routines. Examples of the former include assessments, forms, and digital information systems; examples of the latter include manuals, appendices, and on-line help. Finally, consistent with the notion of dynamic capabilities, these resources are combined to support Deming-like cycles of collaborative diagnosis, planning, implementation, evaluation, and reflection.

While Success for All’s extensive use of routines has been widely interpreted by critics as coercive, our earliest analysis led us to an interpretation of the program as an enabling resource intended to support schools in addressing uncertain and variable needs. This interpretation was rooted in recognition of Success for All as using open routines, information resources, and supplemental guidance to structure decision making, evaluation, and reflection. It was rooted in recognition of “relentlessness” as a first principle encouraging the constant evaluation and adaptation of services for highly variable at-risk students. And it was rooted in guidance to teachers and leaders encouraging adaptive, locally-responsive use of the program. For example, as described in an early version of the Facilitator’s Manual, the primary reference resource for school leaders:

Your leadership, as both a teacher and a learner, is essential for the success of the program. On one hand, this program is highly structured; implementation requires mastering the skills and information detailed in several manuals. Success does in fact depend on using a well-developed and highly-specific curriculum and some methods proven by research in real classrooms to be particularly effective. On the other hand, many of the questions teachers ask are not answered in the manuals. A lot of the problems you encounter are products of your unique situation and are best solved by homemade remedies. For that matter, some of the ideas about making Success for All work have not yet been thought of and are awaiting to be discovered by you and your colleagues. Regular opportunities for team problem-solving help to ensure that everybody feels a part of the solution and emphasize the importance and value of shared ideas….

This program works best in the hands of competent teachers challenged to use their own professional judgment in deciding many important questions about implementation. As the facilitator, you must model the use of good judgment. Nothing this or any manual could possibly include eliminates the need for you to consult with teachers often, to weigh facts carefully, to read relevant research reports and review the principles embedded in the program, and to make your own, sound professional decisions. Effective implementation depends on your reflective and adaptive leadership (Livingston et al. 1996, “Role”, pp. 6–7).

The use of routines emerged and expanded in ways that reflect the combination of satisficing and evolutionary learning central to the knowledge-based logic (Peurach 2011). Over its first decade, staff members from SFAF and from early-implementing schools reported that Success for All initially included more extensive use of closed routines than open routines. This was described largely as an artifact of three exigencies that arose in the context of early, rapid growth: the need to quickly establish a “high floor” under implementation in a network of schools growing at a rate of 50% or more per year; the need to prevent a backslide into past practice; and tacit knowledge of adaptive use emerging only as a high floor was established. Even then, respondents reported that this tacit knowledge was concentrated among experienced developers, trainers, teachers, and school leaders collaborating in tight social networks.

SFAF’s second decade, then, was marked by extensive efforts to formalize tacit knowledge of adaptive use. This took multiple forms: for example, increased formalization of open routines; the introduction of new assessments, information systems, and supplemental guidance; and increased, formal support of dynamic capabilities in schools through structured cycles of planning, implementation, evaluation, reflection, and adaptation.

Developmental sequence

Consistent with the knowledge-based logic, Success for All features a developmental sequence marked by a progression from fidelity of implementation to adaptive, locally-responsive use (Peurach 2011). The developmental sequence is enacted over a standard, 3 year implementation window (and beyond, for schools that elect to continue).²⁴

In the first year, the primary focus is on using routines, information resources, and supplemental guidance faithfully and conventionally. Beginning in the middle of the first year and continuing thereafter, the focus shifts to using program resources to work flexibly and adaptively. The developmental sequence is intended to accomplish multiple, simultaneous goals, including supporting early success through the enactment of tested practices, forestalling early problems (e.g., regressing to past practice and/or introducing uncertainty into early implementation), and establishing common language and experience to support collaboration. Adaptation is intended to increase effectiveness both by addressing local needs and by introducing the variation that drives network-wide learning and improvement.

The process by which SFAF formalized the developmental sequence was marked by the same satisficing and evolutionary learning described immediately above. Over Success for All’s first decade, the developmental sequence was initially formalized as “first principles” and supplemental guidance that encourage schools to exercise professional judgment as their experience and circumstances warranted. Otherwise, SFAF staff members reported that understandings of the developmental sequence were (again) largely tacit, and shared only among communities of experienced developers, trainers, teachers, and school leaders.

However, generally-weak understandings of the transition from fidelity to adaptation interacted with exacerbating conditions to effect “bureaucratic” and “technocratic” interpretations of the program, including generally low levels of expertise and experience in SFAF’s rapidly-growing training organization, cases of district-forced adoption, and policy pressure to rapidly improve achievement.²⁵ Those interpretations, in turn, evoked resistance in some cases and blind compliance in others. As a result, SFAF’s director of implementation estimated that, as Success for All began its second decade of operations, 75% of schools were locked in a pattern of rote, mechanistic implementation.²⁶

Beginning in 1999, SFAF began to improve the formalization of the developmental sequence to support widely-shared understanding of the progression from fidelity to adaptive use. One locus of this work was an effort to develop resources to guide school leaders in supporting adaptive, locally-responsive use of the program. SFAF did not draw on research on educational leadership, which, at the time, was still beginning to address instructional leadership, data-driven improvement, teachers’ professional learning, and other now-current domains of leadership research and practice. Rather, SFAF drew on a panel of experienced school and district leaders who had succeeded in guiding schools in using program resources in locally-responsive ways (Allen et al. 1999). Two members of this team then joined SFAF to collaborate with an experienced SFAF manager (and ex-Success for All principal) to lead a decade-long effort to improve Success for All’s leadership component.

Among the earliest products of their work was an adaptation of the Concerns-Based Adoption Model’s “Levels of Use” framework (Hord et al. 1987). As detailed in Table 1, this framework identifies an eight-step progression from faithful to increasingly-adaptive use of the program, with the leap from “routine” to “refined” use marking the beginning of that transition.²⁷

Table 1 Levels of Use (LoU) of the Success for All program. The Levels of Use component of the Concerns-Based Adoption Model (CBAM) identifies eight distinct levels of the change process. School leaders can use these levels to determine the extent to which teachers and schools are implementing the SFA program. Educators who can accurately assess where individuals or schools are in relation to these levels can provide the support necessary to encourage schools to progress to the next levels. The chart below outlines how the Levels of Use might be described in a Success for All school

The Levels of Use framework was introduced in 1999. Between 2000 and 2004, it was increasingly incorporated into material resources and professional development opportunities that were used to support interpretation and implementation of Success for All as enabling (and not coercive). By 2008, SFAF executives reported anecdotally that formalization of the developmental sequence interacted with other conditions to support an estimated 75% of schools progressing to some level of adaptive use. These other conditions included formalization of open routines and dynamic capabilities, improved professional development for trainers, and an increase in average time-in-program to over 8 year.²⁸ With that, executives reported that more schools were able to participate more fully as collaborators in continuous program improvement.

Dynamic capabilities in the hub organization

Consistent with the knowledge-based logic, all of the preceding depended heavily on the dynamic capabilities of SFAF, both for continuous program improvement and for strategic management. In our analysis, SFAF’s dynamic capabilities were anchored in five activities: active support in schools; information generation and processing; rapid prototyping; rapid, “small scope” knowledge transfer; and constant, formal evaluation (Peurach 2011). These activities were enacted constantly, simultaneously, and in interaction over the entire period of our study, and at such a rapid rate the developers often produced program improvements faster than they could be incorporated effectively into schools.

The general pattern of activity corresponded closely to the knowledge evolution cycle within the knowledge-based logic. Members described trainers’ support of implementation as on-going, qualitative research that constantly generated information about implementation, outcomes, and local environments. Concurrently, developers and executives discerned needs and opportunities for improvement by continuously pooling and analyzing copious flows of information from trainers, schools, and broader academic, professional, and policy environments. Leveraging the on-going, network-wide experimentation and problem-solving described above, relationships between experienced trainers and schools created opportunities to immediately pilot newly-formalized program improvements. With proof-of-concept, improvements were disseminated rapidly via publications, web-based systems, and standing professional development opportunities, and enacted in accordance with the developmental sequence. Finally, the effectiveness of program improvements was evaluated continuously both via formal research and via the re-analysis of state accountability assessments.

The preceding activities also supported the executive team in strategically managing the Success for All enterprise to adapt to rapidly-changing environments (Glazer and Peurach, in press; Peurach 2011). Leveraging capabilities for quickly generating and processing large volumes of information, executives constantly developed interpretations of how federal policy was effecting changes in the decision making of schools, districts, states, and funding agencies. Based on their analysis, executives made decisions regarding a range of interdependent issues, including the targeted clients, program offerings, organizational design and resource allocation, public branding and identification, lobbying and advocacy, and funding. They then adapted the agenda for continuous improvement to align with those decisions. Such work has kept the Success for all enterprise viable for over 20 years, beginning with the initial decision to establish a comprehensive school reform program to recent success pursuing federal i3 funding.

As above, these dynamic capabilities emerged and evolved over a 20 year period (Peurach 2011). They had roots in Success for All’s founding in a preeminent research university, and in the prior development, training, and research experiences of its co-founders. They were supported by practices and norms that developed early in the evolution of the organization, including the participation of executives in all dimensions of the work and the establishment of an informal, collaborative, “can do” culture described by staffers as “hard-wired” into the organization. And they expanded over time with key decisions about the structure and function of SFAF: for example, SFAF’s participation in federally-funded research centers; the incorporation of experienced and professionally-networked staff members; the incorporation of experienced Success for All teachers and leaders as trainers, developers, and managers; and the expansion of the executive team to include expertise in business management, policy analysis, research and development, publishing, and school and district leadership.

Discussion: Value in replication

Thus, in contrast to widely-held interpretations of Success for All as a quick fix or a top-down, one-size-fits-all intervention, our knowledge-based interpretation represents Success for All as a collaborative, long-term enterprise that yields evolving and expanding knowledge of interdependent, enterprise-wide “best practices,” along with knowledge of where and how to replicate them. Our interpretation is grounded in a longitudinal case study suggesting that the program incorporates structures, practices, and understandings characteristic of the knowledge-based logic. Our analysis extends and enriches a decades’ long line of research that frames the success of externally-sponsored school improvement as dependent on reciprocal, mutually-adaptive, learning-focused relationships among external providers and schools.

Again, our argument is not that all who have encountered Success for All have experienced it as interpreted here. As reported above, contrary interpretations of Success for All as bureaucratic or technocratic were instrumental in motivating efforts to formalize support for adaptive, locally-responsive use. It may well be the case that some teachers and school leaders still experience Success for All as bureaucratic or technocratic. Moreover, even those who recognize openness to adaptation may not fully understand how to work adaptively within the program.

Nor is our argument that all educational replication enterprises function in ways consistent with the knowledge-based logic. In fact, SFAF may be an exceptional case. Even if they intend to operate as knowledge-producing networks, other hub organizations may not have the same dynamic capabilities as SFAF. With organizational roots in a leading research university, SFAF appears unusually well-positioned to assume responsibility as the coordinative hub of a knowledge-producing replication enterprise.

Rather, our argument is that a knowledge-based interpretation of Success for All provides insight into the potential for educational replication to function as a collaborative process of developing, reproducing, using, refining, and retaining knowledge. That, in turn, suggests the possibility of educational replication yielding value that lies beyond the field of view of both proponents and critics: specifically, value as manifest in practical knowledge of large-scale school improvement. Further, our analysis suggests that, if individual replication initiatives were to collapse, much (if not most) of that value would be lost. Indeed, our analysis does not suggest that newly-created knowledge resides in the individuals involved in specific replications initiatives, such that it can be remobilized quickly as those individuals move to new contexts. Rather, our analysis suggests that this value resides in the enterprise, itself: the hub, the schools, and their members; their formal routines, tools, and artifacts; their collective dynamic capabilities; and tacit understandings and ways of working retained and reproduced in communities of practice.

As with earlier analysis of the rapid decline in policy support for comprehensive school reform, such findings should give pause to individuals and organizations vested in funding, evaluating, and enacting organizational replication initiatives in education. While consistent with a decades’ long line of research on the challenges and apparent realities of large-scale, externally-supported school improvement, insights from this research have been slow to penetrate analyses of implementation and effectiveness, analyses of returns on investment, and decisions about continuing and/or future investment. The rhetoric of replication has not become a rhetoric of uncertainty, inefficiency, exploration, and experiential learning. Rather, the rhetoric of replication remains a rhetoric of rationality: of effectiveness, efficiency, and economies of scale, and of research-based and research-proven programs.

Consider the federal Investing in Innovation program (U.S. Department of Education 2010c). The i3 initiative establishes high expectations for success, including improving achievement, closing achievement gaps, reducing dropouts, increasing graduation, and increasing college enrollment and completion. The structure of the i3 initiative again implies that the means to these ends is a sequential, stage-wise innovation process, with grants structured to support (in order) development, validation, and scale-up. At each stage, programs are to be based on methods shown by prior research to be effective, and they are required to include rigorous evaluations of implementation and outcomes. And the i3 initiative again establishes the expectation that measurable progress will be made quickly, in three to 5 years.

Yet, 3–5 years hence, our analysis predicts generally weak evaluations of implementation and effectiveness among all but the four largest “scale up” awardees, though with instances and examples that show success and promise. Intersecting this prediction with the notoriously short issue-attention cycle of education reform, we foresee critics ready to use generally-weak-but-variable evidence of effectiveness to claim weak returns on investment and, thus, to advocate moving in new directions. This, despite hundreds of millions of dollars in sunk costs, and despite research suggesting the possibility of knowledge (and, thus, value) having been created and retained in the i3-funding initiatives.

To be clear, our concern is not with rigorous evaluation or accountability for due diligence. We cannot imagine allocating hundreds of millions of dollars absent such things. Rather, our concern is that the knowledge-based logic of replication suggests an additional set of questions that could reveal value that would otherwise go unrecognized. New evidence of value, in turn, should be considered in discriminating among replication initiatives, in examining evidence of implementation and effectiveness, in reflecting on the wisdom of earlier investments, and in weighing options for continued investment.

Thus, the aim of these new questions is not to assess whether a given replication enterprise is or isn’t a knowledge-based replication enterprise, nor is the aim to assess whether a specific replication initiative works or doesn’t work to improve student achievement. Rather, the aim of these new questions is to assess whether a specific replication initiative is working to produce practical knowledge of large-scale school improvement. Such questions include:

• Is there evidence that the program features a design for practice, complemented by designs for organizational infrastructure?

• Is there evidence that the design for practice has evolved over time through collaborative learning among the hub and schools that features both exploration and exploitation?

• Is there evidence that knowledge of practice is being retained by formally codifying it, especially in the form of routines?

• Is there evidence that implementation is structured to support a developmental progression from fidelity to adaptation, including means of managing interpretation of the developmental sequence as enabling (and not coercive)?

• Is there evidence of dynamic capabilities in the hub organization that support both continuous improvement and strategic management?

Conclusion

The time is right to reconsider organizational replication as a strategy for large scale school improvement. Drawing on billions of dollars in public and private funding, many replication enterprises are underway, at different stages of development, and at different scales of operation. These initiatives are sure to come under much scrutiny in the near term: for example, in the evaluations of the federal i3 initiative and the Charter School Grant Program competition; among the philanthropists and investors funding charter management organizations and education management organizations; and, possibly, in the context of the reauthorization of the No Child Left Behind Act of 2001.

The difference between positive and negative returns on formidable public and private investments—and, importantly, the difference between positive and negative rewards for students—may well rest on proponents and critics stepping back from common-but-questionable assumptions to reconsider these initiatives from the perspective of the knowledge-based logic of replication, with a particular focus on better understanding their potential to function as mechanisms for producing, using, improving, and retaining practical knowledge. Identifying value generated through such work will be no small task, with our proposed questions one step in that direction. At the same time, not seeking to identify this potential value risks repeating a cycle of enthusiasm-and-abandonment that will likely squander it.

Appendix: Study design and research methods

Research context

This analysis of Success for All was conducted in the context of the Study of Instructional Improvement (SII), a longitudinal, mixed methods study of three leading comprehensive school reform programs: Success for All, America’s Choice, and the Accelerated Schools Project. SII was composed of three complementary components: a survey component focused on measuring changes in practice and achievement in 115 schools; one case study component focused on evaluating program implementation in 12 schools; and a second case study component (within which this study of Success for All was conducted) focused on understanding the three programs, their sponsoring organizations, and their evolution over time.

Findings from the survey component of SII suggest positive program effects on leadership practice, instructional practice, and student achievement in those curricular areas for which Success for All provided both: (a) extensive, detailed guidance for practice; and (b) extensive, practice-based learning opportunities for teachers and leaders (Camburn et al. 2003; Correnti 2007; Correnti and Rowan 2007; and Rowan et al. 2009a, b; Rowan and Miller 2007).

The second case study component was designed to provide a still-richer account of how the three programs supported coordinated improvement in practice and achievement. To do so, we work inductively, engaging in iterative, interdependent data collection and analysis to construct accounts of the three replication enterprises from the time of their founding through the period of our study.

Data collection

Data collection for this study of Success for All ran from 1996 to 2010. Over this period, we compiled a library of program materials, internal memos, reports, and research on Success for All with publication dates from 1980 to 2010. We generated field notes from participant-observation in 34 one-to-six day Success for All training events over 80 contact days, which included opportunities to observe implementation in ten schools. We conducted 77 semi-structured interviews with 22 SFAF staff members representing a range of responsibilities, professional experience, and tenure in the organization. And we participated in thousands of informal conversations with SFAF and school staff members, primarily in training events but also via email and phone.

We produced structured notes and reflective memos for each training event and interview. We recorded and transcribed 34 interviews. We did not record and transcribe the remaining interviews either at the request of participants or due to the sensitivity of topics under discussion. We maintained notes from informal conversations and pursued interesting points later through continued document analysis, participant-observation, and interviews.

Analysis

Data collection and analysis ran concurrently over the entire period of this study, in interaction with complementary analyses of America’s Choice and the Accelerated Schools Project. Analysis centered on iterative memo writing in which we sorted and re-sorted our data into emerging and evolving sets of categories. This work yielded three conceptual frameworks: a descriptive framework that captures key issues of design and organization across programs; a functional framework that captures five categories of work enacted by SFAF over the course of our study; and an analytic framework that examines the successes and challenges of Success for All as arising from interactions among the program, the installed base of schools, SFAF as the hub organization, and broader environments.

Our research procedures incorporated multiple means for validating our findings, including: extended observation of the enterprise under study; opportunities to investigate “negative cases;” triangulating among categories of evidence; triangulating among the three components of SII; and on-going member checking. Our efforts to establish validity went further: for example, interviewing critics of Success for All to understand and incorporate their perspectives; presenting preliminary findings at conferences; and constantly reviewing the literature to deepen our understandings of key phenomena under investigation.

This analysis

The analysis reported here emerged from our validation efforts: specifically, from on-going efforts to reconcile our work on Success for All with the broader literature on organizational studies, including seminal research on the innovation process and on organizational replication. Our reading and synthesis of research on innovation and replication informed what we were learning about Success for All, and our analysis of Success for All informed by what we were learning from broader research on innovation and replication.

We reported the initial products of these efforts in two conference papers (Peurach 2007; Peurach and Glazer 2010). The analysis reported here leverages subsequent comments, reflection, and continued research to refine our earlier analyses. For the full analysis of Success for All, see Peurach (2011). For the full analysis comparing Success for All, America’s Choice, and the Accelerated Schools Project, see Cohen et al. (in press).