Keywords

Education Reform legislation, passed into law in 2001 (United States Department of Education, 2001), resulted in the introduction of many new approaches to schooling that impacted core functions such as classroom instruction and school leadership. Most of these approaches are not only “new and different,” but, critically, are evidence-based – that is, they are grounded in rigorous research and evaluation that empirically demonstrate their effectiveness. While school leadership and classroom instruction have received the lion’s share of attention from educational reformers and policymakers, they are not the only critical functions of schools. Over the years – recognizing that learning involves more than “the mind” – all schools have generated approaches to supporting and developing the non-academic dimensions of students – their health, mental wellness, safety, peer relationships, family interactions, etc. This function of schools is generally known as “student support” but can be variously labeled by school districts using terms such as “pupil personnel services” or “school guidance counseling”. The purpose of this chapter is to describe the development and impact of a school redesign effort focused specifically on student support.

Recognizing the impact of non-academic aspects of students’ lives on achievement and eventual life chances, schools have provided some type of non-academic support to students (e.g., school health curricula, guidance and counseling services, free and reduced lunch programs, etc.) since the early twentieth century. The student support function in schools typically involves a set of specific personnel (e.g., school counselors, social workers, adjustment counselors, health and wellness staff, etc.) and a wide range of activities (e.g., individual and group counseling, academic support, college and career planning, etc.). However, these efforts have operated at the margins of schooling and in somewhat of an “ad hoc” manner. In contrast to the core functions of leadership and classroom instruction, student support has been seriously neglected in most Education Reform efforts, particularly in the No Child Left Behind legislation. The few references to student support in the field of Education Reform encourage schools to offer “wrap-around services” or to address the needs of “the whole child” – with little focus on specific strategies and evidence-based outcomes.

The absence of a comprehensive and coordinated focus on the out-of-school needs of students is particularly notable in light of the stubbornness of the academic achievement gap for low-income children. The achievement gap between students whose families are economically advantaged and children who live in poverty is wide and deep. Many researchers recognize poverty as a major contributor to the achievement gap (Duncan & Murnane, 2011). However, policymakers have considered the recognition of the impact of poverty on learning as an attempt by leaders and service providers to “make excuses” for the underachievement of students in lower socioeconomic circumstances. Now, after over a decade of intense and pervasive Education Reform efforts to close the achievement gap for low income students, it is finally agreed that schools cannot do so without a systemic approach to addressing out-of-school challenges that are known to negatively impact learning (Becket & Luthar, 2002; Bryk, Sebring, Allensworth, Luppescu, & Easton, 2010; Walsh & Murphy, 2003).

Despite educational policymakers’ lack of attention to the importance of student support, teachers have long recognized the need for more non-academic services for students – especially for those living in poverty. The non-academic needs of students have also been evident to the local human service providers who encounter these students and families in their neighborhoods and communities. However, the recognition of out-of-school challenges leads obviously to the task of determining what to do about it.

Regardless of the best efforts of school-based student support staff, schools simply do not have the capacity to provide all of the services and enrichment opportunities for student thriving. In an attempt to address these needs and promote healthy development, schools at the local level have begun partnering with community agencies and institutions. Most schools, particularly in urban areas, now have an array of community partners who deliver a specific service or a set of services to schoolchildren (e.g., health services, mental health services, violence prevention curricula, after school programs, parent groups, etc.). However, “more” in this case is not necessarily “better.” As the number of community supports available to student have begun to increase, schools are challenged in three major ways: (1) identifying which services and supports are appropriate for individual students, (2) managing partnerships and aligning them in a meaningful way with the work of the school, and (3) measuring the impact of these supports on outcomes such as student achievement and thriving.

As in the realm of curriculum and instruction, student support must be customized to meet the needs of individual students. One size fits one, not all. In the domain of management, schools are often “over-run” with well-intended community partnerships without the structure and processes required to enable the partners to be effective. Surprisingly, there have been few attempts to develop a systemic set of processes in the school to facilitate and support these partnerships. Despite the advocacy of organizations such as the Center for Mental Health in Schools (Adelman & Taylor, 2010), schools nationwide have not engaged in a redesign of the delivery system for student support. In terms of outcomes, there have been very few attempts to measure the impact of student support. While there is considerable evidence of single interventions that focus on one need (e.g., anti-bullying programs, nutrition education, family engagement), there is sparse evidence of the effectiveness of comprehensive approaches that attempt to address the full range of a student’s needs and strengths.

This chapter reports on a new and systematic design for the delivery of student support in schools. The new design has been developed over two decades by a school-community-university partnership in Boston. Known as City Connects, its goal is to have students engage and learn in school by connecting each child with the tailored set of prevention, intervention, and enrichment services he or she needs to thrive. This goal is accomplished by leveraging the resources of a city’s community agencies (City Connects, 2014a). The chapter will describe the development, implementation, evaluation, and future directions of this evidence-based approach to student support.

The Context

Characteristics of the particular context in which City Connects was designed and implemented (the city of Boston and its public schools) are important to understanding the City Connects intervention and outcome evaluation. As is the case in most urban communities, many Boston residents experience social and economic disadvantage, with schoolchildren and their families even more disadvantaged than the population as a whole. Over the many years since City Connects was launched in 2001, the problem of poverty has not lessened. For example, the most recent United States census reported that the poverty rate in Boston was 17 % overall, but 22 % for Boston residents with children under age 18 (United States Census Bureau, 2010). The most recent United States census also revealed that 15 % of Boston residents received food stamps and Supplemental Nutrition Assistance Program (SNAP) benefits, while 76 % of Boston Public Schools students qualified for free or reduced lunch with family incomes at or below 185 % of the poverty level (United States Census Bureau, 2010). Poverty is also evident in growing rates of family homelessness. The Boston Homeless Census reveals that the number of men, women, and children living in emergency shelters or transitional housing increased from 6,992 to 7,255 from 2012 to 2013, a 3.8 % increase; of these individuals, 33 % were children – a 4.3 % increase from 2012 (Boston Public Health Commission, 2013). Further, the number of homeless families in Boston increased from 1,166 to 1,234 the same year, a 5.8 % increase (Boston Public Health Commission, 2013).

Similar to other urban areas in this country, the overlap between poverty and communities of color is substantial. Based on data from the most recent United States census, about 60 % of all Boston residents were White, while 13 % of schoolchildren and families were White; 26 % of Boston residents were African American, while 37 % of the schoolchildren were African American; 16 % of Boston residents were Hispanic /Latino (of any race), compared to 40 % of the schoolchildren (United States Census Bureau, 2010). At this time, about 25 % of the city’s population was foreign born and 34 % spoke a language other than English at home; further, in the Boston schools, English was not the first language for nearly 40 % of students in 2009, and 20 % of school children were classified as limited English proficiency (United States Census Bureau, 2010).

The numbers are similar – and in many cases more dire – in other large American cities. It is no secret that children living in poverty are, on average, less successful in school (Weiss, 2013). Many researchers have begun to identify some of the factors that account for the deleterious effects of poverty on academic achievement. Poverty impacts children’s achievement and growth in at least three noteworthy ways: (1) poverty limits investment – a family’s ability to invest money, time, and energy in fostering children’s growth (e.g., less time to read and talk with their children); (2) poverty can create pervasive stress within families and their neighborhoods, sometimes undermining children’s sense of well-being and safety (e.g., stress may contribute to inconsistent parenting behavior or increased exposure to community violence, ultimately impacting children’s self-regulation, social-emotional stability, and classroom behavior); (3) poverty may contribute to chaotic lifestyles and unpredictable support systems (e.g., less-reliable transportation, municipal services, and businesses) (Brooks-Gunn & Duncan, 1997; Evans, 2004).

The sequelae of poverty, in turn, lead to poor attendance, high mobility, social-emotional dysfunction, lack of readiness for school, and limited cultural capital to understand schools as institutions (Dearing, 2008). Many children also suffer from a lack of exposure to enrichment opportunities. Rothstein (2010) describes the impact on achievement of out-of-school factors relative to in-school factors in the following way: “Decades of social science research have demonstrated that differences in the quality of schools can explain about one-third of the variation in student achievement. But the other two-thirds is attributable to non-school factors” (p. 1). Therefore, academic success is predicated on children’s readiness to engage and thrive in school, with an overlapping impact of the various domains of development on children’s readiness to learn and thrive.

The impact of the poverty experienced by schoolchildren and their families makes it imperative to address out-of-school factors in any educational reform effort (Berliner, 2009; Rothstein, 2010; Walsh & Murphy, 2003). Supporting the whole child and addressing out-of-school needs – albeit in a limited way – is not entirely new to schools. While the proponents of education reform have a laser-like focus on teaching and learning, with only a nod to the impact of student support, schools have been involved in directly addressing the out-of-school needs of children since the late nineteenth century (Walsh & Murphy, 2003). Professions have shaped their preparation programs and intervention strategies so that they are able to contribute to schools’ efforts to address non-academic barriers to learning (City Connects, 2010). These professions include school counseling, school social work, school psychology, school nursing, and school adjustment counseling. However, the work of these student support professionals is typically marginalized in schools, particularly in recent years as educational reforms have narrowed in scope and focus.

The roles of the various student support professionals are typically defined in broad terms by their specific professional organizations. However, their respective and different practices have not been tightly codified. In many schools, the loose definitions of the work of student support professionals create a special paradox. At the same time that “the practice” of teaching has become more circumscribed, focused, and evidence-based, student support professionals’ practices continue to vary from school to school and district to district. Developing and implementing the delivery of evidence-based student support in schools is long overdue. Schools would benefit substantially from a new design for student support that reinvigorates current processes and structures, and results in a defined and evidence-based practice for student support professionals.

In the face of the stubborn achievement gap, some educational policymakers have recently begun to recognize the potential contributions of student support to narrowing achievement differences. After years of focusing on teaching and learning, they are coming to realize that student support may be another critical lever in promoting school change and student achievement, and they are beginning to examine strategies to address the out-of-school factors impacting learning. This shift in the perceived importance of student support contributes to the positive zeitgeist for the redesign of student support in schools. In this context, the process of designing and implementing City Connects, a practice that would result in the effective delivery of student support in an effort to minimize the achievement gap, became possible; this ultimately led a number of schools to transform how they approach student support.

Getting Started

The design of City Connects was carried out by a school-community-university partnership over a 2-year planning period. Implementation and evaluation have occurred over the past 15 years. During the design phase, representatives of the school-community-university partnership deeply engaged other university faculty, local school administrators, teachers and school staff, neighborhood citizens, family members, and community agency staff. The goal of this dialogue was to modify existing student support structures and processes within a geographic group of Boston Public Schools. Early on, it was agreed that the design should involve a systemic collaboration across schools, families, and community agencies. Neither schools nor communities nor families could be the single agent responsible for supporting children. Schools were not in a position from the perspective of their purpose or their budgets to provide all of the supports that children needed. While community agencies could deliver many services, they existed as independent entities and could not provide an integrated structure or system to deliver services to each child and family. Families were limited not only financially, but particularly by a lack of knowledge about and access to available supports. It was important to the team that the new design build upon and transform already-existing school structures and functions. The group saw the potential for eventual success by relying on “evolution” rather than “revolution.”

The design team’s first task was to look for other models of schooling that addressed the out-of-school needs of students. Community Schools offered one of the only models at that time. Developed in the 1990s, the Community Schools model recognized the critical role of health and social services in promoting children’s development, and viewed schools as a vehicle for service-delivery. Their strategy co-located child and family services in the school, especially after-school programs, health initiatives, and early childhood programs. The Community Schools approach represented an early and transformative effort to bring student/family services into the school and to link children with supports (Walsh et al., 2000).

The Community Schools evaluation data available at the time focused on only those students who participated in the school-based after-school programs; the other students in the school were not directly impacted by the intervention. Further, while the results were promising, they were based on average scores for a group of students in contrast to a change in scores for individual students. It was apparent from the literature review that evaluating these types of interventions is challenging and that few rigorous evaluations had – and to this day have – been published in peer-reviewed journals.

Given its central locus in the after-school program of the school, its goal of bringing service organizations into the school, and providing services to children and families who availed themselves, the community-school model was not positioned to systematically reach every child and teacher in the school. Its mission was to promote collaboration between schools and community agencies; it was not intended to transform the internal student support structure and processes of schools. Building on the vision of Community Schools, the City Connects design team recognized a complimentary but distinct set of goals – to reach every children in the school, work with every teacher, and measure impact for individual children on a longitudinal basis.

After looking at several pre-existing models, the design team laid out – albeit in a rudimentary way – two essential components of any intervention – a conceptual framework and a set of best practices. These were critical to informing the shape of the intervention. The result of their efforts was a design for City Connects. Over the nearly 15 years of implementing City Connects, the conceptual framework and the best practices have been deepened and amplified, resulting in a codified intervention or practice for student support staff. After outlining the current conceptual framework, we will review principles of best practice, and describe the City Connects intervention and its measurable impact on student achievement and thriving.

Conceptual Framework

The theory and research of developmental psychology provide the conceptual grounding for the City Connects intervention. Contemporary understandings from the field of human development suggest that a child’s development: (1) occurs in and is impacted by a variety of contexts, including school, neighborhood, and family; (2) is characterized by plasticity, because early development impacts but does not totally dictate later development – in other words, change is possible; (3) incorporates the continuous interaction of risk and protective factors, so that the presence of risk can be “balanced” by protective factors, allowing for positive growth; and (4) occurs simultaneously at multiple levels – biological, psychological, and social – with each level impacting every other level so that intervening in development must be done in a comprehensive way and not isolate a single domain (e.g. mental health) (Cicchetti & Sroufe, 2000; Sroufe, 2013; Walsh & Galassi, 2002).

This conceptual framework suggests an intervention that should be directed at mitigating risk factors and enhancing protective factors for all students. Therefore, modifying the number and types of risk and protective factors is the theoretical goal of the intervention. Research helps us to understand the particular factors that lead to positive outcomes in spite of adversity (that is, resilience) as well as what can be done to support youth. This framework constitutes the major theoretical reason why the design team made the bold assumption that our intervention could alter the course of children’s development. The conceptual approach also highlights the importance of tailoring the intervention to the individual needs and strengths of the child, because the course of development for each child differs. As Cicchetti and Sroufe (2000) assert, “the same risk factors may be associated with different outcomes (i.e. multi-finality) and subgroups of individuals manifesting similar problems arrived at them from different beginnings (i.e. equi-finality)” (p. 257). The intervention strategy for promoting positive development was to reduce or mitigate the risk factors and to increase or enhance the protective factors. As a result, the intervention not only needed to be tailored, but also needed to give as much attention to children’s strengths as it did to their needs. Finally, the conceptual framework pointed toward a comprehensive approach that addressed all of the domains of child’s development: academic, social-emotional, health, and family.

Best Practices in Student Support

The conceptual framework guides City Connects’ research and the articulation of best practices; these, in turn, lead to the development of an intervention or practice. Translating theory and research into the “world of action” is a long road that requires continual feedback from practitioners and from evaluation data. In the case of our student support intervention, some of these principles emerged from sources that represent a distillation of: (1) the recommendations of the Center for Disease Control (Marx, Wooley, & Northrop, 1998), (2) the Center for Mental Health in Schools at UCLA (Adelman & Taylor, 1993); and (3) the Education Trust (2000). Thought leaders such as Joy Dryfoos and the Children’s Aid Society (Dryfoos, 1990) and the Center for Child, Family, and Community Partnerships at Boston College also identified best practices, as did practitioners with substantial experience in the field of student support with whom the team consulted (Walsh et al., 2000). Taken together, these best practices, which flow from the conceptual framework, have universal application in new designs for student support services. We will now identify and describe these best practices.

Student Support Should be Systemic and Coordinated

In providing supports to children, it is incumbent upon the intervention to make certain that no child falls through the cracks, and to do so in a way in which the left hand knows what the right hand is doing. One of the more prominent examples of efforts to bring a systemic approach to the work of student support professionals can be found in the National Model of School Counseling, developed by the American School Counselor Association (ASCA) (American School Counselor Association, 2012). The National Model of School Counseling outlines a framework for developing a systemic practice. ASCA leaders created this framework in response to the critique that school counselors were not typically addressing the needs of all students in a school (The Education Trust, 2009). For example, at the elementary school level, many student support professionals were spending most of their time with a small number of seriously challenged students who often presented with behavior problems. Secondary counselors, on the other hand, often focused on helping high-performing students gain access to college. In either situation, a relatively small number of students were supported, while the large percentage of students was not. A systemic approach ultimately addresses this inequity and makes supports available to all students.

Student Support Should Focus on Strengths as well as Needs

Wise teachers have recognized for years that building on students’ strengths is as important as addressing their needs. Supporting strengths and interests can be transformative in children’s development. Research on children’s competence confirms teachers’ instincts and provides an impetus for all educators and human service providers to move away from an exclusive focus on remediating deficits and balance it with an intentional concern for enhancing strengths (Masten & Tellegen, 2012). Finding and enhancing children’s strengths will lead to resilience when they are faced with adverse situations and relationships.

Recently, some members of the field have begun to advocate for developing students with “grit,” which is defined by Duckworth, Peterson, Matthews, and Kelly (2007) as perseverance and passion for long-term goals (p. 1087). Duckworth and colleagues (2007) explain that grit entails “working strenuously toward challenges, maintaining effort and interest over years despite failure, adversity, and plateaus in progress” (p. 1088). The personality trait of grit has also been shown to predict achievement in academic, vocational, and avocational domains (Von Culin, Tsukayama, & Duckworth, 2014, p. 306). As we attempt to diminish the achievement gap and promote students’ ability to succeed in the face of adversity, it is imperative to value resilience and grit as part of a student support intervention.

Student Support Should be Customized for Each Student

It has become clear that if academic achievement levels are to increase, particularly for children who are poor and of color, student support needs to be tailored to the particular academic needs of each student. Educators have come to understand the importance of finely grained assessments of each student’s academic progress in order to provide appropriate supports. For teachers, individual academic plans have become the norm, and systems such as “response-to-intervention” are operationalizing individualized instruction. However, despite their recognition of the importance of individualized support for students’ academic needs, educational leaders have not come to grips with the need for individualized support to address students’ non-academic needs. In traditional approaches to student support, only students in crisis or serious need received individualized student support plans. The vast majority of students – that is students who were showing early signs of difficulty and students who appeared to have few academic challenges – typically had little or no interaction with student support staff. Tailored plans for every child make this less likely.

Student Support Should be Comprehensive in Addressing the Full Range of Student Needs and Provide a Continuum of Services

Because children function as integrated human beings, it is impossible to intervene in one aspect of development without impacting other aspects. Conversely, interventions that are developed for all students need to consider the child in a holistic way and not arbitrarily split the child into parts, e.g. social/emotional versus health versus family versus academic. Primary care physicians deal with multiple aspects of physical health in the same way schools can provide “primary care” for students by addressing multiple aspects of development simultaneously. This holistic approach can result eventually in more specialized care, but it does not start there. Complementing a holistic assessment of a child is the delivery of a comprehensive continuum of services ranging from enrichment to early intervention to intensive intervention. At different points in their development, children can benefit from supports at each of these levels.

Student Support Should be Culturally Sensitive

Urban schools today are characterized by significant diversity. A large number of students are English Language Learners. In many schools, it is common for 80–100 languages to be represented among the families of the students. Schools also are racially mixed, with each group having its own traditions and culture, and its own approach to child rearing, learning, and behavior. School-based interventions must recognize these differences, train their staff intensively on cultural competence, and – insofar as possible – locate community services that are aligned with the particular racial, cultural, and language identities of the families served.

Student Support Should be Evidence-Based and Continuously Monitored for Effectiveness through Collecting and Analyzing Data to Evaluate and Improve Service Delivery and Student Outcomes

Largely as a result of Education Reform initiatives, evidenced-based approaches in education have become the “coin of the realm.” If teachers are asked to adopt a new practice, they should be assured that the practice is effective, not simply because someone says that it is, but because research has demonstrated it. Consistent with this call for evidence, the leading professional organizations in student support have also advocated strongly for a focus on outcomes (American School Counselor Association, 2012; National Association of School Psychologists, 2010). Despite the profusion of evidence-based curricula in literacy, math, science etc., it is ironic that student support has few measures of effectiveness. Many individual interventions (e.g., violence prevention curricula) utilized by student support professionals are indeed research-based, but the comprehensive practice of holistic student support still depends on individual testimony and long-held beliefs about efficacy, and data that is at best descriptive. Without measures of effectiveness, school counselors and other student support staff become dispensable when budgets are cut. Thus, in addition to directly benefitting students, supporting evidence-based practice reflects counselors’ enlightened self-interest.

Student Support Should be Cost-Effective to Schools by Leveraging the Resources Provided by Community Agencies

The City Connects planning team learned that it is critical to build on existing school structures and processes wherever possible, and adapt them as necessary and appropriate. The City Connects design recognizes that schools already have structures and processes in place to address student needs. The work of student support staff members is clearly important and helps many students; however, many approaches focus primarily on at-risk children without utilizing a systematic practice that measures effectiveness. The design introduced by City Connects aims to modify and enhance the student support structures and processes already existing in a school.

Student Support Should be Implemented Across Schools with Fidelity and Oversight

In order to replicate and scale an intervention, one must demonstrate that the intervention is being implemented as intended. Otherwise, the evidence base, which confirmed the success of the intervention, does not have meaning. While some drift will always occur in an intervention, it must be closely monitored by measuring the fidelity of implementation, and corrections should be made on a regular basis.

Student Support Should Require Direct Teacher Engagement in Student Support Interventions

Because it involves a core function of schools, City Connects is designed to impact and engage the heart of the school – that is, the teachers. Direct teacher engagement in student support proves to be significant and innovative. The typical approach to student support has often had minimal direct engagement with teachers, resulting in two distinct silos of work in the school: one related to classroom instruction and the second related to addressing so-called “non-academic issues.” The goal of the intervention is to directly engage teachers without adding to their burdens. A student support intervention must be efficient, and contribute in a positive way to making teachers’ work easier and more effective.

Description of the City Connects Intervention

Based on this conceptual framework and best practices, the City Connects intervention provides an organized system for coordinating student support in schools. It redesigns and revitalizes traditional approaches to student support by strengthening the involvement of the classroom teacher and leveraging resources in the community. The intervention also provides a clear student support practice in which any school-based student support professional can engage after appropriate training.

At the core of the City Connects intervention is a full-time School Site Coordinator. A coordinator in each school, typically trained as a school counselor or school social worker, connects students to a customized set of services through collaboration with families, teachers, school staff, and community agencies. The ratio of School Site Coordinators to student population is 1:400. The School Site Coordinator follows standardized practices codified in the City Connects Practice Manual. School Site Coordinators are supervised by a Program Manager, who is also trained by City Connects. Each Program Manager is responsible for up to ten schools.

In the fall of each year, the School Site Coordinator works with each classroom teacher to assess and develop a customized support plan for every student. Together, they identify the strengths and needs of each student across major developmental domains (academic, social-emotional, behavior, health, and family), and propose a tailored student support plan, which is discussed with the student’s family. They then connect each child and family to appropriate school- and/or community-based services and enrichments. Students identified as having intensive needs at any point during the school year receive an individual review, which is independent and distinct from a Special Education referral. In this more extensive review, a wider team of education, human services, and health professionals discuss and develop specific measurable goals and strategies for the student (City Connects, 2012).

A critical aspect of the role of the School Site Coordinator is developing and maintaining relationships with children and families throughout the course of the school year, as well as developing and maintaining partnerships with local community agencies and institutions. These partnerships collectively provide a range of prevention, early intervention, and enrichment services. Relationships are formalized through a City Connects Community Resource Advisory Board, comprised of selected citywide agency leaders, and a City Connects Resource Advisory Council, which includes selected agency representatives working at the local neighborhood level. In addition to developing individualized student support plans, School Site Coordinators themselves provide a range of services within the school and classrooms, including healthy life skills groups that address focused topics such as friendships and family relationships, bullying, and healthy eating.

School Site Coordinators document, track, and follow up on the delivery of the tailored set of services and enrichment opportunities, creating a systematic practice that leads to measurable student outcomes. To facilitate this process, and to permit streamlined tracking and follow-up, City Connects developed a proprietary Web-based database, the Student Support Information System (SSIS). The SSIS database allows for secure collection of data on student reviews, individual student plans, service referrals, and providers (both school-based and community agencies) who deliver services. The SSIS system also allows School Site Coordinators to run reports that provide them with critical information on electronic dashboards. This information is used to guide the School Site Coordinators’ practice and develop priorities.

Evaluation Designs and Challenges

The evaluation of City Connects is guided by a theory of change, which is grounded in research. A comprehensive student support intervention that addresses both students’ needs and strengths holistically in the context of urban poverty would – in theory – be expected to achieve positive outcomes in student academic achievement and thriving (Walsh et al., 2014).

As is typical of nearly every school intervention, academic achievement is defined as a major outcome. In addition, because the intervention was anticipated to impact the whole child, student thriving was identified as the second major student outcome. Each of these outcomes was assessed by a number of measures. Measures of academic achievement included report card scores and standardized test scores (e.g. SAT) and high-stakes standardized test scores (e.g. state-wide standards-based assessments). The measures of student thriving included classroom behavior, student work habits, and student effort/motivation to learn. Insofar as possible, the evaluation made use of existing student measures rather than adding the expense and the burden of new measures with students who were already perceived as over-tested. In addition to student outcomes, the evaluation focused on the impact of the intervention on critical stakeholders – teachers, school administrators, community agency partners, and families.

The evaluation was designed to include rigorous quantitative analysis, complemented by a number of qualitative approaches. The evaluation measured outcomes at the individual student level, as well as at the school level. Quantitative analyses have been done in the context of a quasi-experimental design. The analysis has employed a range of statistical methods with control and experimental groups using student-level propensity score matching on a number of characteristics. A systemic approach to data collection made effective use of technology. Implementation was started with a solid description of the intervention, a theory of change that would drive implementation, a plan for evaluating outcomes, and a method for collecting data.

The intervention’s Evaluation Team provides five major functions: (1) monitors data on implementation in order to provide ongoing feedback that would result in changes to the practice or changes in the process of gathering data, (2) provides end-of-year reports to school partners and monitors the implementation through a fidelity system, (3) manages large longitudinal databases and provides the analysis of effectiveness, both immediate and long-term, (4) secures consultants who are experts in various methodological arenas, and (5) seeks feedback from independent external evaluators. It is important to note that the Evaluation Team is distinct from the Implementation Team.

The Evaluation Team for this study is designed as a three-level structure to ensure utmost rigor and independence. First, the Core Evaluation Team includes analysts affiliated with the Center for the Study of Testing, Evaluation, and Educational Policy at Boston College. This team is responsible for data collection, data management, and analysis. Members include the Director of Evaluation, Associate Director of Evaluation, Manager for Data & Analysis, and Manager of Qualitative Research and Fidelity. Supporting staff members include one full-time Research Associate and several Graduate Students. Except for School Site Coordinator data entry into SSIS (the source for student service data but not outcomes data), no member of the Implementation Team has any role in evaluation.

A second layer is the Expert Review Team consisting of university faculty who specialize in associated disciplines including Educational Research, Developmental Psychology, Counseling Psychology, and Economics. This team convenes bi-weekly to review efforts of the core Evaluation Team and provide expert advice regarding study design and analyses. There are five current members of the Expert Review Team.

The final layer for the Evaluation Team is an entirely external Independent Evaluation Board (IEB) consisting of national experts in evaluation of social interventions, research methods, design and analysis of randomized controlled trials and school lottery data, and child development. The IEB receive evaluation results quarterly for comments and convene in person annually to review all evaluation findings. There are four current members of the IEB.

Research Findings and Future Priorities

A wide range of evidence and methods of analysis demonstrates that City Connects significantly impacts student achievement, including report card grades and standardized test scores. In every academic subject (reading, writing, and mathematics), at every grade in elementary school, City Connects students achieve significantly higher mean report card scores than comparison school students (City Connects, 2012). After students have left the City Connects intervention in Grade 5, they score significantly higher on the statewide high-stakes test (Massachusetts Comprehensive Assessment System) than their peers in comparison schools (City Connects, 2014a). Remarkably, these students achieve close to the statewide average for proficiency levels on both Literacy and Mathematics components of this standardized test, and the significant improvement persists into high school through grade 10 (City Connects, 2014b).

City Connects has a significant impact on student dropout rates throughout high school. The cumulative percentage of students who drop out across the 4 years of high school is substantially lower for students who attended an elementary school implementing City Connects than for those who never attended a City Connects school (City Connects, 2014a). Ultimately, this translates to approximately 50 % lower odds of dropping out in high school – an important outcome because high school graduation is widely argued to yield public economic benefits (City Connects). According to Levin and colleagues (2006), a conservative estimate of the benefit is $127,000 per graduate.

The evaluation also demonstrated that students who attend City Connects elementary schools are significantly less likely to be chronically absent or to be retained in grade than students who never attended City Connects schools. This pattern is present at every grade level. In summary, the City Connects evaluation has shown that optimized student support can be delivered in a high-impact, cost-effective way.

In addition to evaluating student outcomes, City Connects evaluations also solicit regular feedback from key stakeholders using electronic surveys. For example, in the 2012–2013 school year in Boston, 100 % of principals, 98 % of teachers, and 99 % of community partners indicated that they were satisfied with the City Connects intervention. In a context in which multiple interventions move in and out of schools, these are very high levels of satisfaction.

Beyond City Connects’ local benefits, its evaluations also are having a demonstrable national impact. For example, Child Trends, a nonpartisan, nonprofit research center that is “focused exclusively on improving the lives of children and youth by conducting rigorous research and sharing the resulting knowledge with key stakeholders,” recently evaluated nine interventions that involve school-community partnerships (Moore, 2014). As a group, these interventions work in the area of linking students and/or families to opportunities in the community that will improve educational outcomes.

Across these nine interventions, Child Trends found 11 studies that meet their standards for rigorous research. Notably, five of the eleven were City Connects studies (Moore, 2014). The overall conclusion of this report was that the interventions constitute “a promising approach for helping more disadvantaged children and youth improve in school and have a brighter path to life,” and that the salutary effects of interventions such as City Connects may be cumulative (Moore, 2014, p. 8).

Scale-up and Sustainability Planning

In recent years, the City Connects intervention has expanded to other school districts in Massachusetts, as well as districts in Ohio and New York. Based on literature on implementation science, the scale-up of any intervention requires documentation of the intervention, capacity to measure outcomes, staff training and professional development, measures of fidelity of implementation, and a plan for sustainability. City Connects has documented its intervention, established a professional development program, demonstrated its capacity to measure outcomes, and provided evidence of significant positive impacts.

Fidelity of implementation is critical to scale-up and sustainability. Expansion with integrity requires that the program be able to measure the degree to which the intervention is implemented in a way that is faithful to the practice as documented in the intervention’s Practice Manual. To respond to this requirement, City Connects developed a system of measuring fidelity of implementation. Expanding City Connects has provided the opportunity to ascertain the degree to which the model and the outcomes can be replicated in another geographic setting.

Throughout the first 2 years of implementation, indicators from the City Connects Fidelity Monitoring System revealed areas of high program fidelity; for example, strong implementation of preparatory steps for the process of reviewing each student with classroom teachers in order to collaboratively assess individual strengths and needs (City Connects, 2012). Information collected via the Fidelity Monitoring System also assisted the Evaluation Team by highlighting areas of potential improvement or need (e.g., the need to support teachers in filling out a required form during the first year of implementation) as the intervention is implemented in new school districts. The Fidelity Monitoring System has regularly informed the content of professional development.

City Connects also has a documented strategy for entering a new district. This process involves several steps, including introducing the program to stakeholders, recruiting School Site Coordinators to serve in schools, conducting a needs assessment of the schools and community and an environmental scan to identify local devices and supports, launching professional development to train new hires, initiating and establishing a plan for evaluation and reporting structures.

The literature on implementation science points out that rigorous evaluation of programs and interventions is critical with respect to sustainability (Halle, Metz, & Martinez-Beck, 2013). City Connects continues to prioritize quality evaluation. Further, according to Halle, Metz, and Martinez-Beck (2013), sustainability planning has been identified as a critical component of the implementation process that should be considered from the outset (p. 9). This suggests the importance of creating a systematized practice that would ultimately be worth sustaining and expanding.

With this in mind, the members of the partnership are also aiming to expand and scale-up the intervention, not only in different geographic locations but also with respect to grade levels. Though the initial focus was on elementary (K-8) school students, the practice has also been adapted and successfully evaluated with early childhood populations. A City Connects approach for secondary schools has been developed as well. This adaptation is currently being evaluated.

Achievements and Current Status

At this point in time, City Connects has designed, developed, and implemented a nationally-recognized evidence-based practice in student support. As an intervention that reorganizes the way student support is delivered in schools, City Connects is leading Education Reform efforts in an important new direction, through the use of a systematic strategy that connects each and every student to a tailored set of enrichment, early intervention, and intensive intervention services (City Connects, 2014a).

City Connects is currently active in 79 sites across 7 districts in 3 states (City Connects, 2014a). The partnership has defined success in terms of a series of outcome measures, including the positive impact of the intervention on students, principals, teachers, and community partners (City Connects, 2014a). The City Connects team has demonstrated that the intervention can be easily replicated in school districts, that it is cost effective, and that it can teach the practice to either new or existing student support members.

The key components of City Connects and programmatic responsibilities of the School Site Coordinators are codified in the Practice Manual and sustained through an ongoing professional development program. This program enables School Site Coordinators to learn and implement all of the critical aspects of the City Connects approach. All newly-hired School Site Coordinators are inducted into the role via a weeklong City Connects Training Institute. The Institute provides an introduction to the City Connects model and an opportunity to begin building a professional network. This professional development program continues bi-weekly throughout the school year, and is delivered at a district level by City Connects Program Managers.

The content for these professional development modules is developed continually and made available to Program Managers via an online information management system. Using this technology, professional development on a regular basis in order to promote collaboration, provide School Site Coordinators with peer support, and ensure fidelity of implementation. The goal of this continuous professional development is to support implementation of the core practice while addressing challenges and opportunities for individual schools and districts.

The City Connects Implementation Team is housed in the Center for Optimized Student Support at Boston College. The Center is responsible for expansion of City Connects to new sites and maintaining the practice of existing sites. The City Connects Implementation Team is responsible for carrying out the processes of entering schools and districts. They work with central administrators, principals, and teachers to explain the City Connects process, to support the hiring of appropriate personnel, and to guide and coach the implementation.

Policy Implications

When the partners from the university, schools, and community began this initiative in the mid-1990s, the national and state policies were focused almost exclusively on academic achievement, with little to no analysis on student support. Teachers were responsible for raising achievement and were told repeatedly that considering the impact of poverty on students and families was a “cop out.” More recently, at both federal and state levels, the language and the policy have begun to shift. Over the past several years, Congress appropriated funds for Promise Neighborhoods and other community schools, suggesting that congressional leadership has recognized their value. The allocated funds will help communities across the country establish and expand schools’ capacity to respond to the non-academic needs of students. The recent federal government requirement that School Improvement Grants (SIG) be given only to evidence-based programs will make high-quality evaluation essential as schools select student support programs as one lever for school improvement.

America’s governmental programs such as Race to the Top have provided opportunities for significant progress in school reform. Some states have built “wrap-around services” into their Race to the Top proposals for low-performing schools. Their efforts have provided small amounts of funding to focus on the out-of-school challenges students face. As Weiss (2013) asserts, school districts that are heavily serving low-income and minority students face some of the most severe challenges with respect to student achievement. Many have advocated for more government resources to address poverty-related impediments to learning, but the message from Washington has not always been consistent with funding decisions.

The 2014 Child Trends report on programs that address non-academic needs introduced several key findings that have direct relevance for policymakers: (a) interventions addressing out-of-school needs can contribute to student academic progress – i.e. decreases in grade retention and dropout, and increases in attendance and overall GPA; (b) it is important for intervention that address children’s out-of-school needs to be firmly grounded in the research on child and youth development and aligned with research on the varied factors that promote educational success; (c) preliminary studies demonstrate a positive return on investment for these types of interventions; and (d) higher quality is related to the effectiveness of programs (Moore, 2014). With respect to implications for policy, the Child Trends report asserts that programs with the aim of addressing children’s out-of-school needs demonstrate that “success in school (and in life) is more likely when young people’s well-being is met across multiple domains – in other words, when their health, safety, social/emotional, and cognitive needs are consistently met,” (Moore, 2014, p. 7). While education reform efforts have been largely focused on academic factors, an assessment of student strengths and needs beyond academics is necessary for policymakers to consider (Moore, 2014, p. 7).

Lessons Learned and Their Import for Others

Developing, implementing, and evaluating a school-based intervention has taught many lessons to the school-community-university partnership involved with City Connects. When a university contributes to the development of new practices in schools, complete collaboration with school partners is essential from the outset (McNall, Reed, Brown, & Allen, 2009). We recognized that these partnerships work best when: (a) a shared conceptual understanding informs the design, (b) there is mutuality in roles and relationships, (c) sound operational strategies guide the work, and (d) both the process of the partnership and its outcomes are evaluated (Walsh & Backe, 2013).

Another major lesson revolved around the speed of change in schools – or lack thereof. Redesigning elements of schooling is a very slow process. Schools, like most big institutions, are slow to adopt significant changes. The glacial pace of change requires that all partners commit for the “long haul.” School change mirrors teachers’ patience with students; they recognize that often “slow and steady” wins the race.

Finally, we have learned the exquisite value of program evaluation. We recognize that an evidence-based intervention should be able to give data immediately to the consumer; in other words, principals should be able to see some immediate outcomes in summative fashion. The program evaluation also highlighted use of data to change and tweak the process, provide feedback to the design team, and measure intermediate and long-term outcomes. Ultimately, the program evaluation is valuable because it taught us how to balance of flexibility in being adaptive to individual schools while faithful to a core practice.