Many schools struggle to promote a safe and supportive learning environment and are challenged by high rates of student disruption, truancy, and school failure. Such problems present a significant concern for schools and for teachers, who must balance the demands of managing student behavior, promoting skills and competencies, and delivering academic content (Pianta 2006). Disruptive and aggressive behaviors are the most common reasons for office referrals and suspensions (Irvin et al. 2006; Pas et al. 2011; Walker et al. 1996), which in turn are major risk factors for truancy, dropout, and subsequent violence. A significant portion of school-aged children have behavioral and mental health problems, making schools an important context for the prevention of aggressive and disruptive behavior problems and a critical service delivery system for children in need (Atkins et al. 2003; Hoagwood et al. 2007).

Although there is growing interest in the implementation of evidence-based prevention programs and mental health services through schools (Hoagwood et al. 2007; O’Connell et al. 2009), schools and districts often lack sufficient resources (e.g., training, money, time) to support or sustain such efforts, and are increasingly turning to the states for leadership. Maryland is one such state that has developed the infrastructure to scale-up evidence-based practices that target behavioral and mental health problems in schools. This effort is possible through a collaboration between the state department of education, a non-profit organization, and a university, and involves multiple stakeholders, including educators, practitioners, and researchers. The collaborative effort is guided by a tiered educational and public health approach to prevention, called Positive Behavioral Interventions and Supports (PBIS; Horner et al. 2005; Sugai and Horner 2002, 2006; Walker et al. 1996). The current paper describes the process by which this 12-year partnership, referred to as PBIS Maryland, was formed to support an extensive network of over 800 schools across the state, as well as the process by which this effort has resulted in high quality implementation of prevention programs, systems of support across the state, and rigorous studies aimed at translating research into practice.

Overview of the Three-Tiered PBIS Model

PBIS is a non-curricular, universal prevention strategy that aims to alter the school environment by creating improved systems and procedures to promote positive change in staff and, consequently, student behaviors. The model draws upon behavioral, social learning, and organizational behavior principles (Lewis and Sugai 1999) that have been traditionally used with individual students and extends and applies them to the entire student body consistently across all school contexts. This whole-school strategy aims to prevent disruptive behavior and enhance the school’s organizational climate by creating and sustaining primary (universal or school-wide), secondary (selective), and tertiary (indicated) systems of support. The three-tiered prevention model follows a public health approach (Mrazek and Haggerty 1994; O’Connell et al. 2009), whereby two levels of selective/targeted group and indicated/individual programs are implemented to complement the universal school-wide components (for a review, see Carr et al. 2002; Horner et al. 2005; Leaf and Keys 2005; Sugai and Horner 2002, 2006). The universal school-wide PBIS model has been widely disseminated throughout the U.S. and has been implemented in over 16,000 schools across 44 states (PBIS 2011).

Although the PBIS model provides a structure for the delivery of more intensive services and programs for children with greater needs, the schools in Maryland and elsewhere in the country have typically focused on the universal or school-wide aspects of the model. Schools often struggle to provide more intensive support services to non-responders because of the increased resources necessary to meet their needs (Barrett et al 2008; Debnam et al. in press a). Consistent with a response to intervention (RtI) approach, non-response to the universal school-wide program signals that additional supports are required for the child to be successful in the general education environment. In fact, the Maryland State Department of Education (MSDE) has adopted PBIS as its RtI approach for behavior.

The Role of Prevention Science and Community-Based Participatory Research in Fostering Collaborative Research Efforts

Many researchers and practitioners have criticized the field of prevention for not achieving public health outcomes. Despite numerous research studies showing the effectiveness of preventive interventions (e.g., O’Connell et al. 2009), the lack of adequate “systems and infrastructure” for marketing and distributing programs has hindered dissemination of efficacious prevention models (Kreuter and Bernhardt 2009, p. 2123; also see Fixsen et al 2005). The field of prevention science was formed in part with the goal of providing a framework for disseminating evidence-based preventive interventions (Kellam et al. 1999). Researchers operating from a prevention science perspective often work in close partnership with potential adopting agencies (e.g., schools, community-based organizations, service providers, states) to help identify what evidence-based programs map onto their needs. But there is also a great need for making research findings and interventions more attractive to users in order to increase uptake (Kreuter and Bernhardt 2009). This may include packaging interventions in a way in which they can be readily used and applied, promoting the intervention, distributing it, and communicating before, during, and after the adoption of the program. Collaborative efforts also include providing proper training, technical assistance, rapid response to needs, and coordinating the provision of services. Prevention scientists can also work in collaboration with community partners to collect and analyze data in order to determine program impacts (Kreuter and Bernhardt 2009).

The PBIS Maryland Initiative serves as an example of such a collaborative research effort, in which prevention scientists from Johns Hopkins University work in partnership with practitioners, educators, and policymakers from the MSDE and Sheppard Pratt Health System (SPHS), both to conduct research on the PBIS model and related prevention programs, and to promote high quality implementation of programs and services through the extensive state-wide PBIS effort. This work has occurred by applying a community-based participatory research (CBPR) approach to partnership. Specifically, the CBPR framework is characterized by its emphasis on the involvement of communities as equal partners with researchers and acknowledges the unique and important role each collaborator plays in the process (Israel et al. 2005). This approach stems from the shared understanding that in working together on a topic of importance to researchers, practitioners, youth, families, and community members, one has an enhanced ability to ensure sustainable change and positive outcomes (Israel et al. 2005; Wallerstein et al 2005).

The PBIS Maryland Initiative also employs a Type II translational approach in its work by helping to move research findings from the fields of public health, education, and mental health research into real-world school settings (SPR MAPS II Task Force 2008) and by evaluating the outcomes of these efforts. Whereas Type I translational research focuses on discovery through clinical trials, Type II translational research examines the process by which efficacious practices, interventions, or treatments become implemented effectively in real-world settings (Woolf 2008). We now describe the history of the PBIS Maryland Initiative and how it utilized a CBPR approach to meet the shared goal of translating research into practice.

History of Collaboration and Partnership

The initial impetus for PBIS in Maryland occurred at a meeting when the CEO of SPHS, Dr. Steven Sharfstein, asked the then Maryland Superintendent of Schools, Dr. Nancy Grasmick, what SPHS could do to support the efforts of MSDE and to help prevent behavioral and mental health problems in Maryland public schools. The partnership was discussed in 1998 between the MSDE, SPHS, and the Johns Hopkins University (JHU), and was formally created in 1999 to provide PBIS training, support, and conduct translational research in Maryland public schools. A series of nested levels of support and leadership has been developed, including PBIS School Teams at each school and a multi-agency PBIS State Leadership Team; these teams provided the basic infrastructure necessary for the state-wide scale-up of PBIS [see Barrett et al. (2008) and Bradshaw and Pas (in press) for a summary of the PBIS Maryland model]. The State Leadership Team currently includes representatives from MSDE, SPHS, JHU, all 24 Maryland school districts, and other state agencies (e.g., Maryland Department of Health and Mental Hygiene, Department of Juvenile Services). There is also a PBIS Maryland Management Team, which includes the lead representatives from MSDE, SPHS, and JHU, and meets weekly. The multiple levels of coordination were developed to ensure proper delivery of training and services and have been utilized in other translational efforts to disseminate programs and achieve high implementation fidelity (for examples, see Bloomquist et al. 2008; Fixsen et al. 2005; Spoth and Greenberg 2005). This type of coordination is considered a key component in scaling-up efforts (Rohrbach et al. 2006; Spoth 2008).

Together, the PBIS State Leadership and Management Teams work to support implementation, sustainability, and expansion of PBIS statewide and to engage in related research and policy work. This type of partnership between a state department, non-profit organization, and university is rare, and has been the cornerstone to our success in Maryland. Such a network allows for the authentic (i.e., genuine, reliable, consistent) and real-time application of research to practice. It has allowed the PBIS Maryland partnership to work collaboratively to rigorously address research questions that are both timely and highly relevant to educational practice and policy. The ongoing data collection, evaluation, and technical assistance provided by the partners regarding implementation and outcomes have also been critical to the success of the PBIS dissemination effort (for other examples, see Rohrbach et al. 2006; Spoth and Greenberg 2005).

In addition, representatives from the SPHS, MSDE, and JHU collaborate in creating and leading the new team training and booster training events for PBIS teams and coaches. The involvement of local school districts, as well as a training approach which involves group training, maximizes dissemination of PBIS through the promotion of exchange between school practitioners. These local “champions” of PBIS can be particularly effective in shaping their colleagues’ opinions (Rogers 2002; also see Schoenwald and Hoagwood 2001).

In fact, local champions and key opinion leaders have played a critical role in PBIS from the start of the initiative (Atkins et al. 2008). The team charged with selecting a prevention model sought input from multiple stakeholders (e.g., teachers, administrators, district leadership) when selecting PBIS for initial adoption. The approach by which stakeholders were gathered to review and select a model was intended to boost stakeholder buy-in and create readiness for change (Adelman and Taylor 1997). Efforts to create readiness are necessary for successful implementation of programmatic change across multiple educational settings [for additional information, see Bradshaw and Pas (in press)]. PBIS was initially viewed as attractive to the state, as well as to school districts and administrators, because it is implemented by teachers rather than only by specialists (e.g., psychologists), can be adapted to be consistent with a school’s culture and climate, and typically requires fewer monetary resources than other standardized prevention curricula. Fewer resources are needed because many of the training and implementation materials are free through the National PBIS Technical Assistance Center (www.pbis.org), which is funded by the U.S. Department of Education’s Office of Special Education Programs. The PBIS framework is flexible and intended to be contextually and culturally appropriate, as well as compatible with each implementing school; this move away from a “one size fits all” approach made both the state and districts feel confident about the decision to adopt PBIS. The tiered framework was also attractive because it acknowledges the need for more intensive supports and services, and is open to integration with other prevention programs.

Initially, implementation was financed, managed, and led by the state team, with Johns Hopkins’ participation supported by its federally-funded research centers and projects, rather than with state funds. However, as the initiative expanded, it was no longer feasible to maintain centralized leadership, and the 24 local school systems took on considerably greater responsibility for sustaining previously trained schools. Meanwhile, the state-level team now focuses on expanding implementation to new schools and to more advanced tiers. Maryland, like most states, has only had sufficient resources to scale-up the universal, school-wide PBIS model, and most prevention efforts have focused on elementary and middle schools (Barrett et al. 2008). Recently, the state team expanded into high schools, where behavioral and school contextual challenges are particularly great [e.g., higher rates of delinquent behaviors, multiple teacher relationships, larger schools, high stakes outcomes (e.g., graduation); Crosnoe 2011; Pianta and Allen 2008]. Prevention programming at the high school level is particularly challenging, as there are relatively few developmentally appropriate evidence-based programs (Greenberg et al. 2001) and limited state and local infrastructure to implement the handful of programs which are available.

Integration of PBIS also has occurred on a policy level, as illustrated by two laws passed by the state legislature that mandated implementation of PBIS or “an alternative behavior modification program developed in collaboration with the Department” (Maryland State Code, Title 13A) for higher risk schools. Interestingly, these laws were not initiated by MSDE, but rather by a legislator whose constituents described the improved outcomes of PBIS for their children. Given the promising effects of PBIS documented in Maryland and other states, along with the increasing pressure to promote safe and orderly learning environments, the Maryland State legislature first passed this legislation in 2004 and focused on schools (K-5) with high suspension rates. More recently, this law was expanded to address habitual truancy rates in middle and high schools in the Maryland State Code §7-304.1 Positive Behavioral Interventions and Support Program. Over 70 Maryland public schools were affected by the truancy legislation in the first year of its implementation. This provides a unique opportunity to examine the effects of legislatively mandated prevention programming, as little research exists currently on this topic.

Collaborative Approach to Leadership and Management

Mutually Negotiated Roles and Responsibilities

CBPR is an integral part of the PBIS Maryland Initiative. Each stakeholder contributes different knowledge and resources to support the initiative. For example, the MSDE has the leverage, access, and reach to institute a state-wide approach to prevention. The SPHS has a structure that allows for external data collection regarding training and implementation, as well as the ability to provide training and technical assistance to the state, districts, and schools. Finally, JHU provides expertise in evidence-based practices and the resources to rigorously examine the effectiveness of PBIS and related programs of interest to MSDE. There is often a braiding of resources to support the initiative and the implementation of evidence-based programs, and to incorporate research projects onto state and local prevention efforts. There is also a strong commitment to shared decision-making among the three partners, along with the districts. For example, when programs are chosen for integration with PBIS, all partners are involved in the selection process.

Common Goals

The prevention of behavior problems and the promotion of a positive school climate are of high importance to all members of the partnership, yet there is also shared recognition that each partnering agency has some of its own priorities, responsibilities, and levels of accountability. The collaborative is supportive in helping to meet both the shared and the agency-specific goals. For example, the MSDE is chiefly accountable to families, policymakers, and local school districts, and highly sensitive to state and federal policies, priorities, and budget cuts. As a result, there is often a need to respond quickly and efficiently to emerging concerns, particularly those identified by policymakers. The university-based partners are expected to pursue external grant funding to support research activities, conduct rigorous studies, and summarize those findings in peer-reviewed publications. It is common for the partners to collaborate on federal grants to support research projects, which address local priorities and emerging concerns. The research projects are designed in partnership and reflect the voices and perspectives of all stakeholders, yet there is a shared commitment to rigor, feasibility, and sustainability in any research effort. It is only through this partnership that it has been possible to launch so many successful translational research efforts, including three large-scale, group-randomized controlled trials, which involved over 150 schools across over half of the state’s school districts. Several examples of collaborative research projects are later described.

Knowledge Sharing

The initiative requires considerable collaboration, shared decision-making, and trust, all of which are core elements of CBPR and the authentic PBIS Maryland partnership. All partners have a commitment to sharing data and knowledge in order to promote effective practice and disseminate findings to multiple audiences. As a result, it is common for members of the collaborative to engage in multiple presentations to various stakeholders, including researchers, policymakers, community members, educators, and practitioners.

Data Systems

Consistent with the PBIS model’s emphasis on data-based decision making (Irvin et al. 2006; Sugai and Horner 2006), a critical element of the PBIS Maryland Initiative has been developing and maintaining a comprehensive data system to monitor and evaluate PBIS state-wide. As a result, the partnership has placed an emphasis on monitoring both implementation fidelity and student and staff outcomes (Patton 1997). The CBPR approach used in the PBIS Maryland Initiative builds on the unique strengths that each agency and stakeholder contributes, and therefore naturally dictates each agency’s roles and responsibilities. For example, the state-wide evaluation activities are chiefly web-based and coordinated by SPHS and JHU. Biannually, schools are required to complete and submit different measures of implementation fidelity, which are used for multiple purposes.

The SPHS is responsible for collecting and organizing data on the training of schools and coaches, as well as the measures of implementation fidelity submitted by the individual schools to monitor the initiative. Criteria were developed by the PBIS Management Team to determine whether a school would be considered “trained” or “active” in its implementation of PBIS. Data-based reports on the implementation status of PBIS schools across the state are generated through the initiative’s interactive website, www.PBISMaryland.org. Through this site, district and state partners can access information on the schools’ implementation status for progress monitoring and to guide coaching and technical support efforts. Annual reports are also generated for the PBIS Maryland collaborative as well as the National Technical Assistance Center for PBIS. The data are also shared among the different partners. For example, the JHU team helps to analyze the fidelity and outcome data to determine the state and local impacts of PBIS. Findings from these analyses are communicated to the state partners through the weekly PBIS Management Team meetings and the monthly PBIS State Leadership Team meetings, in addition to national presentations and peer-reviewed publications (e.g., Barrett et al. 2008; Bradshaw and Pas in press; Pas and Bradshaw, 2011). The state also uses these data to carry out a recognition process to acknowledge schools that are achieving quality implementation and positive outcomes of PBIS. In addition, the PBIS Maryland collaborative uses the data to identify targets for additional support and training.

Dissemination of PBIS in Maryland

School Participation

Currently, there are 1,465 public schools in the state of Maryland, across 24 school districts. Between 1999 and 2010, the PBIS Maryland Initiative trained 819 schools in total, of which 776 were public schools (i.e., 52% of the total public school population) and 43 were private or state operated (e.g., parochial or special education settings). In addition to training school teams in PBIS, the PBIS Maryland Initiative has also trained coaches to provide support to these schools in the implementation and evaluation of PBIS. As of 2010, 594 coaches had been trained. See Fig. 1 for a chart displaying the cumulative number of schools and coaches trained in PBIS. Maryland ranks as the fifth state in the country with regard to the number of schools trained in school-wide PBIS, and ranks second in terms of the concentration of school-wide PBIS schools.

Fig. 1
figure 1

Cumulative number of PBIS Maryland teams and coaches trained by year

Full size image

More than half of the trained schools are elementary schools (i.e., 55%) and one-quarter are middle schools. However, the trained population of elementary schools only encompasses about half of all elementary schools in the state, while nearly 78% of currently operating middle schools have been trained (i.e., there are currently 218 middle schools in the state and 191 middle schools have been trained, of which 169 are open and operating as of 2011). High schools, followed by alternative and charter schools, compose the lowest proportions of trained schools. See Table 1 for a listing of the number of schools in the state and number trained, by school type.

Table 1 Proportion of schools trained by school type and level
Full size table

Participating Coaches

A critical aspect of the PBIS model is the behavior support coaches, who help the school-level teams to implement PBIS with fidelity. In the 2010–11 school year, there were 560 active PBIS coaches, representing 94% of coaches ever trained. The school personnel who serve as coaches have multiple roles within the school context, in addition to their work as a coach. For example, 31% of current coaches are school psychologists, 27% are “other” roles (e.g., special educators, specialists, or academic coaches), and 21% are teachers. The state currently uses a mixture of internal and external coaches. See Fig. 2 for a chart displaying the coaches’ primary roles in their school(s).

Fig. 2
figure 2

Coaches position within the schools. Note The numbers displayed represent the number of coaches by their primary role within the school

Full size image

Implementation Quality

A variety of measures are used by the partnership to track implementation quality. One such measure is the Implementation Phases Inventory [IPI; Bradshaw et al. (2009a)]. PBIS schools are required to biannually (i.e., in fall and spring) submit data on the IPI, which is completed by the district-appointed PBIS coach and submitted electronically to the PBIS Maryland consortium. The measure assesses the presence of 44 key elements of school-wide PBIS following a “stages of change” theoretical model (Prochaska and DiClemente 1982), whereby schools move through a series of four stages: preparation, initiation, implementation, and maintenance. Schools receive a percentage of implemented elements for each stage as well as a total score, such that a higher score indicates greater implementation, and a “predominant phase.” The IPI was developed by members of the State Management Team, and research was conducted demonstrating the psychometric properties of the instrument [see Bradshaw et al. (2009a)].

In the fall of 2010, 665 schools (or 81%) submitted data on the IPI. The majority of schools (i.e., 72%) were rated as being in the final maintenance phase of implementation, based on the IPI scores. Only 6.6% were found to be in the initial preparation phase. Ten and 11% of schools were in the initiation and implementation phases, respectively. Elementary and middle schools had the largest percentage of schools in the maintenance phase in the fall of 2010 (i.e., 76.4 and 79.6%, respectively), whereas only 40.5% of high schools and 68.6% of special or alternative schools were determined to be within the maintenance phase. High schools had a large percentage of schools in the preparation (21.4%) and initiation (28.6%) phases. See Table 2 for a full listing of the percent of schools in each phase of implementation.

Table 2 Predominant phase of implementation as reported on the Implementation Phases Inventory (IPI) for each school type
Full size table

In addition to the IPI, PBIS schools also complete two other measures of implementation fidelity: the School-Wide Evaluation Tool (SET; Horner et al. 2004) and the Benchmarks of Quality (BOQ; Cohen et al. 2007). The SET comprises seven subscales (i.e., expectations defined, behavioral expectations taught, system for rewarding behavioral expectations, system for responding to behavioral violations, monitoring and evaluation, management, and district-level support) that assess the degree to which schools implement the key features of school-wide PBIS. It is conducted by an observer external to the school. Schools receive a percentage score for each scale, as well as an overall score, such that a higher percentage demonstrates a higher level of fidelity. This measure is used throughout the country and studies have documented its reliability and validity (Vincent et al. 2010).

In the spring of 2010, 460 schools submitted SET data and the averages on the SET scales were very high. For the overall SET score, schools on average attained a 94%. The subscale scores ranged from 91.5% (on the violations system subscale) to 96.5% (on the decision making subscale). Of the submitting schools, 276 were elementary or elementary/middle schools; 115 middle schools and 37 high schools submitted SET data, whereas 32 special or alternative school settings submitted SET data. There is relatively little variability in SET scores by school level; however, high schools generally had slightly lower SET scores, with an average overall score of 88%.

The BOQ has been used since 2009 as part of the new team training process, whereby schools complete the BOQ in the spring prior to their summer training and then utilize the measure during the training. Schools are also expected to continue to complete a BOQ on an annual basis. Whereas the IPI and SET are completed by a single individual, the BOQ is completed by multiple raters, including the PBIS team members and coach. Each person completes an independent rating of 53 items regarding 10 areas of implementation. The final score reflects the most commonly endorsed level of implementation (i.e., on a scale of 0–2) by all members and a total percentage of implementation is calculated. A study of its psychometrics documented adequate reliability and concurrent validity with the SET (Cohen et al. 2007). In the spring of 2010, 559 schools submitted BOQ data and, on average, attained a score of 84.6%. The scores appeared slightly higher for the 320 elementary and elementary/middle schools, which, on average, received an 86.7%. The 142 middle schools that submitted BOQ data received an 84.8% on average and the 54 high schools received a 72.7%. Forty-two special or alternative schools submitted the BOQ data and received an 82.4% on average.

Together, these data suggest that the schools in Maryland have reached a high level of implementation quality and have been successful at sustaining this level (Barrett et al. 2008). There is also some evidence that schools trained in school-wide PBIS experienced reductions in suspensions and/or office discipline referrals (Bradshaw et al. 2010). The infrastructure to collect, analyze, and use data is a unique feature of the PBIS Maryland partnership. It also illustrates the state’s commitment to data-based evaluation and research.

Collaborative Research Projects Through PBIS Maryland

Multiple research projects have been launched which take advantage of the existing network of researchers, educators, and practitioners involved in the PBIS Maryland Initiative. For example, the state-wide scale-up efforts have provided the opportunity to conduct effectiveness and translational research on PBIS, which has been supported through federal grants from the Centers for Disease Control and Prevention (CDC), Institute of Education Sciences (IES), National Institute of Mental Health (NIMH), the National Institute on Drug Abuse (NIDA), and some foundation grants from organizations, such as the William T. Grant Foundation. Each of the projects represents an effort to integrate PBIS with other approaches or interventions to ensure both high fidelity implementation and sustainability (Domitrovich et al. 2008). Some of the larger, federally-funded research projects are briefly described below.

Project Target: A Randomized Controlled Effectiveness Trial of SWPBIS

The PBIS Maryland partnership helped to launch a group-randomized controlled trial (RCT) of school-wide PBIS (i.e., SWPBIS). The aim of this five-year (2002–2007) CDC- and NIMH-funded RCT was to examine the main effect of SWPBIS on school climate and student behavioral outcomes. It involved 37 elementary schools, of which 21 were randomly assigned to the intervention condition and 16 to the comparison condition. Schools randomized to the PBIS condition attended the two-day initial training event and annual booster summer training events led by the PBIS State Leadership Team. Ongoing on-site technical assistance was provided by a PBIS behavior support coach and district points of contact. Data were collected on over 3,500 staff and 29,000 students and showed that PBIS significantly reduced suspensions, office referrals (Bradshaw et al. 2010), bullying, and peer rejection (Waasdorp et al. in press), and improved staff members’ perceptions of the schools’ organizational health (Bradshaw et al. 2008, 2009b). Training in PBIS was associated with sustained changes in schools’ internal discipline systems and practices (Bradshaw et al. 2010). Further research was needed to reveal for whom, how, and under what circumstances school-wide PBIS is most effective. This led to the launching of the PBIS Variations Project, which was funded by IES to examine variation in the impact of PBIS. For example, we are exploring characteristics of the child, such as baseline risk profile, which serve as potential moderators of the effects of SWPBIS (e.g., Bradshaw et al. 2011). Research is also underway which aims to determine the extent to which the trial findings generalize to the broader set of schools within the state (Stuart et al. 2011). This work represents an important next step in the research on state-wide dissemination of school-based prevention programs, as it explores the external validity of the findings of the RCTs. This work also highlights the importance of developing an infrastructure to collect data on implementation quality and program outcomes when prevention efforts are brought to scale.

PBISplus Project: A Randomized Controlled Trial of SWPBIS Combined with Tier-Two Supports

The team recently conducted a second RCT to determine the combined impact of the universal, school-wide PBIS model with training in targeted/selective preventive interventions. This IES funded, three-year RCT (called PBISplus) was conducted in 45 Maryland elementary schools to document the impact of a coaching and technical support model focused on targeted/selective interventions for children not responding adequately to the universal level of PBIS. A central focus of this work was on the development of function-based thinking, a generalized (i.e., universally-used) model of functional behavioral assessment (which typically is a formalized process only conducted at the second or third tier of prevention) that could be used school wide (Hershfeldt et al. 2011b). Although the trial has only recently concluded, our preliminary results highlight the importance of principal leadership and support for quality implementation of selective and indicated prevention programs (Debnam et al. in press b), the role coaches play in sustainable school-level changes (Hershfeldt et al. 2011a), and the impact of teacher factors, such as burnout and efficacy, on the use of positive behavior supports in the classroom (Pas et al 2010; Pas et al. in press).

Double Check: A Cultural Proficiency and Student Engagement Model

With funding from IES, the Double Check project builds on the SWPBIS model to promote data-based decision making, professional development on cultural proficiency, and coaching in culturally sensitive classroom management and student engagement. Specifically, through an iterative process, the project aims to augment and combine the data-based decision-making activities of SWPBIS, the Double Check cultural proficiency professional development series (Hershfeldt et al. 2009), and the Classroom Check-up (Reinke et al. 2008) classroom management coaching system to increase the use of culturally-responsive teaching and classroom management strategies, and to promote student engagement in elementary and middle schools. The goal of this work is to reduce rates of culturally and linguistically diverse students being referred for discipline problems and special education services. Consistent with the CBPR approach, this project was developed in direct response to a request from a collaborating Maryland school district, Anne Arundel County Public Schools, which is eager to address concerns related to disproportionality in referrals and disciplinary actions through PBIS.

Maryland Safe and Supportive Schools (MDS3) Initiative: An RCT of PBIS in High Schools

There is increasing interest in the integration of the universal, school-wide PBIS model with other evidence-based selective and indicated prevention programs (Domitrovich et al. 2010). Currently, studies are being conducted in elementary schools on the integration of PBIS with social-emotional learning programs, such as the Promoting Alternative Thinking Strategies model and the Good Behavior Game (see Domitrovich et al. 2010). The PBIS Maryland partnership extended this work to the high schools through a 52-school RCT of PBIS combined with evidence-based prevention programs. This 13-million dollar trial was funded through the U.S. Department of Education’s Safe and Supportive Schools Initiative and aims to develop and administer a statewide web-based measurement system to assess multiple aspects of school climate (e.g., school safety, student engagement, and the school environment), as reported by students, parents, and school staff.

The 30 intervention schools are being trained in the PBIS model and the use of the school climate data to determine the need for tailored evidence-based preventive interventions. The intervention schools receive training, coaching, and the necessary resources to implement a continuum (e.g., universal, selective, and indicated) of evidence-based practices, such as the Olweus Bullying Prevention Program (Olweus et al. 2007), LifeSkills training for high schools (Botvin et al. 2006), Check-In/Check-Out (Hawken and Horner 2003), Check and Connect (Sinclair et al. 2005), and the Cognitive-Behavioral Intervention for Trauma in Schools (Stein et al. 2003), in order to integrate them with PBIS. The 22 comparison high schools will be monitored over a period of three years using this same climate measure; they will receive training at the end of the trial. A second cohort of eight schools is planned, bringing the total sample of schools to 60. This work is also being extended through a grant from the William T. Grant Foundation, which aims to determine the program impacts on multiple classroom and non-classroom observations of setting-level factors (e.g., safety and classroom climate), to examine potential setting-level moderators of program impacts and predictors of intervention fidelity, and to explore the relationship between perceptions of school climate and setting-level measures of school climate. The findings from the MDS3 Project will inform our understanding of the impact of school-wide preventive interventions in high schools, and factors influencing implementation fidelity and the outcomes of those programs. This research also has important implications for Maryland’s Safe and Supportive Schools Initiative in terms of validating the state’s new MDS3 School Climate Survey in relation to the observational data.

All of the research projects reflect the input from several partners, collaborators, and stakeholders at multiple levels, and address state-wide and national priorities related to school-based prevention. Building on the interest and resources of the state and school districts, the research findings are first disseminated locally through the monthly meetings of the PBIS Maryland State Leadership Team. National dissemination occurs jointly by the PBIS Maryland Management Team through presentations at professional meetings. The prevention efforts and policies in Maryland have benefitted tremendously from conducting these effectiveness studies through the PBIS Maryland collaboration. These partnership-focused research efforts also have enabled the development and application of innovative statistical methods to determine the generalizability of findings from randomized trials to the state (Stuart et al. 2011). In the remaining sections, we summarize some lessons learned from the PBIS Maryland partnership and identify some future research directions, both of which we hope will be informative to other researchers and community partners working in collaboration to conduct prevention research in applied settings.

Lessons Learned

The Importance of Authentic Relationships

It is through the development of long-term and authentic relationships between the three partnering agencies that trust and a shared mission have been built. This has enabled the PBIS Maryland Initiative to successfully train over half of the state’s public schools in PBIS, to provide on-going support for and evaluation of PBIS, and to conduct three large-scale RCTs of PBIS. These relationships have been built over time by the varying stakeholders delivering on promises, sharing successes, and working in a collaborative manner on an ongoing basis. From the perspective of each agency and institution involved, it is important to “show up” and engage in conversations with one another about where successes and obstacles lie, what the next steps should be, and demonstrate a shared interest in the goals. These commitments and relationships extend well beyond the life of any particular project or grant. In many ways, the individuals representing the partnering agencies have formed authentic friendships that provide a context for frank discussions about ‘what is’ and ‘what can be.’

Balancing the Priorities of Different Agencies

As noted above, each agency has a unique set of priorities and strengths which are embedded within the broader shared focus and mission of the PBIS Maryland partnership. A key to success is developing a shared agenda that meets the needs of each agency as well as the collective. For example, to successfully work with the state, the non-profit and university partners need to ensure that (a) the data collected are useful to schools and policymakers, (b) data can be collected and analyzed in a non-burdensome way in terms of time and cost, and (c) research designs and questions which respond to the pressing needs of educators and policymakers are developed. The PBIS Maryland Initiative has struck this balance, which in turn has enabled the successful launching of multiple prevention research efforts. The RCTs also provide additional funding that aids schools in implementing the interventions and additional resources for training, support, and coaching. The research has generated data for the schools that they otherwise would not have, in addition to providing outcome data to the state and local school districts regarding the impact of PBIS. It has also provided salary support to faculty and staff members at JHU, SPHS, and MSDE to support their collaborative research efforts. The development of these RCT designs has occurred through the constant communication between the stakeholders, which has enabled the different agencies to share their knowledge, improving the practice of all. School practitioners and state representatives have been provided with exposure to how rigorous research can inform evidence-based practices, while researchers are provided with a better understanding of “on the ground” educational practices and obstacles. This knowledge sharing allows the researchers to disseminate new and innovative research findings which have the potential to impact the fields of prevention and translational research.

Sustainability via Multiple “Champions”

The State Leadership Team involves multiple stakeholders and key opinion leaders from the MSDE, JHU, and SPHS as well as a representative from all 24 school districts. Although this team structure is mainly to ensure that all parties contribute to the process, it also helps to promote sustainability of the initiative. In fact, a large portion of the key members of the PBIS Maryland Initiative have been involved in the initiative since its inception, which established a tone of trust and a commitment to sustainability.

Non-Linear Dissemination and Implementation Process

Although the PBIS Maryland Initiative utilized a theoretical, staged process of diffusion (Adelman and Taylor 1997) to guide its statewide dissemination effort, the process has not been linear (see Bradshaw and Pas in press). The three partnering agencies have invested considerable resources, in terms of time, manpower, and intellectual capital, to develop this sustainable infrastructure. The partnership has weathered leadership and staff changes, shifts in state and federal priorities, and budget cuts. This has often brought the team back to earlier stages, but never threatened the existence of or overall commitment to the partnership. It is because of the existing partnership that the group is able to respond quickly to opportunities and challenges. It is for these reasons that Maryland is considered a ‘national exemplar’ for the successful dissemination and implementation of school-wide PBIS (Bazelon Center for Mental Health Law 2006).

Conclusions and Future Directions

The increasing trend toward state- and district-level coordination and implementation of programs highlights the need for more research on the transition from efficacy of school-based prevention programs to effectiveness research (Flay et al. 2005), and may be best conducted in tandem with research on the scale-up process. The PBIS Maryland Initiative serves as an example of this, where the full range of translational research (i.e., efficacy, effectiveness, and the translation to the real world) is conducted through a state/non-profit/university partnership. Though a similar infrastructure exists in other states, this is still more of an exception than the rule.

The PBIS Maryland Initiative provides a unique opportunity to study the translation of research on prevention in schools via the PBIS framework. By identifying the contextual factors which are associated with the training and adoption of PBIS, as well as the achievement of a high level of fidelity, we will better understand how to target the program to schools most in need and to those most likely to experience successful implementation and outcomes. In addition, studying real-world outcomes and comparing these findings to the results of RCTs helps to assess the external validity of the trials conducted within Maryland (see Bradshaw and Pas in press; Pas and Bradshaw 2011; Stuart et al. 2011). As a result, there is a need for more research on factors which lead to the adoption and adequate implementation of programs (Spoth 2008); this is particularly true in school settings where there is a growing emphasis on the implementation of ‘evidence-based’ prevention programs (Sloboda et al. 2008).

Given the PBIS model’s emphasis on school-wide and systemic change, schools implementing PBIS typically go through a process of getting “buy-in” or support from at least 80% of the school staff prior to adopting it. Maryland is the first state, to our knowledge, to mandate implementation of a specific school-based prevention model through legislative action; however, other states (e.g., Florida) have proposed similar policies and several federal bills related to PBIS have been proposed. Yet there is limited research examining the impact of mandated implementation of school-based prevention programs, and more generally, the effect of legislation on the use of particular prevention models. It is possible that schools mandated to adopt PBIS (or an alternative program) may be more resistant to implementation and thus achieve less favorable outcomes than schools that voluntarily implement PBIS (Pankratz et al. 2002). In an effort to raise educational standards and promote greater consistency across jurisdictions and states, we will likely see more prevention programs mandated by legislators. Consequentially, additional research is needed to understand the impact of such prevention policies on schools and students, and the role of state/non-profit/university partnerships in supporting their implementation.