Improved mental health and other services and outcomes for children and their families are connected to successful implementation of innovative, evidence-based practices (EBPs) (Aarons and Palinkas 2007; Crea et al. 2008). Numerous models and frameworks exist for understanding the complex array of factors responsible for successful implementation of EBPs in organizational settings (Aarons et al. 2011; Fixsen et al. 2005; Greenhalgh et al. 2004; Schoenwald et al. 2008; Simpson 2002). Although inter-organizational networks are included in many of these models as part of the “outer context” of implementation (Aarons et al. 2011; Greenhalgh et al. 2004), an understanding of the role of collaboration between organizations throughout the process of implementation has been somewhat limited. Furthermore, most EBP implementation studies that do focus on inter-organizational collaboration fail to consider the wider context within which collaboration occurs, including factors such as involvement of external stakeholders, socio-political processes, and the roles of relationships and leadership (Horwath and Morrison 2007). Increasingly, this context is characterized by government mandates and fiscal realities that increasingly require collaboration in the form of integrative multidisciplinary practice in the delivery of children’s services (Ehrle et al. 2004; Hogan and Murphy 2002). In a sociopolitical climate in which organizations are facing increasing budget restrictions and are challenged to do more with less, collaboration across agencies and organizations appears to be a critical element to successful EBP implementation. In turn, an understanding of effective collaboration seems to be at the crux of many EBPs developed to improve outcomes in child-serving systems (Prince and Austin 2005).

There exists an extensive literature on the nature of interagency collaboration for the delivery of health and human services in general and child welfare services in particular. Although many consider such collaboration to be essential to the delivery of a complex array of services (Lippitt and Van Til 1981; Stroul and Friedman 1986; Jones et al. 2004), others have questioned its usefulness on both theoretical (Scott 1985) and empirical grounds (Glisson and Hemmelgarn 1998; Longoria 2005). Several studies have pointed to improved access to services and improved outcomes associated with interagency collaboration (Cottrell et al. 2000; Bai et al. 2009). However, Glisson and Himmelgarn (1998) found that efforts to coordinate services between public child-serving agencies in Tennessee were negatively associated with the quality of services provided. Chuang and Wells (2010) found that while inter-agency sharing of administrative data increased the odds of youth receiving inpatient behavioral health services, having a single agency accountable for youth care increased odds of receiving both inpatient and outpatient services.

In part, the inconsistency in findings may be attributed to differences in the definition and operationalization of collaboration as a construct (Grace et al. 2012; Hodges et al. 1999; Dedrick and Greenbaum 2011), and to different conceptual models of such collaboration that emphasize or highlight different determinants and processes (Akhavian et al. 1999; Milbourne et al. 2003; D’Amour et al. 2005; Salmon and Faris 2006). For instance, some researchers have distinguished among collaboration, cooperation, coordination, and networking, whereas others have used these items interchangeably (Grace et al. 2012; Hodges et al. 1999). Others have viewed interagency collaboration as an aspect of organizational culture, defined as “the way things are done in an organization” (Glisson 2007, p. 739). Factors that specifically have been found to contribute to successful interagency collaboration between child welfare and other agencies include shared goals, a high level of trust, mutual responsibility, open lines of communication, and strong leadership (Johnson et al. 2003; Weinberg et al. 2009). Barriers to building effective collaborations include deeply ingrained mistrust and continued lack of other systems’ values, goals, and perspectives, different organizational priorities, confusion over how services should be funded and who has jurisdiction over youth, and difficulty in tracking cases across organizations (Conger and Ross 2006; Sedlak et al. 2006; Green et al. 2008).

To address the inconsistency of findings and lack of information related to the role of interagency collaboration in EBP implementation, we conducted a qualitative investigation of public agencies responsible for providing services to youth in California. Specifically, we examined the experiences of collaboration in EBP implementation of agency directors and senior administrators of county child welfare, mental health and juvenile justice agencies. Our objective was to examine the role of collaboration in implementing new EBPs and identify the determinants and processes of successful collaborations. Specifically, we were interested in the following: (1) what factors led to the creation and sustainment of effective inter-organizational collaborations for the purpose of EBP implementation; (2) what factors contributed to unsuccessful collaborations or served as barriers to successful collaborations; and (3) how did such collaborations function to implement new EBPs? Drawing from Himmelman et al. (2001, p. 3), we define collaboration as “…a process in which organizations exchange information, alter activities, share resources, and enhance each other’s capacity for mutual benefit and a common purpose by sharing risks, responsibilities and rewards”.

In a previous study (Palinkas, Holloway, et al. 2011), we used methods of social network analysis to examine collaboration outcomes. In this study, we conducted qualitative semi-structured interviews to examine collaboration determinants and processes. Such methods have been used previously in studies of interagency collaboration (Grace et al. 2012) and EBP implementation (Aarons and Palinkas 2007, Palinkas et al. 2008). Such methods are especially appropriate when conducting exploratory analysis of a phenomenon lacking a solid conceptual framework or science base (Palinkas, Aarons, et al. 2011).

Methods

This study was part of a larger randomized controlled trial (RCT) known as the Cal-40 Study. The objective of this RCT is to assess the effectiveness of Community Development Teams (CDTs; Sosna and Marsenich 2006) in scaling up the implementation of Multidimensional Treatment Foster Care (MTFC), an EBP designed to reduce out-of-home placement in group and residential care, juvenile arrests, substance abuse, youth violence, pregnancy, and behavioral and emotional problems (Chamberlain et al. 2008; Wang et al. 2010). Public youth-serving systems in 40 California counties that had not already implemented MTFC were invited to participate in the Cal-40 Study. The 40 counties were matched on background factors (e.g., population, rural/urban, poverty, Early Periodic Screening and Diagnosis and Treatment utilization rates) and then were divided into six equivalent clusters: two with six counties and four with seven counties. Each of these six comparable clusters was assigned randomly to one of three sequential cohorts in a waitlist design with staggered start-up timelines. Within each cohort, counties were randomly assigned to CDT or standard implementation conditions. Progress toward implementation was assessed by means of a Stage of Implementation Checklist (SIC) (Chamberlain et al. 2008).

Participants

The present study targeted the 13 counties in the first cohort of the Cal-40 Study. A purposive sampling strategy was employed, beginning with directors of child welfare, mental health, and probation departments of all 13 counties. In some instances, associate directors or senior program managers were recommended by the directors to be interviewed in their place. Of the 45 administrators from all 13 counties invited to participate, 38 representing 12 counties agreed to do so, yielding a response rate of 84 %. After completing informed consent procedures for participating in research, each participant completed a semi-structured interview conducted between July and September of 2008. The number of interviews per county ranged from 2 to 6. Twenty-eight participants were interviewed face-to-face; ten were interviewed by telephone. The research study was approved by the Institutional Review Board at the University of Southern California.

A complete description of the demographic characteristics of study participants is available elsewhere (Palinkas, Holloway et al. 2011). The majority of the participants were female, and ages ranged from 31 to 63 years. Nearly 58 % of the participants were directors or assistant/deputy directors, while the remaining 42 % were program managers. The representation of participants by agency was nearly equal, with a slightly greater number of participants from child welfare services than probation or mental health departments (36.8, 31.6, and 31.6 % respectively).

Data Collection

The semi-structured interview centered on knowledge and implementation of MTFC and other EBPs at the county level. Interviewees were asked if they had ever heard of the Cal-40 Project or MTFC and what their motivations were to participate or not participate in the program. Participants were then asked who they had talked to about participation in MTFC or other EBPs; prompts were given to participants as necessary to identify their relationship to that person, their reasons for talking to that person, and the amount of influence that person had on their decision to participate in MTFC or a similar EBP. Then participants were asked whether they had implemented or attempted to implement any other new EBPs in the past year, whether any of these new initiatives involved collaborations with the other agencies, their characterization of the collaborations with these other agencies, the most important elements of a successful collaboration, and instances, if any, of unsuccessful collaborations.

Data Analysis

A methodology of “Coding Consensus, Co-occurrence, and Comparison” outlined by Willms et al. (1990) and rooted in Grounded Theory (i.e., theory derived from data and then illustrated by characteristic examples of data) (Glaser and Strauss 1967) was used to analyze the semi-structured interviews. Audio-recorded interviews were transcribed and reviewed by three investigators, who developed lists of codes individually. These codes were subsequently discussed, matched and then integrated into a single codebook. Each text was independently coded by at least two investigators and disagreements in assignment or description of codes was resolved through discussion between investigators and enhanced definition of codes. The final list of codes, or codebook, constructed through a consensus of team members, consisted of a numbered list of themes, issues, accounts of behaviors, and opinions that related to organizational and system characteristics that influence inter-organizational collaboration and implementation of MTFC and other EBPs. Inter-rater reliability in the assignment of specific codes to specific transcript segments was assessed for a subset of randomly selected pages from 10 transcripts. For all coded text statements, the coders agreed on the codes 91 % (range = 88–94 %) of the time, indicating good reliability in qualitative research (Boyatzis 1998). The computer program QSR NVivo (Fraser 2000) was used for coding and generating a series of categories arranged in a treelike structure connecting text segments as separate categories of codes or “nodes.” These nodes and trees were used to further the process of axial and pattern coding to examine the association between different a priori and emergent categories related to the topic of collaboration in the context of EBP implementation in general and implementation of MTFC in particular (Strauss and Corbin 1990).

Results

Analyses of interview transcripts revealed three sets of characteristics related to collaboration in implementing EBPs and other innovative programs: (1) characteristics of the collaboration process; (2) characteristics of the external environment in which the collaborations took place; and (3) characteristics of the organizations and their members participating in the collaboration. The characteristics of collaboration process are determined, at least in part, by the characteristics of the external environment and participating organizations and individuals. Each of the three sets of characteristics, in turn, influences the structure of influence networks of implementation collaborators which, in turn, influences the outcome of implementation efforts as outlined in our earlier study (Palinkas, Holloway, et al. 2011). These relationships are illustrated in Fig. 1.

Fig. 1
figure 1

Heuristic model of inter-organizational collaboration for implementation of evidence-based practices

Full size image

Characteristics of the Collaboration

Systems leaders described collaboration as critical to the implementation of EBPs. In no instance did we hear of a successful EBP implementation effort (as defined by movement from adoption to routine use) that did not involve some form of collaboration between two or more organizations, and in no instance did we hear of an instance in which an inter-organizational collaboration hindered or prevented the implementation of an EBP. As described by one chief probation officer,

It really, requires that, if you want to be considered for this grant, you need to have collaborative relationships with this entity, this entity, this one, and I think they’re starting to get it now. And, and my recommendation to any county that really wants to develop a good, worthwhile, comprehensive program is, you’ve got to establish relationships with the other departments who are offering specialized services…Before we even apply for a grant, or get a program off the ground, uh, we bring in our partners from HSA [Human Services Agency], from Behavioral Health, or law enforcement partners, uh, Public Health now…

Accounts of previous experiences with EBP implementation and current efforts to implement MTFC revealed four characteristics of the collaboration process that were relevant to EBP implementation: focus, formality, frequency, and function. Collaboration focus referred to whether the intention was to implement a broad array of innovative programs or practices (broad focus) or a very specific practice (narrow focus). As illustrated by the above quote, the collaborating individuals or agencies may be the same regardless of the specific EBP being implemented (broad focus). In other instances, the collaborations are formed in response to specific EBP requirements or characteristics (narrow focus), as in the case of some of the counties in the Cal-40 Study where child welfare agencies collaborated with mental health agencies to implement MTFC because involvement of the latter was necessary to secure financial support for sustainability through Medi-Cal billing of services.

Collaborative activities also ranged along a continuum of formality, from simple and informal communications to more formalized meetings such as participation in statewide professional associations like the California Mental Health Directors Association, County Welfare Directors Association of California, or Chief Probation Officers of California; or regional and county-wide planning councils or consortiums which were often formed and structured by policies or mandates. System leaders’ participation in committees, consortiums, and advisory councils was also indicative of higher levels of collaboration across agencies in the same county and agencies in different counties. These inter-agency meetings provided a place and occasion for communication, negotiation, and inter-agency strategic planning to take place. In addition to formalized collaborations that involve regularly scheduled interactions, systems leaders often informally contact their counterparts in counties with similar demographic profiles for advice and information on the content of specific EBPs or the process of implementing any innovative program. For instance, a mental health director from a small county commented that counties have their own inter-county networks, based on who can provide information that will be most helpful: “For me, it’s Madera. It’s not necessarily Fresno or Tulare…even though they’re right next door to us. Madera matches us in budget size, demographics, some of the things they do.” Informal contacts are often made with counterparts in other counties that have a reputation for being “early adopters” of innovative programs.

Moreover, collaboration occurred along a continuum of frequency, from infrequent or ad hoc efforts to respond to specific initiatives related to EBP implementation to frequent or continuous efforts to identify and implement innovative programs for the sake of improving service quality. As explained by one child welfare director, “there’s a lot of different levels of collaboration, and how that might look like. I think uhm, certainly agency and department heads talk to each other. They have lots of meetings. Lower level staff from different agencies talk to each other less frequently and usually about specific projects or programs.”

With respect to their functions, collaborations across counties and across agencies within the same county occurred for the purpose of information exchange and pooling of resources in the short term and to alter services delivery and enhance capacity to prevent or treat problem behaviors in at risk youth in the long run. One of the chief probation officers participating in the study provided the following example of a cross-county collaboration:

We have lots of discussions. All the Chiefs in the north[ern part of the] state, we meet quarterly, either by conference call or at a specific location. And uh, we have attempted to consider the possibility of pooling our limited resources. And doing some regional things. Such as, regional trainings for the north state counties. We’ll bring a trainer in and all of us put in some of our resources to bring that trainer in…We have a lot of those discussions between the chief probation officers about those types of possibilities.

Pooling of resources was especially important in small counties that lack the funding or number of clients to sustain an innovative program or specific EBP entirely on their own.

However, the exchange of information and resources are viewed as a means to other ends, which are the alteration of how services are provided and enhancing the capacity to serve youth and produce better outcomes. According to one mental health services program manager, “…I think that’s really, for me, the only reason we would develop a collaboration with other agencies that serve our kids, for some of these evidence-based practices, you know, that we could enter into it, because just traditional things don’t seem to work that well. If I can always get one other system partner to agree, I feel like I’m halfway there to implementing a program that will meet the needs of our clients.” Similarly a child welfare director noted that the desire to deliver better services motivates the development of collaborations across agencies: “I’m going to speak for Mental Health and Child Welfare, since those are the two I know the best, having CBO input and letting them know…‘This is where we have some gaps in. So, anyone out there interested in doing something about this, or, or helping us plan and…’ You know, I think, that, that it’ll be responsive and have come to us with some plans and ideas.”

Characteristics of the External Environment

Characteristics of inter-organizational implementation collaborations were associated with certain features of the external environments in which these collaborations occur. These included the extent to which funding is available to introduce and sustain the EBP, the size of the county, the existence of clients in need of the services provided by each agency, and local, state, or federal mandates that dictated the specific form of collaboration.

Availability of Funding

The availability or lack of availability of funding to hire staff and sustain the EBP was a major factor in the decision to collaborate across agencies in a county and across counties. As one child welfare director noted, “I suspect in many counties, the money brings people to the table.” The ability to leverage dollars across county agencies was an important requirement for the collaboration that was necessary for developing, implementing, and sustaining EBPs and other services. According to another child welfare director, “We look at,…who are the possible partners for those types of projects. So if we don’t have the internal resources, you know, are there other ways to get those resources? You know, by collaborating with somebody else, for instance.” Nevertheless, the acquisition of such resources usually implied some form of reciprocity. According to one mental health director, “So, you know, we’ll put some staff here and we’ll put some money in here, and you put some money in there…We told the Public Guardian, You give us a half-time Deputy Public Guardian, and we’ll take your 10 most difficult clients…So, you know, you give something, you get something.”

The pressure to collaborate in the face of limited funds is especially great during times of budget shortfalls such as the one currently being experienced by the state of California. According to one child welfare director, “it’s forced us to collaborate more, to share our resources”. This pressure is especially intense in small, rural counties who collaborate with neighboring counties in order to maximize the use of limited funds. One example of such collaboration was the decision of two small rural counties to jointly pay for someone to train the staff of both counties in a particular EBP and share the cost because of limited resources.

While restricted funding had the potential to forge collaborations, budget cuts also contributed to severing them. For instance, in one county, “…because we were going through budget restrictions and restraints, we weren’t sure if that program is going to last. And so, that pretty much would sever some of our collaborative relationship with uh, with the other agencies here…[However], because the state came out with a mentally offender MIOCRA grant, we pooled our collaborative back together to address the RFP [request for proposals], and go for the funding” (chief probation officer).

County Size

The small size of a county was an external constraint that demanded collaboration with other counties due to limited resources. However, small counties were at an advantage with respect to collaborations across agencies within the same county. According to the child welfare director of one such county, “We’ve worked well with Probation and Mental Health, and we do work really closely together. That is an advantage of a small county. We know each other very well. We pick up the phone and call each other. We, you know, we connect pretty regularly on a lot of issues.” In some cases, this ease of collaboration was attributed to the relationships that exist in small towns and counties. For example, one small county leader said,

I have the advantage in a small county of having established relationships with the folks that we so desperately need to partner with. And it’s a phone call. And it’s a taking them out to lunch. And it’s a, Okay. You’re booked on all of our meetings. Then come here and we’ll do a lunch meeting. And I’ve got sandwiches and we make it happen. And we’ve launched two projects that way. Over a lunch…But you can’t do that in a large county, it just doesn’t work. It does for us. And we work really hard to keep those relationships established.

In other cases, collaboration was attributed to the easy access and physical proximity among collaborators in small counties: “I mean, we can walk to each other’s offices, as you can see, you know. Uhm, it’s easier to bring everybody together. We’re not spread out and, you know, in larger counties you’ve got some people way over here and some people way over here” (mental health director).

Shared Clients

Collaborations across counties and across agencies within the same county were also facilitated by the presence of “shared clients” in sufficient numbers to make the implementation of the EBP cost-effective. As explained by one child welfare director, “Families who come into our systems, via Mental Health, Probation, Child Welfare, even our Drug and Alcohol Services, uhm, you know, pick any of those doors, and it’s the same family.” It was not surprising to find the same children being simultaneously served by multiple county agencies. Whether to avoid the duplication of similar services and/or address the need for integrated services to address systemic problems, it was the recognition by systems leaders of these shared clients that often led to collaborative efforts or desire for more collaboration between the agencies that they represent. As one child welfare service director stated,

All of these kids, any kid that’s in any one of our systems, is really all of our kids because it’s by chance, bad luck, or happenstance that they landed in Probation versus Child Welfare. And the quicker we agree on that, then the better off we’re going to be in the long run in terms of, you know, blending our funding stream,…taking on innovative initiatives or new ways of doing business.

Government Mandates

Mandates served as both facilitators and barriers to collaboration. Collaboration across agencies within the same county was facilitated by federal, state and local government mandates that required such collaboration to occur. Policies and mandates to improve child welfare services and probation practices around out-of-home care call for self assessments, peer review, and system improvement processes that involve collaboration across these two agencies in each county. Some leaders identified these specific mandates as what prompted collaboration between their agencies. However, conflicting mandates across these agencies also created barriers to inter-agency collaboration. One child welfare director explained, “You know, everybody will always still sort of have their mandates or their place that they have to sort of say, This is all I can do.” Another child welfare service director commented on the difficulty to collaborate with the Regional Centers due to their different mandates surrounding services eligibility.

Characteristics of the Participating Organizations

The third set of characteristics that distinguished implementation collaborations pertained to the organizations represented in the collaborations. These characteristics included features of the ideology or organizational culture and features of the members of these organizations. The first set of features included a common language, common recognition of the problem to be addressed, common goals and values, a buy-in and commitment to innovation and change, and policies and procedures designed to ensure transparency and accountability. The second set of features included the existence of interpersonal relationships and social ties, presence of an individual who could serve as a broker or advocate for the EBP, leadership that was supportive of the implementation, and participants possessing qualities of honesty, credibility, trust and respect for others.

Common Language

Effective communication with and between agencies was described as essential for a successful collaboration. This included the use of clear and precise language that delineated each partner’s expectations, and also entailed formal training that would facilitate the use of a common language across agencies. As one chief probation officer explained,

…we had to learn some different language, talking about milieus and this and that. But, any time our staff were involved in any type of collaborative with our Children’s Mental Health, we always have combined training, so that we can come up, come up with a common language, so that the various, uh, schools of thoughts, or disciplines, understood one another, as opposed to…We call our people ‘probationary’, they call theirs ‘clients’, you know, and things like that.

Common Recognition of the Problem

Interagency collaboration was often contingent on there being a common recognition of the problem and an agreement on the issues to be solved. This entailed having an understanding of other agencies, their issues, and their needs. For example, as one chief probation officer noted, “I think there has to be agreement as to what the issues are. That [if] we’re all going to bring people together, we all have to recognize that there’s, there is a problem,…and that it, it mutually affects all of us, in one way or the other.”

Common Goals and Values

A common set of values and beliefs within and across agencies facilitated inter-agency collaboration. Often expressed as similar philosophy, a shared commitment to make a difference, or working together for the common good, these values were sometimes attributed to individuals, while in other instances attributed to the agencies themselves. In either case, these values were described as the basis of the common goals and priorities to meet the needs of kids. For example, a chief probation officer stated, “I think that, philosophically, we share a common belief about…, that we really do want to see our kids succeed. And we really do want to have better communities and families. We want to make sure that families…not only are we providing services, but we help them become self-sustainable. So, I think we all share those beliefs.” This sharing does not happen by accident but involves the active development of a common culture within an organization. According to the chief probation officer of another county, “You can establish relationships, but to make them long-term you just really have to make sure that you bring in supervisors and managers that understand what you’re trying to do, and this is something [that] the agency values, and this is the direction we want to go in. So, fostering a culture that supports that.”

Commitment

Buy-in and commitment within and between agencies were frequently mentioned as requirements for successful collaboration at different levels. As one child welfare services director explained, “if you’re going to get groups together to collaborate, well you have to have the will and the commitment to do this job.”

Accountability

Accountability to insure that the division of labor among collaborating agencies was equitable was also cited as an important requirement for successful inter-agency collaboration. As explained by one of the child welfare services directors, “we want a collaborative where everybody’s involved. Everybody has accountability. And everybody has contributions.” Some study participants preferred to enforce accountability through memoranda of understanding and written documents; others viewed accountability as a motivation for performance: “rather than using it as a hammer, I try to look at it as a motivation, you know, This is what you do. Why don’t you get credit for it?” (child welfare services director).

Interpersonal Relationships and Social Ties

Social ties and long-term relationships amongst systems leaders led to increased collaboration between their respective agencies. As noted above, this was particularly evident among systems leaders in small, rural counties. However, regardless of the size of a county, collaborations between agencies were occasionally based on individuals in each agency having worked together as peers while in other positions. For instance, one child welfare services director attributed the success of her agency in collaborating with the behavioral health agency director in her county to implement an EBP to the fact that both had worked together as therapists in the latter agency.

Relationships and social ties among systems leaders were also found to be associated with increased support for EBP implementation. For example, a mental health director stated,

We have a new partner now…who is the executive director that came out of Drug and Alcohol. She’s been a friend of mine for years. So, all of these people, well, I would say I have, you know, personal alliances and commitments to. I’m there to support them when they need me. They’re there to support me. We have the same philosophy, whenever we want to try something new…

Furthermore, these relationships allowed for some flexibility in the interpretation of mandates, thus promoting collaboration. As explained by one of the study participants, “I think just kind of working well together and knowing each other allows you to kind of go, Well, okay. We’ll, not really bend the rules, but we can soften the, the ridges a little bit” (child welfare services director).

Collaboration Broker and/or Facilitator

Systems leaders often mentioned a specific individual or type of person that served as a key player in the collaborations in which they have been involved. In some instances, that individual introduced necessary key players to one another. Examples of collaboration brokers included California Institute of Mental Health consultants or an influential county opinion leader. In other instances, this key person served as the collaboration facilitator, who coordinated the necessary meetings and had experience with facilitating the collaboration’s group work processes. According to one of the child welfare services program managers,

One of my personal frustrations…over the years that I’ve been in this work is being put on committees or collaborations, or whatever, where all they do is get together and process the same stuff over and over again. So, I think you need a skilled facilitator, or facilitators, who say, Okay, what are we trying to accomplish today? And what are the steps? And who’s going to commit to it? So, you have to have someone who is really good with group dynamics, and with knowing how to get things done. Or else people get bored, frustrated. And there are so many demands on their time that they stop coming and stop caring. I think that’s something that’s really important in a collaboration.

Supportive Leadership

Agency leadership that was dedicated and supportive of collaboration was cited by participants as a critical element of successful collaborations at all levels between agencies. In establishing a vision and developing and maintaining an organizational culture within their own agencies, “the agency directors have to set the tone for collaboration, working together” (chief probation officer). In addition to a commitment to successful collaboration, other traits identified as characteristic of supportive leadership included clarity in communication, constant nurturing of staff engaged in the project, a willingness to take risks, and a willingness to give up some administrative control.

Honesty, Credibility, Trust, and Respect

Finally, the personal qualities of honesty, credibility, trust and respect, even in the presence of disagreement or controversial decisions, marked a successful collaboration according to agency leaders. As expressed by one of the chief probation officers, “You know, when you think about any relationship, you’re hoping that there is honesty and there’s trust. So, those are critical to collaborating and establishing any relationship.” These qualities gradually developed over time and were largely based on the experience of working with one another.

Collaboration Characteristics and Influence Networks

As evidenced by many of the examples and illustrations provided above, each of the three sets of characteristics is linked to one another in important ways. Thus, characteristics of the external environment such as availability of funding and existence of common clients was associated with more exchanges of information and resources to change patterns of services delivery and improve outcomes for at risk youth. Size of county and existence of state mandates could lead to increased or decreased exchanges depending on whether there was sufficient information or resources to exchange or whether the mandate required or prevented the exchanges from occurring. All of the characteristics of organizations and individual members of these organizations were similarly associated with increased exchanges of information and resources. Frequency of exchanges also appeared to be associated with a broader focus, less formality, and greater frequency of interactions.

All three sets of characteristics, in turn, appear to be associated with characteristics of influence networks. For instance, with respect to characteristics of the collaboration, smaller but more connected (as measured by level of in-degree centrality) networks appeared to be associated with the collaboration processes of a broader focus, less formality, and greater frequency of interaction. The external environmental characteristics of available funding and state mandates were associated with large networks in some instances and small networks in other instances. Smaller counties were associated with smaller networks, while existence of common clients was associated with larger networks. The organizational characteristics of a common language, recognition of the problem, goals and values, commitment, and accountability and the individual characteristics of interpersonal ties, brokering or advocacy, supportive leadership, and personal qualities of honesty, trust, and respect were all associated with larger and more connected networks.

Discussion

In a previous study (Palinkas, Holloway, et al. 2011), we demonstrated a significant association between the structure of influence networks involving individuals from different agencies in the same county and agencies in other counties and the stage of implementation of evidence based practices like MTFC. That study found that county size (i.e., larger counties having more information and resources to exchange) and level of in-degree centrality (i.e., being nominated more frequently by others in a network) were independently associated with the stage of MTFC implementation two years later. In this study, our analysis of information obtained from interviews with leaders of county child welfare, mental health and juvenile justice systems in California revealed collaboration between organizations to be a critical factor in the development and maintenance of these influence networks. In contrast with some studies that found inter-organizatonal collaborations had little or no effect on program outcomes (Chuang and Wells 2010) or may even hinder achievement of successful outcomes (Glisson and Hemmelgarn 1998), we found no instance of where such collaborations hindered the acceptance, adoption or routine use of new EBPs, and several instances of where such implementation could not have been achieved in the absence of the identified characteristics of inter-organizational collaboration. Consistent with the definition of collaboration provided by Hemmelgarn et al. (2001), the collaborations described by these systems leaders were for the purpose of exchanging information, sharing resources, altering behavior, and enhancing capacity to serve at risk youth. They also reflected different degrees of focus, formality and frequency that ranged from statewide (e.g., California Mental Health Directors Association) entities that met once a quarter to county-wide consortiums that met once a week to individual systems leaders who may have limited interactions for the purpose of obtaining or providing specific information or a specific resource.

In this study, we also found that the success of efforts to create such collaborations and the extent to which these collaborations were successful in developing and maintaining the influence networks instrumental in implementing EBPs were dependent on a set of characteristics of the collaboration’s external environment and of the participating organizations and individuals. Borrowing from the implementation models of Greenhalgh et al. (2004) and Aarons et al. (2011) to describe similar sets of factors that influence implementation outcomes, these latter two sets of characteristics constitute the “outer context” and “inner context” of inter-organizational collaboration. The outer context includes the availability of funds and clients necessary to introduce and sustain the EBP, size and proximity of collaborating entities and their administrative jurisdictions, state and local mandates requiring collaboration between agencies, and responsibility for the same group of clients or consumers of provided services. The inner context includes the characteristics of the organizations, including a common language, recognition of problem, goals and values, commitment and accountability, and their individual members, including interpersonal relations and social ties, willingness to serve as a broker or EBP advocate, willingness to exercise supportive leadership, and reputation for honesty, trust, and respect. Our results suggested that these characteristics were, for the most part, associated with increased levels of exchange of information and resources and with larger and more connected networks. However, we also found instances in which the same characteristic (e.g., county size, availability of funding, existence of state mandates) was associated with more or less exchange or with larger or smaller networks. Further research is required to account for these differences across different collaborations operating in different contexts.

As with the networks themselves, the implementation model of Greenhalgh et al. (2004) also places inter-organizational norms and values within the outer context of implementation. In contrast, the inner context of the model of inter-organizational collaboration introduced here includes two sets of characteristics that represent an “organizational culture of collaboration.” As defined by Cooke and Rousseau (1988), organizational cultures are the implicit norms, values, shared behavioral expectations, and assumptions of a work unit that guide behaviors. “It is ‘the way things are done around here,’ as well as the way things are understood, judged, and valued” (Davies et al. 2000, p. 112). As with cultural systems in general, organizational cultures are comprised of sets of shared understandings, arranged in hierarchical order, that serve as models of as well as models for behavior (Palinkas et al. 2005), in this case the behavior that drives the process of collaboration (i.e., focus, formality, frequency and function) and creates and sustains influence networks. An organizational culture of collaboration is shaped by the existence of a common language, common recognition of the problem to be addressed, common goals and values, a buy-in and commitment to innovation and change, and policies and procedures designed to ensure transparency and accountability. It is also shaped by external factors like the availability or lack of availability of funding, state and local mandates and a common client base, as well as internal factors like the presence of individuals who serve as “brokers”, supportive leadership, and social networks that link organizations and their members. The collaboration is the product of the intra- and extra-organizational cultures; in turn, the collaboration also influences the development and evolution of these cultures. Thus, while organizational cultures at both levels may influence the outcomes of implementation efforts (Greenhalgh et al. 2004; Aarons et al. 2011), their ability to exert such an influence is mediated by the collaboration processes and the structure of the influence networks that are created and sustained by these processes.

A number of limitations to the study design should be kept in mind when evaluating these results and their implications. As a qualitative study, both collection and interpretation of data is susceptible to subjective bias and preconceived ideas of the investigators. However, use of multiple coders as well as multiple sources of data (i.e., systems leaders in the same county) to achieve “triangulation” (Denzin 1978) was designed to minimize such bias. Finally, as others have noted (Glasgow et al. 2003), as the processes described were examined in the context of a randomized clinical trial that evaluates the effectiveness of a strategy designed to promote effective collaborations between agencies, the results may not be generalizable to the experience of evidence-based treatment implementation that do not involve RCTs, or may generalize differently depending on the core mediators of the treatments. Further research is required to empirically validate the linkages between the three sets of characteristics in other implementation contexts that do not involve participation in randomized clinical trials. Nevertheless, while there were no specific collaboration requirements for participating in the CAL-40 study, such collaboration was viewed by agencies in many of the study counties as essential to the funding of MTFC. Moreover, the Community Development Team intervention is designed to promote collaboration. Although this specific context may have led to an emphasis on the role of inter-organizational collaboration in implementing MTFC, our data were based on the agencies’ experience in implementing any new EBP in the past year and thus not limited to the specific context of an RCT. Further research is also required to determine whether the collaboration processes found in this study vary based on whether the individuals or agencies involved in the collaboration are early adopters of EBPs or have access to individuals or agencies that have a reputation for being EBP early adopters.

Conclusion

Collaboration between agencies in public youth-serving systems is critical to EBP implementation because it enables an exchange of information and advice and a pooling of resources that individual agencies may require for successful implementation. Although inter-organizational collaborations may vary based on their focus, degree of formality, and frequency of interaction, they are shaped by characteristics of the external environment and of participating organizations and their individual members. All three sets of collaboration characteristics, in turn, shape the structure of influence networks that determine the extent to which implementation moves from dissemination to sustainment.