Introduction

Commonly referred to by many names (e.g., collaboratives, coordinating councils, partnerships), community collaboratives have gained prominence as a vehicle for addressing complex community issues that call for an appreciation of the realities of shared power, responsibility, and interdependencies amongst a diverse network of organizations (e.g., Brown 2000; Bryson et al. 2006). Collaboratives are community-based groups comprised of leaders and staff representing both nonprofit and for profit organizations as well as public agencies and community groups who share a common focus (e.g., mental health services, domestic violence, HIV/AIDS); some include consumer representatives as well. These groups meet regularly to identify and implement strategies for improving their community’s collective response to their targeted issue(s) (Foster-Fishman et al. 2001).

To date, there has emerged a substantial base of research focused on examining the effectiveness of community collaboratives at impacting community/population level outcomes (e.g., Halfors et al. 2002; Roussos and Fawcett 2000) as well as identifying the pre-conditions and characteristics that promote the success of these efforts (e.g., Zakocs and Edwards 2006; Allen 2005; Nowell 2009). Increasingly, researchers, practitioners, and funders are recognizing that effective collaboration is difficult to promote and that a sole focus on community level outcomes—such as population level changes in health behaviors—may mask other important benefits of collaboratives (Merzel and D’Affitti 2003; Roussos and Fawcett 2000). As a result, there is a growing trend in research on community collaboratives to assess their effects on community capacity because these impacts are theorized to be an important intermediate step towards the larger goal of broader community or population level changes (e.g., Allen et al. 2008; Green and Kreuter 2003). Community capacity refers to increases in the ability of community members and institutions to effectively resolve community problems (Butterfoss and Kegler 2002). While still a relatively new framework for assessing collaborative effectiveness, there is a growing body of evidence to suggest that effective collaboratives promote community member skills (Kelger et al. 2007), system linkages (Provan et al. 2005), and systems changes such as shifts in organizational policies and procedures (e.g., Clark et al. 2010). Overall, these changes are critical, given the importance they play in promoting a healthy community (e.g., Florin et al. 2000).

The purpose of this study is to expand on this examination of potential intermediate outcomes for community collaboratives. Building on the notion that collaboratives build local capacity, we investigate the ways in which collaboratives are perceived to strengthen the capacity of the organizations and agencies who participate as members. At this level, the focus is not on how the existence of the collaborative has impacted other organizations or institutions within the community, as is often the focus of studies that target community capacity. Rather, we focus on the ways in which the collaborative has affected the organizations who participate as members.

Interest in the ways in which collaboratives serve to increase the organizational capacity of their members is growing for multiple reasons (Butterfoss et al. 1996; Provan et al. 2005; Provan and Milward 2001). First, there is increasing evidence that improvements in community or population level outcomes depend upon fostering adequate organizational-level capacity. For example, Clark et al. (2010) found that systems changes including organizational and interorganizational improvements within asthma-related institutions corresponded with population-level improvements in asthma symptoms in children and parents’ attitudes about their child’s asthma relative to comparison communities. Second, accrual of organizational member-level outcomes may promote the sustainability of the collaborative through increases in members’ commitment (Butterfoss et al. 1996; Provan et al. 2005). Creating significant community-level change takes time (e.g., Walsh 1998). To the extent that the desired change requires a sustained commitment from members of the collaborative, identifying those factors that promote sustainability of the collaborative becomes paramount to promoting long term success. Last, failure of a collaborative to provide substantive benefits to members has the potential to weaken member organizations by diverting scarce resources away from primary programming (Bailey and Koney 2000).

Despite scholarly interest in the potential of collaboratives to serve as forums for fostering the organizational capacity of members, there is a dearth of research which systematically investigates the nature of impacts on participating organizations as a targeted intermediate outcome of community collaboration. The few studies of community collaboratives that have considered organizational benefits that may accrue through participation (e.g., Florin et al. 2000; Kegler et al. 2007; Mansergh et al. 1996) have tended to utilize a more narrow a priori conception of impact and/or treated member benefits as an independent variable, not the targeted outcome of interest. Consequently, to date, we lack an empirically-grounded understanding of how member organizations view the benefits of their participation in a collaborative, what types of benefits are most prominent, and who benefits most.

Given the importance of organizational capacity building as an intermediate outcome of collaboratives, it is also important to explore whether and how these benefits are distributed among members. Because collaboratives bring together a diverse set of organizations, outcomes may not be experienced equally among participating organizations. Characteristics of the organizations and their representatives such as organizational sector, depth of involvement, and the representative’s management level, experience, and expertise may shape how organizations are affected. For example, scholars have noted that different sectors are affected by different types of resource dependencies and institutional constraints which may influence the benefits an organization gains from a collaborative (e.g.,DiMaggio and Anheier 1990; Rainey and Bozeman 2000; Herranz 2007). In addition, greater levels of engagement and active involvement of higher level executives suggests a different level of organizational commitment to the collaborative and thus may impact the nature of outcomes. Further, human capital has been recognized as an important element of organizational capacity (Nahapiet and Ghoshal 1998). To the extent the collaborative positively impacts organizations through providing representatives access to networking and information sharing opportunities, organizations represented by individuals with less experience and expertise may have the most to gain from the collaborative. Last, it is important to consider how experiences within the collaborative may affect member outcomes. For example, recent theory and research in leadership suggests that, even in a context of shared leadership, there can be ‘in-groups’ and ‘out-groups’, with some representatives being more central and some more peripheral to the power structure of the group (for discussion see Seibert et al. 2003). Those in the ‘in group’ may experience leadership and decision making as more positive and shared relative to those in the ‘out group’. Therefore, representatives who have more positive evaluations of the collaborative’s leadership and decision making practices may themselves have had experiences that led to disproportionate benefit relative to their peers.

The aim of this study is to expand our concept of effectiveness in collaboratives by qualitatively developing a framework for conceptualizing dimensions of organizational capacity that can be enhanced as an outcome of collaborative membership. We then quantitatively examine and validate the factor structure of these dimensions and explore the ways in which—and extent to which—the characteristics of organizations and their representatives explain variation in the degree to which organizations are affected by involvement in the collaborative. This inquiry is guided by the following three research questions:

  1. (1)

    In what ways do representatives perceive their organizations are impacted as a result of their participation in a community collaborative?

  2. (2)

    What are the dimensions of impact?

  3. (3)

    To what extent and in what ways are organizational and representative characteristics associated with degree and type of impact?

Study Context and Research Design

The above research questions were explored in a mixed-methods study of 51 multi-sector Midwestern collaboratives formed to improve their communities’ responses to domestic violence. These collaboratives are predominantly county-based entities comprised of representatives and leaders from an array of criminal justice and human service agencies/organizations (e.g., police, prosecutors, judges, advocates, health providers, shelter providers), as well as community groups invested in addressing domestic violence within their community (Sullivan and Allen 2001). Their intended function is to reduce the prevalence of—and improve the community’s response to—incidences of domestic violence by increasing the level of coordination and collaboration among community stakeholders.

Consistent with a developmental mixed-methods design (Greene et al. 1989), the present study consisted of both exploratory and confirmatory research methods carried out in two phases. In phase 1, we conducted qualitative interviews with key informants to explore the ways in which organizational capacity was affected through participation in a collaborative. This phase culminated in the development of a three dimensional framework for conceptualizing organizational capacity as an intermediate outcome of collaboratives. In phase 2, we developed a measure of organizational capacity based on our qualitative findings that was then administered to 614 organizational representatives across 51 collaboratives. This phase focused on quantitatively validating—and if necessary modifying—the factor structure of the proposed outcome framework and exploring how characteristics of organizations and their representatives relate to the nature and extent of benefits received from participation. In light of the multi-phased nature of the research design, each phase will be presented in sequence.

Phase 1

Methods

Sample

Open-ended interviews were conducted with 15 key informants representing five different stakeholder groups (law enforcement, domestic violence service providers, prosecutors, courts, and batterer intervention programs) prominent within DV collaboratives. The sample for these interviews was attained from referrals by state level informants with the objective of obtaining a diverse sample of information-rich cases of individuals (Patton 2002) who could speak to the range of impacts participation in a DV collaborative can have on the organizational capacity of members. Collectively in their careers, this group of informants had experience working with over 20 different DV collaboratives. On average, informants had 10 years of experience being involved with DV collaboratives (range 3–18 years).

Procedures and Analysis

Semi-structured phone interviews were conducted with key informants. Informants were asked to describe the different ways in which involvement in a collaborative enhances or reduces the capacity of involved organizations. Informants were also asked to discuss the participation costs and benefits to their own organization. Transcripts were analyzed for quotes related to organizational impacts associated with participation in the collaborative and quotes identifying like organizational impacts were assigned a common code. All interviewing and preliminary coding was conducted by the first author. Several iterations of peer debriefings (Lincoln and Guba 1985) were then conducting between the first and second author. During these meetings, data output was examined by the second author for conceptual congruency between the raw quotes and the assigned code and its description. Incongruencies, overlaps, and alternative interpretations were discussed between the first and second authors until consensus was reached. Codes were organized into a framework consisting of three broad categories of impact that we felt parsimoniously captured the nature of impact represented in the data (see Fig. 1). Findings were member checked with community partners to confirm the face validity of the framework. This framework served as the measurement model that was investigated during the second phase of analysis using confirmatory factor analysis.

Fig. 1
figure 1

Conceptual framework of organizational impacts

Full size image

Results

Key informants described an array of ways in which involvement in the collaboratives had enhanced the capacity of their organizations or agencies. We found that organizations participating in DV collaboratives experienced three types of mutually reinforcing outcomes: increased knowledge and awareness, expanded social capital, and heightened opportunity and impact (see Fig. 1). Each of these will be discussed in greater detail below.

Increased Knowledge and Awareness

Collaborative endeavors are often valued for their role in facilitating the dissemination of information (Bailey and Koney 2000). An organization’s ability to obtain information that can help them predict and respond to their environments has been identified as a key element of capacity (e.g., Chen and Grady 2010). Key informants both validated and clarified this claim, noting that participation in DV collaboratives increased their organization’s knowledge and awareness in three important ways. We describe each of these impacts below.

Issue awareness

Informants described how the collaboratives enriched awareness and understanding of the complex dynamics associated with domestic violence. This included helping participating organizations to develop a more holistic understanding of domestic violence and how to more effectively intervene within this problem area. For some organizations, this broader perspective was described as helping staff to better appreciate the rationale behind the actions of their clients, which in turn reduced frustration and boosted motivation.

For the most part, when we [prosecuting attorneys] have a crime victim, we’re trusted more or less. We’re the knight in shining armor. We don’t generally have an armed robbery victim telling us, ‘I don’t trust you and I’m not going to testify.’… We do have that in cases of violence against women. We need to understand that perspective… [The collaborative] helps to give that perspective…the level of frustration goes down.—Prosecuting Attorney

Informants noted that this increased awareness also informed how to best carry out their organization’s work.

[One impact of the collaborative is] how much better we understand the issue of the batterer. By attending the meetings …we were able to learn and understand the tactics of batterers and then explain those tactics to the victims. It helped to provide better services and counseling to the victims because of inside knowledge about batterers—DV Service Provider

System Awareness

Informants reported that participation also fostered a greater understanding of the DV service system and their organization’s role within that system. For example, informants described how the collaborative helped their organizations to become familiar with other agencies and to learn how the whole DV response system operates.

The most effective thing [about the collaborative] is that everybody gets together and understands what each can do within the system… How other agencies are reacting to and taking care of the issue of domestic violence… [For example,] what types of services they contribute, the types of people who are in those agencies, what is the focus of the agency—Prosecuting Attorney

Informants noted the collaborative was particularly valuable in helping participating organizations appreciate the interdependencies between multiple organizations and services, and in gaining feedback about how their actions can affect and are affected by the actions or policies of others. Some noted that this knowledge helped to reduce interagency conflict and improve system functioning and service delivery.

It gives me an understanding of what’s on the other side of the fence. Deputies handle the call and then we’re done with it. We never know if problems arise from what we’ve done. So, through the [collaborative], I get that feedback. And I’m able to sit down with my command staff and say ok, we had this problem on these last few calls,… any ideas? And we come up with a smoother way to deal with this and this does nothing but help the victim—Law Enforcement

Additionally, informants described how knowledge gained via membership in the collaborative enhanced their organization’s ability to make sense of the actions of other participating organizations through understanding the limitations and challenges they face.

It helps other stakeholders understand what happens when a person is placed on probation—what we can and cannot do. It gives the other stakeholders a sense of the limitations of the probation office as well—what are the problems that we have to face as far as the victims are concerned—Probation Office

Information Access

Informants also described how participation in the collaborative gave their organization better access to general information and feedback that could inform their work. This included information about events, access to community data (e.g., the percent of cases that have resulted in arrest), case specific information (e.g., more ready access to police reports), and ideas for improving service delivery.

We identify the need for something and we bring it to the meeting. We get suggestions and ideas and [other members] will say ‘do you know so and so and what they are doin?’…so you get information too so it helps you with the assessment of what the best way to go about implementing it and I think our program has grown because of exactly that kind of input…—DV Service Organization

Social capital

The importance of social capital as an element of organizational capacity has become nearly axiomatic in the organizational literature (e.g., Knoke 2009; Leana and Pil 2006). The term social capital is commonly evoked to describe the value within social-structural relationships that an “actor” such as an individual or an organization can mobilize to make possible the “achievement of certain ends that would not be attainable in its absence” (Coleman 1995, p. 302). However, the characteristics of relationships that constitute social capital often vary by context (Leenders and Gabbay 1999). Informants described social capital in this context, in part, as responsiveness and cooperation from other stakeholders in the absence of any formal authority or structure to mandate such cooperation.

Relationships are important because they are all I can count on—I have no control over a judge or prosecutor. I have to do what I can to get people to listen to what we say. The only leverage that you have within the system is through the relationship with these people—DV Service Provider

You couldn’t get any of this done without [relationships]because there is no accountability structure—if I didn’t show up to the meetings, nothing would really happen to me…—Prosecuting Attorney

Collaboratives have often been described as vehicles for building social capital (e.g., Foster-Fishman et al. 2001; Hardy et al. 2003) and indeed, informants reported that relationships were fostered and strengthened through their involvement in the collaboratives. Informants reflected primarily on two inter-related outcomes created through these relationships. First, informants described their organization’s involvement in the collaborative as providing the opportunity to enhance their organization’s reputation among other participating organizations, resulting in a heightened recognition of their organization as being both respectable and trustworthy. Informants also described increased social capital in the form of other organizations being more accessible to them and responsive to their organization’s needs or concerns.

Access is an opportunity—to have access to prosecutors and judges—to have their respect. We’ve built that through the [collaborative]… If I call a judge they are going to return my call. [My organization] is respected in the area. This is a result of [the collaborative] and our work throughout the community—DV Service Provider

Enhanced Organizational Opportunities and Impact

Lastly, informants talked about impacts that linked specifically to organizational performance in terms of enhanced core capacities such as the ability to attain resources, solve problems, serve clients, and carry out their mission (Sowa et al. 2004). Each of these will be discussed in detail below.

Access to/Capacity to Attain Resources

Informants described their involvement with the collaboratives as playing a role in helping their organization access financial and other resources. For example, some informants spoke of increased competitiveness in getting grants due to their ability to obtain letters of support for their projects. Other informants spoke about their organization benefiting from projects initiated through the collaborative.

Our intensive supervision program was able to be a benefactor of the [grant money received by the collaborative]—Courts

Enhanced Ability to Resolve Problems and Get Things Done

Involvement in the collaborative was also described by informants as something that facilitated their organization’s ability to solve problems and achieve goals. The combination of regular meetings creating a forum for informal problem solving, the opportunity to develop a more comprehensive awareness of the issues, and a chance to foster collaborative, trusting relationships were all described as the mechanisms that led to this advantage.

The biggest and most important thing, three and a half years ago, these people didn’t talk. Nobody wanted to talk to each other and or work with each other. Now the relationships are so wonderful and it makes it very easy for everybody. For example, a woman came into the shelter- she had previously went to police who had done nothing. The shelter contacted the DV prosecutor who then contacted police and the batterer was arrested that same day. Batterers Intervention

Improved Ability to Serve Clients and Fulfill Mission

Ultimately, involvement in the collaborative was described as improving the ability of participating organizations to carry out their respective missions. For some this was described as helping organizations broaden their mission and/or expand their services as a result of the availability of new resources and support.

With the [collaborative], we gained the opportunity to run the program in the jail—it gave us funding to create, develop and facilitate a program for men who are incarcerated—Batterer Intervention

Another informant described being able to draw upon the collective influence of the group which allowed their agency to have a greater impact than they would have had working alone.

For the criminal justice field, an agency thinks of something it wants to do but if you don’t have some other agencies helping you to work on it, it probably won’t do well. In the community corrections, we had an intensive supervision probation program for drunk drivers and there was an idea of extending intensive supervision to include domestic violence. …The program was put together and pitched to the Board of commissioners and they didn’t warm up to the concept—probably because they didn’t want to spend the money. The other agencies kind of rallied to this idea and got some other agencies involved and re-issued this to the Board of Commissions and ended up getting it passed—Courts

Increased Utilization of Services

Informants also linked increases in the utilization of their organization’s services to participation in the collaboratives. Some informants described how networking and developing relationships with other participating organizations in the collaborative helped to increase referrals and access to their services. As one DV service provider stated, “They call us—agencies call us [more] and get us involved in cases”.

Influence into Important Decision Processes

Lastly, informants described how belonging to the collaborative gave their organization access to a forum to participate in important decision processes from which their agency may have previously been excluded. For example,

For new programs that are being developed in the criminal justice program like the DV court—instead of reading about it in the newspaper, we have an opportunity to be at the table and discuss it and there may be some negative that we want to point out and if it’s a good idea rather than having a judge make the decision and announce it. We’re at the table to discuss it. DV Service Provider

Negative Impacts

Surprisingly, when asked what negative impacts their organization had experienced as a result of involvement in the collaboratives, eight of the 15 informants indicated no negative impacts. Several other informants noted only the time requirement involved. However, some informants noted detrimental impacts related to the potential downside of strengthened relationships. In particular, one participant noted that relationships entailed a heightened level of accountability to other stakeholders.

(The collaborative) made the organization open to critique that then had to be dealt with, which was fine. If there is something we’re not doing that we should be doing, then we need to do it—Courts

Another informant described the potential for lines of confidentiality to be crossed.

The relationships with everybody, the lines have become blurred and the issues of confidentiality have become a problem. [For example], talking to the prosecutor about a specific case—Batterers Intervention

Another described how members could be perceived by outsiders as unduly influenced by their associations with others in the collaborative.

[There’s] the possibly of the perception that we’re in the pocket of an advocacy group- which we’re not—but this was a problem in the past. Prosecutors want to be independent of everybody—police, judges, etcetera; we don’t want to be beholden to any particular group no matter who they are—Prosecuting attorney

Lastly, one informant described the potential for the work of the collaborative to compete for grants with service providers.

One potential negative is to try to be competitive for grant money with service providers—we made a rule early on that the most important thing was the direct service providers and that the Council would not go for money that would compete with direct service providers. We always felt that our job was to enhance direct services—Courts

PHASE II

Methods

The goals of this phase of the research were twofold. The first goal was to develop a measure based on the qualitative findings from Phase 1, then quantitatively validate and if necessary, modify, the dimensions of impact resulting from the conceptual framework identified during Phase 1 using confirmatory factor analysis (CFA). The second goal was to explore these dimensions as separate aspects of organizational capacity, examining the relationship of each as it related to characteristics of organizations and their representatives.

Participants

The sample for this phase was recruited from 51 DV collaboratives located within a single Midwestern state; nested within each of these collaboratives were members. For the purposes of this study, “members” are conceptualized at the organizational level because individuals participating in these collaboratives do so as representatives of a particular organization, agency, or group. The term “representative” is used to refer to individuals who attend the collaborative’s meetings on behalf of a given organization, agency, or group. While the collaboratives consisted predominantly of organizational representatives, a few collaboratives also included individuals who were not representatives of an organization, agency or group. These individuals were omitted from the present analysis.

Current membership in the collaborative was based on membership rosters and confirmed during interviews with leaders. In the event there was more than one individual representing the same organization or agency on a collaborative, the one most active in the collaborative was selected as the representative of their organization in the study. Collaboratives, on average, had existed for approximately 10 years but ranged in their ages from 2 to 21 years old. Collaboratives ranged in size from 6 to 61 organizations with an average size of 16 organizations per collaborative. Within the 51 collaboratives, data were collected from representatives of 614 organizations with an average within-group response rate of 86%.

On average, participants had been involved with their collaboratives for approximately 5 years and had been with their organizations for approximately eleven and a half years. Participants attended an average of 65% of the meetings over the past year and 16.6% held some type of leadership position within the collaborative. Approximately 40% were male, and overall, the sample was quite educated with approximately half having obtained a Masters degree or higher. There was good representation across levels of management, with slightly greater numbers of mid-level administrators/coordinators (40%) relative to directors (30%) or program staff (30%). Approximately 30% of participants identified as nonprofit and 58% as public agencies with the remaining sample identifying as for profit or community based groups.

Measures

Organizational capacity was measured based on a scale developed for this study. Scale items were drafted to capture the key themes of organizational impact identified in Phase 1. Data from Phase 1 were summarized in a set of short statements capturing the nature of impact communicated (e.g., organization gained credibility among the other organizations). These short statements were organized into a table by dimension (e.g., social capital). In order to increase the generalizability of the indicators, a review of literature on the proposed benefits of collaboration for organizations was conducted. Statements in the literature (e.g., participation in collaborative networks can help organizations gain legitimacy and acceptability in the community; Provan and Milward 2001) were compared against statements of impact described by participants to inform indicator wording. This process resulted in a scale consisting of 20 items by which participants rate the extent to which their organization has been impacted as a result of participation in the collaborative. Items are summarized in Table 1; each indicator was measured using a five-point Likert-type scale. The resulting scale was piloted with 30 organizational representatives from two DV collaboratives in an adjacent state to confirm the face validity of the indicators. Pilot participants reported items were understandable and relevant in the context of DV collaboratives. Four sub-scales of organizational capacity were calculated based on factor scores resulting from the final CFA solution. Each sub-scale served as a separate dependent variable in subsequent modeling.

Table 1 Organizational capacity indicators
Full size table

Sector was operationalized as four categories: public, private, nonprofit, or community-based group. Representatives’ management level was operationalized as three categories: director, mid-level administrator/coordinator, project/program staff. Given the exploratory nature of this study, depth of involvement in the collaborative was operationalized in a couple of different ways including: percent of meetings the representative attended in the past year, the organization’s tenure with the collaborative, and whether or not the representative held a leadership position in the collaborative. Similarly, measures of a representative’s experience and expertise included their number of years with their organization, the number of years a representative had worked in the field of domestic violence, and their level of education.

Participant evaluation of leadership and decision making practices within the collaborative was measured using a scale comprised of items adapted from previous research (alpha = 0.95). Specifically, the scale asked participants to rate the extent to which they felt the collaborative’s leadership was effective at managing interpersonal dynamics (3 items; Allen 2005; Weiss et al. 2002), accomplishing instrumental tasks (3 items; Allen 2005) and inspiring vision and transformation (9 items; Bass 1985). It also measured the extent to which participants felt decision-making within the collaborative was: shared among members (5 items; Allen 2005); reflexive (6 items; Carter and West 1998; De Dreu 2002); synergistic (2 items; Weiss et al. 2002), and innovative (2 item; Anderson and West 1998).

Control Variable

In order to examine the effect of the representative’s experience with the decision making and leadership practices, average differences between collaboratives in overall leadership and decision making quality were controlled for. Mean scores for leadership and decision-making were calculated separately for each participant and then aggregated to the collaborative level where they were averaged as a measure of the overall leadership and decision making quality of the collaborative.

Procedures

Data were collected primarily via a web-based survey; however, a paper–pencil version was made available to participants who either lacked comfort with or access to the web-based format. Descriptive data about the participants and their organizations were also collected via the survey format.

Analysis

Data analysis consisted of two stages. In the first stage, a series of CFAs were conducted in Amos to validate the outcome framework proposed based on the qualitative findings summarized in Fig. 1. A CFA is a type of structure equation model which can be used to validate the fit of an a priori factor structure to a set of data (Kline 1998). In determining goodness of fit for the models, the following fit indices were examined: CMIN, IFI, TLI, and RMSEA, AIC, PNFI.

Because the predictors of interest for this study contained both categorical and continuous variables, a combination of ANOVA and OLS multiple linear regression methods were utilized to examine member characteristics. Because members are nested within collaboratives, a one way ANOVA was run for each outcome to determine whether a hierarchical approach which controlled for group differences was necessary (Bliese 2001). There were no significant between-collaborative differences for any outcome measure. This is understandable given all outcomes are member-level rather than group- level outcomes. The lack of significant differences between collaboratives indicates that member benefits are not uniform within a collaborative.

Results

As discussed, CFA was used to examine each indicator’s performance in relationship to the hypothesized latent dimensions of organizational capacity as well as to test whether the theorized three dimension (i.e., factor) framework was a superior fitting model to the data relative to contrasting models. Results supported conceptualizing organizational capacity as a multi-dimensional construct; fit indices for the initially proposed three factor model indicated improved fit over the single factor null model and adequate to good fit overall. However; two indicators of the latent variable ‘opportunities and impact’ had comparatively low standardized estimates, suggesting a lower correlation between the latent and measured variables. Both of these indicators concerned acquisition of tangible resources; specifically, the ability to compete for grants and other funding opportunities, and access to more or different types of resources such as office space, staff, or equipment. Because of the theoretical relatedness of these two indicators, the original three dimension framework was modified to include resource acquisition as a separate dimension. Changes in fit indices generally indicated the modification from a three to a four factor model was an improvement in model fit (see Table 2).

Table 2 Comparative model fit statistics for confirmatory factor analysis
Full size table

The four factor model was adopted for use in all subsequent modeling. Of the 20 indicators modeled, 16 have standardized weights of .75 or higher. Reliabilities calculated using Cronbach’s alpha based on the factor scores all indicate good internal consistency with the exception of resource acquisition (see Table 3). This is potentially indicative of the limited number of items and the construct of “resource acquisition” being more additive in nature. For example, increased access to one type of resource (i.e., grant competitiveness) may not be highly predictive of access to another (office space) but the extent to which both have occurred for a given member is still conceptually valuable. The overall pattern of fit statistics for the four factor model suggests adequate fit. Chi-square indicates poor fit however, this index is commonly recognized as unrealistically stringent in models based on sample sizes exceeding 200 cases. IFI and TFI indicate good fit and the RMSEA indicates acceptable fit.

Table 3 Organizational capacity sub-scale reliabilities
Full size table

Frequency distributions for all four outcomes indicated normally distributed variance. Comparison of mean differences between the four outcomes indicated that, on average, participants perceived the greatest benefits with regards to increases in social capital and knowledge and awareness. Consistent with our qualitative framework which posited that dimensions of organizational capacity would be mutually reinforcing, there was significant correlation among the four outcomes. Collectively, these findings suggest that while there is empirical support for examining impacts to organizational capacity as a multi-dimensional construct, these dimensions are not orthogonal. Non-weighted means, standard deviations, and minimum/maximum scores for each scale are as follows: Awareness 3.57 (0.79; 1–5), Opportunity and Impact 3.17 (0.87; 1–5), Social Capital 3.57 (0.86;1–5), and Resources 2.85 (1.11; 1–5). The subscale correlations are presented in Table 4.

Table 4 Correlations among organizational capacity subscales
Full size table

Negative Impacts Resulting from Membership In The Collaborative

Participants were also asked whether their involvement in the collaborative had resulted in any negative impacts to their organization or agency. Only 2.6% of respondents indicated that involvement in the collaborative had any negative impacts on their organization. Participants who responded affirmatively were asked to provide a qualitative description of the negative impacts that resulted from their membership. Two themes emerged from these qualitative responses. The first was a perceived lack of respect within the collaborative for one’s organization. For example, one participant commented “There is a lack of respect and consideration for my agency’s point of view and work”. Consistent with the interview findings, another theme related to having one’s organization or agency more exposed to scrutiny by other organizations.

Exploring Association with Member Characteristics

The final phase of analysis was to explore propositions concerning the relationship of characteristics of member organizations and their representatives to each outcome.

Sector

Interestingly, there were no significant differences between public, for profit, nonprofit, and community-based groups for awareness building (F [3, 589] = 0.148, P=0.931); social capital (F [3, 585] = 1.205, P=0.307), or enhanced opportunity (F [3, 578] = 0.546, P=0.651). There was a trend toward significance for resource acquisition between private for profit and community-based groups (F [3, 562] = 2.323, P = 0.074). This suggests that all sectors are equally likely to benefit from involvement in a collaborative.

Management Level

The level of management was a significant predictor of awareness building (F[2, 565] = 4.23, P = 0.015), as well as increased social capital (F[2,560] = 3.49, P = 0.031). Post hoc analyses indicated that directors reported significantly greater increases for both outcomes relative to program staff. There were no significant differences between levels of management in opportunities (F [2, 554] = 1.96, P = 0.141), nor resource acquisition (F [2, 538] = 0.74, P = 0.477).

Depth of Involvement

ANOVA analyses found that representatives who held a leadership position in their collaborative reported significantly greater impacts associated with increased knowledge and awareness (F[1, 589] = 6.18, P = 0.013), social capital (F[1, 584] = 8.98, P = 0.003), and opportunity (F[1, 578] = 5.80, P = 0.016). There was a trend toward significant impact for resource acquisition (F[1, 562] = 3.06, P = 0.081). Multiple regression analyses for continuous variables (See Table 5) indicated a similar trend. An organization’s tenure with the collaborative was significantly associated with all four outcomes. Representative attendance was a significant predicator of increases in social capital and enhanced opportunity and impact. Attendance also showed a trend toward significance in explaining increased knowledge and awareness.

Table 5 OLS multiple regression analysis
Full size table

Representative Experience and Expertise

Representatives who were newer to their organizations were significantly more likely to report greater benefits in increasing social capital as well as to report heightened opportunities and impact. However, neither the representative’s level of education nor the number of years a representative had worked in the field of domestic violence were significant in predicting impact.

Evaluation of Leadership and Decision Making

Representatives who evaluated leadership and decision making practices more positively reported significantly greater impacts for all four outcomes.

Discussion

While community collaboratives have emerged as a prominent vehicle for fostering a more coordinated community response to complex issues (Brown 2000), research to date suggests that the success of these efforts at achieving their targeted ultimate outcomes—population-level improvements in health behaviors for example—is mediated by improvements in organizational and community capacity (e.g., Clark et al. 2010). As a result, researchers and practitioners are increasing their focus on the intermediate outcomes accomplished by these entities, such as shifts in the environment or broader community system (e.g., Allen et al. 2008). This study makes an important contribution to this area of inquiry by expanding our understanding of the potential intermediate outcomes achieved by community collaboratives. Specifically, this study provides initial evidence to suggest that involvement in a community collaborative builds the capacity of participating organizations and that these capacities appear to improve the targeted service system’s ability to achieve its goals. More importantly, it offers a qualitatively grounded and quantitatively validated framework for conceptualizing the dimensions of these capacity-building outcomes and provides insights into some potential negative impacts that may result. Insights into which organizations are most likely to gain these results from their involvement in a collaborative are also provided.

Results suggest that organizational capacity building outcomes resulting from involvement in a collaborative can be conceptualized into four related, yet distinguishable dimensions: (1) knowledge and awareness, (2) social capital, (3) opportunity and impact, and (4) resource acquisition. Correlations among factors suggests that development in one area of capacity may facilitate development in other areas of capacity as well. An investigation into the co-evolution of different elements of capacity over time is an interesting area for future research. Representatives reported the most impact with regards to increases in social capital and awareness building, suggesting the greatest value organizations get from involvement in a collaborative comes from insights and knowledge gained and relationships fostered.

The findings from this study suggest that collaboratives foster awareness in several important areas, including a more in-depth understanding of the targeted issue and a more comprehensive awareness of the system that is in place within their community for addressing this issue. While this study was limited to domestic violence collaboratives, these findings support the benefits discussed by other collaborative researchers and theorists (e.g., Bouwen and Taillieu 2004). The development of a framework for conceptualizing and measuring these intermediate outcomes makes an important contribution to both practice and future research. Without adequate understanding of how members are impacted as a result of their participation in collaboratives, leaders may miss out on valuable opportunities to track and foster these valuable intermediate outcomes. The effect of these intermediate outcomes on more long term outcomes of interest such as member retention, sustainability, and population level changes, and community resiliency is fertile ground for future research.

The finding that collaboratives in this study were able to promote both issue and system awareness has particularly significant implications for practice. A community system is comprised of a set of actors, activities, and settings that are directly or indirectly perceived to have influence in or be affected by a given problem area (Foster-Fishman et al. 2007). By definition of a system, these actors, activities, and settings are interdependent—creating both the conditions for potential synergy as well as the possibility of destructive interference. In order for community systems to operate effectively in addressing complex problem areas, system members need to collectively appreciate that they are part of a system. This requires each organization to understand their role in the context of the overall community response and how their actions impact and are impacted by other system members. Unfortunately, historical trends that privilege greater differentiation and specialization of practices among organizations have encouraged soloed approaches to service delivery and communities have traditionally provided limited supports to help organizations adopt a more systemic perspective to their work. Scholars in the systems sciences have noted the importance of feedback in enabling complex systems to self-regulate and adjust to changing conditions (Mingers 2006). Findings from this study suggest that community collaboratives can be effective mechanisms for providing this kind of system feedback to member organizations, helping them to understand on an on-going basis how they affect and are affected by the actions of other system members.

Another important finding of the present study is that while participating organizations differed in the extent to which they reported increased capacities as a result of their involvement, very few reported any negative impacts. Nonetheless, results did illuminate some potential negative effects that are worthy of consideration. First, collaboratives increase connectivity among organizations. Such connectivity has been identified as an important element of resilient communities (Allenby and Fink 2005; Chaskin 2008). However, scholars have warned that increasing connectivity can have a dark side as well, potentially serving as an organizational liability (O’Toole and Meier 2004). Key informants spoke to this risk in noting that the collaborative can open an organization up to greater scrutiny by other system members. In addition, for some organizations or agencies for which objectivity, independence and neutrality are critical to their institutional legitimacy, the real or perceived affiliations with other organizations within a collaboration may need to be navigated carefully. Lastly, to the extent that collaboratives choose to seek out grants or create programs, leaders should be aware such efforts may create an additional source of competition for individual organizations.

In addition to identifying what kinds of organizational capacity outcomes and negative impacts may result from involvement in a collaborative, findings from this study help to illuminate some characteristics that may be associated with receiving the greatest benefit out of one’s membership. Importantly, findings suggested that all sectors (public, nonprofit, private, unincorporated community groups) have relatively equal potential to experience the positive benefits on all dimensions of impact. However, there were several conditions under which representatives were more likely to report greater outcomes. For example, executive directors reported significantly greater outcomes related to social capital and knowledge and awareness relative to program staff. This may be explained by the fact that organizational capacities such as organizational reputation, systems awareness and issue awareness have particularly strong implications for higher-order strategic concerns that tend to be the domain of higher level executives in shaping their organization’s practices. It could also be that the groups tended to privilege those in higher positions.

Findings from this study further suggest that depth of participation is a significant factor in the degree of impact associated with participation in a collaborative. Specifically, when organizations had been with the collaborative longer and when representatives held leadership positions and attended meetings more frequently, participants reported greater outcomes related to expanded knowledge and awareness, increased social capital, and enhanced opportunities and impact. This is consistent with hypothesized ‘dosage’ theory of community interventions (Wandersman and Florin 2003) and stands to reason that more intense involvement in the collaborative would provide greater opportunity for the above outcomes to occur.

Importantly, findings from this study also highlight the value of considering the representative’s unique experience with the leadership and decision making practices in the collaborative. When representatives experienced their collaborative’s leadership and decision making practices as effective, they also reported greater positive impact to their organization as a result of their participation in the collaborative. The effect of the representative’s unique experience was significant but the aggregated group-level evaluation of leadership and decision-making quality was not. This indicates that representatives within the same collaborative are not uniform in their experiences of the leadership and decision making practices within the collaborative.

Organizational literatures have begun to examine the heterogeneity of experience of group members nested within the same settings. In team-based structures like collaboratives, leadership is often fluid (Alexander et al. 2001) and representatives may differ in the degree of influence they hold within the group. For example, leader-member exchange (LMX) theory focuses on the unique interpersonal relationship that evolves between leaders and members. Research has found that the quality of the relationship between leaders and a given member is significantly related to job satisfaction (Epitropaki and Martin 2005) and organizational commitment (Martin et al. 2005). Consistent with the tenets of LMX, our research suggests that members of a collaborative do not experience the same level of positive outcomes and representatives’ unique experiences in the leadership and decision making practices of the collaborative are a key factor in understanding these differences.

Lastly, a liability of newness perspective (Hannan and Freeman 1984) suggests representatives who were newer to their organizations, relative to their more tenured counterparts, may lack the reputational legitimacy and insider knowledge necessary to effectively navigate the community system surrounding a complex problem area. Findings provide some support for the proposition that collaboratives would have greater effects on organizations represented by individuals who are less tenured. This supports the notion that community collaboratives may serve as an important mechanism for integrating organizations with less tenured staff into the community network, helping them to build up their professional reputation within the community and providing a venue for problem solving issues that arise as they learn the ropes in their new positions.

Limitations

Interpretation of results from this study must take into consideration several limitations. First, both qualitative and quantitative reports of organizational outcomes are based on self-report from organizational representatives. This design requires representatives who attend meetings on behalf of their organization to serve as a key informant concerning the ways in which and extent to which their organization has been affected as a result of the collaborative. In collaboratives, organizations are represented by individuals and those individuals serve as a primary conduit through which organizational benefits accrue (Manring 1998). Given these representatives are in the unique position of having insider knowledge of both the collaborative and their respective organizations, they offer an important perspective on how their organization has been affected. However, the extent to which and process through which impacts have diffused through the layers of the organization and become institutionalized is largely ambiguous in the present study and an important area for future research. In addition, self-report methods suffer from error variance at the individual level that can inflate correlations between measures. Concerns over common-method bias are more limited in this study as most independent variables were relatively objective descriptors (age, organizational tenure, management level) which are least affected by common-method bias. Representatives’ evaluations of leadership and decision making practices carries the greatest threat for common method bias. As such, the proportionally greater beta for this variable relative to the other predictors should be interpreted with caution.

Lastly, it is important to note that findings from this study were based on data collected from one prominent type of collaborative focused on domestic violence. While collaboratives across different substantive areas share many similarities in terms of their structure, internal processes, and function, they also differ in important ways such as the broader policy contexts within which they operate, the composition of their members, and the focus of their collaborative work. The extent to which and ways in this framework generalizes to collaboratives in other substantive areas remains an interesting empirical question and an important area for future research.

In conclusion, the findings from the present study suggest that community collaboratives are a context for promoting the capacity of participating organizations. When organizations send representative leaders from their organizations and become actively involved in the collaborative over a longer period of time, they are likely to see increases in their social capital, knowledge and awareness, opportunities and impact, and resource acquisition. In other words, through their participation in a community collaborative, organizations appear to gain a variety of benefits that can build their organizational effectiveness. Collaboratives should work to increase the outcomes associated with organizational participation and reduce the costs to ensure that they engage a diversity of stakeholders in the resolution of entrenched community problems.