In recent decades, multicomponent community-based interventions targeting health outcomes have become a prominent vehicle for addressing a variety of public health issues. The earliest of these large prevention-oriented interventions focused on cardiovascular disease, and included the Stanford Three-Community (Stern et al. 1976) and Five-City (Farquhar et al. 1990) projects, the North Karelia Project (Puska et al. 1985), the Minnesota Heart Health Program (Luepker et al. 1994), and the Pawtucket Heart Health Program (Carleton et al. 1995). These projects suggested that multicomponent preventive interventions implemented in broad segments of the community could have measureable and sustained impacts on population health indicators (Winkleby et al. 1996).

Over time, the use of community-based strategies was extended to other behavioral health issues, including substance abuse prevention. Lessons learned from early trials in which highly prescribed interventions were implemented with tight control, such as the Midwestern Prevention Project (MPP) (Pentz et al. 1989) and Project Northland (Perry et al. 1996) led to subsequent interventions in which community-based organizations, often operating as coalitions representing multiple sectors from within the community, were directly involved in planning and executing these projects (Roussos and Fawcett 2000; Wandersman and Florin 2003). These projects include both researcher-initiated studies conducted primarily to test the effectiveness of specific approaches, as well as several large-scale initiatives to which evaluation components were added. Given the widespread promotion and popularity of coalition-based projects, it is safe to say that community coalitions now play a central role in the dissemination and implementation of substance abuse prevention efforts across the US. Community Anti-Drug Coalitions of America (CADCA) reports that over 5000 coalitions are members of its national network.

Are Community Coalitions Effective?

Although coalitions are a popular structure for delivering substance use prevention interventions in community settings, evidence for their effectiveness has been inconsistent. Evaluation results from large-scale substance abuse prevention initiatives involving community coalitions such as Fighting Back (Hallfors et al. 2002), A Matter of Degree (Weitzman et al. 2004), Center for Substance Abuse Prevention (CSAP)’s Community Partnership Program (Yin et al. 1997), SAMHSA’s Drug Free Community Support Program (ICF International 2012), the Enforcing Underage Drinking Laws (EUDL) program (Wolfson et al. 2011), and various state-level evaluations of the CSAP’s Strategic Prevention Framework (SPF) (Flewelling et al. 2005; Collins et al. 2007; Florin et al. 2012), have been mixed, with positive findings often limited to a subset of the targeted outcomes or to selected subgroups only. Results from researcher-initiated projects that included a focus on underage drinking are similarly mixed. Noteworthy examples of rigorously evaluated programs involving community coalitions include the Community Trials Project (Holder et al. 2000), Communities Mobilizing for Change on Alcohol (Wagenaar et al. 2000), Reducing Youth Access to Alcohol (Flewelling et al. 2013), PROSPER (Spoth et al. 2013), and Communities that Care (Hawkins et al. 2014). Collectively, the findings substantiate the common critique that, although popular, the evidence for the effectiveness of community-based substance abuse prevention coalitions is highly variable and far from definitive (Wandersman and Florin 2003; Flewelling et al. 2005; Saxe et al. 2006; Institute of Medicine 2012).

Characteristics of Effective Community Coalitions

Given these mixed findings and the continued ubiquity of community coalitions in substance abuse prevention, it is important to identify and understand the factors, both internal and contextual, that contribute to a community coalition’s success in achieving its objectives. The term “capacity” has been used to broadly characterize organizational attributes that collectively reflect a coalition’s ability to be effective. Indeed, building state and community capacity was one of the primary goals for the massive federal SPF State Incentive Grant (SIG) initiative. As briefly summarized below, efforts to delineate and measure characteristics of community coalitions that reflect their capacity are well represented in the literature. Research that empirically examines the associations between attributes used to define coalition capacity and desired changes in health behavior outcomes, however, are extremely scarce.

It is worth noting that coalition capacity is a different construct than the more general concept of community capacity. The latter has also received considerable attention in the literature and pertains to a variety of community-level attributes that transcend any individual organization (Goodman et al. 1998; Edwards et al. 2000; Chilenski et al. 2007). For the purposes of the current study, we differentiate coalition capacity from community capacity insofar as the former refers to characteristics of the organization(s) within the community charged with coordinating the implementation of specific prevention activities, whereas the latter refers to characteristics of the broader community in which the intervention is implemented.

A variety of definitions for coalition capacity have been proposed. CADCA described coalition capacity as “the various types and levels of resources that an organization or collaborative has at its disposal to meet the implementation demands of specific interventions” (Community Anti-Drug Coalitions of America 2005, p. 41). CADCA then identified four essential components of coalition capacity: participation and membership, leadership, cultural competence, and organizational management and development. Numerous related and sometimes more elaborate conceptualizations of coalition capacity have been presented in the literature (e.g., Chinman et al. 2005; Zakocs and Edwards 2006; Butterfoss and Kegler 2009; Yang et al. 2012; Nargiso et al. 2013; Brown et al. 2012; Shapiro et al. 2013). Commonly cited attributes across these and other perspectives include organizational structure, clearly stated mission and goals, formalized procedures, efficiency of operations, inter-organizational connections and communications, level of member participation, member skills, member diversity, effective leadership, group cohesion, community outreach, and resources. Distinctive features emerge from each of the various treatises as well. For example, Wandersman et al. (2008) differentiate general capacity from innovation-specific capacity, a theme that is further developed and tested by Nargiso et al. (2013). Likewise, Brown et al.(2012) identified member participation benefits and member participation difficulties as important and empirically distinct features of community coalitions.

Regardless of the specific framework, it appears to be generally agreed that capacity is a multidimensional construct. Attributes believed to be important determinants of coalition capacity have been based largely on conceptual justification, in some cases supported by applying knowledge about organizations in general to community prevention coalitions. Furthermore, research has focused primarily on process-oriented, or intermediate, outcomes as indicators of coalition effectiveness in examining the influence of coalition attributes. Specific outcomes examined include the following: success in implementation of annual strategic plans (Kegler et al. 1998), creating system change (Hays et al. 2000), perceived effectiveness by community leaders (Florin et al. 2000), implementing programs and/or policies (Zakocs and Guckenburg 2007), and adoption of evidence-based programs (Brown et al. 2010; Shapiro et al. 2015). In the largest such study identified in the literature, Yang et al. (2012) analyzed data from 551 substance abuse coalitions and found that coalitions with higher levels of capacity (defined as use of essential decision-making processes, development of quality planning products, and expanded coalition membership) pursued more comprehensive strategies than those with lower capacity levels and reported greater implementation of new programs and policies.

Studies that have examined the effects of coalition capacity on distal outcomes such as health behaviors or indicators of population-level health status are far less common. In their review of the literature on coalition effectiveness, Zakocs and Edwards (2006) identified studies published between 1980 and 2004 that examined the effects of coalition attributes and/or implementation activities on various indicators of effectiveness. Of the 26 that met the inclusion criteria, 19 measured various aspects of coalition functioning (i.e., process-oriented outcomes) as indicators of effectiveness, while only 3 examined changes in community-wide health behaviors or health status. Unfortunately, those three studies (Hallfors et al. 2002; Weitzman et al. 2004; SAMHSA 2000) all focused on measures of the quality or intensity of intervention implementation, which are more indicative of intermediate outcomes than underlying organizational attributes reflecting capacity, and therefore did not directly address the underlying question that is the focus of this paper.

Community Coalition Capacity in the Context of the SPF SIG

In 2004, CSAP introduced its Strategic Prevention Framework (SPF), which is a planning and implementation model for states and communities consisting of five key steps: needs assessment, capacity building, strategic planning, implementation of evidence-based programs, policies, and practices (EBPPPs), and evaluation. To promote adoption of the SPF for substance abuse prevention efforts, CSAP initiated the SPF SIG program. Each state, territory, or tribal government receiving an SPF SIG was expected to conduct the five SPF steps at the state level and then again in selected communities. CSAP funded 21 SPF SIGs in 2004 (cohort I) and an additional five in 2005 (cohort II). Additional awards (cohorts III, IV, and V) have been made in subsequent years. This initiative has now provided funding to all 50 states and a number of other jurisdictions including US territories and tribal governments, which in turn have funded community-based organizations, primarily coalitions, to build their prevention capacity and then plan and implement prevention strategies locally. An explicit goal of the SPF SIG was to build prevention capacity and infrastructure at the state and community levels.

The SPF SIG initiative also emphasized the importance of achieving population-level reductions in targeted substance use behaviors among the funded communities. Although states varied with respect to how prescriptive they were in specifying the interventions to be implemented by community grantees in pursuing this goal, most states either required or at least emphasized the use of evidence-based and environmental strategies. Guidance to states and communities regarding the criteria for determining that an intervention is evidence-based was provided by Center for Substance Abuse Prevention (2009), although the criteria are admittedly broad and open to interpretation. Although it is unknown how carefully the criteria were applied, 69 % of all interventions were reported by the community grantee coordinators as being evidence-based. With respect to the types of interventions implemented, process data collected from cohorts I and II states (Buchanan et al. 2010) showed that 44 % of all interventions implemented by funded communities were classified as environmental strategies, followed by information dissemination (16 %), preventive education (16 %), community-based processes (14 %), and alternative drug-free activities (8 %). In addition to considering the research evidence for the effectiveness of interventions, other criteria used by communities in selecting interventions included their capacity to implement them, costs, and fit of the interventions with the demographics of the community. The mean number of interventions implemented per community was five.

The SPF SIG and its national evaluation provide a unique opportunity to examine attributes of community coalitions and whether these attributes contribute to their effectiveness in reducing targeted outcomes. The SPF SIG may have been the first large-scale national substance abuse prevention initiative to fund community-based prevention efforts that also put a strong emphasis on the collection and cross-site analysis of outcome measures representative of the populations in the covered communities. The purpose of the study is to examine data from a relatively large N of community coalitions, collected as part of the SPF SIG cohorts I and II cross-site evaluation, in order to (1) assess whether and to what extent coalition capacity increased during the timeframe of their SPF SIG funding and (2) whether aspects of coalition capacity are correlated with success in reducing selected substance abuse outcomes, in this case, measures of underage drinking.

Methods

Description of Sample

A total of 452 community-based grantees in 26 states provided data for the cross-site evaluation. Of those, 318 grantees in 24 states defined themselves as community-based coalitions, and were responsible for planning and implementing SPF SIG-funded intervention activities in their communities. It is these 318 coalitions that provide the base sample for the present analyses. Other grantees were typically single community-based agencies such as community-based non-profit social service organizations (32 %), local government offices (24 %), or hospitals and other community health care providers (23 %). For assessing the associations between capacity measures and outcomes, the sample was further limited to coalitions that both targeted underage drinking and provided pre-and post-intervention outcome measures for either any past 30-day alcohol use (N = 129) or past 30-day binge drinking (N = 100) by students in some or all grade levels 6 through 12. Underage drinking was the most common priority selected across SPF SIG states and communities and therefore was the best choice of outcomes for providing a sufficient sample size for the planned analyses. The 129 coalitions providing outcome data on any alcohol use were drawn from 13 states, with the number of coalitions per state ranging from 1 to 19. The 100 coalitions providing data on binge drinking were from 10 states. Most of these coalitions (88 coalitions from nine states) were also in the N = 129 sample. The additional 12 coalitions were from one state that provided outcome measures on binge drinking but not any alcohol use.

Data Sources and Measures

Coalition attributes

Community grantees submitted extensive process evaluation information twice a year for 3 years, for a total of six rounds, starting in the spring of 2008, through a Web-based tool referred to as the Community Level Instrument (CLI). Data obtained through the CLI included background items about the grantee and process data pertaining to each step of the SPF. Most coalition capacity measures analyzed for this study were part of the “coalition sub-form,” which contained capacity measures specific to community coalitions and was completed by grant coordinators who self-identified their organizations as coalitions.

Table 1 provides the capacity measures used in the analyses. The measures have been organized according to the broad category (or domain) of organizational capacity to which they appeared to pertain conceptually. Also shown in the table are the labels used in subsequent tables to identify the measures, the wordingFootnote 1 of the specific CLI items used, and the response options and corresponding numerical values used in the analyses. Three items (coalition needs more structure to be effective, there is not enough follow-through, coalition does not monitor whether there is follow-through) were reverse coded so that higher values represent higher capacity for all items used in the analyses.

Table 1 Capacity measures and measurement scales
Full size table

For each community, a baseline and follow-up value was determined for each item. Baseline values were defined as the value from the first round for which a valid response was provided, as long as a valid response was also provided in at least one subsequent round. Likewise, follow-up values were determined by using the value from the last round for which data were available, provided that community provided a valid response on at least one previous round. As noted in the table, exceptions were made for three items that reflect activities that could vary from one round to another based on project needs and therefore are not necessarily expected to show progress over time. These items were converted to cumulative counts of the number of unique response options provided across all rounds (e.g., the number of different community groups targeted for raising awareness at any time throughout the project). The absence of valid responses to items for any particular round was due primarily to either community grantees not yet being operational at the time data were due, or community grantees not self-identifying as a coalition for one or more rounds and therefore not being directed to the coalition sub-form. The majority of coalitions (61 %), however, provided capacity measures for all six rounds of the CLI.

Outcomes

The community-level outcome data analyzed for this study were derived from student surveys conducted in middle and/or high schools. Because the survey data in many states are not publically accessible, or require specialized knowledge and procedures to extract the data, the cross-site project relied on the SPF SIG evaluators for each state to obtain, assemble, and submit their state’s community outcome data. The surveys that generated the data are typically used by states and communities for a variety of purposes, and are generally designed to provide estimates at either the community or school district level. As such, most are based on large samples or censuses of students and are well suited to provide community-level outcome measures as needed for evaluation of initiatives such as the SPF SIG. For the 141 communities contributing outcome data for the analyses reported here, the mean student-level response rate for the student surveys (defined as the actual number of respondents divided by the target sample size) was 73.6 %.

Outcome measures in the form of prevalence rates were reported either annually or biannually, depending on the student survey used. The majority of states employed student surveys that are conducted every 2 years. The student survey measures were based on items inquiring about any use of alcohol in the past 30 days (any use) and about whether five or more drinks were consumed on a single occasion within the past 30 days (binge drinking). Slight variations in the wording of the items were noted across the various surveys used, although many of the states used standard items from national surveys such as the Youth Risk Behavior Survey (YRBS). The measures were aggregated across years in order to produce a single pre-intervention data value and a single post-intervention value for each community. Pre-intervention years were defined as the two most recent survey years up to and including the year in which community residents were first exposed to SPF SIG intervention activity, and post-intervention years were defined as the first two survey years following the first year of exposure to SPF SIG intervention. For some communities, however, only a single survey year was available for calculating the pre-intervention or post-intervention values. The average time interval between pre and post measurements, calculated as the difference between the pre-intervention year and post-intervention year midpoints, was 2.9 years (SD = 0.85) for any alcohol use and 3.0 years (SD = 0.94) for binge drinking.

Change scores for the two outcomes of interested were calculated as the pre-intervention prevalence value minus the analogous post-intervention value, thereby making higher positive change scores desirable. Across the 129 coalitions targeting underage drinking that had pre- and post-intervention measures for any alcohol use, the change scores ranged from −12.2 (i.e., a 12.2 percentage point increase) to 15.0 (a 15.0 percentage point decrease). The mean value was 2.9 (SD = 5.3). For binge drinking (n = 100), the range was from −9.1 to 11.7, with a mean of 1.2 (SD = 5.4). Both measures exhibited approximately normal distributions with only modest levels of skewness (−0.39 and −0.58, respectively) and kurtosis (0.38 and 0.30).

Analytic Procedures

The analyses employed were designed to address the two primary goals of the study previously specified. For the first goal, which was to assess whether coalition capacity increased over time, the baseline and follow-up mean item values are reported for each capacity measure for which an assessment of change over time was appropriate. A change score was then calculated for each capacity measure, defined as the difference between the mean follow-up and mean baseline values (i.e., follow-up minus baseline, thereby again ensuring that positive values reflect desirable change). To determine the statistical significance of the changes over time, we first examined the within-community correlations between the baseline and follow-up values for each measure. Because all correlations were at least moderate, ranging from 0.47 to 0.78, the use of a paired t test was appropriate for this purpose.

To test the significance of the relationships between capacity items and reductions in alcohol use and binge drinking prevalence rates (study goal 2), we employed a mixed model regression approach using SAS Proc MIXED, with state as a random effect to accommodate non-independence of community-level measures within state. The post-test capacity item scores were used as the predictors for these analyses, as they were assumed to better reflect coalition capacity levels during the period when intervention activities were at peak levels of implementation (i.e., generally in the last year of the grants). An initial analysis was conducted in which each capacity item was examined in a separate model with no covariates, thereby providing an overview of the bivariate associations between the items and the outcome measures. We also examined the effect of an overall composite capacity score, calculated as the mean of the standardized values across all the individual capacity items as shown in Table 1. Although numerous bivariate relationships were tested both for this analysis and the pre-post change analysis described previously, the purpose of these analyses was to explore the overall pattern and strength of the relationships observed rather than definitively assess the statistical significance of any particular association. For that reason, no adjustment for the number of tests conducted was applied to the significance levels reported.

We then ran models assessing all capacity measures simultaneously in order to identify features that independently predicted each of the two outcomes. To make the models more parsimonious and avoid potential multicollinearity issues, we first conducted a factor analysis, using principle component extraction and varimax rotation, to identify subsets of items in which the items were highly interrelated. The results of the analysis, conducted using all 318 coalitions, supported the conceptual domains into which the items were initially categorized as shown in Table 1, with three exceptions. First, “paid coalition leadership” was categorized as a separate single-item construct because it was only mildly correlated with the other leadership item and with the other capacity items. The other leadership item (“coalition has collaborative leadership”), was assigned to the organizational structure construct. Second, items associated with mission/vision, organizational structure, and tracking and follow-through were all moderately inter-correlated (with correlations ranging from 0.33 to 0.58) and loaded on the same factor. We therefore combined these domains into a single construct we labeled “internal organization and structure.” Third, because the number of key community partners and total number of partners were very highly correlated (r = 0.86), only the number of key partners was used to calculate the construct score for community connections and outreach. Scores for the six constructs were calculated as the mean value of the standardized items comprising each construct.Footnote 2 As anticipated, the correlations among the six resultant capacity construct scores were all relatively small, the highest being 0.25, indicating that no further consolidation of capacity items was necessary. For each outcome measure, scores for all six capacity constructs were entered simultaneously into a mixed model regression. Community population size, represented as the square root of the total population in order to reduce the influence of large outliers, was included as a covariate. Due to the exploratory nature of these analyses, “marginally” significant effects (i.e., p < 0.10) are noted as such in the text. To achieve greater precision in the estimates, the full models were then reduced by removing predictors using a stepwise elimination procedure if they did not attain a significance level of at least p < 0.10.

Results

The baseline and follow-up mean scores and associated standard deviations for each capacity item, across the 318 community grantees that self-identified as coalitions, are presented in Table 2. The items have been grouped according to the six revised constructs as explained earlier. Also shown are the change scores, t test statistics, and p values. The mean capacity score across the coalitions moved in a favorable direction from baseline to follow-up for all items examined. Additionally, the change was significant at the p < 0.05 level in all cases except for the percentage reporting that the leader of the coalition was a paid position. Although changes in the mean values for the items based on 5-point rating scales appear to be modest, changing only one or two tenths of a point, the relatively small standard deviations for these items and strong pre-post correlations contributed to the finding that even small shifts in perceptions were statistically significant.

Table 2 Changes in capacity item values over time (n = 318)
Full size table

Before assessing the associations between capacity measures and outcomes, we examined whether the 141 coalitions included in these analyses differed from the 177 coalitions that either did not target underage drinking or did not provide the necessary outcome measures needed for the analysis. Coalitions included tended to serve smaller communities, as the mean population size of these communities was 71,000 residents, significantly lower than 137,000 for the communities not contributing to the analysis (p = 0.004). With respect to coalition capacity constructs, those included in the analysis were less likely to have a paid leader (p = 0.022) and also had marginally higher scores on community connections and outreach (p = 0.053). Differences on the other four capacity constructs were negligible and statistically non-significant.

With respect to the bivariate associations between capacity items and outcomes, 29 of the 32 associations examined were positive, thereby depicting a fairly consistent pattern of positive associations between higher capacity and larger decreases in current and binge alcohol use over time. Of the three negative correlations, all were small and non-significant, whereas eight of the positive associations were statistically significant at the p < 0.05 level and five more were marginally significant (p < 0.10). The composite capacity score was a stronger predictor of reductions in both any alcohol use and binge drinking than any of the individual measures, significant at the p < 0.001 level for both outcomes.

Summaries of both the full and the reduced mixed model regressions for each of the two outcome measures examined, using the six construct scores along with population size as predictors, are provided in Table 3. The full model for predicting reductions in any alcohol use prevalence identified three statistically significant predictors. As expected, higher scores on internal organization and structure (p = 0.039) and community connections and outreach (p = 0.011) were both related to greater reductions in any alcohol use. Contrary to expectation, data infrastructure, as measured by the number of different data sources used for assessment purposes, had a negative effect on reductions on any alcohol use prevalence (p = 0.029). The significance levels for these effects did not change appreciably and no additional statistically significant effects emerged after dropping constructs that were not significant at the p < 0.10 level.

Table 3 Full and reduced model results for predicting reductions in any alcohol use and binge drinking prevalence rates
Full size table

For predicting declines in binge drinking rates, two predictors exhibited a marginally significant positive effect in the full model: internal organization and structure (p = 0.099) and funding and sustainability (p = 0.088). In the reduced model, the significance probability level for internal organization and structure improved to p = 0.031.

Although community population size was positively associated with declines in both alcohol use and binge drinking, it was not a statistically significant predictor for either outcome. Finally, even though statistically significant predictors were identified in both models, the percentage reduction in the error variance attributable to the predictors included (which has an interpretation similar to R 2) was 10.8 % for the any alcohol use reduced model and 7.6 % for the binge drinking reduced model.

Discussion

The findings presented here addressed two distinct research questions. The first, regarding whether coalition capacity in SPF SIG-funded communities increased over the life of their projects, reflected the explicit goal of the SPF SIG initiative to increase prevention capacity in states and communities. Previously published findings from the cross-site evaluation of SPF SIG cohorts I and II have demonstrated that state-level capacity did increase during the grant periods (Orwin et al. 2014) and continued to show improvement 1 year following the conclusion of each grant (Edwards et al. 2015). Findings from the study reported here confirmed that capacity of community-based coalitions also increased during the timeframe in which they were funded. The enhancements were surprisingly robust in the sense that significant increases in mean capacity scores across the 318 coalitions were observed for 15 of the 16 items used to tap various aspects of coalition capacity. The single capacity item that showed basically no change was the yes/no item regarding whether the coalition leadership was a paid position. This made sense given that the funding from the state agencies to the community grantees was generally fixed over the life of the community grants, which in turn encouraged grantees to structure their budgets similarly in each year of their grant. Unfortunately, no further data were collected from coalitions after the conclusion of the SPF SIG grants, so it is unknown whether these coalitions maintained their increased levels of capacity after their SPF SIG funding ended as was the case for the state-wide prevention systems.

The second question addressed in this study was whether, and to what degree, measures of coalition capacity are independently associated with the reductions achieved in two population-level measures of the most commonly targeted outcome, underage drinking. One notable feature of the findings addressing this question was the pattern of consistently positive bivariate relationships observed. Disregarding significance levels, 14 of the 16 associations between capacity items and reductions in any past 30-day use of alcohol, and 15 of the 16 associations of these items with binge drinking, were positive. Even though many of these associations were individually not statistically significant, the overall pattern of almost all associations examined being positive was suggestive of an overarching, if modest, association between coalition capacity and success in achieving desired behavioral changes in the target populations. This inference was further supported by the highly significant associations observed between the composite capacity measure and each of the two outcomes.Footnote 3

The regression modeling results were helpful in identifying certain aspects of coalition capacity that are associated with reductions in underage drinking prevalence. In order to do so, it was first necessary to combine individual measures into empirically supported composites. In particular, the data supported the concept of internal organization and structure as being a key coalition attribute comprising multiple interrelated features such as clarity of its mission, collaborative leadership, diverse membership, fair delegation of responsibilities, and procedures for ensuring follow through of plans. This construct was the only one found to be a significant predictor of reductions in both any alcohol use and binge drinking. The other capacity constructs found to positively predict prevalence reductions for one or the other targeted behavior were community connections and outreach, and diversity of funding sources. Both have been identified in the literature as potentially important characteristics of effective coalitions, and in that regard, the finding of their association with prevalence reductions in the targeted behaviors was not unexpected. Paid versus unpaid coalition leadership and cultural competence do not appear to have received as much attention in the literature, and were not found to be significant predictors of the outcomes assessed here. Clearly, however, the limitations of our single-item measure of cultural competence may have mitigated the likelihood of finding a significant effect for this construct. The negative association observed between data infrastructure (i.e., the number of data sources used for assessment) and reductions in any alcohol use was unexpected, and for which no explanation is readily apparent.

The associations reported here are not particularly strong, as indicated by the relatively small reductions in the error variance attributable to the capacity measures. With respect to specific effects tested, differences of one standard deviation in the statistically significant capacity measures were found to be associated with greater declines in prevalence rates of roughly between 1 and 2 percentage points for the two outcomes examined. Because these are population-level outcomes, however, even small reductions in prevalence such as those observed here are of public health significance. Furthermore, the fact that statistically significant associations were observed at all is remarkable considering that the capacity data and the outcome measures were derived from two entirely different data sources. It is unlikely that coalition coordinators were biased or otherwise influenced in how they responded to the capacity items by knowledge of their post-intervention outcome data, given the specificity of the capacity items and the lag that typically occurs between when student surveys are conducted and when community-level estimates from those surveys become available.

How well the findings from this study generalize to other prevention initiatives and contexts is uncertain. The SPF SIG was implemented across states with substantial flexibility regarding which interventions community grantees implemented. Although there was an overarching emphasis on the achievement of population-level change, the funding levels, training and technical support, and requirements placed on community grantees by the state agencies varied from state to state. It is therefore reasonable to assume that the primary findings from this study would be robust with respect to how they might apply to a variety of other initiatives and funding mechanisms that support coalition-based prevention activities. Clearly, however, if the sponsoring agency for a multi-community initiative or project applies an overall strategy and a set of requirements and expectations that are inherently ineffective, even the highest functioning coalitions may be stymied in reaching their objectives. This concern is reflected in the mixed findings from coalition-based projects identified earlier in this paper, and emphasizes the need for well-conceived and conceptually sound overall approaches supported by appropriate levels of funding and high quality training and technical assistance to community grantees (Florin et al. 2012).

Certain limitations to the study methodology should be kept in mind in interpreting the findings just discussed. The findings are based on correlational evidence rather than effects produced through experimentally manipulated variables, and therefore, definitive attribution of causality in the associations between capacity and outcomes achieved is unwarranted. Nevertheless, the associations observed certainly are consistent with a causal interpretation, especially as no other obvious explanations leap to the forefront and because many other studies have identified intermediate outcomes that conceptually mesh with the operation of a causal mechanism (e.g., the finding from previously cited studies that coalitions with higher capacity implement more evidence-based strategies). The capacity measures used are subject to two potential limitations. First, they are based solely on the responses of the coalition coordinator, which could be biased towards a more positive assessment than warranted. Second, the study used a relatively small and previously untested set of capacity measures. The measures analyzed do not provide a comprehensive, psychometrically validated, and theoretically driven assessment of all potentially important aspects of coalition capacity. Coalition coordinator and member knowledge and experience, for example, would seem to be potentially important aspects of coalition capacity. Further analyses with other data sets would be useful to help confirm or refine the constructs identified here, as well as identify other potentially important aspects of coalition capacity.

Results from this study demonstrated that community coalition capacity, even when assessed with a limited set of items, was associated with the level of reductions achieved in a targeted population-level outcome— in this case, underage drinking. This finding represents a significant step forward in community-based prevention research, increasing confidence that attention to coalitions’ organizational capacity is warranted and suggesting that successful efforts to increase coalition capacity may have a meaningful influence on their effectiveness. More specifically, organizational capacity, as applied to community prevention coalitions, is not just an abstract term, but it can be measured, enhanced over time, and it is a significant predictor of a coalition’s level of success in achieving community-level prevention outcomes. Furthermore, certain aspects of capacity appear to drive the association and therefore suggest attributes that warrant heightened attention in efforts to build and maintain coalition capacity. These include a well-organized and effectively managed internal structure, strong community connections and outreach, and multiple funding sources. Left unanswered from this study are the mechanisms through which coalition capacity is linked to success in achieving the desired population-level outcomes. Among the possible explanations, the choice of interventions implemented and the quality or intensity of their implementation would seem to be likely candidates worthy of future research.