Introduction

Major depression is, and is expected to continue to be over the next 20 years, a leading cause of disability for adults in the United States (Kessler et al. 2003; Lopez et al. 2006). Approximately 20.9 million adults (9.5 % of the population age 18 and older) have a mood disorder in a given year (Kessler et al. 2005). Based on the results of a recent epidemiologic survey of U.S. household residents age 18 and older, only about half of Americans diagnosed with major depression annually receive depression treatment (Gonzalez et al. 2010). At the same time, access to, use of, and retention in depression treatment (medications or therapy) are consistently and significantly lower among minorities, such as African Americans and Latinos, compared to Whites (Alegría et al. 2008; Gonzalez et al. 2010; Neighbors et al. 2007).

Promising research has shown that collaborative care interventions intended to improve depression services in primary care settings, such as Partners in Care, can enhance health and employment outcomes for minorities over 5–10 years (Wells et al. 2000, 2004, 2007, 2008). However, widespread implementation of such evidence-based treatments in “real world” community-based care settings where minority populations are likely to receive depression care is often delayed or does not happen due to the complex nature of intervention implementation processes (Aarons et al. 2010; Proctor et al. 2009). Nonetheless, a recent systematic review of community-based models of providing mental health services suggests that, if implemented, such models can improve mental health outcomes (Wiley-Exley 2007), which may help close the gap between the science and practice of depression care (Wandersman 2003) and reduce disparities in mental health treatment affecting underserved communities (Patel et al. 2006; Wang et al. 2005a, b).

Previous research shows that the implementation of evidence-based health treatments and their sustained use in community settings can be facilitated through the use of principles of Community-Based Participatory Research (CBPR) (Lindamer et al. 2009; Stacciarini et al. 2011; Wallerstein and Duran 2010). Studies also suggest that active community engagement in research is associated with the increase in political and community impact of partnered projects that deal with mental health issues (Khodyakov et al. 2011). Because it is extremely difficult to conduct cluster-randomized controlled trials that compare communities assigned into a CBPR process to communities assigned to a more traditional researcher-led intervention (Buchanan et al. 2007), there is a lack of empirical data on the process of implementing mental health services interventions by using CBPR approaches among minority populations (Stacciarini et al. 2011). To fill this gap in the literature, we document and evaluate the process of implementing a series of evidence-based approaches to treating depression in “real world” community-based settings through equal collaboration between academic and community partners in the Community Partners in Care (CPIC) study, which uses a cluster-randomized intervention design (Chung et al. 2010; Khodyakov et al. 2009).

To document and evaluate intervention implementation, we discuss how, and to what extent, the implementation goals of community engagement and collaborative planning were achieved in the intervention arm of the study. In assessing community engagement, we examine agency participation in study meetings, emergence of community leaders, and relationship building among intervention participants. In assessing collaborative intervention planning, we examine the extent of adaptation of study toolkits, development of community-oriented training in collaborative depression care, and development of depression-related community network and resources. Our evaluation not only helps open the “black box” of the community engagement and planning process as implemented in the CPIC study and yields contextual data necessary for interpreting study outcomes at the agency, provider, and client levels (Wells et al. in press), but also helps inform the implementation of future community-partnered mental health services interventions.

Background

The US population suffers from a number of mental health conditions that are highly prevalent, such as depression and anxiety disorders (afflicting roughly 8–10 % (Kessler et al. 2003) and 11 % (Kessler et al. 2005) of the population per year, respectively), and which have been associated with limitation in daily functioning and well-being on par with other chronic conditions, such as diabetes (Ettner et al. 2006; Wells and Sherbourne 1999). While most treatment for these common mental health conditions occurs in primary care and other community practice settings (Wang et al. 2003, 2005b), these disorders often go undetected (Borowsky et al. 2000; Wells et al. 1989). Moreover, approaches to improving mental health care in these settings that have been shown to be both clinically (Katon et al. 1995, 2002) and cost (Pyne et al. 2001; Sherbourne et al. 2001) effective are not widely used (Sullivan et al. 2005) and, even when employed, appear to reach only about half of eligible patients (Unutzer et al. 2002; Wells et al. 2000). This gap is especially disturbing given the proven potential of these interventions to improve access to care and to reduce disparities faced by disadvantaged and underserved minority populations, including African Americans and Latinos (Miranda et al. 2003; Wells et al. 2004), who typically receive care of lower quality and have worse depression outcomes, compared to Whites (Institute of Medicine 2006).

There has been a renewed interest in intervention research in putting effective interventions into practice in community settings (Glasgow et al. 2003; Hohmann and Shear 2002; Lenfant 2003; Schoenwald and Hoagwood 2001; Wandersman et al. 2008), which depends on the knowledge of local settings. A growing number of scholars and community activists advocate for the use of CBPR methods specifically to improve the translation of interventions into practice (Wallerstein and Duran 2010) and improving minority mental health (Stacciarini et al. 2011) because community engagement in a collaborative process of designing, implementing, and disseminating evidence-based health interventions can enhance their relevance, sustainability, and impact for populations in need or at-risk (Lindamer et al. 2009; Stacciarini et al. 2011; Wallerstein and Duran 2010). Although CBPR seeks to enable researchers and community members to forge a common interest in a set of health outcomes and to jointly create interventions that are sensitive to community needs and circumstances by helping to increase the odds of successful program implementation and sustainability (Glasgow and Emmons 2007; Khodyakov et al. 2011), there is a lack of empirical data on the process of implementing mental health services interventions using such approaches. Understanding how CBPR approaches can be implemented in the context of large randomized controlled trials is crucial to accumulating evidence on effective intervention designs and dissemination strategies and on how intervention implementation processes are linked to outcomes for clients, providers, and organizations that serve them. Without this understanding, it is difficult to ascertain why certain outcomes were obtained or to explain variation in outcomes in a manner that can be used to improve future intervention design and dissemination strategies (Hohmann and Shear 2002).

Community Partners in Care

CPIC is an NIMH-funded, five-year, multi-phase, cluster-randomized controlled trial with over 100 participating agencies and over 400 providers from two urban, minority, underserved communities in Los Angeles: Hollywood/Metro, referred to as Service Planning Area 4 (SPA 4), and South Los Angeles (SPA 6). SPA 4 has a population of about 1.2 million people and SPA 6 has over 1 million people. In SPA 4, approximately 57 % of residents are Latino and 6 % are African American. In SPA 6, these percentages are 63 % for Latino and 32 % for African American residents (Los Angeles Country of Public Health 2009).

CPIC is a cluster-randomized controlled trial designed specifically to test an innovative community-academic partnered approach (Jones et al. 2009) to implementing quality improvement (QI) programs for depression at the community level in African American and Latino communities. CPIC randomizes a diverse set of community agencies from both within and outside of the traditional mental health service sector—including social service and faith-based organizations, community health clinics, mental health agencies and substance use agencies—into either a Resources for Services (RS) or a Community Engagement and Planning (CEP) intervention arm (Khodyakov et al. 2009). Agencies randomized into the RS arm receive study materials and technical assistance necessary for developing a QI program for depression care within their individual agency, which typifies a more traditional approach to intervention implementation. Agencies in the CEP arm were expected to work together to develop a multi-agency, collaborative, community-wide approach/plan and network to improve depression care, which represents a CBPR-based approach to intervention implementation (Chung et al. 2010).

CPIC uses a Community-Partnered Participatory Research (CPPR) framework, a variant of CBPR, as a set of guiding principles for community engagement and partnered intervention development and implementation (Jones 2009; Jones and Wells 2007; Khodyakov et al. 2009). CPPR has a structure, a set of principles, and a staged implementation approach assuring equal participation of community and academic partners, while promoting two-way capacity building toward specified health and healthcare improvement objectives. The goals of CPPR are to facilitate respect for community and academic expertise and ensure that community engagement principles, such as power-sharing, mutual respect, and two-way capacity building, are integrated with scientific rigor (Bluthenthal et al. 2006).

Despite the growing body of research in CBPR over the past two decades (Israel et al. 2001; Minkler 2010), CPIC appears to be the first randomized controlled trial that has all of the following characteristics: (1) implementation of equal power sharing and decision-making among community and academic partners in all aspects of the project, (2) reliance on collaboration among diverse community agencies, and (3) focus on a highly stigmatized health condition such as depression. Given these features of the CPIC project and its application in “real life” community-based settings, documenting and evaluating the process of implementing this intervention offer unique insights into the feasibility of using such an innovative CBPR approach for complex evidence-based interventions in public service sectors.

Table 1 describes the three phases of the CPIC study: the partnered planning phase, the randomized trial phase, and the community dissemination phase (Chung et al. 2010). This article reports results of an implementation evaluation of the community planning process within the CEP arm of the randomized trial phase, which occurred over a 4-month period in each SPA and is the core intervention being tested by the CPIC study. The two overarching goals of this planning process were to (1) engage a diverse group of community agencies around depression care and (2) design a collaborative approach or plan for treating it in their community.

Table 1 Three phases of the CPIC study
Full size table

Implementation Evaluation Methodology

In line with a partnered approach to evaluation, a team of academic and community partners, participating equally in the research process, conducted this evaluation between December 2009 and March 2010 in SPA 6 and between April 2010 and July 2010 in SPA 4. We used a mixed-methods approach to data collection and were able to collect data describing the process of community engagement and collaborative intervention planning in the CEP arm of the study in each SPA. Between the two SPAs, we observed and reviewed meeting minutes from 16 two hour-long CEP meetings, reviewed 127 meeting reflection sheets filled out by CEP meeting participants, surveyed 42 CEP participants, conducted 13 individual semi-structured interviews with community CEP participants, and interviewed 18 academic and community intervention leaders either individually or in a group setting. Our data collection is summarized in Table 2. All data collection protocols received IRB approval. In February and June of 2010, the results of a formative, midcourse evaluation in each SPA were presented to CEP participants for the purposes of improving the group process and facilitating the intervention planning. Results of these discussions also informed this evaluation.

Table 2 CEP planning sub-phase: implementation evaluation data sources
Full size table

The collected data were analyzed thematically by a team of academic and community partners to evaluate whether, to what extent, and in what ways the CEP process in each SPA fulfilled the initial goals and objectives of the CEP intervention. These goals and objectives included (a) three engagement process goals (bringing agency representatives to the table; having CEP intervention participants take leadership in the initiative; and building relationship among CEP council members) and (b) three planning process goals (adapting collaborative care models and evidence-based depression toolkits for use by local community agencies; developing community-oriented training in collaborative depression care; and developing community networks and resources to provide depression care). This implementation evaluation also recognized that a key feature built into the CPIC design is to engage participants in developing and modifying study goals throughout the project.

Results

Community Engagement Intervention Goals

Bringing Agency Representatives to the Table

One of the main goals of the CEP process was to engage participants from a diverse set of agencies, some of which do not provide mental health services, to improve depression care among underserved populations. Bi-weekly CEP meetings during a 4-month period were designed to help attain this intervention goal. Both SPAs were able to successfully reach it by bringing to the table a diverse group of agencies, ranging from medical professionals, representatives of the L.A. County Department of Mental Health (DMH), and substance abuse and family counselors, to social workers, clergy and members of religious congregations, representatives of parks and recreation services, and weight loss programs. The SPA 6 CEP council averaged 20 participants per meeting, and SPA 4 averaged 25 participants per meeting. Out of 31 agency programs randomized into the CEP arm in SPA 6, 19 programs (61 %) sent their representatives to at least one meeting; 9 of these 19 programs (47 %) were actively engaged throughout the planning process. In SPA 4, out of 26 agency programs randomized into the CEP arm, 14 (54 %) sent their representative to the meetings at least once; 11 of these 14 programs (79 %) remained engaged in the planning process.

To facilitate active participation of the community, meetings were led by a community partner with extensive experience in community-based projects within each specific SPA and by an academic facilitator experienced in the collaborative care model for depression and chronic illnesses. The facilitators focused on engaging CEP participants and building trustworthy relationships within the group by using ice-breaker exercises, such as the “Stone Soup” (Chung et al. 2010), which were especially well-received in SPA 6. Meeting facilitators prioritized creating a comfortable space for everyone to share experiences and concerns about depression and depression care, sensitive issues, especially within minority communities. This approach appeared to be effective in helping group members overcome reservations they had about the intervention and their participation in CPIC. On meeting reflection sheets, which CEP council members had the option of submitting anonymously after each session, participants often described the meeting facilitators positively, e.g., as “very forthcoming, positively engaging, accessible, friendly and basically very nice people [who]… went out of their way to put people at ease.”

Having CEP Intervention Participants Take Leadership in the Initiative

One of the goals of the CEP process was to engage agency representatives in more formal leadership roles to promote community ownership of the initiative and its sustainability after the conclusion of the project. Although all CEP meetings were co-facilitated by an academic and community study leader, CEP agency representatives were always invited to step up and become more involved in meeting planning and leading training conferences. Two SPA 6 representatives emerged as community leaders and were involved in planning and leading various CEP activities. Overall, community leadership in SPA 6 had a positive impact on group dynamics: it increased meeting interactivity; meeting agendas started to better reflect community needs; and group discussions became focused on issues that were identified as pivotal by agency representatives rather than academic partners. Nonetheless, one of the two community leaders who emerged in SPA 6 changed employers during the third month of the planning phase and could not continue her CPIC participation. The other leader, however, continues to be engaged in the project.

While SPA 6 had two formal community leaders, SPA 4 had a more distributed community leadership structure. Although meeting facilitators regularly encouraged CEP participants to take on leadership roles, only one did so on more than one occasion. This community partner acted as a breakout session leader, participated in planning calls, and volunteered to make a presentation at a CPIC training conference. Another participant was active during all meetings, but did not formally assume any leadership roles. Our meeting observation notes indicate there were many opportunities for taking on leadership roles and several people were potentially able to assume leadership roles, but none actually did so. One person who was active during several meetings lost her job due to the cut in funding at her agency. Other CEP participants may have found the leadership role to be too daunting a task to undertake given that it was not always clear what leaders were supposed to do besides moderating CEP meetings. Finally, a certain degree of confusion over the project goals and high participant turnover may have also contributed to lack of participants assuming leadership roles.

Building Relationship Among CEP Council Members

Results of our group dynamics survey suggest that even by the fourth CEP meeting more than two-thirds of CEP participants in SPA 6, and more than four-fifths in SPA 4, felt comfortable expressing their point of view. More than 80 % of SPA 6 CEP partners reported that their willingness to speak and express opinions increased since they had joined the project. Even a higher percentage of SPA 4 participants (roughly 90 %) either agreed or strongly agreed that council members listen carefully to each others’ points of view, even if they might disagree with them. Open-ended survey comments showed that CEP participants felt that “SPA 4 has good dynamics,” that “everyone gets a chance to express their opinion,” and that participants “feel comfortable sharing [their] ideas and concerns.” Similarly, SPA 6 comments revealed that participants were enjoying “dialogue, networking, and info sharing” and were “developing a relationship, which leads towards trust.”

The process of relationship building, however, was complicated by the fact that some agencies could not send their representatives to all CEP meetings, sent different representatives to different meetings, or stopped their participation in the CPIC initiative. For example, the study details had to be explained during multiple meetings and new CEP participants had to be brought up to speed on a regular basis, delaying the planning process. Moreover, some CEP participants expressed concerns about the amount of time they could devote to CPIC and the strength of relationships they could develop with other agencies if attendance and participation were not consistent. In the words of one participant: “I like the people here, the conversations, the organizations, the ideas. The problem I have, probably a lot of us have, is it’s just one more thing we are doing. I feel loosely connected…It feels scattered.” In interviews conducted with CEP participants after the end of the formal 4-month planning phase, the severe economic recession during the study and related budgetary contractions of funding streams at the national, state, and local levels were frequently cited as the primary reasons why CEP agencies had trouble ensuring consistent attendance of individual representatives to the council meetings.

Collaborative Intervention Planning Goals

Adapting Collaborative Care Models and Evidence-Based Depression Toolkits

Although the study provided general guidance on intervention planning and adaptation of the collaborative care model and toolkits, SPA 6 and SPA 4 took different approaches to these processes because the CEP groups in each community were focused on somewhat different needs. SPA 6 participants were interested in building their capacity in depression care services and therefore were focused on learning about and adapting the existing evidence-based depression toolkits, as well as planning for training conferences. To do so, they spent a significant amount of meeting time in three workgroups—cognitive behavioral therapy (CBT), care management (consolidated from two workgroups—leadership/administrative and care management/outreach), and integrative medicine (an expansion of the initial medication management workgroup focused primarily on more conventional depression medication therapies).Footnote 1

SPA 6 meetings typically concluded with each workgroup reporting back to the larger group on the discussion that took place in their workgroup and developing action items. Although each workgroup was given time to report back, CEP participants expressed the need in virtually every meeting for more time for whole group discussions. The CBT and integrative medicine workgroups met outside of the formal CEP meetings to plan for trainings, ensure training content complied with new County DMH requirements being released at the time, and, in the case of the integrative medicine workgroup, to review the materials related to their expanded focus, i.e. the effectiveness of alternative therapies for depression, such as aromatherapy, yoga, and acupuncture.

In SPA 4, agency representatives had generally better access to depression care resources and some had been previously exposed to CBT and collaborative care. Consequently, the group’s main interest focused on building a collaborative network for depression care. Moreover, compared to SPA 6, SPA 4 facilitators used a less structured approach to intervention planning. Instead of reviewing various components of the CPIC toolkit with participants, they encouraged agency representatives to suggest additional training components that they considered useful for a community-wide network addressing depression care. While this strategy helped increase the sense of ownership among participants over the course decided on by the CEP council during the latter half of the meetings, it created a great deal of ambiguity about project goals, meeting purpose, and feasibility of implementing a coherent community-wide approach to depression. Several community partners noted their skepticism during that time period about the project’s ability to reach its goals. As one commented, “It is very difficult to create service networks when the agencies provide such different services and resources, as well as the fact that different agencies may have different legal requirements to operate.”

To better deal with project ambiguity, SPA 4 leaders decided to try a small group meeting format and discuss project goals and expected outcomes of the planning process. During the fifth meeting, participants chose to break into workgroups to facilitate brainstorming and expedite the planning process. SPA 4 created two breakout groups to think about the strategies that the council should adopt to address depression. At the end of the meeting, the groups shared their ideas by writing them on white boards. The small group format allowed participants to interact with each other in a more meaningful manner, and resulted in the idea of holding case conferences (see below), making it easier to understand how collaboration among a diverse group of agencies may take place. Meeting reflection sheet comments from that meeting revealed that CEP participants enjoyed the small group format and found the brainstorming sessions worthwhile: “Breakout groups work great!” “I think this worked well, and the process of creating a plan has become a bit clearer.”

Developing Community-Oriented Training in Collaborative Depression Care

Because each SPA took a different approach to planning, they also arrived at different outcomes of the CEP process. Given the lack of resources, the primary focus in SPA 6 was agency capacity-building, viewed as helping to improve both the quality of depression care offered by individual agencies and their competitiveness in obtaining DMH service contracts and other external funding. Indeed, developing resources and training people to address depression—in particular learning CBT before other local agencies—were considered by many regular attendees as the main reasons for continuing to participate in the study. As one CEP participant related: “There’s going to be evidence-based CBT mandated no matter what program you are working on, and we have a leg up because we have had this training. I will stay with it as long as I can, and I don’t see the agency pulling us out of the process…I think we are ahead of the curve because the county hasn’t even started the process.”

To build capacity for depression care, SPA 6 adopted a “train the trainer model” in which CEP participants were expected to become leaders of various intervention components and be ready to train other agencies once the CPIC study moves to its Community Dissemination phase. To reach this goal, participants spent time learning about the toolkits, participating in role playing exercises (care management), clinical supervision (CBT), and adapting the existing materials to their needs. The planning phase in SPA 6 culminated in two two-day training conferences intended to expose a wide range of service providers to various components of the CPIC toolkit [for a detailed description of a typical CPIC conference see (Mendel et al. 2011; Khodyakov et al. in press)]. These training conferences included sessions on collaborative care model for depression care, case management, various levels of CBT instruction, and medication management (see Table 3 for conference attendance data).

Table 3 CEP conference attendance data
Full size table

While the original study design intended to keep the interventions in each SPA separate, CEP community leaders decided to open these training conferences to CEP participating agencies in both SPAs. This decision was made to improve capacity-building, increase the study reach, make it easier for agency representatives to attend trainings, and offer different levels of training, especially for CBT. As a result, SPA 4 CEP agencies participated in the SPA 6 CEP trainings early in the intervention planning phase and later invited SPA 6 agency representatives who had not attended trainings previously to join SPA 4 trainings. In addition to offering the core set of CPIC intervention components (CBT, care management, medication management), the training conferences in SPA 4 included sessions on active listening to help non-medical professionals and volunteers better interact with study clients, as well as yoga and drumming sessions intended to help service providers take care of themselves.

Developing Community Networks and Resources to Provide Depression Care

To help build a network among CEP agencies in each SPA, the study investigators developed a Facebook-type social networking tool called OurVillage, introduced to agency representatives during the first CEP meeting. A project staff member was available for one-on-one assistance during meetings and created accounts for all participants who had difficulty doing it themselves. The expectation was that this tool would help agencies interact with each other outside of the formal CEP meetings and receive relevant project messages and information. However, group activity on the site was limited primarily to the project staff posting newspaper and/or magazine articles about depression and materials regarding past and upcoming meetings. Each time a new article was posted, one community member would respond. Yet, there was no ongoing discussion regarding the topic, nor was there follow-up at the next meeting. CEP participants cited a lack of time to learn, become familiar, and/or use a new computer technology; limited or non-existent internet access at work; low levels of familiarity with online social networking tools; and lack of understanding and/or acknowledgment of the relevance of the site to the project goals as the reasons why use of OurVillage was limited.

As an alternative approach to facilitating service networking and providing a blueprint for collaboration between agencies, the care management workgroup in SPA 6 developed a Resources Guide. This guide consisted of (1) a table with information on the services provided, target populations, and eligibility requirements of each program and (2) a table with information about each agency compiled into a quick-to-use guide. Ten CEP programs in SPA 6 submitted information.

The presentation of the Resources Guide during the last few meetings exposed CEP participants to existing resources of other agencies in SPA 6 and raised issues of client referrals and payment for services, a concern among agencies associated with recouping costs of CPIC-related services (such as care management or CBT) was the lack of reimbursement by payers. This discussion revealed challenges encouraging SPA 6 participants to extend CEP meetings for an additional six months to be able to formulate an implementation plan to improve depression care in their community. By the end of this extended planning period, SPA 6 created a strategy that emphasized (1) training non-traditional providers to offer a range of services that can help depressed clients; (2) building agency capacity by offering a range of depression training opportunities; (3) acknowledging complementary/alternative medicine approaches to depression care; and (4) continuing regularly scheduled CEP meetings (see Table 4 for a more detailed overview of the SPA 6 plan).

Table 4 CEP Plans in SPA 6 and SPA 4
Full size table

Similarly, SPA 4 agencies were eager to collaborate with each other to address the problem of depression in their community, but had a difficult time visualizing how the varied types of agencies participating in the CEP council would be able to work together to help a typical client. This problem led to a decision to hold case conferences during the CEP meetings. Each case conference focused on a real client whose name and personal information were de-identified to protect confidentiality. After a brief five-minute presentation of a case, all participants were invited to offer concrete suggestions on how their agency could help in providing care and services to this client (see Table 5 for an example of a case discussed in SPA 4). Case conferences were a catalyst that helped SPA 4 participants better understand how collaboration among diverse agencies might take place, and helped them feel a sense of accomplishment and start drafting a collaborative, community-wide approach to offering depression-related services. This plan emphasized the importance of developing a Resource Guide, such as done in SPA 6, continuation of case conferences and CEP meetings, trainings in evidence-base depression care, and creation of a volunteer program to connect clients to services (see Table 4 for a more detailed overview of the SPA 4 plan).

Table 5 An example of a case conference in SPA 4
Full size table

Discussion

The aim of this paper has been to document and evaluate the implementation process of a key phase of the CEP arm of the CPIC study by focusing on the questions of how and to what extent the goals of community engagement and collaborative planning were achieved. Below we summarize our findings, highlight challenges that study participants encountered, and offer lessons learned that can help inform future attempts to bring evidence-based care to community settings using partnered approaches.

Community Engagement Intervention Goals

Our evaluation suggests that the CEP intervention was able to successfully reach the goal of engagement as indicated by (1) the participation of roughly three-fifths of agencies randomized into the CEP study arm, (2) existence of a core group of CEP participants who remained actively engaged throughout the planning process, (3) diversity among the participating agencies many of which do not offer any depression-related services, (4) emergence of community CEP leaders in each SPA, (5) and community participants’ positive opinions about group dynamics.

Many CEP participants reported that they were actively engaged in CPIC because of the participatory features of the project that distinguish it from traditional academic studies. They emphasized the importance of engaging community partners in the process of building a community plan from scratch and the benefit of networking with other agencies at a time when funding is limited. The emphasis on close collaboration between academic and community, however, was so different from a traditional approach to conducting research that it created confusion among CEP participants who, especially at the beginning of the process, did not fully understand what the project expected of them or how to collaborate with either potential competitors or agencies that do not offer traditional depression care.

While the CEP planning process was engaging, the project, nonetheless, encountered three challenges that complicated the planning process: meeting attendance, participant diversity, and lack of clarity on CEP participants’ roles. First, the difficult economic climate made created a challenge for agencies to send their representatives to all CEP meetings. While the turnover among CEP participants exposed a larger number of agency representatives to CPIC and was aligned with the CPPR principles, it made it difficult to build relationships, ensure continuity from meeting to meeting, and consequently slowed the decision-making and planning process.

Second, the diversity of agencies posed a challenge to participation and engagement because some agencies either did not focus on depression or felt that their existing network of referral sites for depression care was sufficient. Agencies serving different types of clients (i.e. adolescents vs. senior citizens) were also unclear about how they would collaborate with each other, especially if collaboration entails client referrals. As explained by a participant who attended only one or two meetings: “I think CPIC is a wonderful project, and we had a lot of depressed clients. Depression, however, is not our primary area. Not that the [CEP] meetings weren’t a priority…depression is not our priority…We already have our partner whom we utilize for depression-related services; it actually works well for us, we work really well with them.”

Third, CPIC is an unusual project because it relies on CPPR principles that emphasize equal participation of, and power sharing between, academic and community partners. Many agency representatives had never participated in such a project and were not used to having an equal voice in research projects. They needed more time to become comfortable with CPPR principles, learn about their roles in the project, deal with participant turnover, and make decisions about the future of the CEP intervention in their community. Consequently, the emphasis on joint planning and intervention implementation created a certain degree of confusion among the CEP participants who, at first, did not understand their roles. While the CPIC study provided an initial framework and pre-determined timeline of 4 months for the CEP planning portion of the Randomized Trial phase, by design, it assumes a significant degree of community input, where community partners and CEP participants are expected to take the initiative in developing their own roles, tasks, and working processes, in addition to adapting and developing a collaborative approach to depression care. Agencies that do not offer any mental health services or do not directly provide services to clients (i.e., senior citizens centers) seemed to be unclear about their expected contribution to the project. Given the novelty of the CPPR approach and a relatively short timeline, many agency representatives were waiting for more guidance from academic partners, while academic partners expected more community input. This difference in expectations was reconciled only towards the end of the planning phase, when the decision was made to continue CEP meetings and combine the efforts of two SPAs.

Collaborative Intervention Planning Goals

Even with these challenges, the CEP groups in both SPAs were able to draft and act on a formal plan for the intervention activities that are currently taking place in the implementation portion of the Randomized Trial phase. The goal of collaborative planning was achieved because (1) the CPIC toolkits have been adapted to the community needs; (2) CEP agencies were trained on CBT, collaborative care, and other intervention components with several individuals currently performing the roles of CBT trainers; and (3) multiple training conferences on depression care for CEP agency representatives were offered, in which several CEP council members led sessions.

Nonetheless, given the limited amount of time allocated to the planning phase, current economic climate, diversity among agencies, and a low utilization of the OurVillage web portal, the goal of building a community network among CEP agencies to collaboratively improve and provide depression care was much less developed. While the basis for such networks were established during the CEP community planning process, stable relationships and collaboration became visible only after the initial four-month formal planning phase was over; and the strength of these relationships and collaboration, as well as their sustainability, are yet to be determined.

The creation of Resources Guide in SPA 6 and case conferences in SPA 4 helped CEP participants envision how they could refer their clients for services in other agencies and realize the benefits of inter-agency collaboration in serving a specific clientele. A depressed person not only needs medical care, she or he may also require a variety of other services, such as housing assistance, food stamps, and substance abuse treatment. Having trusted referral lines and up-to-date information on services available from other agencies makes coordinating multiple services that a client needs easier, and collectively reviewing specific clients’ needs in case conferences showed practical ways in which diverse providers could support needs of depressed clients, including agencies that may not offer traditional depression treatment.

Although the CPPR process was confusing at first for the community partners who were not used to this approach, the leadership structure of the planning process, which consisted of community and academic partners with prior experience working on partnered projects, was essential for improving the capacity of the CEP agencies to work together to develop an intervention plan. In addition, a four-months planning period and a sequential approach to intervention implementation in two SPAs helped study leaders improve their ability to explain the project design to participating agency representatives. Therefore, the experiences with the planning process in SPA 6 were used to improve the SPA 4 planning process.

Limitations

Given the limited resources of the CPIC project, this implementation evaluation only covers the CEP arm representing the high intensity implementation condition of the study. As a result, it was not possible to examine whether agencies in the RS arm experienced similar processes and the extent to which the community engagement interventions in the CEP arm were uniquely associated with these processes. Other implementation data being collected by the CPIC study, including longitudinal surveys of administrators, providers, and clients, will be used to characterize and test for differences in implementation and care processes between CEP and RS agencies (Wells et al. in press), although not with the level of detail presented in this paper.

The implementation evaluation presented here is also limited to a descriptive analysis. While it utilizes a variety of qualitative and quantitative data and fulfills the call in implementation science and QI research to provide in-depth evidence and insight into the context and process of health interventions (Mendel et al. 2008), we cannot unravel which components of the CEP intervention are more or less effective at engaging community stakeholders or developing community networks of care. This is an issue with many complex, multi-component interventions, such as tested in CPIC, that have a limited ability to identify the exact intervention components that lead to improvements in intervention outcomes (Hawe et al. 2004).

Moreover, as a descriptive analysis of implementation process, the results presented here are not linked to health outcomes of individuals with depression, which is the ultimate focus of the CPIC study. The results of this implementation evaluation, however, will be used to interpret results of outcomes analyses of the study, as mentioned earlier in the paper.

Lastly, the implementation results of this evaluation are limited in generalizability beyond the two communities in Los Angeles in which the CPIC study has been fielded. At the same time, these communities share many similarities with African American and Latino communities in other urban areas. The inclusion of two study areas within the same study also provides insights into the effects that contextual differences between communities may have in the process of implementing community-partnered health interventions like CPIC.

Conclusions

Despite the above limitations, our evaluation not only shows that CBPR approaches can be effective in enhancing collective efficacy to improve depression care, it also illustrates the feasibility of using partnered approaches to adapting and implementing a complex, evidence-based, collaborative care approach for treating depression with agencies working in diverse service sectors (e.g., social services, mental health, primary care, substance use, religious services, and senior centers, among others) in low income communities of color. Therefore, our study fills a noted gap in literature on the implementation and dissemination of health interventions for in-depth understanding of the processes involved in implementing evidence-based practices in public service sectors (Aarons et al. 2010; Proctor et al. 2009). Our work shows the feasibility of a multi-agency planning approach that relies on CPPR principles and structure in designing a community-wide plan to address depression by adapting clinical depression care materials for use in new and innovative ways through sustained committed work overtime to fit agency, provider, and client needs.

Lessons Learned

In conclusion, we summarize lessons learned during this implementation evaluation, which can be helpful for implementing and evaluating future community-partnered interventions:

  1. 1.

    CPPR strategies used in CPIC are useful in engaging community partners around depression.

  2. 2.

    Community engagement and collaborative intervention planning require time and resources.

  3. 3.

    Community-academic partnerships may benefit from a leadership structure that can effectively balance the need to complete the study on time with the need to adjust study plans depending on the needs of participating agencies and a deteriorating economic situation.

  4. 4.

    It may be ambitious to expect community representatives to take on primary leadership roles for the initiative, especially in the context of a large scale research study that could not reimburse for the time they spent on this project.

  5. 5.

    Successful collaboration among diverse agencies requires that they understand what is expected of them, are comfortable with the role they chose to perform, and have organizational support to meaningfully contribute to the project. Study leaders may need to emphasize that roles and functions of agency representatives may vary and stress the value of even the smallest contribution.