1 Introduction—Knowledge co-production and adaptive capacity

Co-production of knowledge is promoted as a potential solution to address climate change impacts, including on working lands (Bremer and Meisch 2017, pp. 3–4; Homsy and Warner 2013, pp. 294–295; Lu et al. 2022, p. 260). We define co-production broadly as a process through which decision-makers or researchers work together with stakeholder groups (people and groups with a stake in decisions or resources) to create actionable knowledge that informs decision-making and/or produces useful outputs for end-users (e.g., Prokopy et al. 2017, pp. 2–3; Lemos et al. 2018, p. 722). Such processes are in contrast to a traditional linear model of knowledge production and transfer (i.e., scholar to decision-maker) (Bacon et al. 2005, pp. 1–2). There are two lenses through which co-production is typically viewed: (1) (our lens) as the collaborative production of knowledge to increase the usability (and use) of the science that is produced (Lemos et al. 2018, p. 722; Prokopy and Floress 2011, p. 90; Wyborn 2015, p. 3); and (2) as a critical approach to understanding how knowledge forms in society and interactions between society and science. In this paper, we describe co-production processes on a continuum from low to high levels of interaction between university affiliated faculty and staff and non-university affiliated stakeholders (i.e., those who have interest in an issue, as an individual or a group member). Where a project sits on this continuum relates to how often non-university affiliated stakeholders are engaged and how much decision-making power stakeholders have in designing the research questions, approach, analysis, and dissemination (e.g., Bacon et al. 2005, pp. 2–4; Mach et al. 2020, pp. 32–33; Reed et al. 2018, pp. s8-s10).

Some literature suggests that better outcomes can be achieved for all (e.g., decision-makers and the public) by empowering stakeholders in decision-making processes as opposed to outcomes from top-down decisions alone (e.g., Arnstein 1969; Watson 2014) (and see also Gagnon et al. (2022) who discuss power distribution through participatory decision-making that enables equal participation by using a shared language). When knowledge is co-produced, it is more likely to be used in policy decisions (e.g., approving plans, restricting practices, implementing incentives) (Armitage et al. 2011, pp. 1002–1003; Lemos et al. 2018; Norström et al. 2020, pp. 188–189) and increase policy support (Lemos and Morehouse 2005, p. 61). In the research context, co-production processes can improve research outcomes due to the utilization of diverse knowledge, skill sets, and networks (Armitage et al. 2011, pp. 999–1000; Lemos et al. 2018, p. 722; Ostrom 1996), while making the outputs generated by the process more likely to be used (Kirchhoff et al. 2013; Lemos and Morehouse 2005, pp. 65–66; Lu et al. 2022, p. 258; Prokopy et al. 2017). Other literature places value in acknowledging the validity of different levels of participation, where participation is dependent on project goals, as well as the structure and capabilities of communities, governments, and stakeholders (e.g., Brix et al. 2020, pp. 175–176; IAP2 2018; Lemos et al. 2018; Neef and Neubert 2011, pp. 182–183; Prokopy and Floress 2011; Reed et al. 2018, pp. s11-s13; Senbel and Church 2011; Watson 2014, pp. 64–67). For example, in the agricultural context, Probst et al. (2000) suggest that rather than a spectrum of engagement there are simply different research approaches. Neef and Neubert (2011, pp. 182–183) suggest six dimensions of participatory research related to project type, research goals, characteristics of researchers and stakeholders, and potential benefits of the project. These dimensions point to the need to develop appropriate processes, plans, and implementation of different approaches to co-production processes, (e.g., Lemos et al. 2018, p. 723; Norström et al. 2020, pp. 187–188). Using terminology we formed during our qualitative coding process, we suggest stakeholder-based project design may be an alternative to highly interactive co-production process (see Table 1 for a list of definitions of the key terms derived from our data analysis and used throughout this paper).

Table 1 Definitions of key terms developed through data analysis
Full size table

Increased social-ecological adaptive capacity is seen as an important factor in resilience to change (Berkes et al. 2008), with co-production processes highlighted as one way to achieve adaptive capacity goals (Bremer and Meisch 2017, p. 10). Adaptation can occur at different spatial (local; state; nation; globe) and temporal (short-term; long-term; responsive; proactive) scales, in different systems (natural; human; public; private), and in different forms (structural, legal, regulatory, financial) (see Smit and Piliosova 2003). Adaptation, in the human context, is the ability of a community or individual to avoid or recover from unusual or unpredicted events (Berkes et al. 2008; Smit and Wandel 2006, pp. 282–283) or act collectively to respond to various threats to natural resources (Armitage 2005, pp. 712–713). Adaptive capacity refers to resources, institutions, and technical/financial ability needed for flexible responses to threats, while allowing for learning and experimentation to address such challenges (Armitage 2005; Berkes et al. 2008; Biagini et al. 2014, p. 99). There are a variety of frameworks that conceptualize adaptive capacity processes. Generally these frameworks either outline conditions that should be present to foster adaptive capacity or document aspects that should be present in an adaptive community (Armitage 2005; Caniglia et al. 2021; Folke et al. 2003; Gupta et al. 2010; Jones et al. 2010; Wall and Marzall 2006). In addition to understanding preconditions for adaptive capacity, we are interested in adaptive capacity actions (building adaptive capacity) and outcomes (management, planning, and policy activities, as well as behavior change) (e.g., Biagini et al. 2014, p. 99; Jagannathan et al. 2020, p. 6; Smit and Wandel 2006).

We explored relationships between co-production and adaptive capacity in research, extension, and education projects relating to water, climate change, and agriculture in the United States (U.S.). Water quality problems such as sediment, nutrient, and chemical loads are a result of practices by both agricultural and non-agricultural actors that negatively impact ecosystems, habitats, drinking water, and recreational and economic activities (Basnyat et al. 1999; Parris 2011; Parry 1998; Shortle et al. 2001). As the climate shifts, there are and will continue to be both direct and indirect effects on agriculture worldwide and in the U.S. (FAO 2021; Hatfield et al. 2014; Melillo et al. 2014). These changes will be felt through increased temperatures, changing precipitation patterns, including droughts and flooding, and geographical shifts in pests and disease, for example. At the same time, agriculture contributes to climate change both directly through greenhouse gas emissions and indirectly through land use changes (OECD 2016). Co-production processes are one possible avenue to help foster adaptation and resilience of working lands systems and communities across the globe.

Recently, stakeholder engagement frameworks that utilize co-production processes have been put forth to promote successful community and stakeholder engagement (Kliskey et al. 2021, pp. 5–12). While co-production has emerged as a strategy to address natural resource challenges, the process has not been well described in practice (Mach et al. 2020, pp. 30–31), leading to a lack of evidence on whether co-production processes create actionable knowledge (Jagannathan et al. 2020, pp. 14–15). Indeed, tensions exist between assumptions of the benefits of co-produced knowledge and the time and resources it takes to do such projects well (e.g., Lemos et al. 2018). As applied social science researchers involved in the working lands context, we continually encounter these tensions—a desire to work hand-in-hand with stakeholders alongside the reality of time and funding constraints. Thus, through this research, we explore whether researchers should invest resources in co-producing knowledge to ensure adaptive capacity outcomes.

2 Data collection and analysis

The data reported here is a subset of a larger project that evaluated successes and challenges of United States Department of Agriculture National Institute of Food and Agriculture (USDA-NIFA) grant funding to advance science related to water and climate outcomes on working lands (see Getson et al. 2020). USDA-NIFA is an important federal governmental agency in the U.S. that funds projects to increase working lands productivity and sustainability and to foster resilient communities. Through this project, we administered surveys to USDA-NIFA Project Directors (n = 1894), and conducted focus groups (n = 28), interviews (n = 9), and case studies (n = 13). Here, we report on results of the 13 case studies that were conducted to understand detailed project design, successes, outcomes, and challenges. We qualitatively analyzed 13 case studies to understand co-production processes and related adaptive capacity outcomes. In this section, we describe how the case studies were selected and conducted, followed with our data analysis process.

2.1 Case study development

We conducted case studies on USDA-NIFA water (2001–2013) and climate (2010–2015) portfolios, which funded projects focused on research, education, and extension related to climate and water issues on working lands (6 climate portfolio; 7 water portfolio). The case studies were conducted by a team of six researchers (all of whom are co-authors on this paper) in 2017 and 2018, entailing interviews, survey responses, and document analysis of USDA Current Research Information System (CRIS) reports written by grant Project Directors (PDs henceforth) and required for each project (https://cris.nifa.usda.gov/).

Surveys were conducted with PDs on all USDA-NIFA projects in the water portfolio (1837 projects funded between 2001 and 2013) and climate portfolio (2241 projects funded between 2010 and 2015) (see Getson et al. 2020 for details; 1894 survey responses across both portfolios). A subset of all water and climate portfolio projects were subsequently selected as potential case studies. To ensure a diversity of case studies, projects were selected by evaluating the following criteria to ensure a collection of cases that would represent diversity of success, scope, geography, and project type: total means of all survey statements where PDs were asked to evaluate project success metrics and how synergies and relationships helped project success; funding category (competitive, non-competitive); funding program; project type (research, extension, education); number of PDs on the project; geographic location; minority institutional status or partnerships; gender identity and academic job rank of the PD. No part of our case selection criteria included specifics about co-produced research or adaptive capacity outcomes. See Supplemental Material for the case list (SM-Table 1).

Each case study included analyses of interviews specific to the case, the PD’s survey responses, and any CRIS reports associated with the project. PD survey data included information about critical research findings, project successes, and lessons learned. CRIS reports are mandatory yearly reports submitted by PDs that include information such as the following: grant amount; USDA-NIFA grant program; project summary, objectives, and approach; reported impacts and outcomes. The number of interviews and CRIS reports varied from case to case.

For each selected project, semi-structured in-depth interviews were conducted with the PD, co-PDs, other key personnel, and any other stakeholders who were involved, benefitted from, or were affected by the research (e.g., community leaders, NGO staff, state and federal agency personnel, K12 educators, etc.). The interviews entailed questions about project successes, challenges, lessons learned, partnerships, stakeholders, and possible capacity building outcomes. A total of 106 interviews were conducted for all 13 projects, ranging from 3 to 23 interviews per project depending on project scope. Interviews were completed in 2017 and 2018. Interviewees included a combination of project team (n = 56) and project stakeholders (n = 50). The cases provide an in-depth look at the successes and challenges of a variety of USDA-NIFA-funded climate and water projects. The projects span across the U.S. landscape, grant type, and project type (e.g., research, extension, education). Project funding ranged from $365,000 to over $7.5 million dollars, with most projects being specific to one state, a few that were set in several states, and two with a national focus. See Supplemental Material for the case list with details of each project (SM-Table 1).

2.2 Codebook development

We used deductive analysis to examine relationships between co-production of knowledge and adaptive capacity outcomes and inductive analysis to understand project design (Bernard and Ryan 2010; Saldaña 2021). Inductive coding was used to identify elements of project success; here, we report on coding within the success framework related to how projects were designed. For our deductive approach, we used Mach et al. (2020) to examine co-production of knowledge and Biagini et al. (2014) and Jagannathan et al. (2020) to examine adaptive capacity outcomes. These frameworks are described next.

2.2.1 Co-production of knowledge

Mach et al. (2020, p. 33) illustrate that co-production of knowledge is practiced along a spectrum of research from contractual and consulting, to collaborative and co-created. The authors outline three aspects of co-produced research: (1) Research question origin: ranging from researcher-developed to stakeholder-developed; (2) Relationship type: ranging from providing services and resources to partnerships; and (3) Interactions over time: ranging from stakeholder participation at specific stages of the project to continuous participation throughout the project. The contractual and consulting end of the co-production research spectrum is exemplified by researcher-developed questions that include stakeholder interactions at specific stages of the research, resulting in provision of services or resources to impacted communities. In contrast, collaborative and co-created research processes entail question development in partnership between researchers and stakeholders, working continuously with researchers over time, enabling co-creation of knowledge with stakeholders positioned to use the knowledge to inform decisions in their community (Mach et al. 2020). Although Mach et al. (2020) look at how decisions are made through various conceptions of knowledge co-production, due to the nature of the original interview process, we were not able to analyze these specific decision points. Rather, we examined how case study interviewees discussed elements of co-production when describing the projects as a whole.

2.2.2 Adaptive capacity

Building off existing theoretical adaptive capacities frameworks, Biagini et al. (2014; p. 104) developed their own typology of climate adaptation action through their analysis of adaptation projects financed through the UN Framework Convention on Climate Change, where they sought to understand adaptation activities in developing countries across the globe. Biagini et al. (2014) put forth ten adaptation categories. We used six categories applicable to our data to examine our case studies: (1) Capacity building: equipping humans, institutions, and communities with the means and capacity to adapt to climate change; (2) Management and planning: data and science used in planning and management efforts and plans; (3) Practice and behavior: adaptive practices and behaviors are present; (4) Policy: new policies developed to allow for climate adaptation; (5) Information: climate information communication systems and tools are present; and (6) Technology: new/improved technologies developed to foster climate resilience.

We also utilized Jagannathan et al.’s (2020, p. 25) categorization of adaptive action outcomes from co-production processes that they term Scope 1 and Scope 2 outcomes. These outcomes differ in scale of impact. Scope 1 outcomes create actionable knowledge that can inform decision-makers, while Scope 2 outcomes challenge the norms and structures of both science and society. Scope 1 outcomes are more common as they are often relatively pragmatic, tangible, and proximate; meaning they address practical needs, are relatively easy to identify, and occur within a comparatively short time frame. Scope 2 outcomes occur less frequently as they are ambitious, extended, and radical; meaning they strive for large-scale impacts, focus on long-term changes, and work towards restructuring both scientific and societal norms. For our purposes, we used the following broad outcomes in our analysis: Scope 1: “catalyzed action” (knowledge used in plans etc. are implemented and/or behaviors are changed), “deepened understanding” (enables integration of local and expert knowledge, prompting increased learning and knowledge of all participants), “strengthened communities” (enables collaborations and fosters community capacity to adapt to change), “utilized knowledge” (incorporation of data, tools, and knowledge into actions, policies, or plans); and Scope 2 outcomes (represents transformation or shifts away from traditional norms). As shown in Table 2, we used Jagannathan et al. ’s (2020) adaptive action outcome categorizations as our broad coding categories for “adaptive capacity outcomes” (i.e., codes). We then grouped specific elements of Biagini et al. ’s (2014) adaptive capacities framework within those broad codes (i.e., subcodes) (see Supplemental Material SM-Table 2 for codes and subcode definitions for the adaptive capacity outcomes category).

2.3 Coding process

The lead researcher first analyzed the case studies inductively to build an initial codebook related to project success and project design. Then the lead researcher, along with a team of three other researchers, collaboratively built a deductive codebook based on co-production of knowledge and adaptive capacity outcomes (Biagini et al. 2014; Jagannathan et al. 2020; Mach et al. 2020). We coded the case studies in NVivo 14 qualitative software and conducted an inter-coder reliability process to ensure that each researcher agreed on code definitions and how they were applied to the data (Church et al. 2019). The process entailed four rounds of iterative coding with four researchers coding the same five case studies. In addition to continual detailed discussion among coders to work out discrepancies, we utilized Cohen’s Kappa as a metric to determine inter-coder agreement—where a score of 0 indicates no agreement and 1 indicates perfect agreement, and a threshold of 0.7 is generally considered adequate (Landis and Koch 1977). In the end, we achieved an overall Kappa score of 0.79 at which point three researchers coded the remaining case studies alone.

In the results that follow, we count codes by case study rather than frequency that the codes occurred in each case. We coded cases as “co-production absent” if no portion of the case indicated co-production of knowledge methods. All of the other codes are not mutually exclusive (e.g., one case may have exhibited all four “project design” codes and/or all “strengthened communities” subcodes). Moreover, some cases had more than one project component that may have entailed different co-production methods and thus we coded for each relevant subcode (e.g., a case could have included one component that had intermittent interactions and continuous interactions and thus one case would have been coded for both subcodes). Table 2 presents the coding framework and associated terminology (category; code; subcode) and case counts by code. Table 1 shows the definitions we use throughout this paper, which were derived from our data analysis.

3 Results—relationships between knowledge co-production and adaptive capacity

3.1 Top co-production, project design, and adaptive capacity outcome categories

We reviewed USDA-NIFA CRIS reports for all the case studies and found that 12 of the 13 cases indicated specific stakeholder engagement goals; 11 cases were coded as stakeholder-based. Eight of the 13 cases’ USDA-NIFA CRIS reports indicated adaptive capacity goals that fit into our coding framework; all 13 cases had at least partial adaptive capacity outcomes (see Supplemental Material for the case list and project goals). We did not see indications that project funding or scope were related to adaptive capacity outcomes.

The overall results for the co-production of knowledge categories, the top four most frequently coded project design criteria, and adaptive capacity outcomes are shown in Table 2. Of the 13 cases we analyzed, seven exhibited co-production of knowledge. Of these seven, only four mentioned that they incorporated continuous interactions with stakeholders, with six having intermittent interactions (some cases discussed both types of interactions). For project question origin, we found that six cases were researcher-driven, three were co-produced, and one was stakeholder-driven (some cases discussed more than one question design origin depending on the particular project component). All seven cases were described as partnerships between the project team and stakeholders.

Table 2 Coding framework and case study code and subcode counts by category (n = 13)
Full size table

In terms of the project design category, almost all cases described the importance of “partnerships” in project success (n = 12). Most cases were “stakeholder-based” (project was designed to meet the needs and strengths of stakeholders or end-users) (n = 11). Most cases also included discussions about the importance of including “diverse expertise” in the project team (inclusion of multiple disciplines, job functions, ethnicities, and/or cultures) (n = 10). Another important aspect of project design described in many cases, was building off of “existing elements” rather than “reinventing the wheel” (n = 9).

Most of the cases were coded in the adaptive capacity outcomes category (Table 2). Almost all cases showed capacity building (n = 12), strengthening or building collaborative networks (n = 12), and knowledge increased (n = 12) outcomes. Although knowledge increased was a prevalent outcome, only some cases indicated that social learning occurred (e.g., learning and coming to new understandings through the project process) (n = 8), and few noted Scope 2 outcomes (e.g., transformative change; worldview change) (n = 4). Many cases discussed both aspects of the catalyzing action code: stakeholders changed behaviors (n = 10) and/or used information (n = 9) generated from the project. Many cases also developed curriculum (n = 10) and reported success in data collected (n = 9). Beyond collecting data, some project interviewees stated they believed knowledge generated from the project had potential for use (e.g., policy change) (n = 7) and several projects developed a decision support tool (n = 5).

3.2 Adaptive capacity outcomes and co-production of knowledge

We sought to determine how many cases with co-production attributes included adaptive capacity outcomes. Table 3 shows relationships between co-production of knowledge category and adaptive capacity outcome category codes. See Supplemental Material (SM-Table 2) for codes and subcode definitions for the adaptive capacity outcomes category, including example quotations.

Table 3 Co-production of knowledge case counts: Adaptive capacity outcomes by co-production present or absent
Full size table

The co-production of knowledge code, “co-production absent” cases had higher counts relative to adaptive capacity outcomes than “co-production present” cases. Knowledge increased, capacity building, collaborative networks, and curriculum development had the highest case counts within the “co-production absent” code (n = 5); these subcodes also had relatively high case counts in the “co-production present” code. Capacity building had slightly lower case counts in the “co-production present” code (n = 4) than the “co-production absent” code (n = 5). Behaviors changed had four cases each in both “co-production absent” and “co-production present” codes.

3.3 Adaptive capacity outcomes and project design

We also analyzed how project design related to adaptive capacity outcomes (Table 4) (see also Supplemental Material SM-Table 3 to see how project design related to co-production of knowledge). We found that projects that are not highly interactive may be more influential for adaptive capacity outcomes than highly interactive co-produced projects—what we named “stakeholder-based” in our coding process. Overall, we found that “stakeholder-based” cases were coded more frequently for adaptive capacity outcomes than “co-production present” cases. Looking at adaptive capacity outcomes that were coded as “stakeholder-based” project design, we found that the capacity building outcome had the most cases coded (n = 8), with collaborative networks next (n = 7), followed by developed curriculum (n = 6).

Table 4 Project design case counts: Adaptive capacity outcomes by project design
Full size table

The highest case count in the analysis of the project design category as related to the adaptive capacity outcomes category was the project design subcode “partnerships” in relation to the adaptive capacity outcome collaborative networks (n = 10); this relationship had more associated cases than “stakeholder-based” project design (n = 7). Indeed, the collaborative networks outcome was coded more frequently in the project design category than any other outcome. The collaborative networks subcode was coded along with several other project design category codes including “existing elements” (n = 5), “diverse funding” (n = 3), “diversity of expertise” (n = 3), and “meeting design” (n = 3).

3.4 Adaptive capacity outcomes reexamined

The two cases with the most adaptive capacity outcomes (Case 5, 14 out of 14 outcome subcodes; Case 8, 12 out of 14 outcome subcodes) were coded in the “co-production present” code. Portions of the project for both of these cases were coded for all of the “co-production present” subcodes: continuous and intermittent interactions; co-produced, researcher-driven, and stakeholder-driven question development; and a partnership relationship between project leaders and stakeholders. Both cases’ adaptive capacity outcomes spanned all of this category’s codes: “catalyzed action”, “deepened understanding”, “strengthened communities”, and “utilized knowledge”. The only adaptive capacity outcomes not present in Case 8 were subcodes within the “utilized knowledge” code: the development of a decision-support tool and results with potential for use (and in this case it was because the project results were already being used). Case 5 (see Table 5) engaged with Indigenous communities as part of the entire project, while Case 8 had a component of the project that integrated an Indigenous community college partner. Both of these cases were coded for the Scope 2 outcomes subcode as well. Only four cases were coded for this outcome out of all 13 cases we analyzed, three of which were coded in the “co-production present” code.

Table 5 Case study highlight: Native Waters on Arid Lands
Full size table

Of the next two cases with the most adaptive capacity outcomes (11 out of 14 outcome subcodes) one was included the “co-production present” code (Case 1—see Table 6) and one was coded for the “co-production absent” code and the “stakeholder-based” code (Case 6—see Table 7). Adaptive capacity outcomes in Case 1 included all the “catalyzed action” subcode outcomes and “utilized knowledge” subcode outcomes, almost all of the “strengthened communities” subcode outcomes (only the trust and credibility subcode was missing), and only one “deepened understanding” subcode outcome (increased knowledge). Case 6 included all the “deepened understanding” subcode outcomes, almost all of the “strengthened communities” and “utilized knowledge” subcode outcomes (only the trust and credibility and decision-support tool subcodes were missing), and one “catalyzed action” subcode outcome (behaviors changed). Of note, Case 6 was one of the four cases coded for Scope 2 outcomes subcode. Two cases did not include the “co-production present” code nor “stakeholder-based” project design (Cases 7, 11), yet they had more adaptive capacity outcomes than three stakeholder-based cases (Cases 2, 4).

Table 6 Case study highlight: Mobile Irrigation Water Management System
Full size table
Table 7 Case study highlight: Urban GEMS
Full size table

4 Highly interactive projects may not be needed for some adaptive capacity outcomes

Co-production of knowledge provides a framework for conducting stakeholder-oriented science that improves the usability of scientific information for adaptation beyond academia. Practically, knowledge co-production has the potential to address socio-environmental risks in agricultural systems affected by climate change. Yet little research has assessed the extent to which co-produced knowledge improves adaptive capacity (for an exception, see Chambers et al. 2021). The work described in this paper responds to multiple calls for critical analyses of the outcomes of knowledge co-production (Lemos et al. 2018; Jagannathan et al. 2020; Wyborn et al. 2019). Through analysis of a unique dataset—water and climate USDA-NIFA project case studies—we found that high interaction co-produced projects were less frequently associated with adaptive capacity outcomes than projects with lower levels of interaction. Indeed, projects that were heavily scientist-led had many adaptive capacity outcomes that were lauded by case study interviewees, including project stakeholders. That said, if looking at transformative shifts that challenge underlying assumptions or result in a worldview change—Scope 2 outcomes—co-production was clearly influential. Jagannathan et al. (2020, pp. 4–5) define Scope 1 outcomes as those that create actionable knowledge (our “deepening understanding”, “strengthening communities”, “utilization of knowledge”, and “catalyzing action” codes) and Scope 2 outcomes as those that challenge the norms and structures of both science and society. Cases in this study that were coded for Scope 2 outcomes challenged societal and scientific norms and three of the four cases with Scope 2 outcomes included co-production processes. Moreover, the two cases with the highest counts of adaptive capacity outcomes were both coded for co-production attributes and both of these cases worked with Indigenous communities [see also (Gagnon et al. 2022) who write about the importance of language in working with Indigenous communities and note that language can both reflect and reinforce worldviews]. Thus, we think our research points to a potential connection between high-interaction co-produced projects and transformative change. Yet, the lack of clear relationships between knowledge co-production and adaptive capacity outcomes in the USDA-NIFA case studies highlights several important considerations for applied science.

Ideally, co-produced projects are “iterative and inclusive processes that are responsive and adaptive as conditions change and as participants acquire better understandings of both the problems they confront and each other’s ways of knowing” (Wyborn et al. 2019, p. 325). However, co-production is expensive in terms of time investment and can even produce negative or unequally distributed outcomes (Popovici et al. 2020). Lemos et al. (2018) make the important point that because scientists must invest time to build relationships with non-academic partners, they may, because of feasibility, focus all their time with certain groups, “privileging familiarity over the uncertainty of new partners or issues” (p. 723). Due to this and other factors, multiple authors have warned of the potential of co-production processes to entrench social inequalities (Musch and von Streit 2020; Turnhout et al. 2020; Järvi et al. 2018).

There are useful alternatives to knowledge co-production processes. For instance, in our cases, we found that projects with low levels of stakeholder interaction resulted in multiple adaptive capacity outcomes like capacity building and collaborative networks. Continuous interactions and partnerships as integral to co-production of knowledge may be warranted for specific groups. Our evidence suggests that including Indigenous communities and partners in projects through intensive co-production processes (if they want to work with collaborators and agree to the project in the first place) (Torso et al. 2020, p. 2342) may positively influence adaptive capacity outcomes, as was exemplified in Case 5 in our data (and should be designed through a partnership format and implemented through highly interactive co-production processes). However, as was shown in Case 6, purely stakeholder-based projects that do not include a partnership design or include stakeholders in project design, although still time-intensive, can still have remarkable adaptive capacity outcomes (including Scope 2 outcomes).

Co-production of knowledge can take multiple forms and include multiple stages, which affect project outcomes, ranging from increased participant knowledge to driving collective action for change (e.g., Mees et al. 2018). We found this range of adaptive capacity outcomes within our data, from projects that increased stakeholder knowledge or where stakeholders used knowledge generated from the project, to changed stakeholder behaviors, or even Scope 2 outcomes. Almost all of the projects we analyzed suggested capacity building and new or strengthened collaborative networks. Both of these categories fall into the “strengthened communities” adaptive capacity outcome code. These outcomes are important because they do just that—build capacity to adapt to change over the long-term, allowing communities to avoid, bounce back from, or collectively respond to natural resource threats (Armitage 2005; Berkes et al. 2008; Smit and Wandel 2006). In the case of these outcomes (capacity building and collaborative networks), most of the projects were categorized as “stakeholder-based”, with few coded as “co-production present”. This finding suggests revisiting the importance of thinking of co-production of knowledge as a continuum of participation with stakeholders, based on goals and project/stakeholder capacity (Lemos et al. 2018; Neef and Neubert 2011, p. 190–191; Reed et al., 2018, pp. s14–s15; Watson 2014 pp. 64–68), because positive outcomes can be achieved whether or not they are co-produced.

Co-production of knowledge that foster Scope 2 outcomes can take a long time relative to discrete project funding periods. Time and cost can make creating and documenting Scope 2 outcomes of projects extremely difficult (Jagannathan et al. 2020). Because of the challenges of documenting these types of outcomes, PDs may be hesitant to make claims about potential transformative changes resulting from a project. Furthermore, Scope 2 outcomes may be more appropriately measured at institutional and societal levels (Harvey et al. 2019). Indeed, sustaining their projects, as well as sustaining collaborative relationships, past the end of grant funding was noted in almost all of our case studies as an overall challenge. This time constraint points to the need for longer-term evaluations of applied projects with the express purpose of understanding long-term impact and change who also note that institutional expectations and time limits on grant funding constrain researchers’ ability to conduct transdisciplinary research).

Although our data do not indicate a consistent relationship between co-production of knowledge and adaptive capacity outcomes, the limitations of and potential biases in our data warrant caution when drawing sweeping conclusions about potential limitations of co-produced processes. First, the idea of examining how co-production of knowledge related to adaptive capacity outcomes emerged from our initial analysis of the case studies. Through that analysis, we saw that adaptive capacity appeared to be a strong outcome for many of the case study projects. We therefore decided to explore whether co-produced projects resulted in adaptive capacity outcomes. The original data collection for the case studies was designed to examine successes and challenges of the projects. It is therefore possible that we did not ask questions that could determine relationships between co-produced projects and adaptive capacity outcomes. Moreover, although we found that stakeholder engagement goals in USDA-NIFA CRIS reports aligned fairly well with cases attempting to engage with stakeholders, adaptive capacity goals were not stated in most cases. Yet despite adaptive capacity goals lacking specificity in government reports, interviews revealed all cases had adaptive capacity outcomes—our case study process revealed outcomes that were perhaps not part of project design from the outset. Second, a plausible explanation for the results we observed is that the conditions necessary for fostering adaptive capacity outcomes likely included factors that go well beyond our research purpose (i.e., whether or not it was co-produced), perhaps more related to elements of project design such as partnerships, building projects from a foundation of existing elements, or including a diversity of expertise on the project team. Indeed, further work that examines how projects with low levels of stakeholder interaction achieved adaptive capacity outcomes is warranted. We also do not claim generalizability of our findings.

Despite the apparent implications and limitations of our results, PDs may be wise to approach their work using principles of knowledge co-production for other reasons than those related to adaptive capacity. For example, co-production approaches may be particularly well-suited to the sorts of “boundary managing” functions, including communication, translation, and mediation, that Cash et al. (2003, p. 38) argue are essential for increasing the salience, credibility, and legitimacy of information designed to bridge the knowledge-action divide [see also Delozier et al. (in review) who explore how people with boundary spanning skills work across disciplines to build trust in collaborative processes—people with these skills would be invaluable in co-production processes]. Moreover, as we stated before, working with Indigenous communities may warrant a co-production approach when appropriate and if desired by the community (Torso et al. 2020). Finally, knowledge co-production processes that build materially substantive partnerships and constituencies beyond the university setting may be especially instrumental for sustaining the legitimacy of public research institutions whose mission and finances are increasingly threatened, in part, by accusations of out-of-touch elitism, culture-war politics, and state and federal austerity budgets (Wu 2017, p. 2).

5 Concluding remarks

In this paper, we have presented evidence that highly interactive co-production project design may not be necessary to achieve adaptive capacity outcomes. Overall, we contend that projects that seek to address complex social-ecological problems should be designed towards stakeholder needs and strengths, whether or not highly interactive co-production strategies are used. Projects should be designed to meet partner goals (including desired outcomes), recognizing time and resource constraints. We suggest this approach for meeting project partner goals, despite challenges that come with stakeholder engagement, as we found in our own data and as others have stated (e.g., projects take more time, concerns with data privacy, and stakeholder interest in and capacity for participating). Instead of touting co-production of knowledge as the only acceptable approach to collaborative processes, we support the conclusion of others who suggest that PDs should step back and think through project goals, project and stakeholder capacity, and what stakeholders desire (Lemos et al. 2018; Norström et al. 2020).