Introduction

Contemporary sustainability challenges in food and farming systems are complex, interconnected, uncertain, and multidisciplinary in nature. Examples of such sustainability challenges are related to, but not confined to, degradation of agro-ecosystems and natural ecosystems; emissions of greenhouse gases and toxic waste; food insecurity; climate variability and change with its associated extreme weather and disruption of expected patterns; decline in ecosystem services; degradation, depletion and spillage of natural resources such as water and soil; loss of biodiversity; and post-harvest losses (Struik et al. 2014).

Associated with these sustainability challenges, there is a growing emphasis in food and farming systems, on building up farmers’ adaptive capacity through an ongoing process of learning, acting and reflecting to cope with, prepare for, and adapt to deal with complexity and uncertainty (Newsham and Thomas 2011; Wise et al. 2014; Grunblatt and Alessa 2017; Hazard et al. 2017). Strengthening farmers’ adaptive capacity depends on tools, processes, and practices that support the integration and co-production of knowledge (Tschakert et al. 2016). Knowledge co-production is defined as “the collaborative process of bringing a plurality of knowledge sources and types together to address a defined problem and build an integrated or systems-understanding of the problem” (Armitage et al. 2011, p. 996). Knowledge integration processes bring together bodies of knowledge that are structured in fundamentally different ways (Godemann 2008, pp. 628–629). Blackstock et al. (2007, p. 279) define participatory sustainability science as “the co-generation of knowledge about socio-ecological systems drawing on multiple understandings in an ongoing collective dialogue to transform practice, where academics and stakeholders are all co-researchers”. Integration and co-production of knowledge aim to connect different knowledge systems, including academic, practitioner, lay, and local knowledge (Fazey et al. 2010) as more inclusive ways of generating relevant, robust, and actionable knowledge for a sustainable future (Hazard et al. 2017).

Bringing together academic, practitioner and other societal actors’ interests, perspectives, and information to address challenges in social–ecological systems is well-served by a transdisciplinary approach (Stokols 2006; Hirsch Hadorn et al. 2006; Jahn et al. 2012; Lang et al. 2012; Mauser et al. 2013; Polk 2014; Scholz and Steiner 2015a; Scoones 2015; Roux et al. 2017). Three phases of transdisciplinary research for sustainability can be differentiated as (1) collaboratively framing the problem and building a research team; (2) co-producing solution-oriented knowledge; and (3) (re) integrating and applying the produced knowledge in both scientific and societal practice (Lang et al. 2012, pp. 27–29). This can be achieved by facilitating a collaborative learning process when the emphasis is on methods that give participants a stake in the research process, integrating and expanding their knowledge and capacities such that actions can be improved (Restrepo et al. 2014).

Several scholars have evaluated transdisciplinary research in land-use systems (Hegger et al. 2012; Wiek et al. 2012; Njoroge et al. 2015; Vilsmaier et al. 2015; Di Iacovo et al. 2016; Schmid et al. 2016; Hubeau et al. 2017; Roux et al. 2017), developed evaluation frameworks (Blackstock et al. 2007; Walter et al. 2007; Polk 2015), quality guidelines (Carew and Wickson 2010; Jahn and Keil 2015; Belcher et al. 2015), and conducted meta-analyses to critically examine transdisciplinary research implementation and outcomes (Brandt et al. 2013; Polk 2014; Zscheischler and Rogga 2015; Schuck-Zöller et al. 2017; Holzer et al. 2018). Some of these evaluations focus on the participatory process, i.e. the quality of the collaboration (Hegger et al. 2012); others on the outcomes, i.e. when and how aims are reached (Walter et al. 2007; Njoroge et al. 2015; Schmid et al. 2016); and sometimes both (Blackstock et al. 2007).

Transdisciplinary research builds on knowledge integration and co-production between academic and non-academic actors. However, the success of this type of research is usually not evaluated based on assessments of the success in facilitating learning processes, namely, knowledge integration and co-production (Jahn and Keil 2015; Zscheischler and Rogga 2015; Westberg and Polk 2016). Difficulties evidencing learning in transdisciplinary research studies have been discussed (e.g. Wiek 2007; Späth 2008; Vilsmaier et al. 2015; Westberg and Polk 2016; Holzer et al. 2018). Involving actors with different reference systems, objectives and interests means there is not one worldview, theory, or methodological canon against which learning can be evaluated (Späth 2008). The discrepancy between the principles guiding how transdisciplinary research is enacted versus how it is evaluated is outlined in two parts.

First, relevant aspects that fostered learning as well as assessments of what participants learn and how they benefit is usually evaluated from the researchers’ point of view. Methods of project evaluation are designed by scientists following their evaluation needs (Roux et al. 2017; Schuck-Zöller et al. 2017; Hubeau et al. 2017; Schneider and Buser 2018). As argued by Jahn and Keil (2015), transdisciplinary research should respond to societal demands, and the evaluation should reflect it. However, there is often little information available from the non-academic actors’ point of view about their engagement in research. Even though research participants are meant to be central in participatory and transdisciplinary approaches, too often, they are not central to the evaluation as put forward by Blackstock et al. (2007), Polk (2015), Seijger et al. (2015), and Zscheischler and Rogga (2015). Specifically, Scholz and Steiner (2015b, p. 663) state that “The evaluation of transdisciplinary processes is a special methodological challenge” in part because “the outcomes, i.e. what has been learned and what has been changed should be determined/measured” and need to be assessed by both people involved in practice and by scientists (Scholz and Steiner 2015b, p. 659).

Second, there is limited empirical information about how knowledge is transformed into action. This lack of information in transdisciplinary research projects is stressed by Blackstock et al. (2007), Lang et al. (2012), Wiek et al. (2012), Cornell et al. (2013), Seijger et al. (2015), Zscheischler and Rogga (2015) and Schmid et al. (2016). The underlying reason for this is twofold. On one side, transdisciplinary research aims at producing transformation knowledge, i.e. the knowledge required for changing a problematic situation to an improved one. Transformation knowledge is assessed based on indicators such as transformation strategies or personal opportunities for action (Walter et al. 2007), the development of relationships or skills (Blackstock et al. 2007; Hubeau et al. 2017), and presence of boundary objects (Hegger et al. 2012). Nevertheless, these indicators do not denote change in actions. On the other side, there are few methods applicable for analyzing the relation between knowledge and action. Methods used include the Learning Loops to explain the stages of learning related to reflection and action (Argyris and Schön 1978), mental models and cognitive maps as internal constructs of farmers’ knowledge in relation with their production systems (Fairweather 2010; Gray et al. 2012; Vanwindekens et al. 2013), and second-order observation used to systematically reveal knowledge underlying actions and change in actions (Kaufmann 2011).

We respond to these knowledge gaps by analyzing: (A) farmers’ perspectives on the collaborative learning process (i.e. relevant aspects that fostered learning, what they learned and how they benefited) and (B) how farmers’ new knowledge served as the basis for changes in action (i.e. how new knowledge is transformed into action). We developed a framework to assess a collaborative learning process based on farmers’ perspectives with the following structure: (1) what farmers found interesting and useful about the process; (2) what farmers learnt (i.e. theory and practice) from the process; (3) how their new knowledge served as the basis for changes in action; and (4) how they benefited from these changes. This framework was applied to evaluate a 2-year collaborative learning process with two smallholder dairy farmer groups in Nakuru County, Kenya.

In this research, we offer an example from food and farming systems of how to strengthen sustainability science by evaluating transdisciplinary research from the perspective of societal actors involved in the learning process. In particular, we assess multiple learning outcomes from farmers’ perspectives with specific attention to knowledge integration and co-production and factors influencing the learning process. This evaluation provides relevant information regarding why and how a collaborative learning process works. It can enable scientists, research funders, and practitioners to analyze and reflect on these processes and the influencing factors to facilitate experimentation, innovation, and adoption as part of a learning process for sustainable change in food and farming systems.

First, we present our evaluation framework for collaborative learning processes. Next, we describe the context in which this collaborative learning process occurred, and then explain how data were collected for the evaluation and analysis. The results show how this evaluation framework was applied with two smallholder dairy groups. Finally, the findings are discussed in light of opportunities and challenges for sustainability-oriented transdisciplinary research more generally.

Conceptual framework: evaluating collaborative learning success

To assess the learning outcomes of a collaborative learning process from the farmers’ perspective, we used a modified version of Kirkpatrick (1998) four-level evaluation of training programs: reaction, learning, change in actions, and impact. Table 1 shows the levels and which variables are measured at each level. The first level includes an assessment of participants’ reaction to the learning process, i.e. what farmers found interesting and useful. The second level refers to what participants learned during the process, i.e. the knowledge and skills gained. The third level refers to how participants changed their actions to apply the new knowledge on their farms. The fourth level refers to the impacts of the learning process. Participatory methods generated qualitative data to examine the four levels from farmers’ perspectives (Table 1).

Table 1 Collaborative learning evaluation framework.
Full size table

To systematically analyze how new knowledge served as a basis for changes in actions (third level), we combined components of Argyris and Schön’s Learning Loops (1978) with second-order cybernetics using the Control Loop Model (Kaufmann 2007, 2011). The Control Loop Model helps to systematically reveal the knowledge driving actions when managing a system. In this case, what farmers observe and the cause–effect relations on which they base their actions (Fig. 1). Hence, we are not observing the system, but the farmer’s observations of the system. When using second-order observations with Learning Loops, we analyze changes in actions after a process of knowledge co-production and integration, as will be explained below.

Fig. 1
figure 1

(modified from Kaufmann 2007)

Control Loop Model used analyzing for second-order observations.

Full size image

Central to Argyris and Schön (1978) approach is the relationship between knowledge and action in a learning process, where individuals (or organizations) take an action and simultaneously reflect on it to learn. Learning starts when individuals compare the observed outcome with the desired outcome of an action (Argyris and Schön 1978, p. 2). A discrepancy between the observed and desired outcome of an action is considered an error and leads to a problematic situation, which calls for a period of joint and collaborative reflection, i.e. the detection and correction of the error. Depending on the degree of reflection, learning processes can be differentiated into single-loop, double-loop, or triple-loop learning (Table 2; Fig. 2).

Table 2 Relationship between Learning Loops (Argyris and Schön 1978; Flood and Romm 1996; Hummelbrunner 2015) and the Control Loop Model (Kaufmann 2007)
Full size table
Fig. 2
figure 2

(modified from Argyris and Schön 1978; Flood and Romm 1996)

Learning Loops adapted to the activity level.

Full size image

Single-loop learning is achieved when errors in conducting an activity are detected and the activity is improved by altering actions (Argyris and Schön 1978), e.g. correcting or improving livestock management practices. It provides short-term solutions to implementation problems, dealing with symptoms rather than with root causes. Hence, this reflection level is often used to correct quality and efficiency problems (Hummelbrunner 2015).

However, when dealing with uncertainty and change, e.g. due to climate variability or market instability, simply correcting errors in routine actions is usually insufficient. It requires a change in the understanding of the system (March 2006; Howden et al. 2007), and thus reflection at higher levels. Double-loop learning occurs “when mismatches are corrected by first examining and altering the governing variables and then the actions” (Argyris 1999, p. 68). Thus, actions are replaced with new actions based on farmers’ recognition of new cause–effect relations (Restrepo et al. 2016). It implies a reflection on problems and how aims can be achieved, incorporating experience into feedback to inform planning (Pahl-Wostl 2009).

Argyris and Schön’s original work did not include triple-loop learning. Other authors have conceived it as reflecting on how we organize ourselves to make decisions, what kinds of principles we apply to decide whether something is right or better, and whether such principles need to be changed (Hawkins 1991; Flood and Romm 1996; Romme and Van Witteloostuijn 1999). At the activity level, triple-loop learning occurs when actors reflect on the aims of their actions, i.e. “why I do what I do”, and adjust them or replace them. Double- and triple-loop learning help people understand which strategy works better to achieve a goal, leading to better mid- and long-term solutions (Hummelbrunner 2015). Thus, these reflection levels are useful for making informed decisions while coping with rapidly changing conditions, and aid in enhancing adaptive capacity. Intrinsic are trust-building work and willingness to take risks (King and Jiggins 2002; Armitage et al. 2008).

Transdisciplinary research: context and design

Complex problematic situation: wicked problem in smallholder milk production in Nakuru County, Kenya

Smallholder dairy production is typically part of mixed crop–livestock farming systems that operate with low-external inputs. Milk production is important for household liquidity, as income is obtained both daily and monthly depending on the quantity of milk allocated to different buyers. During the dry season, milk production is the only source of income for many smallholder farmers as opportunities for casual labor off-farm are reduced in rural areas and there are no farm related incomes.

Smallholder dairy farmers in the study area generally keep a maximum of three crossbred cows, for both milk sales and home consumption. Climate change and growing population contributed to a shift in land-use that led to a sharp decline in communal lands in which animals were grazing. Cows are now commonly fed napier grass, crop residues (i.e. maize stalks, bean and pea stubble, as well as residues from carrots, cabbage, and potatoes), and weeds. Access to inputs is constrained by poor infrastructure and low capital endowment. Dairy production in the area is influenced by cyclical patterns of rainfall and fodder availability. During the rainy season, when fodder is abundant, milk production increases to the point that there is an oversupply, with its negative impact on selling prices. Furthermore, milk tested at collection points is often rejected. During the dry period, farmers often do not have enough fodder for their cow(s), which sharply increases labor demand for fodder collection and leads to low milk yields. This makes it difficult to profit from favourable prices during the 3 months long dry period.

On-farm milk losses were defined by all participating actors (scientists and farmers) as the difference between potential and actual milk yield, mostly during the dry season, and random rejections during the rainy season (Restrepo et al. 2016). In this study area, we found that the milk produced is low in protein and high in fat. This makes milk density low, Footnote 1 and is a possible explanation for milk rejection during the rainy season when only tested with a lactometer (measuring density).

Collaborative learning process

This project was conceptualized as four interconnected phases (Restrepo et al. 2014): (A) establish the collaboration; (B) process of dialogue; (C) process of discovery; and (D) applying the new knowledge (Fig. 3).

Fig. 3
figure 3

Collaborative learning process (Restrepo et al. 2014, p. 45)

Full size image

The collaborative learning process consisted of a partnership between two smallholder dairy farmer groups, a social agricultural scientist, and a young animal scientist from the area who acted as translator/interpreter. Smallholder dairy farmers from the Mukinduri Self-Help Group (SHG) and Lare Community-Based Organization (CBO) in Nakuru County, Kenya (Fig. 4) were engaged in a collaborative learning process led by the first author for 2 years as part of the umbrella transdisciplinary research project for reducing food losses and adding value (RELOAD). The first author had a role similar to the transdisciplinary champion described in Miah et al. (2015), as she acted as the main researcher, and facilitated group activities throughout the process described in this article.

Fig. 4
figure 4

Map of Nakuru County, Kenya depicting the location of group 1 (Mukinduri SHG) and group 2 (Lare Livelihoods CBO)

Full size image

In response to the topics put forward by the farmers, different academic and non-academic actors were involved during various phases of the process. Academic actors included: animal scientists from the local university that facilitated workshops on drought tolerant fodder crops, home-made concentrates, and yogurt making; and a team of dairy scientists/microbiologists from the local university that analyzed milk samples from every lactating cow owned by Mukinduri group members. Non-academic actors included: extension officers from the Ministry of Public Service, Youth and Gender Affairs offering training in group dynamics; local farmers from neighboring villages leading farmer-to-farmer exchanges regarding silage production, fodder production, construction of zero-grazing units, and record keeping; and representatives from the main dairy companies that gave information about different marketing possibilities. Table 3 shows the methodological sequence of tools and methods used and the different actors participating in this transdisciplinary research.

Table 3 Methodological sequence, duration, and actors in this collaborative learning process, 2013–2015
Full size table

During the process of establishing the collaboration (A), the relation between the farmer groups and researchers was institutionalized. The collaboration was established through a mutual selection process. Guided by explicit and implicit selection criteria including a willingness to learn, researchers built relationships with two smallholder farmer bottom–up initiatives. The two farmer initiatives also chose the researchers by pro-actively engaging in the process and requesting a collaboration contract. Farmers had the status of co-researchers, they had a voice in the process of defining, designing, testing, implementing and evaluating sustainable solutions for jointly defined real-world problems, i.e. they had equal decision rights in the process (Restrepo et al. 2015).

During the course of the project, groups organized themselves into sub-groups, e.g. in one group by gender and age, and in the other by geographical proximity. Sub-groups met to plan for and reflect upon activities implemented. In smaller groups, discussion and argumentation were more profound because members were better able to express themselves. These dynamics enhanced participation of members and reinforced learning together. It promoted all voices to be heard, which aided in group decision-making. To facilitate the operationalization of planned activities, group members decided to distribute functions and responsibilities among themselves. All members had a specific responsibility and were enthusiastic about following through. Groups met regularly, and sub-groups reported what they completed during each period. This meeting structure allowed for adequate time to share and plan as a group, as well to solve problems. Meeting locations rotated between the farms of different members, creating an opportunity to learn different approaches used on each farm. Under these conditions, members grew to know each other better, new friendships were initiated, and confidence among the group was strengthened. Together, these factors helped to balance power relations among members and between group members and researchers.

To promote knowledge integration during the dialogue phase (B), group members and researchers (including dairy and animal scientists) arrived at a common understanding of the sustainability challenges and co-identified contextualized solutions, e.g. different types of fodder and silage to increase milk quantity, and monitoring and evaluation tools to test on-farm milk quality and udder health (Restrepo et al. 2016). We used participatory methods to depict an in-depth picture of actors´ perceptions and attitudes towards a certain issue (Table 3). These methods and tools allowed farmers to express the aspects that were most relevant to them, enhancing the exchange of ideas, knowledge, and perspectives. A respectful and trustful dialogue thus was established between researchers and farmers.

Through the discovery phase (C), farmers were able to fill knowledge gaps through a knowledge co-production process. The farmer groups applied for, and were granted with self-managed innovation funds to test self-defined solutions. Knowledge co-production in the discovery process consisted of (1) farmer-to-farmer exchange sessions with peers making silage, growing different types of fodder or who had built a zero-grazing unit; (2) farmer-led experimentation with different types of fodder and silage; (3) a participatory monitoring and evaluation system to record variations in milk quantity and to test milk density and mastitis (Restrepo et al. 2015); and finally at the end of the whole process (4) sharing stories of change using the Most Significant Change. Applying new knowledge (D) consolidates the new activity into a more broadly recognized social practice. This phase, led by the farmer’s groups, is ongoing Footnote 2. It includes other activities that are part of transdisciplinary processes in the larger RELOAD research project. For example, (1) a continuation in building group members’ capacity through the participation of two representatives from each group in small-scale dairy multi-stakeholder platforms, and other activities such as visits to agricultural fairs and (2) knowledge dissemination in the area (other villagers and other villages) via farmer-to-farmer exchange sessions (Albrecht 2017; Krause 2017).

Methodology

Data collection

Case studies are used in evaluation research to allow the phenomenon studied to be addressed in context with all its complexity (Yin 2013). We systematically documented farmers’ own perspectives and knowledge from February to November 2015 using different oral inquiry methods (Table 1). These consisted of 40 semi-structured interviews with group members (20 in Mukinduri and 20 in Lare) including critical incident questions, where farmers narrated their most remarkable learning experience—including both a satisfactory and an unsatisfactory day (Brookfield 1995)—19 narrative interviews (12 in Mukinduri and seven in Lare) exploring farmers’ experiences during the collaborative learning process (Jovchelovitch and Bauer 2000), and two sessions of the Most Significant Change (MSC) technique with each group (March 2015–20 farmers in Mukinduri and 14 in Lare—and October 2015–20 farmers in Mukinduri and 16 in Lare). This technique is a form of participatory monitoring and evaluation that provides data on impact and outcomes from actors’ own perspectives (Davies and Dart 2005). The different inquiry methods stimulated farmers to narrate the events they considered relevant, referred to as relevance fixation (Jovchelovitch and Bauer 2000). The duration of semi-structured (SSI) and narrative (NI) interviews was between 45 and 90 min. The MSC sessions lasted ca. 120 min. With permission, each individual interview and group session was audio recorded.

A co-inquiry tool was developed with the Mukinduri group members because they expressed an interest in knowing how much milk they produced together. Once a month for a year, every group member recorded how much morning and evening milk was produced and its use (e.g. amount sold, home consumption, given to calf).

The percentage increase in milk production was calculated with farmers by comparing two points in their own records; i.e. farmers compared the amount of milk before and during the feeding strategy introduced. This testing occurred during the same season.

Data analysis

Audio recordings, totaling 54 h, were translated, transcribed, and analyzed using qualitative content analysis. This method was developed to analyze perspectives on issues and processes in social groups (Attride-Stirling 2001; Braun and Clarke 2006; Flick 2009).

To analyze farmers’ perspectives on the collaborative learning process, codes were inductively identified to develop the coding framework. Codes pertain to themes such as inclusion, exchange, practice, learning topics, and benefits from the learning process. To analyze how new knowledge was transformed into action, a deductive coding frame was developed using the Control Loop Model (Kaufmann 2007). As the information collected covered different topics, the analysis was restricted to changes in knowledge related to trying different feeding strategies and monitoring and evaluation tools. We selected (a) the topic of feeding strategy as it was a high priority for farmers, and ranked first in their knowledge gains (see Fig. 6) and (b) the topic of monitoring and evaluation tools as these directly enhanced learning, playing an important role in the reflective observation stage of Kolb's (1984) experiential learning cycle. The codes developed refer to farmers’ aims, observations, cause–effect assumptions, and actions. Coding involves identifying ‘if… then’ and ‘in order to’ relations.

Once codes were applied to the transcribed material, quotations were abstracted and patterns were identified. Results are presented around key themes that emerged from participants and were consistent across the data set. Direct quotations that are representative of recurrent themes add farmers’ voices to the results. These are labeled with a system that designates: gender, group membership (L for Lare or M for Mukinduri), a random number #, and the inquiry method [e.g. (fe)male farmer—L/M#, SSI/NI/MSC].

Results

The results are presented according to Kirkpatrick’s four levels for evaluating learning processes: reaction, learning, change in actions, and impact.

What farmers found interesting and useful from the process

Farmers’ overall reaction towards the collaborative learning process was positive, “we were discussing… that knowledge is more than money. Because if it was money we would have shared amongst us, spent and forgot” (male farmer M37, SSI). Farmers specifically valued (1) their inclusion in planning the approach, (2) the farmer-to-farmer exchange sessions, and (3) learning from practice, as outlined below.

Inclusion

Farmers appreciated their decision power to direct the approach, i.e. arriving at a common understanding of problematic situations, and agreeing on goals and strategies to address them. As a farmer stressed, “after we met, we had the idea that we could combine forces and test our milk, source ideas from outside, change the mode of feeding, construct a unit… take care of the cow so that it could give us good produce” (male farmer—M39, SSI). Through farmer-managed innovation grants, farmer groups experimented with feeding strategies, including different types of fodder and silage, and the construction of zero-grazing units for group members. In a participant’s words, “through the grant, we have been able to experiment” (male farmer—L187, SSI).

Exchange

Sharing information via farmer-to-farmer exchange sessions was beneficial, as expressed by a farmer, “we are able to move out and visit other farmers and I think as we visit more farmers we gain new knowledge” (male farmer—M179, SSI). Regarding the importance of these exchanges, a farmer commented: “those who have not gone out think it is only their mother who knows how to cook well” (male farmer—M197, NI). During such exchanges, farmers acquired information related to both the production context and the process of achieving a certain outcome, e.g. making silage. The importance of learning processes, not only outcomes, can be seen by the following comments: “we even did silage together and shared how to do it… so, I was able to follow from the first step to the last steps” (male farmer—L25, SSI), and “I have attended agricultural shows and there you are shown the finished product. Here we have learned the process ourselves” (male farmer—L196, SSI). Stressing the importance of their own production context when implementing a new farming practice, a farmer remarked, “so from visiting other farmers you gather a lot... so if I need to implement, it will be my original idea. I choose what best suits me” (male farmer—M197, NI).

Practice

Farmers emphasized the value of learning by doing. Indeed, farmers distinguished learning from practice, as can be seen by the following comments: “the most important thing is to apply. You can learn new things, but if you do not apply, you are missing the point” (male farmer—M35, NI), and “it’s a lot of power to learn and to practice” (male farmer—M32, NI). During the farmer-to-farmer exchanges and farmer-led experimentation, farmers gained practical experience. This was complimented by dialogue as farmers actively monitored results and shared findings with other group members. As emphasized by a farmer: “the most satisfying part was to experiment and learn with the fodder we received. I was also satisfied as we shared the results of our experiments and we learned from each other” (male farmer—L195, SSI).

What farmers learned from the collaborative learning process

Because of the importance of learning by doing for group members, both theory and practice are included in this section. Farmers shared with the researchers during the semi-structured interviews that they gained diverse knowledge and skills, with a total of 23 different topics and 291 quotations (Fig. 5a). Learning was related to (1) fodder production, (2) dairy farming, (3), innovation capabilities, and (4) organizational skills. In Mukinduri, knowledge and skills gained regarding fodder production was emphasized by 95% of the group, and construction and management of zero-grazing units by 75%. In Lare, 65% of the group reported gaining knowledge and skills on silage making and 50% on fodder production. In both groups, the overall participation in the experimentation was high (Fig. 5b). The collaborative learning process was described as enabling participants to “see things in a different way” in open interviews with 25% of Mukinduri group members and 15% of Lare group members.

Fig. 5
figure 5

a Knowledge and skills gained as stated by group members (frequency of responses; multiple answers per respondent; *number of quotations) and b participation in the experimentation process

Full size image

Farmers’ new knowledge as the basis for changing actions

When analyzing knowledge that underlies changed actions, we found that single-, double- and triple-loop learning after group members tried (1) feeding strategies (Table 4) and (2) monitoring and evaluation tools (Table 5).

Table 4 Feeding strategies learned during a collaborative learning process that led to changed actions
Full size table
Table 5 M&E tools learned during a collaborative learning process that led to changed actions
Full size table

After experimenting with different feeding strategies, farmers changed their practices (Table 4) by (1) implementing corrective actions (e.g. cutting fodder into small pieces); (2) learning new cause–effect relations (e.g. intercropping fodder); or (3) changing aims (e.g. allocating land for fodder production). Allocating land to grow fodder is a new action entailing a new way of rearing cows, hence, triple-loop learning. Changing aims led to further changes in farmers’ practices to adapt to the new aims. Planting fodder for the cow often led to extra fodder, contributing to adjustments from learning new cause–effect relations related to storing extra fodder and corrective actions such as building a feeding trough.

After trying different monitoring and evaluation tools, farmers changed their practices mainly based on learning new cause–effect relations and being able to test their effect (e.g. monitoring and improving fodder quality). Milk production records increased farmers’ opportunities for monitoring dairy cow performance, incomes, and household expenditures as explained by a farmer: “with the records… now I am able to learn what I am getting from the cow and… take care of my home expenditures; buy salt and pay school fees for my kids. With the records, I… account for every coin… I am even able to save” (female farmer—M34, NI). Some farmers from both groups complemented their experiments by monitoring milk quality: “I have used the lactometer. I wanted to know whether the density [of the milk] is good. It went to 29, even 31. This was after feeding the cow with the new feed” (female farmer—M40, SSI). Table 5 shows more examples of monitoring and evaluation (M&E) and changed practices.

Benefits to farmers from these changes

Figures 6 and 7 show group members’ perception of multiple benefits from a collaborative learning process: (1) farmers’ change in practice (12 different benefits, 184 quotations) and (2) farmers’ capacity to act (nine different benefits, 154 quotations).

Fig. 6
figure 6

Perceived benefits related to farmers’ change of practice [Narrated in two rounds of stories of change during a collaborative learning process in Nakuru County, Kenya (March 2015 - 20 farmers in Mukinduri and 14 in Lare—and October 2015—20 farmers in Mukinduri and 16 in Lare)] (frequency of response; multiple answers per respondent)

Full size image
Fig. 7
figure 7

Perceived benefits related to farmers’ capacity to act [Narrated in two rounds of stories of change during a collaborative learning process in Nakuru County, Kenya (March 2015 - 20 farmers in Mukinduri and 14 in Lare—and October 2015—20 farmers in Mukinduri and 16 in Lare)] (frequency of response; multiple answers per respondent; *number of quotations)

Full size image

Due to changed practices, milk production increased on average by 80% in both groups (Fig. 9). In Mukinduri, participants also observed healthier cows and a reduction in workload (Fig. 6). This reduced workload was especially important for women who bore responsibility for fodder collection. This led to reduced household conflict, as can be seen in the following comment, “since we started planting fodder, the wife is not bothered, even she has time to relax. Otherwise, she would be searching for fodder... it has reduced conflict” (male farmer—M37, SSI). In Lare, members felt better prepared to cope with the dry season, and observed increased economic benefit and healthier cows (Fig. 6). A farmer recalled from the farmer-led experimentation, “I remember one day during the dry season, the fodder we planted, like sorghum, was able to survive. It was too hot and I was satisfied with the fodder because I was able to feed the cow for three weeks, so I saw the importance of sorghum...” (male farmer—L24, SSI).

From the knowledge co-produced and the change in practices, farmers’ capacity to act was augmented, as can be seen by the following comments: “through sharing, I have been able to learn a lot about cows and I am motivated to continue because I saw others are doing a lot. And if they are doing it, why not me?” (male farmer—M209, NI) and “I am now like a giraffe, observing very far, I am able to project for the future, I am ready to be an example to the other youths” (male farmer,—L43, MSC). In Mukinduri, during the second MSC round of stories of change (October 2015), group members recognized the efficiency of collective action (Fig. 7). A farmer explained: “There are many things that you can do as a group compared to individually… like constructing the zero-grazing units or planting new fodder. It is easier to come up with ideas as a group” (male farmer—M179, SSI). Another benefit farmers shared was a feeling of increased friendship and empathy among members. During the first MSC round of stories of change (March 2015), few Lare farmers narrated stories associated with their own capacity to act. However, during the second MSC round, they brought up the importance of having stronger networks and more trust (Fig. 7).

Figure 8 shows the average milk production in liters per cow/day and average milk commercialization in liters per farmer/day. Average yields for the area were 10.4 ± 5.3 l milk/day/farm, while average milk sales were 7.3 ± 3.9 l milk/day/farm n = 199 and 197, respectively, recorded by 20 farmers.

Fig. 8
figure 8

Average milk production (l) cow/day and milk commercialization (l) farmer/day, co-inquiry with Mukinduri SHG (20 group members, once per month over a year)

Full size image

After trying different types of fodder, Mukinduri group members compared the amount of milk before and during the time they were testing different feeding strategies. 90% of Mukinduri group members and 80% of those in Lare recorded an increase in milk production greater than 50% (Fig. 9). In Lare, four members did not have increased production because their cows were dry, while in Mukinduri, two members described an increase of 25% even as their cows were about to become dry.

Fig. 9
figure 9

Percentage of change in milk production during farmer-led experimentation in a collaborative learning approach in Nakuru County, Kenya (data collected from farmers’ own milk records)

Full size image

Discussion

The four-stage collaborative learning evaluation framework applied in this transdisciplinary research shows what smallholder dairy farmers in Nakuru County found attractive and useful, what they learned, what they applied, and how they benefited from the process. This method is a way of assessing the process and impacts from the perspective of those involved (Sect. “Farmers’ perspectives of the collaborative learning process”). Combined with second-order cybernetics using the Control Loop Model and the organizational learning theories with the Learning Loops, it can further serve as an example of evaluating knowledge integration and co-production in transdisciplinary research (Sect. “New knowledge as the basis for changing actions”).

Farmers’ perspectives of the collaborative learning process

Participatory methods allowed for data about participating farmers’ subjective experience to show relevant aspects that fostered learning, what participants learned and how they benefited.

Relevant aspects that fostered learning: how and why the process works

Results revealed how and why a collaborative learning process works. Learning was fostered by: farmers’ inclusion, farmer-to-farmer exchange sessions, learning from practice and sharing results.

First, farmers’ inclusion as co-researchers in all steps of the learning process is an integral component of a transdisciplinary research design. In this research, male and female group members set their own goals, managed their own activities, and assessed their own performance. This collaborative learning process represents a shift from researchers having power over the process, to researchers and group members working together. Using Self Determination Theory, Restrepo et al. (in preparation) analyzed farmers’ perspectives on the collaborative learning process and revealed the importance of giving autonomy to the research participants and building trustful relationships. Development of trustful relationships is related to less hierarchical patterns of communication (Rist et al. 2006). Sharing power with farmers from the beginning of the project by including them as co-researchers, i.e. sharing decision power over the process, is also suggested by other scholars working in transdisciplinary research as a way of addressing sustainability challenges and enacting change (Wiek et al. 2012; Sewell et al. 2014; Njoroge et al. 2015; Schodl et al. 2015; Dolinska and d’Aquino 2016; Restrepo et al. 2016; Siew et al. 2016; Chaudhury et al. 2017; Fielke and Srinivasan 2017; Hazard et al. 2017; Ortiz et al. 2017; Toth et al. 2017). In the same line, Chilisa (2017) campaigns for decolonizing mainstream methodologies and recognizing African philosophies and worldviews to disrupt asymmetrical power relations between indigenous and western academic knowledge when seeking to address Africa’s sustainability challenges. For balancing the asymmetry in power distribution that often characterizes agricultural research and development, we advocate for explicit attention to managing power through continuous critical reflection, e.g. capturing and reflecting on experiences related to the complexities of participation, power, privilege and relationships such as through a fieldwork journal.

Second, farmer-to-farmer exchange sessions represent a way to learn, share, decide, and gain experience. Farmers in these groups expressed their preference to learn from peers sharing similar context in informal settings, as also stressed by Kilpatrick and Johns (2003), Lankester (2013), and Jones et al. (2014). Sharing knowledge by learning from peers strengthened networks beyond group members’ villages. Networks can facilitate access to external information sources, which is key in fostering learning and change within food and farming systems (Olivera and Straus 2004; Rist et al. 2007; Pahl-Wostl 2009; Pai et al. 2015; Dolinska and d’Aquino 2016; Cliffe et al. 2016).

Third, learning from practice, or applying lessons learned by experimenting, shows how the experience itself is the real teacher (Kolb 1984). As argued by Leeuwis and Van de Ban (2004, p. 149), “conclusions drawn by people themselves on the basis of their own experiences tend to have a greater impact than insights formulated by others on the basis of experiences that learners cannot identify with”. Farmers appreciated learning (more than money), as it was from practice, context specific, and took place during a step by step process. This learning from practice and the ability to experiment is what enhances farmers’ adaptive capacity, which is necessary for transitions towards sustainability. We demonstrate that if the learning process is organized along these principles, farmers are willing to learn and change, and do not need to be “convinced” and motivated to participate. Thus, following this approach is a way to overcome a challenge known as the “adoption problem” faced by scientific and development institutions that use linear innovation approaches.

Fourth, sharing results from practice among participating actors facilitates mutual learning. The exchange of empowering stories increased researchers’ and farmers’ understanding of the transformed situation. For example in the MSC sessions in this research (Sect. “Benefits to farmers from these changes”), group members demonstrated many benefits from the process and led to the co-construction of a shared narrative among them. Leeuwis and Aarts (2011) found that empowering stories have a direct effect on innovation processes, while Blissett et al. (2004) point to the importance of accumulating (shared) experiences, hence, the need to remain within an experiential learning process for a sufficient amount of time. Thus, it is crucial for funding institutions to adopt policies that support a structure in which researchers can engage in such long-term transdisciplinary collaboration (Blissett et al. 2004; Wiek et al. 2012). Likewise, researchers also need to propose for and plan longer term collaborations to create the conditions in which such experiential learning can thrive.

These four aspects stimulated participation and commitment to the learning process, promoted knowledge integration and co-production, and increased the usability of results due to their contextualized relevance and accessibility. Thus, transdisciplinary research in food and farming systems effectively fosters learning by including farmers as co-researchers, facilitating farmer-to-farmer exchange sessions, and other diverse learning opportunities ranging from hands-on practice to dialogue about experienced outcomes and changes. This learning is critical for creating the possibility for systems to change.

What participants learned and how they benefited: impacts

Our evaluation revealed multiple impacts of the collaborative learning process; i.e. from analyzing detailed production data to societal dynamics, overcoming a deficit in sustainability research projects that deal with both societal and ecological impacts (Bäckstrand 2003; Wiek et al. 2012). These results show what farmers learned and how they benefited. The learning process further expanded farmers’ ability to change management practices in their own farming system. Farmers constructed contextual knowledge by testing different feeding strategies and testing milk quality and udder health (Fig. 5). Among the multiple benefits, farmers experienced an average increase in milk production of 80% (Fig. 9), and associated benefits such as healthier cows, reduced workload and improved cow fertility (Fig. 6). Results show that the collaborative learning process empowered farmers to enact change by strengthening their capacity to innovate and communicate with other actors, hence building up farmers’ adaptive capacity. The enhanced capacity to act was not restricted to the tested management practices. New action possibilities emerged for instance, by coming together to achieve individual and collective goals, by sharing the knowledge co-produced with other actors (Fig. 7), or using monitoring and evaluation tools (Fig. 5). Hence, farmers experienced that they learned and can change, which enhanced their general problem solving capabilities. This is of main concern when it comes to improving low-external input farming systems in variable and heterogeneous environments, inherently much more complex than intensive systems and require context specific solutions (van Keulen 2006). Farmers’ adaptive capacity was enhanced through the collaborative learning process as it created a participatory learning environment in which farmers could decide what they wanted to learn, how they wanted to learn, and from whom they wanted to learn.

New knowledge as the basis for changing actions

The aim of transdisciplinary research is to create both scientific and actionable knowledge that serves to improve complex problematic real-world situations. Although (re-)integrating and applying knowledge produced in transdisciplinary research in societal practice is conceptualized as an integral part of transdisciplinary research (Lang et al. 2012), the application phase might not actually be included in some participatory and transdisciplinary innovation projects dealing with food and farming systems, as also previously noted by Restrepo et al. (2014). Implementation of actionable knowledge might happen after the project end, meaning that societal impact might remain uncertain and undocumented. However, the proof of the usefulness of new knowledge for changing practices only results from testing it under specific contextual conditions. If this phase is not included in transdisciplinary projects, practical recommendations might resemble those of project that follow a linear innovation approach.

In the collaborative learning process, we evaluated, implementing the solutions and testing them started from the discovery phase, overcoming the knowledge-action gap. In the analysis, we assessed how the newly co-created knowledge was transformed into action. As an analytical method, we integrated the Control Loop Model (Kaufmann 2007) and the Learning Loops (Ashby 1952; Bateson 1972; Argyris and Schön 1978; Hawkins 1991; Flood and Romm 1996; Ison et al. 2000; Armitage et al. 2008; Pahl-Wostl 2009). Farmers learned as they regulated their farming system by implementing corrective actions still based on the same cause–effect relations, by discovering new cause–effect relations and testing their effect, or by questioning and changing their aims, i.e. by single, double or triple-loop learning (Tables 4, 5). The example of farmers shifting the approach to feeding their cows by planting their own fodder and storing it illustrates a transformative change that improved their ability to deal with complexity and context-dependence of sustainability challenges in food and farming systems. In Nakuru County, climate change coupled with growing population pressures has created a shift in land-use leading to the disappearance of communal grazing lands. For this reason, the changed practices related to the semi-zero grazing and zero grazing require co-producing knowledge to identify and implement practices that are feasible for smallholder farmers with low physical and financial capital. Change in one practice led to changes in other practices and hence to change in the production system, so farmers are not only learning a new solution but also learned how to learn and change.

When farmers learned how to use different monitoring and evaluation tools, it strengthened their experiential learning possibilities (Leeuwis and Van den Ban 2004). By monitoring farm activities, farmers were able to (1) detect earlier differences between the observed and aimed value, e.g. low milk density; (2) reflect on the causes of deviation, e.g. the cow is not getting all nutrients needed to produce good milk quality; and (3) apply possible solutions, e.g. increase the ratio of lucerne, a protein rich fodder. By trying different monitoring and evaluation tools, farmers discovered cause–effect relations that they did not know before, e.g. the effect of new fodder on milk quantity and quality. This shows that facilitating the use of different monitoring and evaluation tools enhanced farmers’ reflection levels, hence promoting mainly double-loop learning, as can be seen in Table 5. The application of these learning tools was possible for people with low literacy levels, as is common especially among older farmers in rural areas of sub-Saharan Africa. Overall, monitoring and evaluating activities helped farmers observe previously unperceived information, e.g. allowed them to differentiate what they could not differentiate before, hence acquiring a deeper understanding of the farming system. Access to new information influences how smallholders understand, shape and alter their farming systems through their actions (Kaufmann and Hülsebusch 2015; Restrepo et al. 2016). Monitoring and evaluation tools allowed farmers to see whether they achieved their goals, hence to evaluating the usefulness of knowledge gained during transdisciplinary research engagement in their own context. The evaluation framework presented in this paper allows for understanding what conditions and factors support the facilitation of double- and triple-loop learning, crucial to building up farmers’ adaptive capacity to cope with, prepare for, adapt to, and deal with complexity and uncertainty associated with sustainability challenges.

After trying different fodder options in conjunction with monitoring and evaluation tools, some group members expressed that they “see things in a different way” (Sect. “What farmers learned from the collaborative learning process”). This denotes a transformation of their relevance system, i.e. the structures of assumptions through which we understand our experiences (Mezirow 1997, p. 5). This transformation can potentially lead to conscientization Footnote 3, empowerment, and emancipation (Mezirow 1996). As farmers establish, shape, and maintain their respective farming systems through their actions and their underlying knowledge, a change in management actions implies a change in the farming system. This shows how a second-order cybernetic analysis can be paired with the Learning Loops to better understand how to facilitate different reflection levels needed for a collaborative learning process that can enable participants to create change and adapt to changing social–ecological conditions.

Evidence for double- and triple-loop learning in this paper corroborates that the collaborative learning process challenged farmers’ assumptions and beliefs, in opposition to the linear thinking in the transfer of knowledge and technology approach. This usually only promotes single-loop learning, as it is content-led and delivered through transmissive methods (Sterling 2011). Thus, the collaborative learning process represents a move away from the classical transfer of knowledge and technologies, to a process of co-creating transitions towards sustainability through knowledge integration and co-production processes.

Conclusions

This paper offers a methodological contribution demonstrating how farmers’ perspectives can be harnessed for a knowledge analysis that contributes to an evaluation of collaborative learning processes. Analyzing farmers’ perspectives of the learning process means that knowledge was not extracted from the farmers by the researchers, but rather built-up and owned by all actors involved, a characteristic of transdisciplinary research. Farmers highlighted the following factors, which appear to be critical when facilitating learning processes that aim for sustainable change: (1) active participation of farmers as co-researchers from beginning to end; (2) farmer-to-farmer exchange sessions; (3) learning from practice; (4) learning monitoring and evaluation tools; and (5) sharing results. Farmers acquired new capacities and knowledge, which included improvements in the dairy production system and an enhanced adaptive capacity that triggered positive social and ecological outcomes in the face of sustainability challenges. Results show that the collaborative learning process met group members’ needs and learning objectives. Hence, promoting sustainable change requires learning methods that integrate and expand farmers’ existing knowledge and enhance their capacities to act, which can then contribute to an increased room for maneuver.

Integrating the Control Loop Model from second-order cybernetic analysis with Learning Loops from organizational theory revealed the cognitive change associated with knowledge integration and co-production processes. This new analytical method permits changes in knowledge underlying changes in actions to be revealed. Results show the important role of farmers’ experimentation, as well as their own monitoring and evaluation for promoting learning and building adaptive capacity, important aspects in addressing sustainable challenges and creating transformational change. This could start with the implementation of corrective actions based on the same cause–effect relations, then learning new cause–effect relations, and further, questioning and possibly changing their aims. By questioning aims, farmers transformed their relevance systems, indicating that transformative learning occurred. Hence, learning and subsequent changes in practices are best achieved through experiential learning which can be made apparent through analysis that uses the Control Loop Model and Learning Loops. The collaborative learning cases assessed here reflect the knowledge integration and co-production linked to a specific sustainability challenge, but are well-suited to deal with complexity, context-dependence, and rising social and ecological uncertainty in food and farming systems. This work articulates a deeper understanding of a contextualized real-life problem and transitions towards sustainability enacted by smallholder farmers operating under low-external input conditions.

The four-level evaluation scheme used in this research offers a comprehensive way to assess the perception of farmers on what they liked, learned, applied and how they benefited from a collaborative learning process. The evaluation scheme is open enough to accommodate aspects of relevance to the farmers. Hence, it contributes to refining transdisciplinary research with a methodology that can be used to redistribute power towards participant/co-researchers during the evaluation of societal impact. This shows how the collaborative learning process enabled farmers to jointly develop relevant solutions for real-world sustainability challenges. Thus, we connect to a wider discourse regarding why and how transdisciplinary research works from the perspective of the participants/co-researchers and how it can more effectively be evaluated. This research contributes methodological innovations for a better understanding of processes and outcomes of collaborative learning that are necessary for enacting effective transdisciplinary research for sustainability.