Evaluating Facilitated Modelling Processes and Outcomes: An Experiment Comparing a Single and a Multimethod Approach in Group Model Building

Abstract

Facilitated modelling approaches have been suggested as helpful tools to support negotiation in strategic analysis processes due to their potential to facilitate cognitive change and enhance consensus and commitment with final decisions. In the present research, we developed an experimental framework to compare what two of these approaches, that is, group model building and a multimethod approach, contribute to the process and outcomes in the negotiation of strategies. In the multimethod approach, we combined strategic options development and analysis with computer simulations of the group model building approach. We explored the differences between these two modelling approaches in facilitating cognitive change, consensus and commitment by building an experimental research design with real clients, working on their organisation’s problem. Furthermore, we compared the type and content of participants’ contributions in the strategic conversation. The lessons from the experiment conducted are twofold. On one hand, the multimethod approach encouraged more divergent contributions and produced a higher degree of cognitive change than group model building (i.e., the single approach). On the other hand, group model building encouraged more contributions about content related to causes of the problem and enhanced more commitment to the final solution than the multimethod approach. Hence, the conducted experiment brings new insights into the benefits of using multimethods and possible losses resulting from such combinations. Accordingly, we have presented opportunities for further research regarding the combination of facilitated modelling approaches.

1 Introduction

In order to support organisations in managing complex and messy problems, facilitated modelling approaches have been developed since the 1970s (Eden and Ackermann 2006; Rouwette et al. 2011a, b). The term facilitated modelling is used to characterise approaches where a client group jointly builds models with the assistance of a group facilitator (Franco and Montibeller 2010; Franco and Rouwette 2011; Rouwette 2011; Tavella and Franco 2015). These models, built during the process, are used as “transitional objects” that facilitate the shifting of positions and the negotiation that can lead to consensus (Ackermann and Eden 2011, p. 24). Facilitated modelling approaches aim to help managers and stakeholders to structure a complex and messy problem situation. These are challenging aims because it is not only required to open up participants’ minds for different and competing perspectives, but also, to move toward alignment on what is wise to do or to refrain from. These points are of crucial significance during the process when managers and stakeholders come together to “manage the complexity” (cf. Eden and Ackermann 2006, p. 767).

Facilitated modelling approaches are particularly applicable to strategic analysis processes—processes of identification of strategies for a business (Fleisher and Bensoussan 2007). Typically, strategic analysis takes place under high uncertainty; managers use diverse definitions and have to deal with complex systems (Eden and Ackermann 2001; Rouwette 2011; Warren 2009). By supporting the communication during the process and helping participants to reflect on and change their previous assumptions, facilitated modelling supports the negotiation of successful solutions. For reaching successful solutions, the alignment of managers’ goals is necessary as well as the negotiation of agreements between participants about what the problem is and possible paths of action (Akkermans and Vennix 1997; Black and Andersen 2012). During the problem solving process, facilitated modelling approaches represent these goals and agreements visually in a model: By using a model as visual representation of the group discussion, and through the mediation of an independent facilitator, facilitated modelling enhances confidence in the strategies identified and, therefore, fosters participants’ commitment with implementation (Franco and Montibeller 2010; Rouwette 2011). Facilitated modelling has shown potential to improve the quality of the outcomes of the analysis during a strategic analysis process by reshaping participants’ preconceived ideas (cognitive change) and enhancing the exchange of knowledge and information between them (Akkermans and Vennix 1997; Rouwette 2011; Vennix 1996). Furthermore, facilitated modelling has demonstrated its potential to increase the political feasibility of the agreed strategies along with consensus and commitment with these strategies (e.g. Ackermann and Eden 2010a; Eden and Ackermann 2013; Rouwette 2011).

A number of facilitated modelling approaches have been developed. Franco and Montibeller (2010) identified at least three big divisions: facilitated problem structuring (e.g., strategic options development and analysis), facilitated system dynamics (e.g., group model building) and facilitated decision analysis. Moreover, a group of multimethods has been identified, combining at least two facilitated modelling approaches with the aim, for instance, to strengthen specific elements of the process (Howick and Ackermann 2011). Different terms have been used to refer to these methods. For example, Howick and Ackermann (2011) referred to them as mixed methods, and Mingers and Brocklesby (1997) referred to them as multimethodologies. In this paper, we use the term multimethod proposed by Ackermann et al. (2011) for the particular combination of facilitated modelling approaches that we investigated.

Facilitated modelling approaches have important theoretical and practical differences among them. Furthermore, it is not clear which elements of their process and structure contribute to successful outcomes (Bérad 2010; Franco and Montibeller 2010; Rouwette 2011). This lack of transparency is a problem for spreading facilitated modelling practices and replicating the results obtained by expert facilitators (Rouwette 2011). In addition, the current body of knowledge about the outcomes of facilitated modelling approaches and benefits of multimethods is based on single cases examined in different contexts, and evaluated with different criteria. These conditions complicate the comparison between approaches and the extrapolation of successful experiences to different problems and contexts. For this reason, the present research supports the claim of authors like Tavella and Franco (2015), McCardle-Keurentjes et al. (2008), Midgley et al. (2013) and Rouwette et al. (2011a) for more systematic research and the use of an experimental design to compare and evaluate the contributions of facilitated modelling approaches to strategic analysis processes.

Our study builds on the comparison of the process and contributions of group model building (GMB) and strategic options development and analysis (SODA) by Rouwette et al. (2011a). In their research, the differences in the process and the outcomes between both approaches were clarified on a theoretical and practical level. Interestingly, the approaches were contrasted for one and the same real-life problem situation. Yet, the difficulty to conduct research in a comparable research situation can be noticed. For instance, the participants were not the problem owners themselves. Some participants in the SODA meeting were also present in the GMB workshop and thus, the data are not independent. Furthermore, the number of meetings in the study differed (i.e., four GMB meetings, whereas three meetings—actually two—were used for SODA) as well as the experience of the facilitator (i.e., a high-experienced facilitator in GMB meetings vs. two low-experienced facilitators in SODA meetings). Our study tried to overcome some of the limitations described by Rouwette et al. (2011a) by splitting two management teams confronted with a real-life problem in their organisation into two comparable groups of problem-owners. In each of the two organisations, during two workshops one team used GMB and the other used the multimethod approach, both supported by the same facilitator (the first author). Note that the context was similar for the teams of one organisation. The strategic conversations taking place during the process and their final outcomes were analysed with the purpose of exploring the research question: “What are the differences in the process and the outcomes between the multimethod approach and GMB?”

The remainder of this paper proceeds as follows. Section 2 provides a concise theoretical background of the approaches examined in this study and the main findings of previous studies. Then in Sect. 3, we introduce the particular framework used in our research. We explain the variables measured and the instruments used. Section 4 presents the main findings and discusses their relevance, as well as theoretical and practical implications. Finally, based on this experience, we draw conclusions, and consider the limitations of our study and the future work we envision in this topic.

2 Facilitated Modelling Approaches

Facilitated modelling approaches, like other “soft” operational research (OR) approaches, assume that problems are socially constructed and that the inclusion of different perspectives is needed to identify and implement successful solutions (Franco and Montibeller 2010). Therefore, facilitated modelling approaches recognise that the way in which strategic issues are defined is crucial to determine the nature of strategic agendas to be addressed. The elements of negotiation should be taken into consideration in strategy formulation in order to increase their probabilities of being successful (Ackermann and Eden 2011). Typically, the tangible outcomes of facilitated modelling workshops include one or a set of models (e.g., system dynamics simulation models, stock and flow diagrams or a cognitive map depicting means and ends relations), figures, theories and lists of plausible solutions. Beside these tangible outcomes, Franco and Montibeller (2010) identified the inclusion of different perspectives in the discussion, the creation of shared agreements, and the ownership of the problem definition and solutions as intangible outcomes commonly associated with different facilitated modelling approaches.

As said, within the wide family of facilitated modelling approaches this research focused on two specific ones, GMB (cf. Andersen et al. 2007b; Vennix 1995, 1996), and SODA in the multimethod approach (cf. Ackermann and Eden 2005, 2010a). GMB is grounded in the system dynamics methodology, originally presented by Forrester (1961). System dynamics is based on the assumption that the behaviour of complex systems arises from their causal structure (Meadows 1976). This structure is a representation of the interconnections between the strategic resources of an organisation (Warren 2005). By analysing the structure and its feedback mechanisms it is possible to understand system’s behaviour. Computer simulation models serve to explore the effects of possible strategies. GMB constitutes “a bundle of techniques used to construct system dynamics models working directly with client groups on key strategic decisions” (Andersen et al. 2007b, p. 691). Depending on the nature of the problem, GMB can make use only of diagrams (qualitative GMB) or can combine them with computer simulations (quantitative GMB) (Vennix 1996). As will be clarified in more detail in the following section, we used GMB in both the qualitative (i.e., in the first workshop) and the quantitative mode (i.e., in the second workshop). SODA is a facilitated modelling approach that builds a “graphic representation of a problematic situation” to explore options in complex systems (Ackermann and Eden 2010a, p. 135). Ackerman and Eden noted that SODA is used to explore complex problematic situations before making a decision. To explore these situations, SODA models build graphic representations known as cognitive maps. By using these maps, SODA allows participants to explore the problems with a holistic perspective and to arrive at a negotiated agreement about how to act (Ackermann and Eden 2010a). Note, in the multimethod approach, we used SODA in the first workshop, to be followed by quantitative GMB in the second workshop.

The results of Rouwette et al. (2011a) indicated that the main theoretical difference between GMB and SODA lies in their assumed understanding of the world and, hence, the perspective they take to analyse the problem. On one hand, SODA is clearly subjective (Ackermann and Eden 2011; Eden 1988) and aims to achieve an agreement on participants’ perceptions and priorities by capturing each individual’s interpretation of the problem (Rouwette et al. 2011a). On the other hand, GMB has a dualistic perspective (subjective and objective) (Andersen et al. 2007b) and aims to incorporate not only individuals’ perspectives, but also to replicate the real world with some accuracy (Richardson and Andersen 2010; Rouwette 2011). In this sense, GMB does not only aim to build a common definition of the problem, but also to confront this agreed upon definition with data observed in the real world. This difference determines the focus of the model: While SODA concentrates on the social world and the individual perceptions, GMB aims to integrate these perceptions with the real world (Rouwette et al. 2011a). The theoretical difference between both approaches is transferred into practical differences in the process. The main practical differences found by Rouwette et al. (2011a) are:

• (a) Starting question GMB starts by identifying the reference behaviour mode (past observed behaviour) and asks for explanations. SODA on the other hand starts the process by asking participants about their perspectives regarding current and future issues;

• (b) Model analysis While GMB focuses its attention on identifying the feedback loop structures responsible for the observed behaviour (reference mode), SODA focuses on assessment of actions and goals (p. 797);

• (c) Option analysis In SODA the actions are evaluated from a static perspective; changing a part of the model will influence the intermediate ends and therefore the final goals (p. 798). GMB on the other hand, assesses the options from a dynamic perspective; building first the structure and then identifying steering points and plausible behaviours.

Due to these differences, the authors pointed out that a combination of the two approaches could be beneficial for both methods since GMB can supplement SODA by providing a framework for the quantification of variables and the identification of feedback loops and leverage points. And, in a similar way, SODA can supplement GMB by providing a framework to identify the actions that can “steer the future behaviour in the preferred direction” (Rouwette et al. 2011a, p. 800).

Many attempts have been made so far to incorporate SODA into the GMB process (e.g., Ackermann and Eden 1997; Ackermann et al. 2010, 2005; Andersen et al. 2007a; Howick et al. 2006, 2008). Therefore, it is surprising that little work has been done to measure the benefits of using multimethod approaches and to evaluate possible losses that can result from such combinations (Howick and Ackermann 2011; Mingers and Brocklesby 1997). Exploring the implications of multimethod approaches is important, in particular with regard to their effects on the final commitment of participants. Particularly, the translation of tangible outcomes from one method to the other (e.g., the means and ends of a SODA model into causal relations in a System Dynamics diagram) needs to be understandable for participants. Note that authors like Luna Reyes et al. (2006) and Richardson and Andersen (2010) pointed to the importance of diagram consistency in order to create ownership of the model and the solutions explored from it. Similarly, Videira et al. (2012) underscored the importance of participants’ involvement in the final model construction for creating commitment with the strategies identified. Both features, that is, the high level of involvement in model construction and diagram consistency can be compromised when multimethod approaches are designed.

So far, case study research has been the favourite research strategy used to evaluate the benefits of mixed methods (Howick and Ackermann 2011) and facilitated modelling approaches in general (Midgley et al. 2013). Typically, the studies have offered a description of particular facilitated modelling approaches, and the reflections about the perceived impact of this type of decision support have been based on the perspective of the facilitator (Tavella and Franco 2015). Even though researchers in the field are highly experienced and their perspectives are valuable for the evaluation of benefits of the approaches, it is problematic that they use different criteria (Midgley et al. 2013). Due to these differences, if the evaluations are only based on facilitator(s)’ personal opinion(s), “they may miss evidence that does not fit their current thinking about what is important” (Midgley et al. 2013, p 144). As an illustration of the differences in the research processes and tools of previous case studies, some examples of studies concerning specifically the two facilitated modelling approaches discussed in this paper (SODA and GMB) including combinations of the two, are summarised in Table 1.

Table 1 Previous case study-based research in SODA, GMB or in combination

Even though the list of examples presented in Table 1 is not exhaustive, it clearly shows the diverse perspectives, instruments and variables used in the evaluation of facilitated modelling approaches. Consequently, each experience, reflected in the variables measured, is the result of different conditions, problem(s), procedure(s) and, most of the time, different criteria to evaluate them (Midgley et al. 2013). Under those circumstances, comparing and aggregating data across cases becomes a complex task. Therefore, it is not possible to conclude that the (combination of) approach(es) is the cause of the outcomes observed in a case, or that these outcomes are the result of other conditions, such as, the facilitator’s personal skills, team dynamics, or the nature of the problem (Howick and Ackermann 2011). Since the degree of participants’ stake in the strategic issue has important implications for the level of conflict between participants, “one would ideally like to use approaches in exactly the same problem with exactly the same client group” (Rouwette et al. 2011a, p. 778). However, as Rouwette et al. (2011a, p. 788) pointed out, “It is unlikely (...) that a management team confronted with a real life problem would voluntarily split up in two or more (comparable) groups which then each use a specific method to work on the problem”. It is probably for this reason that we could not find any previous research that compares groups in the same client organisation working on the same issue using different approaches.

Within this context, in the present research, we opted for an experimental research design, because that would allow a comparison based on findings collected in a controlled setting. Moreover, we decided to conduct the research with real clients, working on their organisation’s problem. This design aims to contribute to the current body of knowledge about the gains and drawbacks of combining two single facilitated modelling approaches into a multimethod approach by comparing under controlled settings the multimethod proposed by Ackermann et al. (2010) that combines SODA with system dynamics computer simulations (i.e., using SODA first, and then GMB restricted to the quantitative mode) with GMB (i.e., first in the qualitative mode, and then in the quantitative mode). This comparison was done in terms of (a) the strategic analysis process by assessing the type and content of the contributions made in the group conversation during the workshops and (b) intangible process outcomes, that is, cognitive change, consensus and commitment.

By making a comparison with real clients, working on their organisation’s problem, our research design provides direct means to quantify and compare what happened during the different processes. Hence, we aim to address the limitations and gaps described above for case study-based research, quantifying in comparable settings the process and outcomes of a single and a multimethod approach. This is expected to bring a novel contribution to the field, while allowing to draw hypotheses that may be further validated in future work replicating the proposed research design. Several hypotheses and suggestions for further research that we identified are described in the last section.

3 Methods and Experiment Design

3.1 Design and Participants

To explore the possible benefits and drawbacks of the combination of facilitating modelling approaches to the negotiation of strategies, the present research conducted an exploratory field experiment in two organisations in Lisbon, Portugal. Experiment is hereby understood as “an inquiry for which the investigator plans, builds, or otherwise controls the conditions under which phenomena are observed and measured” (Webster and Sell 2007, p. 2). Under experimental conditions it is possible to select participants to create comparable groups and to control which approach and which scripts are used to support the negotiation process, allowing relatively objective comparisons.

However, it is important to be aware that some limitations are associated with experimental research, for instance, the trade-off between realism and controllable and comparable situations (Franco and Rouwette 2011). While seeking control, researchers can easily turn experiments into situations too artificial to have any meaning in real-life. This point is particularly important when evaluating facilitated modelling approaches because the validity of the results highly depends on the level of engagement the participants have with the issue they are working on (Midgley et al. 2013). For this reason, the present research used participants’ real problem during the experiment with the aim of maintaining as much realism as possible.

The field experiment was conducted in the spring of 2014 with managers of two Portuguese organisations that agreed to use the experimental framework proposed to analyse a strategic issue they were facing at the time the experiment was conducted. Overcoming the challenge of having real customers discussing their problem simultaneously using two approaches was possible thanks to the high interest of the participating organisations’ top managers in learning more about facilitated modelling methods. Their engagement and approval for conducting the experiment was obtained following an invitation and presentation of the research plan by the first author, with whom they had collaborated in the past.

Of the two organisations, organisation 1 is a small private firm in the hospitality business and the other, organisation 2 is a medium-sized governmental office responsible for the management of 60 % of the domestic waste generated in Portugal. In each organisation, two groups analysed one and the same strategic issue of their organisation during two different workshops. To perform this analysis, one group used GMB and the other one used the multimethod approach developed by Ackermann et al. (2010). In this study, GMB combined the use of a system dynamics diagram (used in the qualitative mode) to define the structure of the problem with system dynamics computer simulations (used in the quantitative mode) in order to explore unintended consequences of the solutions proposed. Similarly, the multimethod proposed combined SODA (constructing a cognitive map) to structure the problem and system dynamics computer simulations to identify pressures and policies (Ackermann et al. 2010). Figure 1 shows the roadmap followed during the experiment by the two groups in both organisations.

Fig. 1

General road map of the experiment

In total, eighteen participants were engaged in the experiment. During the first stage (see Fig. 1, Experiment Set up), the first author and the gatekeeper of each organisation—a member of the organisation “who carries internal responsibility for the project” (Richardson and Andersen 1995, p. 115)—invited a group of potential participants to participate in the project. The selection of these potential participants was based on their knowledge, degree of involvement in the strategic issue to discuss and the time they had available to participate during the entire process. From the list of managers who accepted the invitation ($n = 27$), the first author and the gatekeeper selected a team of eight and ten candidates in organisation 1 and 2 to participate in the experiment, and split them in two comparable groups based on their age, gender and professional background. The demographic characteristics of these groups are presented in Table 2.

Table 2 Group demographic characteristics

Additionally, important characteristics common to the participants were that they (a) did not have previous experience working with facilitated modelling approaches, (b) had adequate English language skills to understand and participate in the workshops, and (c) were not aware which specific strategic issue would be analysed before the experiment started. The last condition means that, even though the participants had enough knowledge to address the problem, they were not informed in advance about the specific strategic issue to be discussed during the meeting in order to avoid any negotiation between participants previous to the workshops.

The workshops were conducted by a team of one facilitator and two recorders per workshop. The first author served as facilitator in all the workshops. The facilitator had been trained in system dynamics, and group modelling facilitation, and thus was familiar with the two methods used in the experiment (GMB and SODA). The recorders were six first-year students of the European Master in System Dynamics programme and participated voluntarily in the meetings “writing down or sketching the important parts of the group proceedings” (Richardson and Andersen 1995, p. 115).

3.2 Procedure and Scripts

In our experimental framework a set of scripts was used to ensure the consistency of the process among the groups and its comparability. Scripts are a predefined and documented set of behaviours in facilitated modelling workshops (Andersen and Richardson 1997) that contribute to increase the transparency, replication and the transmission of effective practices (Hovmand et al. 2012). During the first workshop, the groups working with GMB and the groups working with the multimethod approach followed different scripts. In this stage, the groups working with GMB built causal loop diagrams (CLD) including the causal explanations of the problem, whereas the groups working with the multimethod built a cognitive map including the main issues and alternatives for the problem (see Fig. 1). During the second workshop, in all four groups the same scripts were used to evaluate the strategies identified with simulations of a high-level (simple and highly aggregated) system dynamics model. The summarised outline of the workshops is presented in Table 3 and a more detailed description of key scripts employed can be found in “Appendix 1”.

Table 3 Workshop outline

3.3 Measures

Following the recommendations of Franco (2007) and Rouwette (2011), we used triangulation between three different data collection methods in order to evaluate the process and its outcomes. The methods used were observations, questionnaires and interviews.

3.3.1 Observations

Observations are descriptions of what is happening in a particular situation from the perspective of an observer (Saunders et al. 2006). These descriptions can be in the form of qualitative data (narrative descriptions) or quantitative data (coded descriptions). In this research, we classified the participants’ contributions by coding them, based on their type and content. The term participant’s contribution was used to label the participant’s action to verbally express an idea, concept, question or emotion during the workshop. Each participant’s contribution represents one participants’ turn, from the moment he or she started to speak until finished. Therefore, a contribution may include only one word or it may include many sentences. Examples of participant’s contributions are:

Exactly!!!. (Financial Manager, organisation 2, GMB workshop 2);

The important is to focus on our essence, on what we are. Not pay attention to the competitors. For that reason I think we should not focus in the benchmark but in our internal vision. (Marketing Manager, organisation 1, multimethod workshop 1);

I think one cause [of the problem] is the high expectations we have created in our customers. They think they can ask for an utopic quality since they pay more than the rest (Marketing manager organisation 1, GMB workshop 1);

You see, as I see it, there is not a direct connection between the elements. What do you have there is more like spiral. On a spiral relationship everything is connecting with everything. You cannot recognise clear cause and effect relationships there. You see, everything is connected. You cannot not identify the beginning and the end of the relationships. For instance, the lack of financial resources is related to the lack of leadership, but also the lack of leadership affects the lack of financial resources. Everything is connected (Financial Manager, organisation 2, multimethod workshop 1).

In total 3163 participants’ contributions (thus excluding the contributions made by the facilitator) were coded during the workshops in the two conditions (multimethod contributions $n = 1743$, and GMB contributions $n = 1420$). In addition to the video or audio recording, the notes made by the recorders present in the room during the workshops enabled us to reconstruct the process followed by the participants.

To analyse the contributions, we coded them in the categories or types presented in Table 4. We recognised the benefits (e.g., efficiency gains) of building on existing coding schemes based on theories that fit to the research question and aim of our study. Therefore, our coding scheme was constructed using the approach described by Folger et al. (1984) for group communication research. The primary types of the coding scheme are: D—divergent, Co—convergent, CSu—clarifying/summarising, A—assessment and N—negative. The category assessment was divided into subtypes in order to better describe the contributions during the process. In addition, the code scheme presented by Dwyer and Stave (2008) was used to code the content of each contribution (see Table 4). The content of the contributions was identified as: Problem formulation (PF), causes (C), alternative/solutions development (AS) or others (O).¹

Table 4 Coding scheme: type and content of contribution

Using this coding scheme, the first author and an independent coder, to the best of their knowledge, exhaustively coded the contributions recorded in audio and videotapes during the workshops. The independent coder was a volunteer student on economics proficient in both English and Portuguese, unaware of the aims of this research. Previous to the coding work, the first author trained the independent coder in order to ensure the coder understood the coding scheme. To determine the interreliability, we calculated the percent agreement using Holsti’s method² (Neuendorf 2002). There were only minor disagreements between the two coders, as the interrecoder reliability values showed (i.e., between 91 and 96 %). In addition, the results reported by the two coders were validated with the observations reported in notes made by the two recorders in real time.

3.3.2 Questionnaires and Interviews

The outcomes of the workshops were measured in terms of the previously mentioned intangible outcomes: cognitive change, consensus and commitment (Rouwette 2011; Akkermans and Vennix 1997). Cognitive change was understood as the change in participants’ perceptions, thoughts, believes and long term knowledge (Johnson-Laird 2013). Consensus was considered as the creation of “agreement on content or actions” about the organisation’s priorities (cf. Rouwette 2011, p. 881). Commitment was understood as “the extent to which the team members accept the strategic decision reached and intend to cooperate to carrying out it” (Korsgaard et al. 1995, p. 61).

The questionnaires and interviews were used to elicit participants’ perspectives regarding the intangible outcomes of the workshops. We used two questionnaires: a prequestionnaire (prior to the first workshop) and a postquestionnaire (at the end of each workshop). Moreover, semistructured interviews were conducted after the experiment was finalized.

In the prequestionnaire, we asked participants to give their opinion about: (a) the best solutions for the strategic issue defined during the experiment set up and (b) the issues that are important for the discussion.

The postquestionnaire was based on the one developed by Midgley et al. (2013) to evaluate systemic problem structuring methods. This tool was selected because it aims to support “locally meaningful evaluations” between approaches in field experiments (Midgley et al. 2013). Nevertheless, the questionnaire was modified to get into more detail on the variables under consideration in the present research (i.e., cognitive change, consensus and commitment). To make the necessary modifications, an extra section was added to elicit the participants’ perceptions about (a) the best solution for the strategic issue of the workshops, (b) what issues were important to include in the discussion, (c) the degree of consensus and (d) their level of commitment. The questionnaire items used for the results presented in this paper are listed in “Appendix 2”.

Finally, using semistructured interviews at the end of the workshops, we evaluated participants’ perceptions of the group decision (i.e., consensus) and their willingness to implement the decision (i.e., commitment). The first author interviewed five participants of the nine participating in the GMB condition and five participants, of the nine participating in the multimethod condition, in each organisation randomly selected from both organisations. The specific interview items used for the results reported in this paper are presented in “Appendix 2”.

4 Results and Discussion

The experiment started by defining the strategic issue to discuss during the workshop with the gatekeeper of each organisation. The strategic issues were selected since they were important for the long-term future of the organisations participating and also because they involved, according to the criteria of the gatekeepers, decisions dealing with complexity and uncertainty. The strategic issues selected were:

• Organisation 1: How to increase the organisation profits by 2020?

• Organisation 2: How to reach the waste source separated materials goal by 2020?

After defining the issues to discuss, the groups were formed and the workshops were scheduled according to the availability of the participants. Each group participated in two workshops separated by two weeks of modelling work in between (see Fig. 1), the first workshop dedicated to structuring the problem and the second one to explore solutions to address it.

During the first workshop the participants used either (a) qualitative GMB (i.e., in the GMB condition), or (b) SODA (i.e., in the multimethod condition) to create a preliminary boundary of the problem. To perform this task, the GMB groups built a causal loop diagram representing the causes and consequences of the problem under analysis (for an example, see Fig. 2). In contrast, the multimethod groups built a cognitive map with the relevant issues for the problem and used this map to rank strategic priorities (for an example, see Fig. 3). To illustrate the differences in the models built during the experiment, Figs. 2 and 3 present a causal loop diagram and a cognitive map built during the workshops.³

Fig. 2

Causal loop diagram for organisation 1 developed by the group working with GMB (output of workshop 1). Note B balancing loop, R reinforcing loop

Fig. 3

Cognitive map for organisation 1 developed by the group working with the multimethod approach (output of workshop 1)

Both models represent the group assessment of the strategic issue: How to increase the organisation profits by 2020? The cognitive map (Fig. 3) shows that the group working with the multimethod approach focused on the actions (internal and external) that will increase the number of customers and organised them into a means to end network. It is important to highlight that even though SODA does not focus on feedback loop relationships (Rouwette et al. 2011a, b), participants identified three feedback loops in the diagram (represented by blue lines in Fig. 3). The causal loop diagram in Fig. 2, on the other hand, shows that the group working with GMB focused on explanation of the mechanisms by which the organisation gains and serves customers. The diagram shows that the group identified seven feedback loops.

In addition to the map or diagram, the groups drew graphs over time regarding the most important concepts in their model. Graphs over time are graphic representations of how the behaviour of an element has developed in the past and how it could be expected to unfold in the future (Andersen and Richardson 1997; Luna Reyes et al. 2004). Figure 4 shows a graph overtime drawn by the participants of organisation 2 during the workshop. In the graph overtime, the participants represented the previous behaviour of one of the target variables (recyclable waste) as well as the desired and the expected behaviour for the upcoming five years.

Fig. 4

Graph over time drawn by one of the participants of organisation 2 in the group working with the GMB approach (output of workshop 1)

During the time between the two workshops, the facilitator used the outcomes of the first workshops to build four (one for each group) system dynamics models behind the scenes (see Fig. 1). The models were built partially following the scripts of Howick et al. (2006) and mainly based on the experience of the facilitator (i.e., the first author) as a modeller. To ensure that all the ideas were properly represented in the simulation model, a workbook was prepared containing the main concepts and diagram explanations of each model and directing questions. These workbooks were distributed among the participants and collected one week later with participants’ comments and answers. Participants’ comments, answers and suggestions stated in the workbook were used to refine and calibrate the final system dynamics (quantitative) model to be used during the second workshop. Figure 5 shows, as an example, the diagram representation of the system dynamics model built for the group working with GMB in organisation 2.

Fig. 5

Stock and flow diagram representation of the system dynamics model built behind the scenes for the group working with GMB in organisation 2, based on the outputs of the first workshop

Hence, in the second workshop, all the groups used a system dynamics (quantitative) model to explore strategies to solve the problem identified by the organisations. As mentioned, these quantitative models used were built by the first author without the presence of participants from the organisation based on the causal loop diagram or cognitive map produced in the first workshop. Therefore, the second workshop started with a review of the causal loop diagram or cognitive map built during the first workshop and the main results of the workbooks. It continued with the presentation of the (quantitative) system dynamics model and the explanation of the main concepts and feedback loops in the model. To do so, the facilitator rolled out the quantitative model, starting the presentation with a simple structure and progressively adding more details. In this process, the facilitator used graphics and animations to explain the iconography of the model and computer simulations to show the behaviour resulting from the model structure. For example, as depicted in Fig. 5, the system dynamics model of organisation 2 (GMB condition), represented the strategic issue: How to reach the source separated waste materials goal by 2020? The group focused on the creation of awareness of the target population and its sensitisation regarding the environmental consequences of the domestic waste. The amount of source separated materials collected by organisation 2 depends, according to the group, mainly on the number of people who are aware of the importance of recycling and the number of people who are sensible to this importance. The expenses on communication campaigns were identified as the main drivers of the population awareness and sensitisation. Moreover, the model also incorporated logistics constrains that potentially limit the collection of the separated materials, and economic limitations due to the current size of the market for recycled materials in the country. By using this simple model the participants were able to grasp not only the system dynamics iconography represented in the simulation model, but also got important insights about delays, accumulations and feedback loop mechanisms present in the system. The detailed script used is presented in “Appendix 1”, with the title Script 1 Workshop 2.

Fig. 6

Contributions by type: GMB versus multimethod. Note Multimethod, organisation 1, $n = 837$; GMB, organisation 1, $n = 553$; multimethod, organisation 2, $n = 906$; GMB, organisation 2, $n = 867$. Holti’s level of agreement: MM $=$ 95 %; GMB $=$ 91 %

Fig. 7

Contributions by content: multimethod versus GMB. Note Multimethod, organisation 1, $n = 837$; GMB, organisation 1, $n = 553$; multimethod, organisation 2, $n = 906$; GMB, organisation 2, $n = 867$. Holti’s level of agreement: MM $=$ 96 %; GMB $=$ 95 %

Once the team understood the structure represented by the model, the facilitator asked participants to propose and test different strategies. Then, these strategies were included in the model and simulated to observe their consequences. Previous to each simulation, the facilitator asked participants what they expected the simulation result to be. After running the simulation, the facilitator returned to these predictions and explained the simulation results based on the model structure. After all the strategies were simulated, the facilitator asked the participants to think about and to propose different exogenous variables or combinations of events (scenarios) that could threaten the solutions proposed to the initial problem of the workshop. Finally, the facilitator summarised the results of the two workshops, connecting the results between the two rounds of events. The detailed scripts, used in this part of the workshop, are presented in “Appendix 1”, with the titles Script 2 Workshop 2, and Script 3 Workshop 2.

4.1 Type and Content of Contributions

In total, 3163 participants’ contributions were reported during the GMB and the multimethod workshops, and coded according to their type and content. Due to the small size of the sample (only two groups in two organisations) no special statistics treatment was applied to the data. In Figs. 6 and 7 the differences between the contributions made by participants in the two approaches can be seen. Figure 6 shows differences in the contribution types: divergent and assessment. More divergent and assessment contributions were made during multimethod workshops than during the GMB workshops. Further, it is important to mention some other relevant differences in the contribution types between the two organisations. While the group working with GMB in organisation 1 showed a similar amount of convergent contributions to the group working with the multimethod, in organisation 2 the group working with GMB reported more of this type of contributions than the group working with the multimethod. Conversely, a noticeable difference was found between the groups working with GMB and the multimethod in organisation 1 for clarifying/summarising contributions, however this time, no such difference was observed for this type of contribution between the two groups in organisation 2.

Regarding the content of the contributions, in Fig. 7, it can be seen that in the multimethod workshops, participants contributed more statements with regard to the problem formulation (e.g., concerning the issues important to discuss) than in the GMB workshops. In contrast, the same figure shows that participants who used GMB contributed more statements about the causes of the problem (i.e., expressing causal explanations) than the participants who used the multimethod approach. Though, this difference is smaller between the groups of organisation 1.

In addition the results show that in the case of organisation 1 more contributions were provided during the multimethod process than during the GMB process. These results could be expected since Ackermann et al. (2010, p. 338) described the first workshop of the multimethod approach as a “fast and efficient” way to elicit and rank many ideas. Additionally, the positivist perspective of GMB could block some emergent conversations (Rouwette and Vennix 2006) and, therefore, reduce the number of divergent contributions by narrowing them to variables that represent causes or consequences. However, despite that the facilitator and the scripts were the same for the two organisations, in organisation 2 there was only a slight difference between the number of contributions provided by the participants in the two methods.

The results above support the findings of previous case study-based research (see Table 1) and were somewhat expected since both methods (SODA and GMB) stand on different theoretical and methodological assumptions, as outlined in Sect. 2. However, these results also bring novel insights since they arise from: a) the systematic, objective and transparent quantification of the differences between a single (i.e., GMB) and multimethod approach (i.e., the combination of SODA and GMB), and b) the direct comparability of the two approaches in a controlled experimental setting, wherein managers from the same company were asked to tackle the same problem.

4.2 Cognitive change

The degree of cognitive change was assessed by comparing the answers of the participants to the questionnaires before and after the experiment regarding what were in their opinion the three best alternatives to solve the strategic issue under analysis. By comparing the two questionnaires it was possible to quantify how many participants changed one, two, or all (three) of their suggestions after participating in the experiment. In Fig. 8, it can be seen that the participants in the multimethod workshops exhibited a higher degree of cognitive change because they shifted most of their original suggestions after the experiment. Figure 8 shows that while most of the participants from the multimethod workshop (seven out of nine participants) changed all of their proposals after the experiment, only two of the eight participants who participated in the GMB workshops changed all of their proposals.

Fig. 8

Degree of cognitive change (see “Appendix 2”, Item 1 and 2): GMB versus multimethod. Note Multimethod, $n= 9$; GMB, $n= 8$

Nevertheless, it is important to recall how we measured the degree of cognitive change. Our measurement of cognitive change reflects the degree to which participants—in their perception—changed their original proposals; it did not measure the extent to which participants validated their previous beliefs, learned about the causes of the problem or changed their ranking of possible alternatives. Further research may explore this issue and illuminate on the causes of cognitive change.

4.3 Consensus

The consensus was measured based on participants’ answers in the postquestionnaire ($n = 17$) and during the interview ($n= 10$). These answers are summarised in Figs. 9 and 10 for the interviews and postquestionnaires, respectively. These results show that the level of consensus perceived by the participants in the GMB workshops was higher than the degree of consensus indicated by those from the multimethod workshops.

Fig. 9

Degree of consensus reported in the interviews (see “Appendix 2”, Items 5 and 6): GMB versus multimethod. Note Multimethod, $n= 5$; GMB, $n= 5$

Fig. 10

Degree of consensus reported in the postquestionnaires (see “Appendix 2”, Item 3): GMB versus multimethod. Note Multimethod, $n= 9$; GMB, $n= 8$

Fig. 11

Content of consensus in organisation 1 (see “Appendix 2”, Item 2): GMB versus multimethod. Note Multimethod, organisation 1, $n= 10$; GMB, organisation 1, $n= 12$ (up to three alternatives per participant). C1 $=$ expansion, C2 $=$ real estate market, C3 $=$ finance results, C4 $=$ advertising, C5 $=$ customer satisfaction

Fig. 12

Content of consensus in organisation 2 (see “Appendix 2”, Item 2): GMB versus multimethod. Note Multimethod, organisation 2, $n= 15$; GMB, organisation 2, $n= 12$ (up to three alternatives per participant). C1 $=$ internal strategy, C2 $=$ communication and awareness, C3 $=$ technic and logistic solutions, C4 $=$ budget control, C5 $=$ stakeholder involvement

In addition, we analysed the content of the consensus reached during the experiment by clustering and comparing the solutions proposed by the participants at the end of the experiment. When the content of these solutions was analysed, for both approaches, the majority of the proposed solutions could be clustered under the same category as can be seen in Figs. 11 and 12. For example, six participants from the multimethod workshops in organisation 1 (see Fig. 11) agreed that the final solution of the problem was to perform activities in the category “Expansion” (C1).

Moreover, Figs. 11 and 12 show that participants in the multimethod approach provided more solutions that could be grouped under the same cluster. This could give the impression that more participants agreed about the same solution for the multimethod approach; hence, their degree of consensus seemed higher than the degree of consensus reached for the participants working with the GMB approach. However, this result seems at odds with the degree of consensus reported in the interviews and postquestionnaires (see Figs. 9, 10). Hence it is not possible to conclude which approach (i.e., the single or the multimethod) excels in fostering consensus.

Finally, it is interesting that the analysis of the content of consensus shows that the groups agreed on completely different solutions and reached different conclusions from the models they built. These findings point to the important effect of the problem structuring stage (Ackermann and Eden 2011) on the definition of strategic agendas. Our results indicate that when SODA was used in the first workshop, participants were encouraged to explore a wider range of ideas, which might explain the differences in content of the solutions proposed. For instance, the multimethod group sought for more alternatives outside the boundaries of the organisation, such as expansion and stakeholder engagement (see Figs. 11, 12, respectively). On the other hand, when in the first workshop (qualitative) GMB was used to frame the problem, participants focused on the causes of the current behaviour within their influence and, hence, they were inclined to choose potential solutions inside the boundaries of the organisation, such as communication expenses, (i.e., advertising, see Fig. 12, category C2) and financial results of the organisation (see Fig.  11, category C3).

These conclusions are supported by the answers of the participants during the interviews. For example, when they were asked, “What would you consider the main contributions of the approach used in the first workshop?”, participants working with GMB emphasised the enhancement of problem understanding:

I think we understood better some problems of the company... (Marketing Manager, organisation 1);

I think the discussion was good...we also came up with some ideas of what were the reasons for the problem, like what things were preventing us from achieving the desired results... (Human Resources Manager, organisation 2);

Well, we discuss a lot about the causes of the problem...that was important...because before making decisions we should analyse what is the current situation and how we ended here (Financial Manager, organisation 2).

On the other hand, when asked the same question, participants working with the multimethod approach highlighted that this approach helped them to bring many ideas to the discussion and to prioritise them:

I think the cognitive map was really good. The map really help us to focus on the important issues....you see at the beginning we have a lot of different ideas... (Financial Manager, organisation 2);

Clearly the cognitive map give us a better perspective...you saw we have many issues but when we arrange them in this framework with the goals on top, it became clear some issues were not important (Financial Manager, organisation 2);

We cannot focus on everything that we would like, so we need to identify what is coming first (Human Resources Manager, organisation 1).

Even though these results are not unequivocal, they seem to indicate a relationship between the approach used to structure the problem and the resulting model that participants built and, hence, the outcomes, of the process. Such relationships should be studied in more detail in further research.

4.4 Commitment

To evaluate the degree of participants’ commitment with the final solution, the participants were asked in the postquestionnaire to what extent they felt committed with the final solution of the workshop, and in the interviews to what extent they were willing to implement this solution. The results of the postquestionnaire are presented in Fig. 13 and show that participants working with GMB demonstrated a higher degree of commitment than participants in the multimethod condition.

Fig. 13

Degree of commitment: GMB versus multimethod (see “Appendix 2”, Item 4). Note Multimethod, $n= 9$; GMB, $n= 8$

It has been discussed in the literature that commitment is closely related to the ownership over the model built by the group and its results (e.g., Akkermans and Vennix 1997; Franco 2007). Moreover, Richardson and Andersen (2010) have rightly pointed out that the ownership of the resulting simulation model will be closely related to the consistency between the maps drawn by the group and the resulting simulation model. In this sense, the relatively low commitment of the participants working with the multimethod could be a result of the inconsistency between different representations of the problem (cognitive maps vs. system dynamics model) used in the multimethod approach. This is an important disadvantage of this approach because, in the context of the strategic analysis process, commitment to implementation is a key ingredient for the success of the strategies developed in the process. Similarly, authors like Videira et al. (2012) and Luna Reyes et al. (2006) have pointed out the importance of active involvement of the participants in the simulation model construction to create ownership of the resulting model and its results. Therefore, important challenges to be addressed in designing multimethods that involve different types of models, are the trade-offs between model consistency and client involvement, and the resulting commitment. Especially when simulation models are included, attention to these trade-offs seems warranted.

In particular, we recognised lack of ownership in: (a) participants’ lack of confidence in the outputs generated by the workshop (see Fig. 14) and (b) participants’ perception that the final model was incomplete or needed to include particular elements they mentioned during the first workshop and that were not explicitly present in the final model (“Appendix 2”, items 7 and 8).

Fig. 14

Confidence (We use the variable “confidence” as a measure related to model ownership (Howick et al. 2008; Rouwette et al. 2011b). Note: The results in the figure correspond to the answers of participants in the postquestionnaire to the question: to what extent do you agree or disagree that the workshop has helped you to... Have confidence that the outputs generated by the workshop will make a difference?) of participants in the outputs generated: GMB versus multimethod. Multimethod, $n= 9$; GMB, $n = 8$

Evidence regarding ownership issues was found in the interviews, for instance, when participants were asked about their desire to help to carry out the implementation of some the results of the workshops (see “Appendix 2”, Items 7 and 8).

Also answers to questions about moments of the discussion that the participant disliked, or elements that were not included and could improve the results of the meeting, suggested that ownership was at stake. Participants working with the multimethod approach said, for instance:

I think the model could have more details...was too general....many things stayed out of the model...think we should invest more time including the details that are missing and then try the solutions.... I’m sure the results would be different (General Manager, organisation 1);

Mmmmm...think I will need to understand more what it requires of me...so far is not clear...I think everybody in the team will have to contribute in one way or another... (Financial Manager, organisation 2);

Well...we need to put more details in the model...and talk more about implementation for sure... I think for now it is a good idea to start...but still more work is needed (Designer, organisation 1).

On the other hand, participants working with GMB provided statements like:

Yes...I will be happy to do so...I think it is important to show the investment should be constant and also the conditions...many times we have these events or these campaigns...but these are just single shots...this should be something we keep doing (Communications Manager, organisation 2).

Yes...I will like to do it...think I know about our competitors’ quality proposition and can assess what should be the quality we propose, and also the prices...I know about a lot of cheapest places but with really bad quality (Operations Manager, organisation 1);

Finally, also the expertise of the facilitator needs to be considered. Finding a facilitator with enough knowledge, training and experience with different methods is not always possible and could constitute a limitation for replicating this type of research. In our case, the facilitator had more experience in facilitating GMB workshops than SODA workshops and it must be noted that we did not formally measure the facilitator’s performance. Nevertheless, observation in the room (e.g., by the third author) and evaluations of participants indicated that the facilitator’s competence to perform both methods did not constitute a limitation in this study.

5 Conclusions

The present study offers a first step in the process of systematically comparing facilitated modelling approaches under comparable settings while addressing real-life problems and strategic decisions in organisations. For this purpose, we advanced a novel experimental research design which allowed to validate previous case study-based findings and provide important lessons for further applications by academics and facilitated modelling practitioners.

The results showed that combining SODA and GMB in the multimethod proposed by Ackermann et al. (2010) stimulates a process that encourages participants to provide a broad range of ideas. This stimulus, including the benefits of computer simulations, influenced the final outcomes. In particular, we found a higher degree of cognitive change. However, the multimethod exhibited an important disadvantage when compared with the GMB approach (i.e., in the qualitative and the quantitative mode) in terms of commitment to the final solution. This lack of commitment could be attributed to the negligible degree of participant involvement in the quantitative system dynamics model building process. Moreover, our results provide empirical evidence to support the statement of Richardson and Andersen (2010) that diagrammatic consistency is important to extend the ownership from the maps to the resulting simulation model. Like the results show, the participants of the multimethod condition were less willing to implement the solutions proposed during the workshops. Participants said that they needed to interact more carefully with the model before trusting it and, more importantly, they wanted to directly contribute to it by adding “missing” elements to its structure. We understand these statements as a manifestation of the lack of ownership some participants had towards the final model. Consequently, the difficulty to realise sufficient diagram consistency can constitute an important limitation at the moment of integrating facilitated modelling approaches that use different diagrams and icons (like it is the case for SODA and system dynamics facilitated modelling) and should be carefully considered at the moment of planning the workshops and selecting the scripts.

In addition, the results suggest that the approach used during the problem structuring stage has an effect on the strategies that are analysed in the model and the conclusions the group draws from them. Participants’ focus on problem causes in GMB was joined by a search of solutions inside the boundaries established for the selected strategic problems. On the other hand, the divergent dialogue promoted during the SODA workshop encouraged the emergence of solutions outside the boundaries of the model. As we examined only two groups in two organisations, there is not enough evidence to build explanations about the mechanisms behind such differences. More research is needed to understand these differences and to formulate practical recommendations about how to effectively use or combine different approaches to support the negotiation process. Therefore, we recommend conducting more experimental research with comparable groups using different approaches to structure the problem in order to understand how the different approaches’ perspectives or scripts influence the final decisions of the managers’ team. In order to evaluate facilitated modelling approaches based on aggregation of data of future studies, it is of the utmost importance that organisation and group characteristics as well as the scripts used to support the strategic analysis processes are clearly described in the reports of future studies.

In a more practical sense, the present results also provide practitioners with insightful hints of what to expect when combining different facilitated modelling approaches, in particular with GMB. In case SODA is part of the combination, the different way to frame the problem could contribute to promoting wider discussions and to create models that are more open to different perspectives. However, practitioners should keep in mind that time and effort should be dedicated to creating a smooth transition between the methods in order to foster ownership towards the final modelling outputs.

In short, by working in our experiment with real clients and their organisation’s problem, our research design provides at least two novel contributions to the literature on facilitated modelling with GMB. In practice for organisations, and practitioners who use a facilitated modelling approach, our study suggests a relationship between the approach used and participants’ commitment toward the final solution. Our hypotheses with this respect are that: a) the level of commitment is affected by the degree of participants’ ownership toward the final model, b) ownership decreases when shifting from one diagramming tool to another in a combined approach.

For facilitated modelling researchers, our results also suggest that by encouraging participants in the GMB approach to think about causality, this approach might lead participants to only seek solutions within the initial boundaries established for the selected strategic problem. In this case, the problem structuring method needs to strike a balance in the setting of boundaries, so that these are closed enough to effectively analyse endogenous problem causes, but also sufficiently broad to promote deliberation on innovative solutions. We hypothesize that, due to its impact on setting boundaries, the approach (a single vs. multimethod) influences the scope of the strategies that are identified as paths of action. Testing this hypothesis might lead to (a) a better understanding of the effect of the way in which issues are elicited in a workshop (e.g., in a multimethod vs. GMB approach) on participants’ awareness of the bigger picture, and (b) criteria for deciding whether a single or a multimethod (and if so, which combination) is appropriate for a particular context and problem.

The present results are bound to be exploratory due to two main limitations; as mentioned, (a) the sample size, and (b) participants’ time. The sample size was constrained to only four groups participating in the experiment and participants’ time investment was restricted to their participation in two workshops and filling in the workbook (approximately eight hours per person). Notwithstanding these limitations, there is considerable potential for further research using the insights provided by the current experiment. Additional studies can extend the present findings by repeating the experiment in other cases compensating, if possible, for the limitations referred to above. Specifically, more research is needed to discover to which extend the explanatory hypotheses proposed in this paper are meaningful.

Finally, there are also opportunities for a more explicit consideration of the scripts including the techniques used to make the transition from cognitive maps, used in SODA and other facilitated modelling approaches, to system dynamics diagrams. With the exception of the scripts presented by Howick et al. (2006) there are no scripts, as far as the authors have knowledge of it, that allow to effectively link cognitive maps with system dynamics representations. Moreover, there is a lack of evidence on the effectiveness of the currently existing scripts. This, for sure, constitutes an important area of research since the sense of ownership, and therefore the commitment of the participants with the results of facilitated modelling workshops, could be compromised when a multimethod is used.

Appendices

Appendix 1

 Script 1 Workshop 1 (GMB): graphs over time

Context	At the beginning of a group model building session as it is a springboard for discussion about the problem to be modelled
Purpose	To engage participants in a group model building session in framing the problem, initiating mapping, eliciting variables and gathering input in deciding the reference modes for the study
Primary nature of group task	Divergent
Time	Preparation: 10 min
	Session: 30 min
	Follow-up: None
Materials	Camera or other method to capture the graphs
	Stacks of $8.5 \times 11$ white paper with X and Y axes drawn on them
	Large blank wall $(8'\times 10')$
	Thick markers
	Glue sticks, tacks or painter’s tape
Inputs	None
Outputs	Candidate variables for the dynamic model or the map
Roles	Facilitator works with the group and has some experience with SD recorder to document the session and photograph the clustered graphs
People needed in the room	Participants
	Facilitator
	Recorder
Steps	1. Based on group size, was decided to work individually
	2. The modelling team hands out sheets of white paper to each participant or group
	3. The facilitator gives an example of how to draw a graph over time, carefully labelling X-axis “Time” with start time, end time, and now indicated with a vertical dashed line. The Y-axis is labelled with a variable name. The facilitator then sketches the behaviour over time
	4. The facilitator then asks participants to draw one variable over time per piece of paper. The participants should be given the option of including hoped for behaviour, expected behaviour, and feared behaviour on the same graph
	5. The facilitator and wall-builder walk around and help participants with the task if they need it. Allow 15 min or until the group runs out of steam to complete the task
	6. Reconvene as a large group. Facilitator takes one graph at a time from each participant, holds it up in front of entire group and asks him/her to talk about it. Ask for participants to share the “best stuff” first. Clarify timescale, variable names, etc.
	7. The facilitator pastes the graph on the wall
	8. The facilitator repeats steps 6 and 7 with each participant, taking one graph at a time until all graphs are shown or time has run out. Finish by asking if any participant has something else that really ought to be shown
	9. During the steps 7 and 8, the facilitator tries to cluster the graphs meaningfully on the fly based on themes and variables
	10. Facilitator explains the clusters of graphs on the wall, trying to summarize dynamics that help to characterize the problem that emerge from the participants’ graphs
	11. The facilitator enables the participants to talk about the clusters and the characterization of the problem they imply
	12. Consider labelling the clusters based on themes or related variables
Evaluation criteria	Interesting, self-sustaining group discussion after clusters described by the wall builder
	Meaningful clusters identified
	Graphs tend to converge to a clear dynamic problem
	Some key dynamic variables emerge from reflecting on the graphs and thematic clusters
	Modelling team can begin to see key stocks and perhaps important feedback loops
	Members of the group appear to have better understandings of the issues of interest to other members
Author(s)	George P. Richardson and David F. Andersen
History	First described in Luna Reyes et al. (2006)
References	Andersen, D. F., & Richardson, G. P. (1997). Scripts for group model building. System Dynamics Review, 13(2), 107–129
Notes	None

1. Adapted from Andersen and Richardson (1997, p. 118)

 Script 2 Workshop 1 (GMB): variable elicitation

Context	Early in the modelling process
Purpose	To facilitate consensus-based group discussion about the model problem and boundaries. It elicits key variables that become the input for other activities
Primary nature of group task	Divergent
Time	Preparation: None
	Session: 20 min
	Follow up: None
Materials	Markers
	Stacks of plain paper
	Chalk/whiteboard markers
Inputs	None
Outputs	Prioritized list of variables
Roles	Facilitator familiar with system dynamics modelling process and next steps of project
	Recorder to take notes on any questions, list of next steps, and who is going to do the next steps and by when they are due
People needed in the room	Facilitator
	Modeller
	Participants
Steps	Part I
	1. The facilitator gives each participant sheets of blank paper and markers
	2. The facilitator writes a task focusing question on the whiteboard or flipchart, such as, “What are the key variables affecting the process and outcomes of the [project name] project?”
	3. The facilitator asks participants to write as many problem-related variables as they can on the sheets of paper. Participants are given a few minutes to work individually on their lists
	4. Once they have finished the individual exercise, the facilitator uses the same process used in the hopes and fears script to put all individual variables on the board. When a variable name is open to several interpretations, the facilitator asks for a brief description or definition of the variable, including the units in which the variable can be measured
	5. The facilitator writes the variable name on the board, including any additional information in parenthesis
	Part II
	6. The facilitator asks the participants to prioritize the variables by simple voting mechanisms. Individuals can vote for as many variables as they want. The number of votes for each variable is also written down on the board
	7. The facilitator makes a summary of the variables on the board, while the recorder captures the products of the process either photographically or in a word processor
	8. The facilitator suggests which variables can be considered stocks as they are mentioned. If the participants agree, the facilitator can add the words “level of” to these variables
Evaluation criteria	Identification of key variables and stocks
Author(s)	Andersen and Richardson
History	Originally described in Luna Reyes et al. (2006)
Revisions	Unknown
References	Luna-Reyes, L. F., Martinez-Moyano, I. J., Pardo, T. A., Cresswell, A. M., Andersen, D. F., & Richardson, G. P. (2006). Anatomy of a group model-building intervention: Building dynamic theory from case study research. System Dynamics Review, 22(4), 291–320
Notes	None

1. Adapted from Scriptapedia (2015)

 Script 3 Workshop 1 (GMB): structure elicitation

Context	This script fits after exercises to elicit reference modes and a break have been completed
Purpose	To capture the key endogenous mechanisms elicited during a discussion that have the potential to explain the observed behaviours or dynamic hypotheses
Primary nature of group task	Convergent
Time	Preparation: 15–20 min
	Session: 40 min
	Follow up: None
Materials	1. Chalk/whiteboard markers
	2. Flip chart/whiteboard
Inputs	Graphs over time
Outputs	Causal loop diagram structure
Roles	Facilitator familiar with system dynamics modelling process and next steps of project
	Recorder to take notes on any questions, list of next steps, and who is going to do the next steps and by when they are due
People needed in the room	Facilitator
	Recorder
	Participants
Steps	1. During the break after the graph over time script, the modelling team selects a couple of key behaviours from the reference mode elicitation exercise
	2. The facilitator starts the structure elicitation by suggesting two variables. The facilitator explains that these stocks are initial simplifications of the system
	3. The facilitator asks the group to identify the variables that help to increase or decrease the initial value of those variables. Participants suggest causal relations linked to these two initial stocks and their corresponding rates
	4. The facilitator clarifies the nature of the causal relationships with the group while drawing them on the board
	5. After adding a couple of variables and causal relations, the facilitator summarizes by telling the story embedded in the model so far and asks the group to add further causal explanations, stressing the importance of selective thinking about causality with the purpose of reaching a powerful and parsimonious explanation of the project success
Evaluation criteria	A basic causal structure has been produced
Author(s)	George P. Richardson and David F. Andersen
History	Luna-Reyes, L. F., Martinez-Moyano, I. J., Pardo, T. A., Cresswell, A. M., Andersen, D. F., & Richardson, G. P. (2006). Anatomy of a group model-building intervention: Building dynamic theory from case study research. System Dynamics Review, 22(4), 291–320
Revisions	None
References	None
Notes	This script based entirely on Luna Reyes et al. (2006)
	The main limitation of this script is the risk of having a discussion guided by the group facilitator. The main advantage is that it is flexible and easy to prepare for. Initial aggregations can create conflict with the client group
	Usually, the facilitator or the reflector differentiates between detail complexity (many disaggregated processes), and feedback complexity (a rich feedback story with many loops), explaining that system dynamics modellers have found that it is much easier to increase the detail complexity once an appropriate level of feedback complexity has been reached than to increase feedback complexity when the desired level of detail complexity has been reached
	A very important element in the process is to write down (or erase) all group ideas on the board, even if they cannot be included easily as part of the feedback story

1. Adapted from Scriptapedia (2015)

 Script 1 Workshop 1 (multimethod): initial issue elicitation

Context	At the beginning of a strategy as and issues management forum
Purpose	The objectives of this script is to make the managers aware of the bigger picture
Primary nature of group task	Divergent
Time	Preparation: 10–15 min to prepare
	Session: 35–40 min
	Follow up: Causal linking process
Materials	Computer with Decision Explorer
	Projector
Inputs	Starter question
Outputs	Printout of issues surfaced and the emergent strategic themes
Roles	Facilitator familiar with strategic options development and analysis and cognitive mapping
	Recorder to take notes on any questions and issues definitions
People needed in the room	Participants
	Facilitator
	Recorder
Steps	Set up of the forum
	1. The facilitator thanks participants for accepting to participate on the forum and introduces the issues management forum process, the first phases of issue surfacing
	2. Then the facilitator reviews the agenda and the process
	3. The facilitator encourages participants to use their judgment to surface their experience and wisdom as well as hard evidence during the workshop
	4. Introduction of strategic issue
	5. The facilitator introduces the strategic issue showing it on a flip-chart sheet and/or placing it in a central position on Decision Explorer
	6. It is important that the facilitator checks whether all the participants are clear in their understanding of the starter question
	7. Initial issue surfacing
	8. Then the facilitator provides a brief description of what an issue might be, for example: Issues may be seen as past, present or anticipated events or circumstances, internal or external which cause some concern to the future
	9. The facilitator asks the participants to identify three of this issues related with the starter question and to write them down on a piece of paper
	10. The facilitator encourages the participants, where possible to use around 6–15 words per contribution
	11. Then the facilitator asks each participant for one of the issues they wrote down making sure that all participants contribute going around the group and asking each participant (a round robin)
	12. During the process the facilitator encourages the participants to contribute in a “yes and” mode rather than a “yes but” mode
	13. As each of the issue statement is made, the facilitator captures the issues on the Decision Explorer so that all participants are able to continuously keep it in mind
	Issues review
	14. After the first round of issue surfacing the facilitator reviews the issues by spending a little time returning to all the issues mentioned
	15. At this point the facilitator may ask to reword some issues so that they are more actionable. Adding a verb to the statement can do this
	16. Next issue elicitation rounds
	17. Then the facilitator stimulates the participants to think of deeper considerations and go beyond the everyday problems, and starts a second robin round to surface more issues
	18. Facilitator pays attention to ensure that both internal and external perspectives, have been taken into consideration
	19. The exercise continues for two or three more rounds. Typically by the end of this stage, with a group of 5 or 6 participants, around 15–20 issues will have been surfaced
	Development and review of thematic clustering
	20. Either during, or once the initial flood of issues has significantly slowed, the facilitator begins to move the issue statement into thematic clusters. This activity helps the group to make sense of the growing body of material and not feel overwhelmed
	Final review
	21. The facilitator finishes the script doing a quick review of all the issues surfaced and asking the participants if they understand the meaning of all of them
	22. In case there are questions about the meaning of a particular statement, the facilitator tries to clarify these with help of the group
Evaluation criteria	An initial map containing 15–20 issues has been produced
Author(s)	Fran Ackermann and Colin Eden, 2011
History	None
Revisions	None
References	Ackermann, F., & Eden, C. (2011). Making strategy, mapping out strategic success. 2nd Edition. Sage: London, pp. 70, 81
Notes	None

1. Adapted from Ackermann and Eden (2011, pp. 70–81)

 Script 2 Workshop 1 (multimethod): causal linking process

Context	After the Initial issue elicitation script
Purpose	The objectives of this script is to make the team aware the strategic issues are not independent, they have an impact on each other
Primary nature of group task	Convergent
Time	Preparation: 5 min to prepare
	Session: 35–40 min
	Follow up: None
Materials	Map containing the issues
	Computer with Decision Explorer
	Projector
Inputs	Issue map with content related clusters
Outputs	Printout of issues surfaced, clustered into emergent strategic themes
Roles	Facilitator familiar with strategic options development and analysis and cognitive mapping
	Recorder to take notes on any questions and issues definitions
People needed in the room	Participants
	Facilitator
	Recorder
Steps	Explain the causal linking process to participants
	1. The facilitator starts providing a clear idea of what the links represents. In issue management the links represent causality. Therefore they can be interpreted as “x” may lead to “y” or may also described “x” is a means to the end “y”
	2. The facilitator uses an explanation in support of a view already made by a participant to illustrate a causal link
	Carrying out issues linking
	3. Then the facilitator asks participants how the different issue statements in the cluster relate to each other
	4. The recorder captures the explanatory material that participants provide to explain the reasoning behind the links they propose
	5. It is recommended to use a round robin process to establish links by asking each participant, in turn, to propose a link
	6. During the process the facilitator checks that participants have a basic understanding (not necessarily agreement) of the chains of argument
	7. The linking process continues until all the issues have been linked. If there are some issues of which participants think they are completely unrelated, they can be put a part for further linking
	8. When the linking process has finished, the facilitator creates a tidy map by moving the statement with no out-arrow to the top, reducing crossing arrows and moving those statements with no in-arrows to the bottom of the map
	Carrying out analyses of the issue network and refine the map
	9. The facilitator leads the analysis of the map looking for those issues that do not impact other issue statements for clues regarding possible goals
	10. Then the facilitator identifies if there are any feedback loops in the map. If it is the case that some feedback loops are found the facilitator explores the nature of the feedback loop(s) (reinforcing or balancing) and helps the group to consider the means for intervening to exploit or minimize/manage the impact of feedbacks
	11. Next the facilitator helps the group to identify key issues. These are issues that have a lot of links around them. Different views of Decision Explorer could be used with this purpose
	Produce an overview map
	12. Finally, the facilitator summarises the map highlighting those issues identified as priorities. If it is necessary the facilitator can reduce the complexity by producing a summary map including only those issues
Evaluation criteria	An initial map containing 15–20 issues has been produced
Author(s)	Fran Ackermann and Colin Eden, 2011
References	Ackermann, F., & Eden, C. (2011). Making strategy, mapping out Strategic success. 2nd Edition. Sage: London. pp. 81–89
Notes	None

1. Adapted from Ackermann and Eden (2011, pp. 81–89)

 Script 1 Workshop 2: Presenting a high-level system dynamics model

Context	At the beginning of the workshop to explain a system dynamics model
Purpose	The objectives of this script is to introduce the group to a system dynamics model built or refined
Primary nature of group task	Convergent
Time	Preparation: 60 min to prepare
	Session: 35–40 min
	Follow up: Developing strategic options
Materials	System Dynamics Model
	Computer with a system dynamics software
	Projector
Inputs	Stock and flow diagram
Outputs	Participants have clear understanding of the structure in the model
Roles	Facilitator with medium experience on system dynamics modelling
	Recorder to take notes on any questions and issues definitions
People needed in the room	Participants
	Facilitator
	Recorder
Steps	1. The facilitator starts presenting a simple structure of the model. This structure can be a stock with two flows (inflow, outflow) or two stocks connected by one flow
	2. The facilitator uses this structure to introduce the team to the system dynamics iconography, the meaning of stocks and flows
	3. The facilitator uses a plumbing metaphor, with the stocks capacity seen as a tub and the flows as water in the tub. And then refers these terms to variables with real meaning in the model (for instance population and adopters)
	4. Once the group is clear about the iconography of the model the facilitator continues rolling up the model and gradually introducing the other chunks of the model
	5. During the process facilitator checks continuously if the participants fully understand what the structure represents
Evaluation criteria	Participants have understood the main elements of the model structure and the system dynamics iconography
Author(s)	Frank Ackermann, David Andersen, Colin Eden and George Richardson
History	None
Revisions	None
References	Ackermann, F., Andersen, D. F., Eden, C. & Richardson, G. P. (2010). Using a group decision support system to add value to group model building. System Dynamics Review, 26(4), 335–346
Notes	None

1. Adapted from Ackermann et al. (2010, pp. 339–343)

 Script 2 Workshop 2: developing strategies

Context	After the “Presenting a high-level system flow view” script
Purpose	The objectives of this script is to explore possible alternatives to the problem addressed in a system dynamics simulation model
Primary nature of group task	Divergent
Time	Preparation: 20 min to prepare
	Session: 15–20 min
	Follow up: None
Materials	System dynamics model
	Computer with a system dynamics software
	Projector
Inputs	Stock and flow diagram
Outputs	Participants have a clear understanding of the structure in the model
Roles	Facilitator with high experience on system dynamics modelling
	Recorder to take notes on any questions and issues definitions
People needed in the room	Participants
	Facilitator
	Recorder
Steps	1. The facilitator sets up task by asking participants to write short phrases naming policies that participants would like to see discussed, modelled, and simulated in the course of the work. They could be policies tried in the past or currently, or policies being talked about for the future including both ideas that are considered realistic or “wild” ideas that exceed expectations for what is feasible
	2. The participants may work in pairs to build confidence and share thinking while still keeping the divergent nature of the group task
	3. The facilitator asks each participant (or pair) to present one of the policies they suggest to the group
	4. In the meanwhile the facilitator takes notes of the implications for the model of the policies proposed. These implications can be: to modify a variable value, add a piece of structure or add missing links between variables
	5. Once all the participants have presented at least one policy, the facilitator explains the implication of each policy in the model structure, and introduces it in the model
	6. Then the facilitator asks the participants what they think the resulting behaviour will be when the policy is introduced in the model
	7. After the participants have expressed their assumptions regarding the expected behaviour, the facilitator runs the simulation and shows the results to the groups
	8. Then the facilitator discusses the results with the group and reviews with them the structure responsible for the observed behaviour
	9. The facilitator repeats the steps 6–8 for all the other policies proposed by the group
Evaluation criteria	Participants have tested different strategic options in the simulation model and discussed their effects
Author(s)	Frank Ackermann, David Andersen, Colin Eden and George Richardson
History	None
Revisions	None
References	Ackermann, F., Andersen, D. F., Eden, C.&Richardson, G. P. (2010). Using a group decision support system to add value to group model building. System Dynamics Review, 26(4), 335–346
Notes	This script needs the facilitator to have complete understanding of the model since the facilitator will have to introduce changes and run it during the workshop. It is not recommended to use medium or big models, since they could be too complex to manage accurately during the workshop. In those cases, the policies can be simply voiced by the group and mentally simulated in the diagram (Vennix 1996, p. 178). Then, in back office, the modeller can adjust the model to present the result in further sessions to the group

1. Adapted from Ackermann et al. (2010, pp. 339–343)

 Script 3 Workshop 2: exploring scenarios

Context	Toward the end of the policy design stage
Purpose	To help the team to explore different scenarios for the policy implementation stage
Primary nature of group task	Convergent–divergent
Time	Preparation: 5 min (paper and markers)
	Session: 20–30 min
	Follow up time: None
Materials	Markers
	$8.5 \times 11''$ (or A4) paper
	Masking tape for posting on wall
	Wall or window for posting
Inputs	List of policies to implement
	System dynamics diagram (SFD or CLD)
Outputs	Scenarios to test the policies chosen by the group
Roles	Facilitator with basic experience in SD recorder
People needed in the room	Participants
	Facilitator
	Recorder
Steps	1. The facilitator sets up task by asking participants in subgroups of two or three to identify the exogenous variables in the model they found are more relevant to the success of the policies previously proposed
	2. The facilitator asks each subgroup to explain to the entire group the two variables they found most important and why. (If there are many groups, one variable per group could be enough)
	3. The recorder takes note of all the variables the subgroups propose on the board
	4. Then the facilitator asks the participants to identify what the different development are those variables could take in the future and what could be the effect of that development on the expected results of the policies proposed previously in the workshop. It is particular important that the facilitator encourages the participants to think on a creative way
	5. Then the group should classify all the variables that can result in negative effects on the policy proposed in a pessimistic scenario and those with potential positive effects in optimistic scenario
	6. The facilitator invites the subgroups to create coherent histories for both scenarios
	7. If the group is big enough the steps 4–6 can be performed in small groups of 3 or 4 participants
	8. To conclude, the facilitator summarises the scenarios presented by using the system dynamics structure
Evaluation criteria	List of variables found
	Scenarios developed by the group
Author(s)	First author
History	Based on the scenario building techniques combine with System Dynamics for an assignment. Newly developed in this study
Revisions	None
References	Schoemaker, H., & Paul J. (1995). Scenario planning: A tool for strategic thinking. Sloan Management Review, 36(2), 25
Notes	Script developed by the first author for the experiment purposes

Appendix 2

Per instrument, questions and interview items used to measure the main variables

Questionnaire	Item no.	Question	Variable
Prequestionnaire	1	Which are in your opinion the best alternatives in order to assess the initial question of this workshop? Please name three.	Cognitive change
Postquestionnaire	2	Which are in your opinion the best alternatives in order to assess the initial question of the workshop? Please name three.	Consensus
			Cognitive change
	3	To which extent do you agree with the final solution of the workshop?	Consensus
	4	To which extent do you agree with the follow statement:	Commitment
		I feel committed with the implementation of the final solution of the workshop?
Interviews	5	What do you think about the decision made at the end of the workshops?	Consensus
	6	So do you agree that it is the best alternative?	Consensus
	7	If we will have the opportunity to present this alternative to (name of the C.E.O.), will you be willing to do so?	Commitment
	8	And if he decides to go on with this alternative, and we make a team to implement it...would you like to join?	Commitment

1. Postquestionnaire adapted from Midgley et al. (2013)