Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

The first wave of group model building research consisted of over 130 studies describing single applications, brought together and analysed in two review papers. Research on exploring the underlying mechanism can broadly be placed into three groups: studies focusing on participants as recipients of information, those focusing on participants as sources of information, and those focusing on participants looking at the interaction between receiving and contributing information. The second wave of studies uses theories from social psychology to explain how modeling impacts knowledge and behavior. In modeling sessions, participants receive information, which might persuade them to change their evaluation of the issue at hand. Changes in evaluations in turn lead to changes in intentions and actions. While these studies focus on receiving information, the third wave of studies looks at participants as actively constructing information. A model helps to bring pieces of information together, but a necessary precondition is that information is brought out into the open. As each individual participant knows only a fragment of the total set of information, pieces of the puzzle need to be brought together to get an overview. This means that their decision to share information is crucial to the modeling effort. Research shows that members of freely interacting groups often do not share essential information, resulting in suboptimal decisions. Interaction in modeling groups is less free in the sense that participants are led through a series of steps designed to elicit and combine relevant information. It seems logical to expect that compared to freely interacting groups, modeling groups exchange more information and come to better decisions. Finally, a fourth wave of studies looks at the interaction between receiving and contributing information. How does the gradual emergence of model structure influence communication between participants? Do participants share information with all others equally, or are participants higher in the hierarchy more likely to send and receive?

This chapter starts by explaining the practice of group model building in more depth. The main part describes the four waves in turn. Ideas for further research are formulated at the end.

2 Group Model Building in Practice

An example may serve to show how group model building is used in practice. In 2012 a company active in the part-time labour market feared that the economic downturn that had started in 2008 would eventually impact their organisation (Bachurina 2012). The strategy of the company in essence came down to bringing together two types of clients: temporary workers looking for a job and company clients looking to fill temporary positions. In a growing economy, temporary workers would visit the company offices in increasing numbers. Companies often could not find new recruits fast enough and therefore hired the part-time labour organisation to find temporary workers. Some managers were worried that while this mechanism increased revenues in a growing economy, it would also lead to increasing losses in a downturn. A group model building project typically starts with a conversation between a contact client and a modeler. The client relates how he or she sees the problematic situation and the desired outcomes of a potential project. If facilitated System Dynamics modeling is found to be a suitable approach, a topic area is chosen and potential participants are invited to a series of sessions. A rule of thumb is to invite participants from all areas of expertise that bear on the topic, in addition to people who have a role in the implementation of conclusions.

In the first session, the aim of the project is introduced to the participants. Participants are invited to narrow down the issue that the modeling project will focus on. In System Dynamics, a problem is expressed in the form of a reference mode: the behavior of a performance measure over time. This reference mode may take the form of a drawing by participants or be constructed on the basis of data from information systems. The left hand side of Fig. 11.1 shows profit as the central reference mode in this case.

Fig. 11.1
figure 1

Reference mode of behavior (left) and causal diagram at end of first session

Full size image

Expressing the central issue of interest in the form of a reference mode of behavior sets the stage for the rest of the modeling effort. In essence, participants are asked to identify how this behavior came about, by jointly building the model structure that is responsible for the problem. In this sense, system dynamicists strongly believe in operational thinking: those factors that are connected to the problem should be identified and related to one another. An example by Richmond (1993) may clarify what is meant by operational thinking. An economic journal published a study on a sophisticated econometric model designed to predict milk production in the United States. The model included a large set of variables linked together in complex equations, but the number of cows was not included in the model. “If one asks how milk is actually generated, one discovers that cows are absolutely essential to the process” (Richmond 1993, p. 128). The focus on operational thinking is different from other (facilitated) operational research modeling approaches that focus on mapping, for instance, ideal systems or personal beliefs on means–ends relations. Models that are created in group model building have a dual identity (Zagonel 2004). On the one hand, they can be seen as tools that align views of stakeholders (the boundary object view). On the other hand, models may be said to represent reality (the micro world view). Which of the two views is emphasised depends, among other things, on the aim of the modeling project.

In group model building, as in other facilitated modeling approaches, the person guiding the group through the steps of modeling remains neutral with regard to content. The facilitator helps the group to articulate their ideas and relate these to each other in a series of steps. Participants are asked to individually note down variables that relate to the issue of interest. These are collected and noted down on a whiteboard or computer screen. Next, the central variable, in this case the company’s profit, is placed in the middle of the board or screen. The facilitator then asks the group members to suggest a variable that impacts the central variable. When one participant suggests a variable and its relation to the central variable, the facilitator notes this down on the screen and then asks the rest of the group if they agree. Other group members may suggest changes and additional variables, but the ground rule is that a relation is drawn only if all participants agree. In this way, a model is incrementally built and the list of unconnected variables grows smaller. The model at any time captures what has been discussed and agreed upon so far. The right hand side of Fig. 11.1 shows a causal loop diagram that emerged over the course of one session.

In follow-up sessions, the model may be expanded until the point where the group has sufficient confidence that the structure that has been built can explain the observed behavior. Analysis of the model concentrates on feedback loops. At the end of a group model building project, a model typically consists of multiple interacting feedback loops. When participants have adequate confidence in the model structure, policies to change the situation in a desired direction can be added to the model and their effects analysed. Projects may stop short of formal modeling when the client’s goal of increased understanding has been reached. Nevertheless, most system dynamicists would agree that formal modeling, even without extensive reference data, will always contribute to the consistency of the model and improve understanding. Formalising the model comes down to expressing each relation in mathematical form and assigning parameter values.

Figure 11.2 shows a stock and flow diagram, which is used to visualise formal models. Formal models may be simulated over time, so that model behavior can be compared to the reference mode. This comparison is one of several validation tests that need to be passed if a model is to be used as a micro world. Still, a formal model can operate as a boundary object. Vennix, one of the founders of group model building, once explained the benefits of a formal model to clients as follows: “What it brings to the process is one additional participant. This participant is rather dumb, as he only knows what you have told him. But he is also very consistent: he can tell you exactly what the consequences of your assumptions are.” In the case of the temporary work organisation, the initial causal loop diagram was translated into a formal model. Data from the internal ERP system was used to populate the model with data. Testing ideas against data revealed several inconsistencies in the participants’ reasoning. Different scenarios of economic growth were simulated and compared to the business-as-usual scenario. Contrary to expectations, the scenarios did not show large differences in number of clients or resulting profits. Consultant visits to prospective client organisations turned out to have a larger impact than initially assumed.

Fig. 11.2
figure 2

Stock and flow diagram on client acquisition

Full size image

This example illustrates both one particular approach to working with participants and some of the core ideas of System Dynamics. System dynamicists assume that feedback loops are important elements of a system’s structure and responsible for its unexpected behavior. As human beings lack the ability to predict how a system consisting of multiple interacting feedback loops will behave, mathematical models are necessary to infer behavior from structure. If the role of mathematical models is emphasised, it may seem a straightforward conclusion that the most important information on messy problems consists of precise, numerical data. We need numbers in order to build a mathematical model. What is far more important, however, is qualitative information on how decisions by actors in the system are made. To a large extent, this information cannot be found in information systems or databases but is part of stakeholders’ mental data. “Searching questions, asked at points throughout the organisation under study by one skilled in knowing what is critical in System Dynamics, can divulge far more useful information than is apt to exist in recorded data” (Forrester 1961, p. 58). In other words, the idea that stakeholders are important sources of information has been around from the start of the System Dynamics field. Another role of stakeholders is in receiving and accepting model results and is closely related to implementation. Roberts (1973) highlighted the importance of choosing a problem that is relevant to a decision maker, otherwise he or she will not bother with the modeling process or the resulting recommendations. Apart from showing how the core assumptions of System Dynamics play out in practice, the example also shows one particular process of involving clients in System Dynamics modeling. A wide range of approaches to working with clients, from generic approaches to quite specific elements of modeling sessions, is reported in the literature. While participation in building System Dynamics models has been around since the start of the field in the 1950s, the term group model building was first used in a paper by Richardson and Andersen in 1995. Group model building now serves as a generic label for at least six distinct facilitated modeling formats, which are described in more depth by Andersen et al. (2007). Recently the focus of discussion has shifted to fine-grained analysis of short pieces of interaction. Andersen and Richardson (1997) introduced the idea of so-called scripts: precise descriptions of a specific phase in a modeling session of 20 min or less. Scripts have an aim, a step-by-step outline of what to say and do with clients and a specified product. By combining scripts, the agenda for a single session or project can be developed. Hovmand and colleagues (2012) have compiled a list of scripts and advice on how to use them, and made all material freely available via Wikibooks.Footnote 1

3 First Wave: Reviews of Assessment Studies

The previous section indicated that although group model building applications have a set of core ideas in common, a wide variety of ways to involve clients may be used in practice. At least six different approaches have emerged, and a facilitator can choose from a list of scripts when designing a session or project. It is not surprising that the first wave of group model building evaluation has focused on bringing together different group model building applications and comparing them with regard to process and outcomes. Two reviews are available: Rouwette et al. (2002) gather group model building studies published up until 1999; Scott et al. (2015) look at studies published between 2001 and 2014.

Studies were included if they described a System Dynamics modeling project involving a client team in at least the stage of conceptualisation, and empirical results on its effectiveness were described. Rouwette et al. find a total of 107 studies, which in the main (84) address organisational problems, and strive for implementation of results. Those for which no implementation is expected are usually training or demonstration sessions, often with student participants. Studies also differ with regard to research design:

  • 88 studies are qualitative case studies gathering data through observation (all 88), individual assessment interviews (six) and group interviews (two);

  • 19 studies use a quantitative estimation of results, through a posttest survey (14) or through questionnaires employed at two points in time (five).

Before addressing the results of the review, four issues are important to address. First, it is likely that studies are biased towards successful interventions. Second, it is important to note that the majority of these studies depend on participants’ self-assessment of results after the intervention. This is problematic, as people are poor judges of both the extent and the causes of learning (Nisbett and Wilson 1977). Only five studies collect data before and after the project. Third, group model building is not a uniform intervention but, as described in the previous section, uses a range of processes and scripts. Each of the applications addresses a particular problem and works with a particular group of participants, and the temporary workers case reported above offers one example. The range of available scripts and ways to design the process are reflected in the cases. About one in four starts from a preliminary model, the others from a blank sheet of paper. A total of 22 studies result in qualitative models; 85 result in a quantitative model of which 56 involve the client in the formalisation phase. About one half of the projects are completed within three months, and two out of three in six months. Fourth, studies look at a range of group model building outcomes, but no single study addresses the full set of outcomes. Given the variety in context and process of modeling interventions, outcomes are remarkably similar. These are some of the key outcomes reported in the review:

  • Communication: measured in 40 studies, of which 39 indicate a positive effect

  • Learning: 96 of 101 indicate a positive effect

  • Consensus: 49 of 53 indicate a positive effect

  • Commitment: 31 of 35 report a positive effect

  • Changes in behavior: 29 of 30 report a positive effect

  • Implementation of results: 42 of 84 report a positive effect

There are few differences in outcomes among types of studies. Qualitative models seem to be less likely to lead to commitment, consensus or system changes than (small or large) quantitative models. The context in which qualitative models are built is different and time investment is lower than for full quantification. Differences between types of modeling interventions may therefore also be due to differences in context or the time participants spent in sessions. On other outcome measures there are no differences. A recent review (Scott et al. 2015) looks at quantitative assessment studies published from 2001 to 2014. A total of 26 studies are found. Where studies in the previous review are to a large extent based on self-assessment of results after the intervention, 15 studies in this review use measurements at two points in time. Results are in line with the previous review, in that Scott and colleagues also find that group model building achieves a range of outcomes such as communication, learning, consensus, behavioral change and implementation. Four studies in the review compare the approach to “normal meetings” and find that group model building is more effective. No studies were found that compare effectiveness of group model building to other modeling interventions.

Several studies that are included in these reviews attempt to explain why outcomes were created. One causal mechanism, formulated at a quite generic level, is the following. Ultimately, the aim of facilitated System Dynamics is to change the problematic situation for the better. In order for system improvement to materialise, someone will have to implement system changes. These may be in line with recommendations from the modeling project or may come down to (conscious or unconscious) changes in individual behavior. Implementation of system changes is more likely if insight into the problem of interest has shifted (or in other words, if learning has occurred). Another influence on implementation may be the group consensus that has developed over the course of the modeling project. Consensus and insight may develop on the basis of the communication process between participants, which is supported by both the model and facilitation (Fig. 11.3).

Fig. 11.3
figure 3

A possible causal mechanism relating the group model building process

Full size image

In the next wave of evaluations several authors zoom in on particular elements of this causal chain, compare elements and relations to existing theories and test to what extent these explain group model building results.

4 Second Wave: Participants as Recipients of Information

The second wave of evaluation studies brings together those contributions that look at how people’s opinions change due to the information they receive in the modeling engagement. Here the focus is still on participant behavior after the project, but an explanation is sought in the information that is exchanged during modeling. Two theories have been proposed. The first centres on the concept of mental models. This is a central concept in System Dynamics, as many in the field assume that lasting improvement in decision making can follow only from a significant change in decision makers’ mental models (e.g. Doyle and Ford 1999; Geurts and Vennix 1989). Doyle and Ford (1999, p. 414) consider a number of different interpretations of the term used in System Dynamics publications and beyond, and ultimately arrive at the following definition: “A mental model of a dynamic system is a relatively enduring and accessible, but limited, internal conceptual representation of an external system (historical, existing or projected) whose structure is analogous to the perceived structure of that system.” Richardson et al. (1994) specify in more detail which elements a mental model contains. They separate mental models into means, ends and means–ends models. Goals are stored in the ends model, while strategies, tactics and policy levers are part of the means model. The means–ends model connects these two and consists of detailed causal relations (design logic) as well as more simple if–then statements (operator logic). In driving a car, design logic refers, for instance, to the inner workings of the engine. An example of operator logic would be that if you brake hard on a wet road, your car is likely to skid. Andersen et al.’s (1994) preliminary conclusion is that providing operator logic is necessary for improving decisions in complex situations. This is surprising, as many system dynamicists would assume that making participants familiar with detailed model structure and its corresponding behavior is the key to increasing insight and changing behavior. In terms of Andersen and colleagues, this constitutes design logic and is not likely to be effective.

A second theory also focuses on how information changes participants’ minds but, in addition, makes the link from changes in insights to changes in behavior explicit. This line of study (Rouwette et al. 2011; Rouwette et al. 2009) looks at the relation between attitudes and behavior and the impact of persuasion on attitude change. The impact of attitudes on behavior is shown in the right hand side of Fig. 11.4 below. In Ajzen’s theory of planned behavior, (Ajzen 1991; Fishbein and Ajzen 2011) intentions are the immediate antecedent of behavior. Intentions are in turn explained by attitude toward behavior, subjective norm and perceived behavioral control.

Fig. 11.4
figure 4

The impact of group model building on persuasion, attitudes and behavior (based on Rouwette 2003, p. 116)

Full size image

Let’s take a manager of the part-time labour company described in the example above as an example. Ajzen’s theory addresses particular behaviors. Imagine the manager is considering hiring more personnel. The theory then assumes that her intention to hire personnel becomes stronger if:

  • attitude toward behavior, or the evaluation of the outcomes of this action, becomes more positive; for instance, when he or she expects more personnel to be able to attract more company clients and eventually lead to more turnover;

  • subjective norm, or the degree to which he or she expects significant others to think he or she should engage in this behavior, grows stronger; for instance, when he or she realises senior management is more positive about hiring than he or she expected;

  • perceived behavioral control, or the evaluation of control over the behavior, increases; for instance, when he or she realises that employees can be hired faster than initially predicted.

Ajzen’s theory is probably one of the most widely used in social psychology and has been tested in a multitude of studies. In addition to its conceptual structure, it also comes with recommendations on empirical testing. An example is Fishbein and Ajzen’s (1975) emphasis on compatibility of measures in order to ensure a substantial correlation. They suggest that general attitudes with respect to organisations, institutions, groups, individuals or ideas are good predictors of general behavioral categories summed over multiple behaviors. In contrast, specific attitudes will be good predictors of specific actions.

Intentions, attitudes, norms and control can be related to the group model building outcomes discussed earlier. Intention is similar to commitment in that both capture the effort a person wants to exert in order to reach a goal. Attitude toward behavior is closely related to the ends model described before. The subjective norm and consensus are similar in their emphasis on the subjective or personal definition of a situation. Perceived behavioral control seems related to the means model mentioned earlier.

The left hand side of Fig. 11.4 shows how modeling and facilitation are related to changes in attitude, norm and control. Theories on persuasion (Chaiken et al. 1996; Petty and Cacioppo 1986; Petty and Wegener 1998) specify two routes through which attitudes can be changed: the central and the peripheral route. The central route consists of understanding and evaluation of arguments. A persuasive message is received; arguments in the message are identified, contrasted with existing knowledge and judged on their validity. Quality of arguments and their persuasiveness have an influence only when taking this first route. Following the peripheral route, evaluations are changed on the basis of simple decision rules, or heuristics. An example of a heuristic is: “if a large number of studies support these conclusions, I accept them as valid”. The decision on which route will be used depends on the person’s motivation and ability to process information. If both motivation and ability are high, the central route will be more effective in changing attitudes. Motivation is high when, for example, the situation is high in personal relevance. Ability to process is high when a person can understand the message, deduce arguments and compare these to her own ideas. Rouwette (2003) assumes that ability to process information is where group model building makes an essential contribution, as it helps participants to integrate and structure available information about a problem.

What evidence has been found that group model building effects actually materialise along these lines? Rouwette (2003) uses the concepts described above to assess the effectiveness of modeling in five applied cases. A total of 29 participants and 86 behavioral options are included in the analysis. In line with expectations, participants perceive a high ability to process information and exchange of arguments. Attitudes and subjective norm change in line with project recommendations; perceived behavioral control does not change. Rouwette et al. (2011) test relations in the model proposed above in seven modeling cases (five from Rouwette’s study and two additional cases), with a total of 42 participants and 124 behavioral options. As expected, participants are motivated and able to process information exchanged in the sessions. Information contained persuasive arguments. Ability to process information, however, impacts only one of the three variables, as expected. A structural equations analysis shows that ability has only a weak relation to attitude and no relation to subjective norm or perceived behavioral control. Both studies conclude that control does not change, and several reasons for this lack of impact come to mind. It may be that participants who before the modeling engagement see only a limited part of the issue, over the course of the project learn about other aspects and come to realise that the problem is even more complex than they initially thought. However, even if this is the case in qualitative projects, one would expect that the simulation of policies helps participants to identify levers for change and therefore increases their sense of control. Both qualitative and quantitative projects may suffer from an emphasis on design logic at the cost of operator logic and therefore may not give participants concrete guidelines to improve their situation (Andersen et al. 1994). With regard to attitudes, Rouwette (2003) does see a change in line with recommendations. But Rouwette et al. (2011) find that attitudes are only weakly related to ability to process and in addition are negatively impacted by argument quality. At first sight this result is difficult to understand: if there are better arguments for a proposed action, participants’ support declines? One explanation may be the compatibility of measurements. Fishbein and Ajzen’s (1975) recommendations on compatibility were followed with regard to all variables in Ajzen’s theory except for communication. The measurement of ability to process is generic, but actions and corresponding intentions, attitudes, norms and control were formulated at a much more specific level. It may thus be the case that some participants felt that communication in general was quite open but that with regard to the particular action they were interested in, they did not hear anything that was both new and relevant.

The second wave of evaluation leaves us with a better understanding of what kind of information is particularly likely to change the opinion of participants in a modeling session. It also specifies the path from opinion to behavior after the modeling intervention. The causal mechanisms have been tested in a limited number of studies, yielding limited support but also pointing to measurement problems and possibly to unexpected impacts of modeling. It is also clear that the causal mechanisms presented here tell only part of the story. In particular, they give us little to go on when trying to pinpoint exactly which piece of modeling output is likely to sway participants. The general idea is that information needs to be relevant and novel to someone if it is to impact his opinion and may be more effective when formulated as operator logic. But in order to be persuasive, information apparently needs to be tailored to the person and even to the particular actions that person is considering. This means that a piece of information may change one person’s opinion but not another’s, or may change one type of behavior but not a slightly different type. Researchers in facilitated modeling may be most interested in a more generic question: in comparison to unsupported decision making, such as a free discussion, why does modeling seem to work better? In terms of the concepts introduced in this section, how does modeling help to identify effective arguments?

5 Third Wave: Participants as Sources of Information

Where the previous wave of studies tried to discover the causes of changes in behavior after sessions, studies in this third group focus squarely on behavior during sessions—in particular communicative behavior. The temporary workers case described at the beginning of this chapter showed how participants over the course of the project jointly construct a model of their situation of interest. The facilitator designs a process, typically with the help of scripts, which invites participants to identify relevant information and share it with others. Information is confronted and combined, and aspects that participants all agree to end up in the model. Participant opinions may also be compared against available data, contributing to further refinement of the model. Since the facilitator is neutral with regard to content and moreover does not have the detailed content knowledge that participants have, relevant variables, relations and loops will have to be suggested by participants. If a piece of information is not mentioned and not revealed by other data later, it will not be part of the model. At any moment during the modeling process, a participant has to decide if her personal expertise and opinion is relevant to the topic that is being discussed and, if so, formulate it in terms of the model. Fig. 11.2 shows a particular part of the temporary workers model: the part related to client acquisition. If the model is to represent client acquisition in a valid manner, participants with information on this topic will need to speak up so that their suggestions can be incorporated into the model. As participants come from different departments or organisations, it is not a given that they immediately see how their personal opinions and expertise are relevant to a particular topic.

This situation is similar to a line of research known as hidden profile studies. Stasser and Titus (1985, 2003) set out to study information sharing in groups. They provided group members with pieces of information, some of them known to one individual only and others known to more group members or to all of them. For instance, let’s imagine there is a group of three people that want to choose between options A and B. There are four pieces of information in favour of option A. This information is shared, meaning that it is known to all three of the members. There are seven pieces of information in favour of option B. Only one of these is shared, and in addition each group member has two pieces that are only known to him or her. This is the unique information. If group members share all of their information, they will realise that there is more information in favour of B than of A (seven against four). However, before the discussion starts, each member has four pieces in favour of A and three in favour of B (one shared and two unique). Initially, he or she will think A is the best alternative. A hidden profile is created when each group member has unique information and the best alternative is hidden from members. They will have to pool their information in order to identify the best alternative. Typically, group members discuss shared information and only a minority of groups (around one in five) choose the best option. These findings have been supported by a series of studies (Stasser and Titus 2003). Some of these studies focused on ways to increase information exchange and prevent groups from falling into the hidden profile trap. Factors such as facilitation, assignment of expert roles, process accountability, a shared task representation, critical thinking norms and counterfactual thinking have been explored (McCardle-Keurentjes et al. 2008).

Many of these factors seem an inherent part of a facilitated modeling process. As a consequence, it does not seem too far-fetched to assume that participants in group model building are more likely than unsupported groups to exchange information and identify the best solution. McCardle-Keurentjes (2015; McCardle-Keurentjes et al. 2008) has tested this assumption in two group levelFootnote 2 and one individualFootnote 3 experiments. Participants in her study construct causal loop diagrams. Two outcomes in particular are relevant here: coverage of information (the extent to which task information is mentioned at least once) and focus of discussion (which part of the discussion focused on a particular type of information, for instance, unique or shared information). As the latter also includes mentioning a particular piece of information more than once, this outcome also fits into the next wave of studies, which addresses interaction between sending and receiving information. Contrary to expectations, model building groups had no better coverage of unique information and neither did they focus more of their discussion on unique information. Modeling groups also did not make decisions of higher quality than unsupported groups. Modeling groups did spend more time on long term information and less time on discussing solutions. The main outcomes expected of the individual experiment are likewise not found. McCardle-Keurentjes suggests several possible reasons for the lack of differences between modeling and unsupported groups. The participants in her controlled experiments were students, with no stake nor substantial experience in the problem to be discussed. The time for discussion and model construction was limited to one hour.

The third wave of evaluation leaves us with somewhat of a puzzle. As McCardle-Keurentjes (2015) notes, testing whether unique information would be exchanged more in group model building than in unsupported meetings would seem to constitute an easy test. However, the intervention failed that test. Part of the explanation may indeed be that in her experiment time was limited (one hour versus a minimum of two times three hours for qualitative modeling in real-life settings). But why facilitated System Dynamics did not contribute to better coverage of unique information, even if in only one hour, is unclear. The next wave again evaluates modeling in applied settings, looking at how contribution and reception of information interact.

6 Fourth Wave: Interaction Between Contributing and Receiving of Information

The description of the second wave of studies ended with the question of how group model building helps to identify arguments. From the third wave no definite conclusion could be drawn: facilitated modeling does not seem to make it more likely that unique information is identified. Which other explanations for the effects of modeling on insight, attitudes and behavior were suggested? Three ideas are put forward in the literature.Footnote 4 Black and Andersen (2012) propose that models can function as boundary objects. De Gooyert (2016) understands the modeling process as the construction of a shared frame of reference. Van Nistelrooij et al. (2012) turn to social exchange theory to better understand the role of power distance in communication.

According to Black and Andersen (2012), the importance of boundary objects follows from their use as a tangible representation of dependencies across disciplinary, organisational, social or cultural lines that can be transformed by all discussion participants. A representation functions as a boundary object if it is a tangible two- or three-dimensional shared object; depicts dependencies among participants’ objectives, expertise, resources and actions; and—importantly—can be changed by all involved. Black and Andersen describe how a boundary object is incrementally built, using examples of modeling groups struggling with conflict. “The visible script products, wielded as boundary objects, provide early and growing evidence that participants are being heard by facilitators and by one another. This evidence builds trust and at least a limited sense of psychological safety […]” (Black and Andersen 2012, p. 203). The first stage of building the boundary object is to generate tangible ideas for the group to consider. In the second stage, group members identify interdependencies between ideas and perspectives, showing consequences of the ideas identified so far. Black and Andersen describe how two groups with opposing points of view managed to work together in listing their ideas and identifying interdependencies, using a computer system that allowed ideas to be represented anonymously. By uncoupling ideas from people, the group managed to build on each other’s contributions. The third stage is a discussion that transforms some of the ideas, by modifying what was gathered so far on the basis of the group’s shared input. The emerging diagram helps to depersonalise conflict and in one case ran directly counter to the ideas of a powerful executive in the meeting, without challenging him directly. Finally, in the fourth stage the group uses the transformed ideas and prioritisation to identify ways forward. In a session with representatives of different agencies, the first three stages had been completed and a shared representation built, to some extent bridging the differences in goals, areas of expertise and actions of participants. When the close of the session drew near, the commissioner who had convened the meeting decided to bypass the shared visual representation and unilaterally proposed a list of eight actions to take the results further. The participants never followed up on the discussion, and the actions were not implemented. Black and Andersen assume that the commissioner’s unilateral proposal took away the opportunity for the participants to transform the shared representation and that therefore the fourth stage, identifying actions together, was never completed. By laying out four stages of information exchange in modeling sessions, each stage building on the former and all four necessary if the group wants to identify joint actions, Black and Andersen (2012) enrich our understanding of how group model building helps to identify arguments. In effect, when information shared by participants is solidified in the form of a visual representation, this establishes a level of trust. Trust in turn allows the group to move on to exchanging another kind of information, in turn enriching the diagram, and so on.

De Gooyert (2016) draws on the framing literature to conceptualise what is going on in modeling sessions (e.g. Kaplan 2008; Snow et al. 1986). He analyses eight sessions with a total of 96 participants. Each session lasted about five hours and brought together 8–15 participants from a range of organisations in the energy sector. On the basis of video recordings and transcriptions of the conversations in the workshops, De Gooyert analyses how participants engage in frame building and frame relating. Frame building comes down to identifying important cues and expressing the meaning attached to these cues, justifying ideas using analogies, metaphors or other sources of authority. Interestingly, listening plays an important role in frame building, as it helps to confirm and amplify suggested frames. As soon as a frame is relatively stable, participants start to connect it to other frames. De Gooyert finds several frame relating strategies: translating, extending, dissecting, appealing and merging, thereby refining the work of Snow et al. (1986). Strategies for frame building and relating explain why some workshops result in more shared cognitions and others fail to achieve convergence.

Van Nistelrooij et al. (2012) offer another perspective on how sending and receiving of information interact. They build on social exchange theory (Lawler et al. 2008; Lawler and Yoon 1998), which looks at how social exchanges take place in a network. For each interacting dyad in the network, the difference in power between the partners in the dyad shapes their exchange relation. A higher power difference will lead to a lower number of exchanges. Successful exchanges will in turn lead both partners in the dyad to attach more positive emotions to the relation. This in turn fosters commitment to the relation and a feeling of cohesion. In a pilot study, Van Nistelrooij et al. compare meetings in a Dutch government organisation with a total of 11 participants. Participants met once in a regular meeting and once in a group model building meeting. The first half hour of each meeting was transcribed, coded by a single coder and analysed with regard to interactions. Power was measured by asking organisation members to indicate the perceived power of each meeting participant. Employees of the focus organisation were presented with a matrix of 16 members of their organisation. People were presented in pairs, and for each pair the employees were asked which one was higher in authority. As expected, in the regular meeting the interaction between partners in a dyad dropped off fast with increasing power distance. In the group model building session, the decline was much less prominent. These results provide some evidence for the idea that in facilitated modeling participants interact on a more equal level than in a meeting as usual. However, the content of the exchanges was not yet analysed, so it remained to be seen how important or relevant the information exchanged in dyads was. Ideally, one would like to see that a participant who is perceived to be in a lower power position reveals crucial information that makes the model more relevant to the problem at stake.

The fourth wave of studies offers three pathways in which contributing and receiving of information interact. Four incremental stages of constructing shared visual representations help participants to build trust and joint understanding. Frame building and relating help to achieve convergence in opinions. There is some indication that facilitation and modeling neutralise the effect of power differences: even partners in a dyad that are very different in power exchange information in modeling sessions, but less so in meetings as usual.

7 Conclusion

In this contribution I reviewed studies on the impact of facilitated System Dynamics modeling, with a particular emphasis on behavior. Behavior has been studied from two perspectives. On the one hand, System Dynamics modeling aims to change a problematic situation for the better, which necessitates implementation of results. Implementation assumes that at least some stakeholders in the situation at hand change their behavior. On the other hand, a facilitated approach also encourages particular behavior of participants in sessions while discouraging other types of behavior. For instance, information sharing and equal participation are supported, high levels of cognitive conflict and politicking are avoided. Early evaluation studies of group model building concentrated on implementation, or behavior after the sessions. To explain (lack of) implementation, researchers and practitioners frequently referred to the interaction between participants, the problem and the model, much of which can be observed during modeling sessions. To check assumptions on effective ingredients, most early studies relied on opinions of participants assessed in interviews or questionnaires after the sessions. Only recently have studies tried to open the black box by capturing and analysing what goes on in model-supported meetings. This contribution described four phases of evaluation of group model building: reviews of assessment studies, the receiver perspective, the sender perspective, and interaction of sending and receiving information. Table 11.1 presents the key topics.

Table 11.1 Main topics in four phases of group model building evaluation and selected references
Full size table

The picture that emerges after describing these four phases of evaluation is more consistent than perhaps expected. Theories and studies, some of them preliminary, seem to build on each other and fill in each other’s blind spots. In broad lines, and with some ideas more supported by evidence than others, the impact of group model building on behavior seems to materialise along the following lines: A group of participants is brought together because of their knowledge, power and/or interest in a dynamic problem. There may be a degree of conflict between participants, but all commit to spending a limited time on trying to better understand the problem. A facilitator guides them through a process of building a model that attempts to explain the problematic behavior over time. Participants share their ideas on the problem, first drawing up a list and then relating ideas. The resulting diagram is modified on the basis of the group discussion, may be compared to available data, and ultimately points to actions that may improve the situation. Each phase that is completed successfully creates trust and lays the groundwork for the next stage. In the process, participants build a joint understanding by constructing and relating frames. Facilitation and modeling help participants, despite their differences in power, to bring relevant information out into the open. Unique information is shared, but not more than in regular meetings. So far, behavior in meetings was discussed. Because participants receive new and relevant information that may lead them to reconsider some of their opinions, behavior outside of sessions is also impacted. Participants change their ideas on desirable ends and about how means and ends relate. This is closely related to changes in attitudes and subjective norm. If the information in the session represents not only design logic but also operator logic, perceptions of means and of behavioral control may also change. Opinions on ends (attitudes), means (perceived control) and means–ends relations converge and create a strong subjective norm. All of these contribute to changed intentions and ultimately changed behavior. Provided that the quality of the model is sufficient, implementation of proposed recommendations will help to change the situation for the better.

There are several spots in which details are missing from this picture. Possibly, on closer inspection, inconsistencies or impossibilities will emerge, as in the works of Escher and Magritte. It is likely to be too much too hope for that facilitated modeling turns out to be a purely democratic process in which the truth is jointly discovered and recommendations are implemented. What sounds more realistic is that group model building helps to counter some biases in human decision making, by exploiting others. This is similar to Schoemaker’s (1993) discovery that the use of multiple scenarios reduces overconfidence by reinforcing the conjunction fallacy.

Several limitations, puzzles and avenues for further research stand out. McCardle-Keurentjes (2015) arrives at the surprising conclusion that students participating in group model building do not exchange more unique information than students participating in meetings as usual. In addition, many of the positive results of the reviews (Rouwette et al. 2002; Scott et al. 2015) follow from participants’ self-assessment of results after the intervention, while we know that people are poor judges of learning (Nisbett and Wilson 1977). De Gooyert (2016) points out that System Dynamics seems to have a blind spot in the sense that it does not address the political dimension of the policy process. Alternative paths through which group model building influences participants’ behavior inside and outside of sessions can be identified. The fact that participants in System Dynamics modeling are asked a descriptive or explanatory question may be important: How are decisions made in this part of the problem? How can we explain the observed data? This is different from asking how future goals may be achieved, or who was involved in/is responsible for the problem or any number of other questions. Another factor may be the level of formality of the models used: formal enough to provide some structure to the conversation, but not so formal as to stifle discussion (Andersen et al. 2007). Finally, a lot can be learned from a comparison between group model building and other facilitated modeling approaches. For instance, Tavella and Franco (2015) also look at micro level interactions between participants, and between participants and the model. Franco, Rouwette and Korzilius (2015) use interaction analysis to understand how consensus develops in modeling groups.

An earlier study (Rouwette and Vennix 2006) concluded by saying that the most promising path forward was to determine the “differences that matter”, between problems, between client groups and between modeling interventions. Ten years later there is more clarity on possible causal paths, starting from behavior in modeling sessions, via opinions and attitudes of participants, to behavior in and effects on the problem of interest. Maybe, in addition to conducting more fine-grained empirical studies, we also need further development in terms of conceptual understanding. Perhaps it is time to turn our sketch of causal mechanisms into a simulation model and to test its dynamic implications.