Data Analysis of Social Simulations Outputs - Interpreting the Dispersion of Variables

Abstract

In the domain of social simulation, there are very few papers reporting on the statistical analysis of simulation results, while it is very common in empirical social sciences. The paper advocates the recourse to the statistical analysis of social simulation outputs, as a very efficient way to improve the interpretation of simulation results and so the understanding of the system that is the model’s target. This is illustrated by the study of a simulation model designed to analyze a real case regarding the management of a river in South West of France. Several standard statistics methods are used to shed light on the possible outcomes of the debate between the stakeholders.

1 Introduction

The standard way to build a social simulation model may be sketched as this: you are concerned by or interested in a phenomenon that occurs in some system of reference which, according to the classification of [5], may be either a particular “empirical space-time circumscribed” case, the application of a “theoretical construct intended to investigate some properties that apply to a wide range of empirical phenomena” or “focus[es] on general social phenomena”, or stylized fact. This phenomenon of interest is characterized by indexes, which allow to measure various features of each occurrence of the phenomenon and whose value is either directly measured, given by an expert, collected in any way or resulting of a treatment of these. Then, the simulation model is built to produce outputs, also issued directly or resulting from additional treatments, which values are as close as possible to the indexes when it is run in the appropriate conditions.

To validate the model and to ensure that the mechanisms it embeds are able to reproduce the phenomenon of interest, the model building process includes the use of statistics to identify the model’s parameters whose initial values influence the model’s surface response (sensitivity analysis) and to give to each parameter the most suitable value (calibration). The simulation model is run many times, and optimization techniques are used to calibrate the parameters in such a way that the (mainly univariate) statistical properties of each output variable (i.e., mean, standard deviation or distribution, auto-correlation if the variable is distributed in space or time, ...) are as close as possible, (i.e., with a reasonable confidence interval) to the properties of the corresponding index in the system of reference. The same holds when statistics is used for the replication of a model as e.g., [15].

In simulation models designed for the study of systems such as the customer waiting times of a queuing system, bottleneck in a mass transit system during rush hour, the throughput of a production workshop or the mean time to failure of a machine, it is very common to perform statistical analyses for operation research regarding the system performances (duration of the start-up phase, steady-state analysis, confidence intervals, ...). In these engineering cases, the model is analyzed to know its own internal control logic, not in order to improve its mirroring (or whatever relationship between the model and its target) of a system of reference; see [11] or [1] or [10] as instances of many papers on this topic at the Winter Simulation Conference.

Many authors advocate the systematic analysis of simulation models in accordance with [3] and [9]. The need for engineering principles and tools to improve the practice of Multi-Agent Based Simulation and enhance the knowledge obtained in this way is increasingly recognized; see for instance [12, 13] regarding the Design of Experiments. In this line, we claim that this kind of statistical analysis is beneficial for the study of social simulation models as well. While the system of reference (SR) is most often observed only once, since experimentation in social affairs are rarely feasible, the simulation model is run many times (at least thirty times or more) and thus we can proceed to a statistical analysis of the data produced by these runs as if we had thirty exemplars of the system of reference. In fact, a statistical analysis is very common in empirical social sciences for the analysis of questionnaire surveys, data collected from archival records or dataset pick up in whatever way.

Then, the question is no longer to compare the unique value of each index observed in the SR with the mean value of the corresponding simulation output; it is to consider each run as an observation of a SR’s numeric analog and to proceed to a statistical analysis of this dataset. Beyond the matching between the SR indexes and the simulation outcomes, the purpose of the analysis is to uncover structural or behavioral patterns that could be buried into the dataset [8]. This knowledge comes from the operating mode of the simulation model under various circumstances (instantiated by the series of random values for each run) and thus it bears on its deep characteristics, features which can not be identified in the course of a single observation.

Then, three cases can occur:

• the feature is already identified and well known by the experts of the SR, so that the data analysis brings a new piece to the validation of the model;

• the feature is in contradiction with what the experts know about the SR, so that the data analysis questions, more or less severely, the validity of the model;

• the feature is not a fact known by the experts of the SR but it is consistent with their actual understanding, so that the data analysis is likely to bring a new piece of knowledge about the SR, to the extent that the relationships between the SR and the model are well-defined.

We guess there is no uniform way to conduct the statistical analysis of any simulation model results, because the methods likely to provide interesting findings depend on the very nature of the SR and the hypotheses that the model is aiming to test. Moreover, statistics is a quickly growing discipline and new analysis techniques are continuously developed to tackle specific questions.

So, in the following of this paper, we will just illustrate how statistical analysis techniques can be exploited to enhance the knowledge about social simulation models, and in this way the knowledge about the system of references under consideration. The model and the simulation outputs have been produced using a social simulation platform, SocLab¹, designed for analyzing power and collaboration relationships within social organizations. This platform allows the user to edit models of organizations, to study the properties of models with analytic tools, and to compute by simulation the behaviors that the members of the organization could adopt each other.

The remaining of the paper is as follows. We first present the SocLab modeling framework, the main features of an organization it allows to consider and the questions it intends to address. Section 3 presents our real-world case study related to water management in a French area and the simulation outputs. The following sections show how quite simple statistic analyses bring answers to a number of questions about social features of the system. A discussion and a conclusion are finally provided in Sect. 7.

2 The SocLab Modelling of Social Organizations

The SocLab framework formalizes and slightly extends the Sociology of Organized Action (SOA), introduced by [6]. For space limitation we just outline the syntax and semantics of SocLab models, a comprehensive presentation of this framework and its use may be found in [17].

Roughly speaking, SOA proposes to explain why people behave as they do, especially when they do not behave as they are supposed to, with regard to the organization’s rules. An organization is defined as a set of actors and a set of relationships based on the access to resources. Each actor has some goals, which are a mix of his own objectives and his organizational roles, and he needs some resources to reach these goals. On the other hand, each actor controls the access to some resources, and so determines to what extent those needing these resources have the means to achieve their goals. Actors are reciprocally dependent on each other.

Actors are assumed to be rational, that is to say their behavior is driven by their beliefs on the best way to achieve their goals. So, each actor cooperates with others in the management of resources under its control to get from them access to the resources it needs. We call the process, by which they mutually adjust their behaviors with respect to others, the social actor game. The well-known regulation phenomenon results from this process: the adjustments drive actors to exhibit quite steady behaviors as if they obey to external rules. So, SocLab proposes to shed light to the regulation of organizations, how and why they occur, with what shape.

Figure 1 shows the SocLab metamodel of the structure of organizations. Accordingly, the model of an organization includes:

• the list of the actors;

• the list of the resources: each resource is controlled (or managed) by one actor². This actor behaves in a more or less cooperative way and the state of a resource measures (on a scale of $-10$ to 10) how much he tends (or not) to cooperate with others by favoring (or hindering) accesses to the resource;

• the stake of every actor on every resource: this quantity measures the importance of the resource for the actor; a not null stake means that the actor actually depends on the resource. The more a resource is needed to achieve an actor’s important goal, the higher the corresponding stake (on a scale of zero to ten; the sum of the stakes for every actor sums to ten);

• the effect function of a resource on an actor who has a not null stake on this resource: this function quantifies how well the actor can use the resource to achieve his goals, depending on the state of the resource;

• the solidarities of every actor towards each of the others.

Fig. 1.

The SocLab metamodel of organizations as a UML class diagram

A configuration (or state) of the organization is defined as the vector of the resource states. Thus, a configuration is characterized by the level of cooperation of each actor with regard to others. In any configuration of an organization, every actor gets from others some capacity to access the resources it needs to achieve his objectives. This capability of an actor $a$ when the organization is in a configuration $s=(s_r)_{r\in R}$ is calculated as the sum on the resources of the values of effect functions weighted by the actor’s stakes:

$$\begin{aligned} \mathrm{capability }(a, s) = \sum _{r\in R} \left( \mathrm{stake }(a,r) \times \mathrm{effect }_r(a,s_r)\right) . \end{aligned}$$

This raw capability is weighted by the solidarities between actors, so that the important output we will consider is the satisfaction of actors, where:

$$\begin{aligned} \mathrm{satisfaction }(a, s) = \sum _{b\in A} \left[ \mathrm{solidarity }(a, b) \times \sum _{r\in R} \left( \mathrm{stake }(b,r) \times \mathrm{effect }_r(b,s_r)\right) \right] . \end{aligned}$$

(1)

To compute the configurations that are likely to be issued by the regulation process within an organization, the SocLab platform includes a simulation engine which implements the social game and so computes which behavior each actor is likely to adopt [7, 16]. To this end, a multi-agent implementation of the model of an organization provides the actors with rationality for playing the social actor game. Social actor agents try, as a meta-goal, to get a high level of satisfaction, i.e., to have the means needed to achieve their concrete goals. However, according to the bounded rationality assumption [19], they just look for a “satisficing” level of satisfaction, not an illusory optimal one. So, within a trial-error reinforcement learning process, each actor maintains a dynamic level of aspiration, and a simulation terminates when a stationary state is reached because every actor has a satisfaction that is over his level of aspiration. A state is stationary if actors manage the resources they control in such a way that every one accepts his level of satisfaction: then, the organization can work in this way, a regulated configuration has been found. The length of a simulation, i.e., the number of steps necessary to reach a stationary state, indicates how difficult it is for the actors to jointly find how to cooperate.

3 The Management of the Touch River

The simulation model to which we will apply statistical tools concerns the management of a river called Touch. A detailed presentation of the case is given in [17, 18], including the empirical and theoretical dimensions of the system of reference and a detailed presentation of the SocLab model.

Touch is a tributary of the Garonne in which it flows downstream of Toulouse, an agglomeration of one million inhabitants in the South West of France. Its catchment area covers 60 municipalities and its course crosses 29 municipalities. Three fourth of these municipalities stand upstream and are mainly agricultural villages or small towns. Other municipalities, located downstream, form a dense urban area of the Toulouse agglomeration. Downstream municipalities have had to deal with several episodes of flooding during the past decades. They have tried to protect themselves by building dikes that, even if expensive, are not sufficient to eliminate the flooding risks, and they consider that upstream municipalities do not cooperate enough. On the contrary, upstream municipalities, strongly influenced by farmers, consider that they have done their best for preventing flooding by letting some land lying uncultivated in order to absorb the excess of water in case of flooding.

Since 1995, the French water policy requires the elaboration of a flood risk prevention plan (FRPP) for each river, and this obligation was reinforced by the European Water Framework Directive (WFD 2000/60/EC), transposed into the French law as the Law on Water and Aquatic Ecosystems (LEMA, Law of 30 December 2006). On the occasion of the establishment of the PRPP of the Touch, B. Baldet [4] studied the difficulties to reach an agreement that combines the views of all the field stakeholders and administrative authorities. He analyzed the field observations to the light of several sociological theories. The SocLab model, whose simulation results are analyzed in this paper, describes the system of organized action devoted to the elaboration of a new Touch’s FRPP. It has been designed in order to formally confirm (or infirm) the empirical findings of the sociological study and the possibility of hypotheses about the future management of the Touch.

The simulation model includes 10 actors which are involved in the management of the river and so are interested in the definition and the application of the FRPP. In this model, each actor controls a single resource that synthesizes its means of actions:

• actor 1: Departmental Territory Direction (DDT) acts as the State representative and will instruct the new FRPP; it controls the Validation resource;

• actor 2: National Office for Water and Aquatic Ecosystem (ONEMA) is the reference agency for the monitoring of water and aquatic environment; it controls the Expertise resource;

• actor 3: Adour-Garonne Water Agency (AEAG) is the operational authority in charge of strategic plans at the basin level. Accounting for the requirements of the various water uses and of the protection of aquatic ecosystems, it defines, supervises and funds the water policy; it controls the Funding resource;

• actor 4: a citizen organization of riparian farmers in the upstream area. They own floodplain lands and, as riparian, they have the right to use the river and must maintain the banks; it controls the Lobbying resource;

• actor 5: the group of 25 upstream municipalities that have 21,000 inhabitants; it controls the Control of flow resource;

• actor 6: the group of 4 downstream municipalities (75,000 inhabitants) that are incriminated at each occurrence of a natural catastrophe. Due to flooding threats, they must prohibit any building on a portion of their territory; it controls the Self funding resource;

• actor 7: the inter-communal association for water civil engineering (SIAH), in charge of the management of Touch³. It has to maintain the river bed and banks and is funded by actor 3. It includes representatives of the 29 riparian municipalities and is managed by an active technician who favors the cooperation among municipalities while worrying about the Good Ecological Status of the river; it controls the River management resource;

• actors 8 and 9: political authorities, the regional and departmental councils, respectively. They can bring additional financial support to civil engineering measures; each one controls an Additional funding resource;

• actor 10: an engineering consulting firm, specialized in water, energy and environment, in charge of technical studies. It controls the Studies resource;

The actors who are the most engaged in the negotiation are actors 6, 4 and 5 from the population point of view, and actors 7, 3 and 9 from the institutional point of view. All these actors are strongly concerned by both the elaboration and the further implementation of the FRPP. Actors 1, 2, 8 and 10 are less concerned. In this model, each actor controls a single resource that summarizes its means to influence the discussion. We skip a technical presentation of the SocLab model resulting from the sociological analysis - it is useless for the following of this paper - and all details may be found in [18].

The analysis of the debates, notably within the SIAH, shows three main options for the Touch management, each one supported by committed actors:

Upstream and downstream municipalities are interdependent, though their respective interests are different or even conflicting. So the elaboration of the FRPP includes a fourth option which is probably the main issue of the discussions:

The SocLab platform provides tools for the analytical investigation of (the model of) organizations. For instance, it computes indexes about structural or state-dependent properties of an organization and allows to interactively explore the space of the configurations by computing, e.g., the configurations which optimize or minimize the satisfaction of a given actor or the Nash equilibria. These analytical results frame the context where the regulation process takes place. They contribute to the interpretation of simulation results by placing what actually happens in the range of what could happen.

The dataset we examine contains the outputs of 100 simulation experiments with the same initial values: the resources are put in the neutral state (i.e., the value 0, and runs with different values give same results). Thus, experiments vary just by the seed of the random numbers generator. The output variables are the number of steps to reach convergence (a stationary configuration), the state of the 10 resources and the resulting satisfaction of actors at the end of the simulation. The higher the state of a relation, the more cooperative the controlling actor is. The level of cooperation of a resource is evaluated as the total satisfaction it provides, accounting for the fact that most relations are conflictive: most states provide a positive satisfaction to some of the (dependent) actors and a negative satisfaction to others.

The satisfaction of each actor, i.e., its capability to reach its goals, is determined according to Eq. (1) by its solidarities, the state of the resources he depends on, its stakes and the effect functions of the resources it depends on. A quick sensitivity analysis of these parameters (without checking interactions) shows that the model’s response is not sensible to a variation of 15 % of their values. Each actors put 3 or 4 stake points on the relation it controls so that its satisfaction depends on about one third of its own behavior. The range of values of actors’ satisfactions are quite dispersed, from 90 (actor 2) to 195 (actor 6). The lower bounds (the worst configuration for each of them) range from $-25$ (actor 2) to $-85$ (actor 6) and the upper bounds (the best configuration) from 60 (actor 8) to 110 (actor 6). The dataset may be found in [20] together with some other results⁴.

4 Univariate Statistical Analysis

A quick overview of the variables distributions is provided in Fig. 2. The number of steps is strongly skewed with a small number of simulations having a very large number of steps; so, the mean or median are not a good summary of the distribution.

Most resource states (except for “Self funding”, “River management” and “Additional funding”) also have a skewed distribution, with several outliers having small values. The scattering of the state variables is very varied: some variables have a very small dispersion, like “Validation” (which is frequently equal to 10, its maximum possible value) or “Additional funding 2” which is almost always equal to 6 (also its upper bound value). For these resources, as well as for “Lobbying”, “Control of flow” and “River management” (but to a lesser extent), the management of these resources seem almost fixed in advance by organizational constraints that strongly determine the behavior of the controller actors. On the contrary, “Expertise”, or even “Self funding” and “Additional funding” are resources that have a larger dispersion (for “Expertise” the values spread on the whole range from $-8$ to 8): the actors which control these resources are less constrained by the organization and a deeper analysis is necessary to decide whether they hesitate between quite equivalent attitudes or whether they have enough power to strategically adapt their behavior to the context.

Fig. 2.

Boxplots for the number of steps before the simulations converge (left), the resource states (middle) and the actor satisfactions (right, grey boxplots indicate a median satisfaction below 50).

The satisfactions of actors are given in value, but they could be considered in proportion to account for the disparity of their respective range of values. Most of them are approximately symmetric, but with a small variability regarding their range. Actors 3, 8 and 9 are the most satisfied in almost all simulations (actor 8 appears the most satisfied, but it is less committed in the game), while actors 4, 6 and 10 are frequently poorly satisfied. As actor 6 has a low satisfaction, the option (O1) will probably not prevail. The same holds for actors 4 and 5 and the option (O2), but to a lesser extent. As the satisfaction of actors 2 and 3 is slightly better, the option (O3) seems to be the most likely. As the satisfaction of actor 7 is medium, it seems that a compromise that would be acceptable by most actors is possible (O4), and this is compliant with the fact that none of the options (O1), (O2) or (O3) strongly prevails upon the others.

The dispersion of the actors’ satisfactions shows that the position of actors 4, 9, 6 and 3 are well settled, while the positions of actors 5 and 7 are more precarious. Regarding the respective range of values, the actor satisfactions are globally more steady (with smaller dispersions) than the resource states: the variation coefficients of actor satisfactions have a range of 0.02 to 0.06, whereas those of resource states have a range of 0.06 to 0.98 (except for “Additional Funding 2”). This fact might be interpreted as a form of fairness among actors ensured by a complex system effect: actors compensate a decrease of accessibility to a needed resource by a better access to other ones.

5 Correlation Analysis

Figure 3 shows a graphical representation of the correlation coefficients between all pairs of variables. The number of steps has a slight negative influence on all actors (and resources), except for upstream actors 4 and 5. This is a general and meaningful property of the simulation algorithm: long simulations indicate that actors struggle to find a configuration that provides each of them with an acceptable level of satisfaction, and this difficulty to cooperate entails lower levels of satisfaction for most actors. Here, Actors 4 and 5 are the beneficiaries of delay in convergence, and their conflict with the remaining of the organization will be confirmed by further analysis in the following section.

Fig. 3.

Graphical representation of the correlation coefficients between pairs of variables: the thinner the ellipse, the larger the absolute value of the correlation coefficient (see [14], implemented in the R package ). The grey level matches the absolute value of the correlation coefficient (darker colors are used for larger values).

The correlations between the actors’ satisfactions show two groups of strongly related actors: actors 1, 2 and 3; actors 8 and 9. Actors 1, 2 and 3 are organizations that represent the State and carry out public policies. The positive correlation between their satisfactions means that their main interests are consistent and that these three domains of the State policy strengthen one another. Moreover, actor 7, instituted by actor 1 and funded by actor 3, is shown to be in accordance with the State services. Actors 8 and 9 are political institutions and it is satisfactory that they have similar interests on topics such as the river management, despite their political divergence. Moreover, the correlation between the two groups is positive: there is no conflict between the State representatives and the local authorities.

As for actors 4, 5 and 6, the most concerned with the river functioning, they have to be regarded in conjunction with actor 7, which is the place where they can build a compromise together. Actor 5 seems careful; surprisingly, it does not support the farmer association nor is it in conflict with downstream municipalities. The case of actor 4 requires a specific attention: it is in conflict with actors 6 and 7, and also with most of other actors. It is responsible for long-lasting simulations but we will see that it is not powerful enough to make its interests to prevail (this is because the effect functions of the relation it controls have a small amplitude). The satisfaction of actor 7 is positively correlated with those of actors 5 and 6, also in addition to those of the state representatives group: these actors support the options (O1), (O2) or (O3). This fact confirms the possibility of a compromise (O4), which has already been pointed out in the analysis of the actor satisfactions.

Regarding the correlation between actors and resources, let us recall that, through the effect of solidarities (see Eq. (1)), each actor depends more or less on most resources. Table 1 shows the global influence of each resource on actors. Some actor satisfactions are strongly correlated with the state of a specific resource: “Expertise” conditions the satisfaction of Actor 1, in accordance with its strong concern with ecological issues that are important in the management of this resource. The satisfaction of Actors 8 and 9 is strongly correlated with “Funding” because a higher financial engagement from Actor 3 means a lesser need for their financial effort; moreover, we have seen that the commitment of Actor 3 on ecological issues meets the concerns of Actors 8 and 9. “Control of flow” is the most influential resource on actors, in value and in proportion regarding its small dispersion (see Fig. 1): it is positively correlated with the satisfaction of Actor 4 and strongly negatively correlated with the satisfaction of all other actors, except Actor 5: a low level of this resource means a stronger control on the river and thus a higher decision power for Actors 2, 3, 6 and 7. “Self funding” is strongly negatively correlated with the satisfactions of Actors 5 and 7: a high level for this resource means a higher decision power for Actor 6 which reduces the decision power of actors 5 and 7. Finally, “Lobbying” is not very influential on the actors’ satisfaction and thus, while actor 4 is in conflict with others, it does not have the means to make its point of view to prevail.

These results show that, despite its analytical structure, the behavior-selecting processes of actors make the model strongly non-linear: the column “Influence on the controlling actor” of Table 1 shows that the satisfaction of most actors is not so much correlated with the resource it controls and, as expected, actors compensate losses due to their concessions by a better access to others’ resources. As an other conclusion, the correlation between the “Absolute Influence” and “Relevance” columns is 0.47: if the control of an highly relevant resource is, of course an advantage for an actor, that does not ensure to him with a high influence; to this end the actor must also properly use this resource and their capability⁵ to do that varies from 0.06 (actor 4) to 0.41 (actor 2). There is no remarkable correlation between any pair of relations. Despite the significant correlations between their satisfactions, actors behave independently one another. There is no coordination or coalition within a subgroup of actors, no actor seems to influence the behavior of another one; in other words, each actor is autonomous with regard to others. A Principal Component Analysis of the resources’ states confirms this fact since the first two components explain just 29.7 percent of the variance: no relation plays a preponderant role.

Table 1. The influences of each relation as: the (absolute in case of the “Absolute Influence”) sum of the values of its correlations with actors’ satisfactions); its correlation with the satisfaction of its controlling actor; its relevance (as the sum of the stakes actors put on it).

Regarding actors, the first two components of the Principal Component Analysis explain 69 percent of the variance and confirm the analysis of pairwise correlations between actors’ satisfactions (see Fig. 4). Actors 10 and 5 are not very influential; actor 4 is in conflict with all others, especially 5, 6 and 7; actors 8 and 9 are on their side; actors 1, 2 and 3 also go together. Finally, the position of actor 7 is very noticeable, since it is the key actor for the option (O4): being very close to actors 5 and 6 and not so far from actor 3, it seems to have the means to promote this option.

Fig. 4.

The position of each actor shows its relative contribution to the variance of actors’ satisfactions. Actor 4 is in clear opposition with all others, mainly Actors 5, 6, 7 and 3.

6 The Modes of Simulations’ Outputs

According to SOA, simulations may include runs whose outputs are quite far from the actual observations of the system of reference: they correspond to “potentialities”, possible ways of operating of the organization, to configurations which do not actually occur but might be observed in the future. Indeed, the regulated behavior exhibited by a social system is the result of past events, occasional opportunities or constraints, random choices made at bifurcation points or whatever contingent circumstance while, to the extent of its adaptability and latent variety [2], the organization could as well operate in another way under other circumstances. A tight matching between simulation outputs and the indexes of the organization is just interpreted as a structural property of the organization: a strong regulation which prevents actors to depart from a normative behavior. When simulation results are dispersed, the simulation algorithm can exhibit runs which are either widely scattered corresponding to an organization that is little regulated or are clustered corresponding to alternative functionings and possible futures. Regarding our case study, it is of first importance to know whether runs are uniformly distributed or whether there are modes in correspondence with the possible options (O1) to (O4). In the latter case, their respective frequencies may serve as a forecast of the possible issues in the debate.

This requires that the analysis does no longer focus on the variables but on the vector provided by each simulation run as a whole. To this end, we use the hierarchical clustering method which seeks to build clusters containing similar simulations. Pairwise distance between simulations are computed using the Euclidean distance of the vectors of scaled variables (i.e., variables are centered and reduced to unit variance) containing all satisfactions, all resources and the number of steps (so that observations in the same cluster are alike for all these values) and a bottom up approach is used to hierarchically aggregate the simulations into clusters in a greedy way. A linkage criterion specifies the dissimilarity between clusters: we used the Ward’s method which minimizes the total within-cluster variance at each step of the hierarchy. The number of clusters is chosen classically by cutting the hierarchical tree at the smallest height that corresponds to a large increase in within-cluster variance. The result of this analysis is shown in Fig. 5 where four clusters can be identified.

Fig. 5.

Hierarchical clustering. Experiments are partitioned into 4 clusters (numbered from 1 (at left) to 4 (at right)) as represented by the colored rectangles. An outlier (simulation number 16) can also be identified (at the right hand side of the figure).

Then, for each cluster, the mean value of the actors and resources scaled variables can be calculated for this class, as shown in Fig. 6. For instance, regarding the resource plot, the state of the “River Management” resource decreases from cluster 2 to cluster 3. Using the same method as for the constitution of the 4 clusters, the resources are compared regarding their values in the clusters, resulting in the dendrogram at the top of the panel. According to this super-classification, the “Studies” and “Validation” resources are very close; the same holds for the “River Management” and “Add. Funding 2” resources, which are also close to “Add. Funding 1”. The smallest the length of the dendrogram path between two resources, the most similar values these resources have among the four clusters. Similarly, the dendrogram at the left side of each plot shows the similarity between the clusters regarding the values of the resources or the actors. This display of the clustering should be completed by boxplots gathered either by clusters or by elements.

Fig. 6.

Mean scaled satisfactions of actors (left) and states of resources (right) in the 4 clusters (darker colors correspond to higher values). As the values are scaled values by variable (actors, resources), the colors should be compared by columns, not by rows.

From these figures, the following conclusions can be made:

• It appears that the upstream municipalities are rather in opposition with the other actors of the organization since actor 4 is the most far from other actors and the “Control of flow” resource (controlled by actor 5) is the most far from other resources.

• The hierarchical clustering identifies four clusters that can be related to the four options emerging from the empirical analysis.

• Cluster 2 corresponds to reaching the best compromise in the process of elaborating the new public policy of Touch (O4); simulations are shorter (857 steps instead of about 1300 for other clusters) and the average satisfaction of actors is higher (56 instead of about 52.7). Unfortunately, it is not the most likely outcome, since the cluster includes only 7 % of simulations, but it is a possible outcome.

• Cluster 1 (containing 20 simulations) contains simulations that are almost the opposite of cluster 2: in this cluster, all actors except actor 4 have a lower satisfaction than in other cases (51.6). In these simulations, the state of “Control of flow” is high and the state of “Funding” and “Expertise” is low. These simulations correspond to the success of option (O2) over the other options: actors 4 and 5 succeed in making their interest prevails over other actors’ interests.

• Clusters 3 and 4 (respectively, 42 and 31 simulations) are the closest clusters (see Fig. 5 and the left side dendrogram of Fig. 6) with mostly average values, where most actor satisfactions and resource states take an intermediate value between those of clusters 1 and 2. These clusters gather 75 % of the simulations and thus correspond to the most likely outcome of the negotiation process.

• Cluster 3 is characterized by a stable low satisfaction for actors 5 and 7 and by a high state for “Self funding”. These simulations are rather in favor of option (O1). In cluster 4, actors 4 and 5 are more satisfied than in the other clusters and the state of “Self funding” and “Studies” is low. These simulations are rather in favor of option (O2).

7 Conclusion and Discussion

The paper has given an example of the benefice of the statistical analysis of simulation outputs distributions to reveal causality patterns and improve the understanding the operating mode of the system of reference. Mostly, the relevance of the model of the considered organizational setting has been confirmed as well as the likelihood of the behavior-selecting learning process of actors. In addition, some unexpected, but consistent facts have been revealed such as the singularity of upstream municipalities with regard to the whole organization.

When a simulation model is an abstraction of a phenomenon modeled as a “stylized fact”, the purpose is to propose a mechanism, as simple as possible, able to generate the phenomenon as an emergence from the interactions between the system’s components. In this case, the study of the simulation outputs aims mainly to verify whether outputs are steadily focused with a small standard deviation, since their dispersion means that the proposed mechanism is not a good explanation for the phenomenon.

When the simulation model refers to a concrete system as the case considered in this paper, a “good fit” between the system of reference indexes and the simulations outputs must also be checked first to confirm the relevance of the model. In this case, the model includes a wider heterogeneity of agents, with a variety of individual features and goals, and so a larger number of properties are to be investigated. Then, further data analyses of simulation outputs can bring new knowledge and a much deeper understanding of structural and behavioral properties of the model. These model’s properties may then be interpreted in terms of properties of the system of reference to the extent the matching between the elements of the model and the system of reference is well defined. The nature of the properties of interest straight depends on the questions that motivate the elaboration of the model. Thus, there is no method that could be applied in a systematic way and, among the huge number of tools and derived data that could be calculated, it is to the designer of the model to find the ones allowing to shed light on the question he considers.