Keywords

1 Extended Abstract

We propose a novel approach to the statistical analysis of economic agent-based models (ABMs). The analysis of ABMs is often constrained by problems of (i) computational time, (ii) correct construction of confidence bands, (iii) detection of model ergodicity, and (iv) identification of transient behaviour. All these issues are pivotal to the validity of a model, both when it is used for thought-experiments, and when it aims at delivering policy insights. Nevertheless, they are often overlooked [17] or solved informally without a commonly shared procedure [10].

We propose fast, easy-to-use, automated, and statistically rigorous procedures to address all these problems. We implement such procedures in MultiVeStA, a model-agnostic statistical analyser which can be easily tool-chained with existing ABMs. Independently from the nature of the ABM at hand, the analyser performs simulations, distributing them in the cores of a machine or a network, computes statistical estimators, and implements the minimum number of simulations necessary to satisfy given conditions on confidence intervals.

The above-mentioned problems are not specific to the ABM context; they affect most simulation-based analysis approaches and were therefore tackled by many scientific communities in the past. In computer science, several automated procedures have been proposed to mitigate these problems. An example is the family of techniques known as statistical model checking (SMC) [1, 18]. Roughly speaking, SMC can be seen as an automated Monte Carlo analysis guided by a property of interest given in an external property specification language. Here we focus on the statistical model checker MultiVeStA [11, 16, 20]. While previous versions of MultiVeStA have been successfully applied in a wide range of domains including, e.g., threat analysis models [4], highly-configurable systems [2, 3, 19], public transportation systems [9, 11, 12], robotic scenarios with planning capabilities [5, 6], and crowd steering scenarios [15], it has never been employed for the analysis of ABMs. Here, we have redesigned and extended MultiVeStA to target analyses of interest for the ABM community (e.g. [13]). For example, we integrated a series of tests that allow for (i) counterfactual analysis, (ii) detection of ergodicity, and (iii) estimation of the transient period.

We demonstrate our approach in [20] using two ABMs from the literature. The first is a macro stock-flow consistent ABM from [8]. We first replicate the results from the original contribution, scaling the runtime analysis from 15 days to 15 h thanks to the automated parallelization of simulations. We also show how the statistical reliability of our approach allows us to perform meaningful counterfactual analysis.

The second ABM is a simple financial market model from [14]. This model has analytical solutions [7] which we use to assess the effectiveness of our approach. Contrarily to computational analyses reported in prior literature [14], which were biased by erroneous under-estimations of the transient period duration and of the process autocorrelation, we match the correct analytical results of the model.

In the near future, we plan to extend the number of tasks performed in an automated and user-friendly manner by our tool, e.g., including the identification of multiple stationary points. We also plan to use our tool for the analysis of other classical and novel ABMs.

The tool, models, and more information are available at: github.com/andrea-vandin/MultiVeStA/wiki.