1 Introduction

This paper presents evidence that the distribution of stock returns of the US stock market, proxied by the S&P 500 index—an index that measures the stock performance of 500 publicly listed US companies—is statistically indistinguishable from a mixed Gaussian distribution. Following this, a new computational agent-based model is developed that can produce such probabilistic features and that can replicate the complexity and entropy properties of US stock returns. It has been established that stock prices resemble a nonlinear time-series and financial markets are complex and dynamic, see Xiao and Wang [35] who developed a novel financial price model based on a well-known particle system in statistical physics, the contact system, and showed they can replicate the complex and entropy properties of stock returns. Their work, however, does not aim to develop a model that can replicate the nonlinear/complex behaviours of stock prices and their probabilistic features (mixed Gaussian returns). This is the principal void we attempt to fill.

There are numerous surveys of computerised agent-based models of financial markets, see LeBaron [19], Hommes [18], Chiarella et al. [9], Hommes and Wagener [17], Lux [22], Fagiolo and Roventini [12] for example. Some of the key models in this literature are discussed below. Farmer et al. [13] presented a minimally intelligent agent model with a unique method of stock price determination, namely a continuous double auction. They modelled the limit order book directly where agents post bids and offers randomly within some constrained set. They showed that their model can explain a sizable portion of the variance between the best buying and selling prices in a cross section of almost a dozen stocks. Furthermore, Raberto et al. [29] presented a random order generation single stock model similar to Farmer et al. [13] in the sense that the model assumes that agents randomly choose whether to buy or sell. While such assumptions may seem at odds with the perception that a significant portion of stock market trading volume arises from the actions of large institutional investors, each of whom base their trading and investing decisions on their (non-empty) information set, their model can produce a leptokurtic-shaped probability density of log price returns as well as volatility clustering. However, their model does not produce an autocorrelogram of returns that resembles real-world markets. There exist agent-based models that produce simulated price series whose returns resemble the Autoregressive Conditional Heteroscedasticity (ARCH) and the Generalised Autoregressive Conditional Heteroscedasticity (GARCH) volatility effects of Engle [11] and Bollerslev [6]. For example, Youssefmir and Huberman [36] presented a multi-agent model of (random) clustered volatility which arises from traders’ switching strategies. In addition, Lux and Marchesi [23] presented a multi-agent, single stock agent-based model of the stock market and showed that scaling and GARCH effects can arise out of agent interactions. There has been some work to model other financial markets, instead of stock markets, within a simulation agent-based framework. Most of this work has focused on exchange rate markets. One such example is the model of Arifovic [2] that presented a two period model of univariate exchange rate dynamics in an economy populated by agents which vary regarding their rules for first-period consumption and first-period relative currency investments, each of which is represented by bit-strings. The primary and interesting finding of Arifovic [2] is that even if traders employ genetic algorithms, the exchange rate will never settle down to a stable exchange rate. Marengo and Tordjman [24] also presented a simulation model of univariate exchange rate dynamics. By using a classifier system to model evolutionary learning, they showed that their model can produce periods of market stability and periods of market turbulence. Geanakoplos et al. [14] presented a model where they show that leverage, not interest rates, was the underlying driver of the housing boom and bust, using counterfactual reasoning with insights from an agent-based model of the housing market. One of the earliest works within agent-based computational finance is the single stock model of Day and Huang [10] where demand exceeds supply. Whilst the simulated price series of the Day and Huang [10] model does not appear to correspond to price series that are observed in real-world markets, their model is capable of producing switching bull and bear markets. Arifovic [1] showed that in a single stock market, based on the model of Bray [7], if agents use genetic algorithms to learn about the structure of the stock market then prices and quantities converge to Rational Expectations Equilibrium (REE) quantities and prices. A paper with a similar research agenda, in some sense, is Routledge [31]. In the single stock model of Routledge [31], agent types are represented by specific linear forecasts of future dividends as well as by the decision of the agent to acquire a costly informative signal. The primary result is that some parameters will lead to convergence of the REE solution, whereas others will not. The model of Routledge [31] has connections with the Grossman-Stiglitz model [15] which is itself linked to the informational efficiency of asset prices. Souissi et al. [32] presented an agent-based model to study the volume of market transactions in a financial market where agents can either be zero intelligent traders, fundamental traders or technical traders. They showed that the existence of zero intelligent traders may lead to market instability and high volatility levels. Assenza et al. [4] presented a macroeconomic agent-based model with capital and credit (CC-MABM) and illustrated that interaction of upstream and downstream firms and the evolution of their financial conditions (capital and credit) are drivers of a “crisis”.

Whilst each of these models advances the understanding of financial markets, they do not aim to replicate the complex/nonlinear properties of real-world financial markets and their probabilistic features, namely that US stock returns (as documented empirically in this paper) are mixed Gaussian. This is precisely the aim of this paper. To replicate the nonlinear, complex and entropy features of stock returns, as well as their probabilistic features, a novel computational agent-based model of the stock model is derived that is then solved through Monte Carlo simulations. A computational agent-based approach is used due to its numerous advantages. Given the recent financial crisis, conventional macroeconomic and financial economic methods failed to explain or forecast the crisis. This was, in large part, due to assumptions such as agent homogeneity and the existence of purely rational agents which implied that market prices coincided with linear equilibrium prices. Such assumptions allowed for analytic tractability at the expense, it seems, of accuracy. Computational agent-based models allow for agent heterogeneity and bounded rationality, and the majority use the power of a computer to run simulations from which models and forecasts can be derived. Market prices need not match up with rationally expected prices that are linear. Further discussion of the increasing use of complexity theory and agent-based modelling in economics and finance modelling is contained in Battiston et al. [5] and Turrell [33].

The model is developed by taking the seminal Santa Fe Artificial Stock Market (SFASM) model of LeBaron et al. [20] and modifying it significantly so that it can produce the probabilistic features and nonlinear/complex features of stock markets. The view of the stock market presented in Palmer et al. [26], LeBaron et al. [20], Palmer et al. [27] and Arthur et al. [3] is one of a market that can be approximated by an agent-based model with many different types of agents, each of whom behaves according to a different decision making rule. In these models, jointly referred to as the Santa Fe model, agents are represented by the parameters governing their linear forecasts of prices plus dividends and associated estimated variances. Using the influential Santa Fe Artificial Stock Market (SFASM) model of LeBaron et al. [20] as a base model is sensible because there is substantial pre-existing research, suggesting that the SFASM is a reasonable approximation of real-world financial markets in the sense that it can replicate some financial phenomena we observe. In particular, it has been documented in the literature that the SFASM model is capable of producing volatility persistence, significant trading volumes, bubbles, crashes and GARCH behaviour of stock returns, see LeBaron et al. [20, 27], Palmer et al. [26] and Arthur et al. [3]. Each of these stock market phenomena has been documented, in the empirical finance literature, to exist in real-world stock markets.

It is useful to make clear two primary differences between the proposed model and the model by LeBaron et al. [20]. The first primary difference, and one that has major implications for the dynamics of prices and returns, is that stocks in the newly proposed model are driven by a regime-shifting dividend process where regimes are determined by a hidden state process which follows a Markov chain. This is the key assumption that leads to a mixed Gaussian distribution of stock returns, a result that is unobtainable within the classic SFASM structure. Through this assumption, the model can replicate the probabilistic features of stock returns (mixed Gaussian distribution of stock returns). One of the key motivations for using a regime-shifting model is that it can capture “good“ states of the world (e.g. stock market boom and high dividends) and it can capture “bad“ states of the world (e.g. stock market decline and low dividends). Ignoring these regimes may result in a model that fails to capture the various peaks in the US stock return distribution that we show exist. Secondly, as a result of this regime shifting dividend process, agents condition on the state of the current world when they form conditional expectations of the future. It is not assumed that agents have perfect knowledge of states of the world. They may have perfect knowledge, or they may have imperfect knowledge, and the results of the paper do not depend on which case is true. In both cases, the distribution of returns generated by the model matches up with the distribution of real-world stock returns.

Numerous nonlinear analyses of the simulated model are then conducted, and the results are compared to the real US stock market. As mentioned above, for analysis of the stock return distribution, it is shown that simulated returns are mixed Gaussian, both qualitatively and statistically. Additional return distribution analyses and complexity and entropy analyses are conducted in a spirit similar to Xiao and Wang [35]. It is shown that simulated returns are fat-tailed (kurtosis greater than 3) and power-law fitting is also studied. To study complexity, various entropy measures developed by Xiao and Wang [35] are utilized such as fuzzy entropy (FuzEn) and multiscale fuzzy entropies based on moments which are efficient in detecting nonlinearity in time series. Rich complexity properties are discovered in the returns generated by the model, which match up well to real US stock returns, for plausible mixture probabilities. This paper produces a candidate stock market model that can replicate the (a) probability distribution of stock returns and (b) the nonlinear and complex behaviour of stock prices, that are observed in the real-world US stock market.

It is important to highlight how this new approach and proposed model can produce new and interesting insights relative to existing approaches and models in the literature. Firstly. whilst the existing models have greatly increased the understanding of financial markets, there is a clear void in that none of the existing models can replicate the nonlinear and complex features of stock prices as well as the probability distribution of stock returns (mixed Gaussian). This is the drawback of existing literature, including the papers discussed above, and is the main contribution of the proposed model.

The proposed model is suitable to achieve this task for the following reasons. Firstly, it explicitly models the key variable that determines the probability of distribution of stock returns: the dividend process. In particular, it is assumed that the dividend process is determined by a hidden state process which follows a Markov chain. This is a reasonable assumption as it is able to capture that average dividends can be higher in good states (e.g. during periods of economic growth) and lower in bad states (such as economic recessions). Secondly, the model is reasonable as it incorporates an agent-based approach allowing for heterogeneity in agent behaviour and the possibility that agents have bounded rationality. The proposed model is the first to incorporate a framework that incorporates both these features and is the first to replicate these probabilistic, nonlinear and complex phenomena of US stock returns. Furthermore, the use of a genetic algorithm to model agent behaviour and the assumption of evolutionary behaviour allows the model to capture the learning ability of sophisticated traders and investors, who account for a large share of stock market traded volume.

The paper is structured as follows. Section 3 provides a motivation for the work by providing further empirical evidence that real-world stock returns have a distribution that is mixed Gaussian. Section 4 introduces the agent-based model including, inter alia, details of the stock market, the dividend process (and the Markov description of the model), agents, forecasting rules and classifier system, price determination and the genetic algorithm. Section 5 contains extensive simulation results of the model including evidence that it can produce stock return distributions that match real-world distributions, complexity analyses and sensitivity analyses. Section 6 provides a comparison between our work and other seminal papers in the literature. Finally, Sect. 7 concludes the paper.

2 Data

The dataset used in this study is monthly stock market data (prices and dividends) of the S&P 500 (to proxy the US stock market) from Robert Shiller’s website (www.econ.yale.edu/\(\sim \)shiller), starting January 1871. As detailed on Robert Shiller’s website, Monthly dividend and earnings data are computed from the S&P four-quarter totals for the quarter since 1926, with linear interpolation to monthly figures. “dividend data before 1926 are from Cowles and associates, interpolated from annual data. Stock price data are monthly averages of daily closing prices”.

The statistical and nonlinearity behaviour of real-world stock returns was analysed, and the results reported below.

Distribution and fat tails

First, recall some properties of distributions. The kurtosis is defined as

$$\begin{aligned} \kappa&= \frac{\sum _{t=1}^{T}(r_{t}-{\bar{r}})^{4}}{(T-1)\sigma ^{4}} \end{aligned}$$

where \(r_{t}\) is the return at time t, \({\bar{r}}\) is the mean of these returns and \(\sigma ^{2}\) is the variance.

The fat-tail can be measured by a power-law exponent \(\alpha \) defined by,

$$\begin{aligned} P(|R_{t}|>x)\sim x^{-\alpha } \end{aligned}$$

Furthermore, statistical tests are carried out to deduce if US stock return distributions are statistically different to non-Gaussian distributions. To do this, the Kolmogorov–Smirnov statistic is used,

$$\begin{aligned} \text {KS}&= \max _{x}(|{\hat{F}}(x)-G(x)|) \end{aligned}$$

where \({\hat{F}}(x)\) is the cumulative distribution of the sample data and G(x) is the cumulative distribution of the normal distribution.

Table 1 contains such statistical properties of monthly logarithmic returns of the S&P 500, decade by decade. The average monthly return tends to be approximately 2% and extreme observations can be as large as a loss of 30.75% and as large as a gain of 40.75%. Not surprisingly, most decades (12 of 15), kurtosis exceeds 3 (the value for a normal distribution), with a maximum value of 15.17 during the 1920s. This is indicative that monthly stock market returns contain more extreme observations than would be the case if returns followed a normal distribution.

The skewness results show that stock returns were negatively skewed during 13 of the 15 decades with the exceptions being the 1870s and 1930s. This presents a second departure from normality. These results are confirmed by Kolmogorov-Smirnov tests where, for each decade, the null hypothesis that returns are normally distributed is rejected at a 5% level of significance.

Tables 2 and  3 contain summary statistics of these US stock market returns at four various time frequencies (monthly, quarterly, semi-annual and annual). Annual returns, exclusive of dividends, have a mean of 5.90% but this almost doubles (10.35%) when dividends are included. Standard deviations are similar whether or not dividends are included. Returns are positively skewed at higher frequencies, and each return series exhibits excess kurtosis relative to a normal distribution.

3 Motivation: stock return distributions

To explain the motivation of this work, the following two definitions are introduced:

Definition 1

Mixed Gaussian random variable.

If T is a random variable satisfying the following probability law

$$\begin{aligned} {T} \sim \sum _{j=1}^{J} w_{j} P_{{T}_{j}} \end{aligned}$$

where the \(w_{j}\) are weights (i.e. \(w_{1}+...+w_{J}=1\) and \(w_{j} \ge 0\)) and the \(P_{{T}_{j}}\)’s are the probability measures generated by the normally distributed random variables \({T}_{j}\) (i.e. \(P_{{T}_{j}}(A) := P({T}_{j} \in A\)) for any measurable event A), then T is a mixed Gaussian random variable (with J number of components). Note that, in this case

$$\begin{aligned} P({T} \in A)&= \sum _{j=1}^{J} w_{j} P_{{T}_{j}}(A) \end{aligned}$$

for any measurable event A.

Definition 2

Mixed Gaussian stochastic process.

If \(\{{T}_{t}\}_{t}^{}\) is a stochastic process such that each \({T}_{t}\) is a mixed Gaussian random variable, then \(\{{T}_{t}\}_{t}^{}\) is a mixed Gaussian stochastic process.

3.1 Kernel density estimation

This section documents some new empirical observations which are the motivation for the work in this paper. Using monthly stock market data of the S&P 500 (to proxy the US stock market) from Robert Shiller’s website (www.econ.yale.edu/\(\sim \)shiller), it is qualitatively shown that various stock return distributions, gross with dividends, are mixed Gaussian (in the sense defined above, that is a weighted average of normal distributions). In Figs. 3, 4, 5, 6, it is shown, by estimating nonparametric kernel densities, that gross returns during various periods and over various frequencies, appear to be generated by mixed Gaussian distributions. The density is estimated using a nonparametric kernel regression with an Epanechnikov kernel, which is mean square optimal. As can be observed, this empirical observation is robust to various time periods and various frequencies. Various important financial economic time periods appear to have gross stock return distributions that are a mixture of distributions, with their associated frequency, notably the Great Depression, the dot-com bubble, the recent financial crisis and the post-crisis period.

3.2 Kolmogorov–Smirnov (Goodness-of-Fit) statistical tests

To assess this more rigorously, Kolmogorov–Smirnov (goodness-of-fit) tests were conducted to assess whether stock returns for the periods and frequencies documented above are generated by mixed Gaussian distributions (i.e. weighted sum of normal distributions). For pure comparison purposes, tests were also conducted to determine whether stock returns are generated from a Gaussian (i.e. normal) distribution. To test whether stock returns are generated from a normal distribution (with mean and variance being the sample mean and sample variance, respectively), this is carried out in two stages. In the first, the sample mean and sample variance are estimated. In the second, the Kolmogorov–Smirnov statistical test is used to assess whether stock returns are generated by the estimated Gaussian distribution under the null hypothesis that that stock returns are drawn from the estimated distribution. To test whether stock returns are generated from a mixed Gaussian distribution, this methodology is extended using the following three stages. In the first, the number of components (J) is estimated from the respective kernel density estimates above. The J sample means and the (size J) covariance-variance matrix of the stock returns are then estimated by Maximum Likelihood, using the Expectation-Maximization (EM) algorithm. The Kolmogorov–Smirnov statistical test is then used to assess whether stock returns are generated by the estimated mixed Gaussian distribution under the null hypothesis that stock returns are drawn from the estimated distribution. Table 4 reports the result of the Kolmogorov–Smirnov test that stock returns, for each of the above documented periods and frequencies, are generated by a normal distribution. As can be seen, the null hypothesis that stock returns are generated by a normal distribution is strongly rejected. This is true for each period and frequency documented. Table 5 reports the result of the Kolmogorov–Smirnov test that stock returns, for each of the above documented periods and frequencies, are generated by a mixed Gaussian distribution. As can be seen, the null hypothesis that stock returns are generated by each respective mixed Gaussian distribution cannot be rejected. This is true for each period and frequency documented, providing robust evidence that a mixed Gaussian distribution is a sound approximation to the real distribution of US stock returns.

4 Stock market model

In this section, an agent-based asset pricing model is developed that can produce such distributions of stock returns. A crucial aspect of the validation of the model will be to show it produces mixed Gaussian distributions and to show that the complexity and nonlinear features of the stock prices produced by the new model resemble the complexity and nonlinear features of real-world stock returns.

4.1 Summary of modelling process

A summary of the modelling process is contained in the flow chart in Fig. 1. There are two key parts to the overall model: (1) the model of the structure of the stock market itself, specifically of the stocks and their associated dividends and (2) the model of agent behaviour, trading and learning. When these two models are combined, stock prices and returns can be computed and then analysed. Key details are contained in the flow chart, and complete details are contained in the following sections.

Fig. 1
figure 1

Flow chart showing summary of modelling process

Full size image

The key ideas are that the model (a) captures different states of the stock market by modelling dividends as a hidden state process which follows a Markov chain, (b) does not assume specific types of rationality and allows for either fully rational agents or even boundedly rational agents, and (c) allows for the possibility that agents learn from their forecasting errors relating to future prices and dividends by using a genetic algorithm. Put together, these assumptions will be shown to generate a rich set of outcomes that are shown to be consistent with real US stock markets, in particular, consistency in terms of probability distribution of stock returns (mixed Gaussian) and consistency of nonlinear and complex properties.

4.2 Stock market

The stock market consists of two assets including one risky dividend-paying stock and one risk-free bond. The model is of discrete time and infinite period. The price of the stock at time t is denoted by \(P_{t}\). The dividend of the stock at time t is denoted by \(D_{t}\). The risky asset is in fixed finite supply of N units. The risk-free asset pays interest at a rate \(r_{f}\) per period.

4.3 Dividend process

We introduce a regime-shifting model for the dividend process, which captures “good“ and “bad“ dividend states, e.g. during stock market booms and busts, respectively. Because such regimes are driven by unobserved states, the regimes are determined by a hidden state process \(s_{t}\) which follows a Markov chain taking values in 1, ..., J with transition probabilities

$$\begin{aligned} P(s_{t}=j|{\mathcal {I}}_{t-1})=P(s_{t}=j)=w_{j} \end{aligned}$$

The dividend process is modelled as the following

$$\begin{aligned} {D}_{t} = \bar{{D}}_{j} + \rho ({D}_{t-1} - \bar{{D}}_{j}) + {u}_{t} \ \ \ \ \ \ \ \text {if } s_{t} = j \end{aligned}$$
(1)

with i.i.d. random noise \(u_{t} \sim {N}({0},{\sigma }^{2}_{{u}})\), which blends the standard assumption in the literature that the change in returns is an AR(1) process with the new assumption that dividends are driven by a hidden state process which follows a Markov chain.

Hence, at time t, the dividend process has the following probabilistic structure

$$\begin{aligned} {D}_{t} = \left\{ \begin{array}{ll} \bar{{D}}_{1} + \rho ({D}_{t-1} - \bar{{D}}_{1}) + {u}_{t} &{} \text {if } s_{t} = 1 \\ \bar{{D}}_{2} + \rho ({D}_{t-1} - \bar{{D}}_{2}) + {u}_{t} &{} \text {if } s_{t} = 2 \\ \vdots &{} \vdots \\ \bar{{D}}_{J} + \rho ({D}_{t-1} - \bar{{D}}_{J}) + {u}_{t} &{} \text {if } s_{t} = J \\ \end{array} \right. \end{aligned}$$
(2)

Note that this implies that the conditional distribution of dividends is mixed Gaussian. This can be seen as follows:

$$\begin{aligned} P(D_{t}\le x | {\mathcal {I}}_{t-1})&= \sum _{j=1}^{J} w_{j} P(\bar{{D}}_{j} + \rho ({D}_{t-1} - \bar{{D}}_{j}) \\&+ {u}_{t}\le x | {\mathcal {I}}_{t-1})\\&= \sum _{j=1}^{J} w_{j} P(V_{j,t}\le x | {\mathcal {I}}_{t-1}) \end{aligned}$$

where \(V_{j,t}| {\mathcal {I}}_{t-1} \sim N((\bar{{D}}_{j} + \rho ({D}_{t-1} - \bar{{D}}_{j}),{\sigma }^{2}_{{u}}\)).

This regime-shifting model for the dividend process drastically alters some of the key properties of existing artificial stock market models including the process for Rational Expectations Equilibrium (REE) prices, the process for actual prices, the manner in which agents will form beliefs about the stock market and their trading actions as well as the resulting distribution of stock returns. Details of each of these are provided in subsequent sections.

Observe that in the degenerate case when \(J=1\), the model of LeBaron et al. [20] is a degenerate case of the much more general model that is being proposed.

Fig. 2
figure 2

Flow chart showing summary of verification process

Full size image

4.4 Agents

There are N smart traders in this model indexed by i. It is assumed that these traders are univariate in that they only invest in one risky stock. Each of them is assumed one-period myopic with constant absolute risk aversion (CARA) preferences over their one-period ahead wealth with CARA coefficient \(\lambda \). Each agent i has wealth \(W_{t,i}\) at the start of time t and must then divide their wealth between the risky asset and the risk-free asset through trading in the stock market. Their asset holdings of the stock at time t are denoted \({X}_{t,i}\), and variation over time in holdings of each asset is achieved by trades in the stock market.

The trading (i.e. portfolio) optimization problem faced by each agent i can be described as follows. It is assumed that agents in the model have some degree of rationality. At the very least, they are boundedly rational (but the model also works if they are perfectly rational). The traders in the model, at the very least, are boundedly rational in the sense that they have beliefs that are almost consistent with the stock market. More specifically, agents in the model know that dividends are regime-shifting where the states are driven by a hidden state process, but they may not know the exact objective transition probabilities. That is, they may not know \(w_{j}\), but they make the assumption that \(P(s_{t}=j|{\mathcal {I}}_{t-1,i})=P(s_{t}=j)=w_{j}^{i}\) where \({\mathcal {I}}_{t-1,i}\) is agent i’s information set at time \(t-1\) and \(w_{j}^{i}\) is agent i’s subjective assessment of \(P(s_{t}=j|{\mathcal {I}}_{t-1,i})\). If they were perfectly rational and had perfect knowledge, then it would be the case that \(w_{j}^{i}=w_{j} \ \forall i,j\). We believe that, as we only need bounded rationality to verify the model, the model has a wide degree of applicability.

Given the structure of dividends in the model, traders in the model assume that future (gross discounted) returns are conditionally mixed Gaussian, conditional on their information set, \({\mathcal {I}}_{t,i}\). As described below, one-period ahead wealth will also be conditionally mixed Gaussian, conditional on \({\mathcal {I}}_{t,i}\). As will be seen, the portfolio optimization that agents carry out in the model results in a new form of expectation formation (agent heterogeneity), much different to that of the model of LeBaron et al. [20]. Due to this difference, and the other differences above such as the structure on the dividend process, a new computational agent-based model is developed as the model of LeBaron et al. [20] cannot account for these differences. It will also be illustrated that the model of LeBaron et al. [20] is insufficient from an alternate perspective: specifically, it cannot produce the distributions of real-world stock returns that were empirically documented in Sect. 3.

Each agent assumes that discounted one-period ahead gross returns are conditionally mixed Gaussian, conditional on their information set \({\mathcal {I}}_{t,i}\) at time t. As will be seen, this assumption, combined with CARA preferences and the expectation of a log sum of weighted normal random variables, allows agents to solve for optimal holdings of the assets in terms of the first two moments of the price plus dividend distribution, conditional on \({\mathcal {I}}_{t,i}\), as well as the transition probabilities. Since prices at t and the risk-free rate are known to agent i at t (i.e. that \({P}_{t} \in {\mathcal {I}}_{t,i}\) and \(r_{f}\in {\mathcal {I}}_{t,i}\)), agent i thus deduces that

$$\begin{aligned} {P}_{t+1} + {D}_{t+1}\ \Big | \ {\mathcal {I}}_{t,i} \ \sim \ \sum _{j=1}^{J} w_{j}^{i} P_{U_{j,t+1}} \end{aligned}$$
(3)

where the \(w_{j}^{i}\) are weights (i.e. \(w_{1}^{i}+...+w_{J}^{i}=1\) and \(w_{j}^{i} \ge 0\)) and the \(P_{{U}_{j,t+1}}\)’s are the probability measures generated by the normally distributed random variables \(U_{j,t+1}\) (i.e. \(P_{U_{j,t+1}}(A) := P(U_{j,t+1} \in A\)) for any measurable event A) and where \(U_{j,t+1}| {\mathcal {I}}_{t,i} \sim N({\mathbb {E}} (P_{t+1}+\bar{{D}}_{j} + \rho ({D}_{t} - \bar{{D}}_{j}),{\mathbb {V}} (P_{t+1}+\bar{{D}}_{j} + \rho ({D}_{t} - \bar{{D}}_{j})\)).

Since agents have one-period ahead CARA utility over their wealth, their utility functions over their wealth at time \(t+1\) has the form \(U(W_{t+1,i}) = -\exp (-\lambda W_{t+1,i})\). Given each agent’s portfolio choice and trading optimization problem discussed above, their wealth at \(t+1\) satisfies:

$$\begin{aligned} W_{t+1,i} = {X}_{t,i} ({P}_{t+1} + {D}_{t+1}) +(1+r_{f})(W_{t,i}-{X}_{t,i} {P}_{t}). \end{aligned}$$

Finally, it can be stated that each agent i’s trading optimisation problem at each time t concisely as:

$$\begin{aligned}&\max _{{X}_{t,i}}{\mathbb {E}}_{t,i}\Bigg (-\exp \Big \{-\lambda \Big ( {X}_{t,i} ({P}_{t+1} + {D}_{t+1})\\&\qquad +(1+r_{f}) (W_{t,i}-{X}_{t,i} {P}_{t}) \Big )\Big )\Big \}\Bigg ) \end{aligned}$$

Now define:

$$\begin{aligned} {X}^{*}_{t,i}&\equiv {{\,\mathrm{arg\,max}\,}}_{{X}_{t,i}}{\mathbb {E}}_{t,i}\Bigg (-\exp \Big \{-\lambda \Big ( {X}_{t,i} ({P}_{t+1} + {D}_{t+1})\\&+(1+r_{f})(W_{t,i}-{X}_{t,i} {P}_{t}) \Big )\Big )\Big \}\Bigg ). \end{aligned}$$

And where

$$\begin{aligned} {P}_{t+1} + {D}_{t+1}\ \Big | \ {\mathcal {I}}_{t,i} \ \sim \ \sum _{j=1}^{J} w_{j}^{i} P_{U_{j,t+1}} \end{aligned}$$

as defined above. Note that if \(J=1\), the usual portfolio optimization emerges. However, in the more general model, with \(J>1\), a novel portfolio optimization problem, demand function and pricing function emerges which are solved for below.

Traders’ optimal holdings can now be solved and are given by the following proposition.

Proposition 1

Under the assumptions maintained in the discussion, and assuming that agents have subjective assessments regarding transition probabilities, and assuming that, for agent i,

$$\begin{aligned} \begin{pmatrix}P_{t+1}+\bar{{D}}_{1} + \rho ({D}_{t} - \bar{{D}}_{1}) + u_{t} \\ P_{t+1}+\bar{{D}}_{2} + \rho ({D}_{t} - \bar{{D}}_{2}) + u_{t} \\ \cdot \\ \cdot \\ P_{t+1}+\bar{{D}}_{J} + \rho ({D}_{t} - \bar{{D}}_{J}) + u_{t} \end{pmatrix} \Bigg | {\mathcal {I}}_{t,i} \sim \mathbf{N } \big ( \mu ^{i}, \Sigma ^{i}\big ) \end{aligned}$$

where \(\mu ^{i}\) is agent i’s (subjective) expectation of state-dependent prices plus dividends and where \(\Sigma ^{i}\) is agent i’s (subjective) estimate of the conditional variance-covariance matrix of state-dependent prices plus dividends; agent i’s optimal stock holdings at t satisfy

$$\begin{aligned} {{X}_{t,i}^{*}}&= \frac{\lambda ^{-1} \big ( \sum _{j=1}^{J} w_{j}^{i}\mu _{j}^{i} - (1+r_{f})P_{t} \big ) }{ \sum _{j=1}^{J} \big (w_{j}^{i}\big ) ^{2} \Sigma _{j,j}^{i}+ \sum _{j=1, k \ne j} w_{j} w_{k} \Sigma _{j,k}^{i} } \end{aligned}$$

where \(w_{j}^{i}\) is agent i’s conjecture regarding the transition probability for state j. Alternatively stated,

$$\begin{aligned} {{X}_{t,i}^{*}}&= \frac{\lambda ^{-1} \big ( \mathbb {{\hat{E}}}_{t,i} \big ( (P_{t+1}+D_{t+1})) - (1+r_{f}) P_{t} \big ) }{ \sum _{j=1}^{J} \big ( w_{j}^{i}\big ) ^{2} \Sigma _{j,j}^{i}+ \sum _{j=1, k \ne j} w_{j}^{i}w_{k}^{i} \Sigma _{j,k}^{i} } \end{aligned}$$
(4)

where \(\hat{{\mathbb {E}}}(\cdot | {\mathcal {I}}_{t,i})\) denotes i’s subjective conditional expectation.

Proof

See Appendix. \(\square \)

Note that the above solution assumes that while agents do know that the dividend process is regime-shifting, where the regimes are driven by a (hidden) state process which follows a Markov chain, they need not know the exact transition probability of each Markov state and the statistical moments of state-dependent prices plus dividends. That is, agents can be boundedly rational. Furthermore, they form their own estimates of transition probabilities and have their own conditional expectations as well as estimates of variances and covariances. That is, agents are heterogeneous in the model.

In later sections, model-based prices will be compared to prices that would have realised in a REE. To achieve this, the analytic solution for the REE pricing function is derived by imposing additional restrictions on the model—specifically homogeneity and rationality. The solution to this is presented in the Proposition below. Each agent is assumed to have 1 share of the stock.

Proposition 2

Under the maintained assumptions above, the REE pricing function satisfies:

$$\begin{aligned} P_{t}&= \frac{\rho }{1+r_{f}-\rho }{D}_{t} + \frac{(A+1) \Big ( ((1-\rho ) \sum _{j=1}^{J} w_{j} {\bar{D}}_{j} \Big ) - \lambda \Bigg ( \sum _{j=1}^{J} \big (w_{j}\big ) ^{2} \Sigma _{j,j}+ \sum _{j=1, i \ne j} w_{i} w_{j} \Sigma _{i,j} \Bigg )}{r_{f}} \end{aligned}$$

Proof

See Appendix. \(\square \)

This provides us with a process of theoretical stock prices in the stock model based on a homogeneous REE pricing function. Note that the homogeneous REE pricing function that is derived is a generalization of the one derived in LeBaron et al. [20]. This is due to the regime-shifting process for dividends. In computations, the homogeneous pricing solution is used to obtain theoretical prices from which the simulated prices obtained from the agent-based model are compared.

Based on the REE pricing function, one can compute optimal mean squared error forecasts of prices plus dividends in the class of homogeneous forecasts. This has the solution given by the following result:

Proposition 3

Under the maintained assumptions above, the optimal mean squared error (MSE) forecasts of prices plus dividends in the class of homogeneous forecasts satisfies:

$$\begin{aligned} \mathbb {{E}}_{t} \big ( {P}_{t+1} + {D}_{t+1} \big )&= \rho \big ( {P}_{t} + {D}_{t} \big ) + (1-\rho )\\&\quad \quad \Big ( ({A}+{1}) \sum _{j=1}^{J} w_{j} {{\bar{D}}}_{j} + {B} \Big ) \end{aligned}$$

Proof

See Appendix. \(\square \)

These are used when constructing rules for agents, as described fully in Sect. 4.5. We will denote \(A_{re} = \rho \) and \({B}_{re} = (1-\rho ) \Big ( ({A}+{1}) \sum _{j=1}^{J}w_{j} {{\bar{D}}}_{j} + {B} \Big )\).

4.5 Forecasting rules and classifier system

LeBaron et al. [20] incorporated the use of forecasting rules and the classifier system first developed by Holland [16]. These are now modified it so that it is consistent with the regime-shifting structure of the new stock market model and the (linear) homogeneous REE solution that applies to it.

The key idea is that agents form forecasts of future prices and dividends by using candidate forecasting rules. Each agent i is assumed to have an information set at time t that consists of various publicly available information of the stock market. Such information can include \(P_{t}\) at t, the dividend \(D_{t}\) paid at t moving averages of the price series and so on. Agents condense this information into world bits, and each agent is assumed to have twelve world bits of price-dividend rations and price moving averages. At each point in time, each world bit is either true or false. Bits that involve just the price series are referred to as technical bits. Bits involving dividends are referred to as fundamental bits. We also include two control bits. For example, bit one could be “\(P_{t}\ge 100\)”. At time t, this is either true or false.

Each agent is assumed to make linear forecasts. This means forecasts will be of the form

$$\begin{aligned} {\hat{\mathbb {E}}}_{t,i} (P_{t+1} + D_{t+1}) = a_{i} (P_{t} + D_{t}) + b_{i}. \end{aligned}$$

A classifier forecasting system is used to model how agents make these forecasts as it allows the model to capture that agents may form forecasts differently in different states of the world. This means forecasts at t are dependent on whichever bits the agent i decides is important. Each agent has twelve bits they believe are important. This is collected into a ternary twelve string of the symbols \(\{0,1,\# \}\). The \(k^{th}\) component of the ternary twelve string corresponds to the \(k^{th}\) world bit. A value of 0 means that the world bit must be false for the condition to match. A value of 1 means the world bit must be true and a value of \(\#\) means indifference to that world bit. So, if the world only contained five world bits, a ternary string of \(\#10\#1\) would match the world bit string 11001 but not 10001. Trading rules consist of ternary strings, as described, as well as a forecasting equation of prices plus dividends (and an estimated variance) connected with that rule. From now on, denote rules by the subscript j. Rule j for example could refer to the state \(01101\#1\#\#10\) and the following forecasting equation (which activates when the state \(01101\#1\#\#10\) is activated)

$$\begin{aligned} \mathbb {{\hat{E}}}_{t,i,j} (P_{t+1} + D_{t+1}) = a_{i,j} (P_{t} + D_{t}) + b_{i,j} \end{aligned}$$

with associated variance estimate \(\sigma ^{2}_{i,j}\). For each rule j (and for each agent i), its associated parameters \(a_{i,j}\) and \(b_{i,j}\) are initially set to random values uniformly distributed in the intervals \(A \text { range}\) and \(B \text { range}\), respectively, where \(A \text { range}\) and \(B \text { range}\) are intervals centered about the mean-squared error optimal solutions in the (linear) homogeneous REE of the agent-based model, see below.

$$\begin{aligned} \mathbb {{E}}_{t} ( P_{t+1}+D_{t+1} )&= \rho (P_{t}+ D_{t}) + (1-\rho ) \\&\Bigg ( (A+1) \sum _{j=1}^{J} w_{j} D_{j} + B \Bigg ) \end{aligned}$$

Combined with this uniform distribution, \(A \text { range}\) and \(B \text { range}\) thus represent the beliefs of the agents regarding \(A_{re}\) and \(B_{re}\). At each point in time t, agent i may have several rules that are activated by the world bit string at t. Agent i chooses the rule that has least variance and from this optimal trading rule the agent can now compute their optimal forecast of prices plus dividends (i.e. their optimal computation of \(\hat{{\mathbb {E}}}_{t,i} (P_{t+1} + D_{t+1})\)). Each agent also has an estimate of the variance \(\sigma ^{2}_{i,j}\) associated with this forecast. Given these, optimal trading positions can be computed using optimal stock holdings in the stock market driven by regime-shifting dividends, which is given by Eq. 5.

4.6 Trading and price determination

Prices are set by a specialist who observes the demands (order flow) of the stock and sets \(P_{t}\) such that the market clears (demand equals supply). That is, the specialist sets prices such that \(\sum _{i=1}^{N}{X}^{*}_{t,i} = N\). After trading, each agent i updates the accuracy of forecast rules according to an exponentially weighted average of squared forecast error,

$$\begin{aligned}&\nu ^{2}_{t,i,j} = \Big (1-\frac{1}{\tau } \Big ) \nu ^{2}_{t-1,i,j} \\&\quad +\frac{1}{\tau } \big ( P_{t} +D_{t} - (a_{i,j} (P_{t-1} + D_{t-1}) +b_{i,j}) \big )^{2}; \end{aligned}$$

where \(\tau \) is usually set to 75.

4.7 Learning genetic algorithm

To allow for evolutionary learning on the part of agents, a genetic algorithmic learning approach is incorporated into the agent-based model. The genetic algorithm is the same as that discussed in Palmer et al. [26] to ensure that effects of altering the structure of the stock model can be analysed and keeping other aspects constant. An overview of the genetic algorithm used by Palmer et al. [26] is provided here. After each K periods, agents update their rules according to the following genetic algorithm. For each agent i, 20 rules are removed according to a penalized variance criterion and exactly 20 new rules are added. These 20 new rules are obtained by the set of surviving 81 rules and are added through one of two approaches: crossover or mutation. Crossover occurs with probability 0.1 and mutation with probability 0.9. Crossover means selecting two parent rules, according to tournament selection and uniformly crossing the two rules - bit by bit for the ternary string and real number by real number for the vectors \((a_{i,j}, b_{i,j}, \sigma ^{2}_{i,j})\). Mutation means selecting a single parent by tournament selection and modifying that parent’s string randomly bit by bit and modifying that parent’s \((a_{i,j}, b_{i,j}, \sigma ^{2}_{i,j})\) by random amounts. The same probabilities are used as given in Palmer et al. [26] to maintain some degree of uniformity and for comparison of results.

5 Simulation results

In this section, results are presented that show that the new model can produce stock return distributions that are mixed Gaussian (weighted sum of normal random variables), similar to the distributions of real-world stock returns, as documented in Sect. 3. It is also shown that the Santa Fe model of Palmer et al. [26] produces a distribution that does not replicate the mixed Gaussian distributions of stock returns documented in Sect. 3. Monte Carlo simulations are also conducted to carry out a sensitivity analysis to understand whether the model simulated prices vary with salient parameters (such as transition probabilities and agent risk aversion) in a plausible manner and to understand how, through the use of an agent-based model, mispricing of stocks depends on agent beliefs.

5.1 Summary of verification process

A summary of the verification process is contained in the flow chart in Fig. 2. There are three key parts to the verification of the model developed: (1) verify that the probability distribution of simulated stock returns is the same as real US stock returns (mixed Gaussian), (2) verify that the nonlinear and complex behaviour of simulated stock returns is the same as real US stock returns and 3) conduct a sensitivity analysis to verify that simulated stock prices/returns vary in a plausible with key parameters of the model. Key details are contained in the flow chart, and complete details are contained in the following sections.

5.2 Stock return distributions

One of the motivations behind this paper, as mentioned in Sect. 3, is to develop an artificial stock market model that is capable of producing a distribution of stock returns that is similar to historical US stock market return distributions. In this section, it is shown that the Santa Fe artificial stock market model, with parameter values as used in Palmer et al. [26], produces a distribution that is distinctly different to that observed in real-world stock markets. It is then shown that the model developed in this paper produces a distribution which is similar to the distribution of real-world stock returns we documented in an early section of this paper. This is shown through kernel density estimations and Kolmogorov–Smirnov statistical tests.

The single-stock Santa Fe artificial stock market as given in LeBaron et al. [20] was simulated with the parameter values as used in their paper. This is a degenerate case of the model developed here with parameter values given by Table 6. Note that \(A \text { range} \ [A_{re}-0.25, A_{re}+0.25 ]\) and \(B \text { range} \ [B_{re}-14.5, B_{re}+14.5 ]\) and thus contain \(A_{re}\) and \(B_{re}\), respectively. The model simulation is run for 2500 time periods to ensure sufficient learning with \(K=10\) and is then allowed to run for a further 2500 periods after which the last 2500 simulated price observations are extracted and the density estimate for the corresponding stock returns is plotted in Fig. 7a. T and K are chosen to ensure \(T/K=250\), as in LeBaron et al. [20]. (Alternate values for T and K can be used, and they produce similar results.) The density estimate of the simulated stock returns produced by the degenerate Santa Fe model has a shape that is symmetric and non-mixture in shape. This is confirmed by Kolmogorov–Smirnov statistical tests. The null hypothesis that the Santa Fe simulated returns are generated from a mixed Gaussian distribution is then tested. The resulting Kolmogorov–Smirnov statistic is 14.8412, with a p-value of 0.00, indicating that the null hypothesis is rejected at even a 99% significance level. This provides convincing statistical evidence that the distribution of stock returns produced by the Santa Fe model is not a mixture of Gaussian distributions unlike the real-world stock return distributions documented in Sect. 3.

Following on from this exercise, a similar simulation for the new agent-based model is conducted. Using the parameters including the transition probabilities as in Table 7, the new model is simulated for 2500 time periods to ensure sufficient learning with \(K=10\). Then, the model is run for a further 2500 periods after which the last 2500 simulated price observations are extracted and the density estimate for the corresponding stock returns is plotted in Fig. 7b. (Alternate ranges for T and K can be used and they produce similar results.)

The density estimate of the simulated stock returns produced by the new model has a shape that resembles a two-component mixed Gaussian distribution. This is confirmed by Kolmogorov–Smirnov statistical tests. The null hypothesis that the simulated returns from the new model are generated from a mixed Gaussian distribution is tested. The resulting Kolmogorov–Smirnov statistic is 0.0419, indicating that the null hypothesis is not rejected at a 99% significance (it is not even rejected at a 90% significance). This provides fairly compelling statistical evidence that the distribution of stock returns produced by the new model is statistically indistinguishable from a mixed Gaussian distribution.

By altering the parameter values, we are able to replicate each of the real-world stock return distributions we documented in Sect. 3. Furthermore, if real-world US stock returns were obtained from various other distributions (e.g. Gaussian, student t, Hansen skewed t), the model can still replicate these distributions by changing the fitting distribution.

5.3 Nonlinearity and complexity behaviour of real returns and simulated returns

In this section, the nonlinearity and complexity behaviour of real-world stock returns and model-simulated returns are analysed.

Multiscale and multimoment complexity

Following the analysis in Xiao and Wang [35], simulated price process and real-world price process are compared using the multiscale fuzzy entropy in mean method (\(\text {MFE}_{\mu }\)) which is based on fuzzy entropy (FuzEn). This is a verification check to ensure the stock prices and stock returns generated by the new model have similar nonlinear and complexity properties to real-world US stock prices and stock returns.

Quantification of unpredictability of financial time series is an important task in finance, and prior work may be found in Xiao and Wang [35], Ormos and Zibriczky [25], and Wu and Zhang [34]. Various entropy-based metrics have been found to be useful in conveying information about complexity and information in financial time series. Some entropy-based approaches that have been used include approximate entropy [28], sample entropy [28], fuzzy entropy [8], permutation entropy [8], distribution entropy [21] and dispersion entropy [30]. FuzEn has recently become popular due to its advantage that it is reliant on sample size yet maintains performance. To account for volatility in financial time series and possible nonlinearities, Xiao and Wang [35] build upon the FuzEn approach and develop the multiscale fuzzy entropy in mean (\(\text {MFE}_{\mu }\)) method. The simulated stock return series generated by the mixed Gaussian agent-based model was analysed using \(\text {MFE}_{\mu }\), for various mixture probabilities, and results were compared to real-world S&P 500 returns. Details of the full algorithm for computing \(\text {MFE}_{\mu }\) may be found in Xiao and Wang [35].

For the estimation of \(\text {MFE}_{\mu }\), the parameters were set as in Xiao and Wang [35] so that the threshold of tolerance is the standard deviation of the return series, the fuzzy power function is 2, and the embedding dimension is also 2. The scale factor \(\tau \) is varied from 1 to 20 in step sizes of 1. Eleven different price series were generated, each with a different \(w_{1}\) (probability of high dividend state) ranging from 0 to 100% in steps sizes of 10%. The \(\text {MFE}_{\mu }\) for each of these series were then studied and were then compared to the \(\text {MFE}_{\mu }\) that is generated by real-world monthly US stock returns. This serves two purposes: (1) as a test of whether the new simulated model can produce a time series of stock returns with similar complexity and predictability to real-world stock returns and (2) to try and infer which \(w_{1}\) (probability of high dividend state) matches best to real-world US stock returns.

Figures 8 and 9 containing the results for the simulated \(\text {MFE}_{\mu }\) series show that the \(\text {MFE}_{\mu }\) series decrease with the scale parameter \(\tau \), suggesting that complexity falls with a rise in \(\tau \). This is also true for real US stock returns (Fig. 10).

The \(\text {MFE}_{\mu }\) series generated on the simulated data were then compared to that generated by the real US stock returns. Following this, the mean-squared error of each was compared and it was found that \(w_{1}=80\%\) provided the best calibration.

The next analysis carried out was to study the complexity structure of model-simulated returns and to compare these to the complexity structure of real-world US returns using \(\text {MFE}_{m^{p}}\). Figure 11 shows the complexity results for S&P 500 returns. As can be seen, the relationship between \(\text {MFE}_{m^{p}}\) and \(\tau \) changes. This is consistent with the findings of Xiao and Wang [35]. For low values of p, complexity tends to decrease as \(\tau \) increases. For larger values of p, complexity decreases initially as \(\tau \) increases but then increases again for large values of \(\tau \).

Figures 12,  13, and  14 show the complexity results for the model-simulated series using \(\text {MFE}_{m^{p}}\). The proposed model with either \(w_{1}=20\%\) or \(w_{1}=40\%\) seems to show a complexity structure quite similar to real-world S&P 500 returns.

5.4 Sensitivity analyses

In this subsection, results of the sensitivity analyses carried out are reported. In these sensitivity analyses, the relationship between stock prices produced by the mixed Gaussian agent-based model and salient underlying parameters is analysed, within the simulated environment. Key parameters that determine stock prices in the model are agents’ risk aversion coefficient (\(\lambda \)), agents’ beliefs regarding the variance of future prices plus dividends (\({\sigma ^{2}}_{{P+D}}\)), the risk-free rate (\(r_{f}\)), the mean of the stock dividend stochastic process (\(\bar{{D}}\)) and the transition probabilities \(w_{j}\). The effects of \(A \text { range}\) and \(B \text { range}\) on simulated stock prices will be examined in the following subsection. To analyse the relationship between stock prices and the first group of salient underlying parameters, a base case of the simulated model was fixed with the following parameters. These parameters are initially set as shown in Table 8.

Given the set of base case parameters, the model was then simulated numerous times by varying, one at a time, for each of \(\lambda \), \({\sigma ^{2}}_{{P+D}}\), \({\sigma ^{2}}_{{u}_{t}}\), \(\rho \), \(r_{f}\), \({{\bar{D}}}_{j}\) and \(w_{j}\) whilst holding the remaining parameters fixed at their base case values. To ensure the results are not dependent on the values drawn for the various random variables, 600 price paths were simulated for each sensitivity analysis.

As can be seen, from Figure 15a, as the level of risk aversion of agents rises, stock prices fall monotonically (since agents are risk averse). It can also be seen, from Figure 15b, that the stock price falls as the agents’ variance beliefs of the future price plus dividend of the stock increase (since agents are risk-averse). For reasons relating to the structure of CARA utility of investors and traders, a negative relationship between the risk-free rate and stock prices and a positive relationship between the mean of the dividend process and stock prices can be expected. This model is also consistent with both these expectations as depicted in Figs. 15c and 13d. Lastly, the higher the probability of the high mean dividend state, the higher the stock price (since stocks are a claim to dividends) as shown in Figure 15e.

5.5 Stock mispricing depends on agent beliefs

In this subsection, evidence is provided that stock mispricing, relative to REE prices, is possible in the new agent-based model (consistent with the findings in the empirical behavioural finance literature) and that the degree of such mispricing is larger if agents’ beliefs of the stock market (i.e. of \(A_{re} \) and \(B_{re}\)) are more likely to have a greater deviation from their rationally expected values. Three different versions of the model are simulated, each of which is given a different range for the draws of \(a_{i,j}\) and \(b_{i,j}\). The remaining parameters are each fixed as shown in Table 9.

Each model simulation is run for 2500 time periods to ensure sufficient learning with \(K=10\) and is then allowed to run for a further 2500 periods after which the last 250 simulated price observations are extracted and plotted. Figures 16a–c contain the time series plots. The corresponding ranges for the \(a_{i,j}\) and \(b_{i,j}\) parameters are included alongside the time series plots. The plots in blue are the simulated price series. The plots in green are the theoretical price series obtained using the (linear) homogeneous rational expectations pricing equation from Sect. 4. When agents’ stock market beliefs are more likely to have a greater deviation from their rationally expected values (given by a greater A range and/or B range), the larger is mispricing relative to REE prices. This can be seen by, for example, comparing Fig. 16b with Fig. 16c.

6 Comparison to other seminal agent-based models of financial markets

In this section, a detailed comparison is provided between our paper and other seminal agent-based models of financial markets. Existing papers in the literature have provided explanations for various real-world phenomena observed in financial markets, including, for example, volatility clustering, the stability of prices, market states, and housing booms and busts. As the comparison below shows, the contribution of our work is that it is the sole paper that can replicate the probabilistic properties (mixed Gaussian distribution) of US stock returns as well as its nonlinear, complex features.

Paper

Model Type/Summary

Main Finding(s)

Replicates Nonlinear/Complex Properties of Stock Markets

Replicates Distribution of Real Stock Market Returns

This paper—Seedat and Abelman (2021)

Model of the US stock market derived from Markov chain methods with heterogenous agents

Replicates the nonlinear/complex properties of stock returns; replicates the mixed Gaussian distribution of stock returns; price series generated by model vary in plausible way with key variables

  

Xiao and Wang (2020)

Financial price model based on a particle system in physics incorporating communication range heterogeneity of market agents

Replicated the complex and entropy properties of stock returns; replicated fat-tailed distribution of financial returns

  

Farmer et al. (2005)

Minimally intelligent agent model with a double auction

Explained a sizable portion of the variance between the best buying and selling prices in a cross-section of almost a dozen stocks and their diffusion rates

  

Raberto et al. (2001)

Random order generation single stock model where total financial resources are constrained

Replicated volatility clustering and leptokurtic shape of returns

  

Youssefmir and Huberman (1997)

Multi-agent single stock model with switching traders’ strategies

Replicated volatility clustering and sudden bursts of activity

  

Arifovic (1996)

Two-period Kareken-Wallace overlapping generations model of the exchange rate market

Showed that the exchange rate never settles to a stable rate if agents employ genetic algorithms which is the result of fluctuations in the portfolio fractions

  

Marengo and Tordjman (1996)

Model of the exchange rate market with evolutionary learning

Model explained how periods of market stability and periods of market turbulence may arise

  

Geanakoplos et al. (2012)

Model of the housing market utilizing counterfactual reasoning, with data from Washington DC

Showed that leverage, not interest rates, were the underlying driver of the1997-2007 housing boom and bust

  

Paper

Model Type/Summary

Main Finding(s)

Replicates Nonlinear/Complex Properties of Stock Markets

Replicates Distribution of Real Stock Market Returns

Day and Huang (1990)

Deterministic model of a single stock market with excess demand

Model showed how switching bull and bear markets may arise

  

Arifovic (1991)

Model of a single stock market where agents’ behavior is modelled using genetic algorithms

Model showed how prices and quantities can converge to Rational Expectations Equilibrium (REE) quantities and prices

  

Routledge (2001)

Model of a single stock market where agents form linear forecasts, in a spirit similar to a Grossman-Stiglitz model

Model shows under what conditions prices and quantities can converge to Rational Expectations Equilibrium (REE) quantities and prices

  

Souissi et al. (2018)

Model of a financial market with one asset where agents can either be zero intelligent traders, fundamental traders or technical traders

Model showed that the existence of zero intelligent traders may lead to market instability and high volatility levels

  

Assenza et al. (2015)

Macroeconomic agent-based model with capital and credit (CC-MABM) and with upstream and downstream firms

Model illustrated that interaction of these firms and the dynamics of capital and credit are drivers of a crisis

  

7 Conclusion

In this paper, it was documented that US stock return distributions, over various periods and time frequencies, are mixed Gaussian (a weighted sum of normal random variables). This was achieved by kernel density estimation and Kolmogorov–Smirnov statistical tests. To explain this, an agent-based asset pricing model was developed where the dividend process is regime-shifting and where regimes are driven by a hidden state process which follows a Markov chain. In this model, agents are boundedly rational and heterogeneous and trade the stock and the risk-free asset. By computing optimal trader demands and by Monte Carlo simulations, it was shown that the model can replicate the observed real-world US stock return distributions, even if agents are boundedly rational and may not have perfect knowledge of the primitives of the stock market. It was also shown that the model replicates the nonlinear and complex features of US stock returns. Furthermore, it was shown that the price series generated by the simulated model varies in a plausible way with salient financial economic variables and that asset mispricing, relative to REE prices, are larger the more uncertain agents’ beliefs of the stock market.