A financial trading system with optimized indicator setting, trading rule definition, and signal aggregation through Particle Swarm Optimization

Abstract

Algorithmic trading, a widespread practice in the financial industry, is based on the automatic signal generation based on trading rules of one or more technical analysis indicators. Generally, the parameters for computing the indicators (such as the time windows), the trading rules (converting the indicator into a trading signal) and the weights for signal aggregation (for combining the signals from a plurality of indicators) are established by the trader based on her experience and are treated as fixed inputs of the trading algorithm. In recent literature, simple optimization systems are introduced by varying only one category of parameters at a time, that is only the indicators setting, only the trading rules definition, or only the signal aggregation while keeping the remaining parameters fixed. Our research goes further and proposes an automated trading system based on simultaneous optimization of the three categories of parameters. More precisely, we consider four technical indicators widely used in financial practice, the Exponential Moving Average, the Relative Strength Index, the Moving Average Convergence/Divergence, and the Bollinger Bands and we determine the optimal signal aggregation, trading rule definition and indicator setting using the Particle Swarm Optimization metaheuristic over a commonly used fitness function, that is the net capital at the end of the trading period. We apply our trading system to the Italian index FTSE MIB and to a set of financial stocks belonging to the FTSE MIB over a multi-year period for training and testing. We generally achieve superior performance both in sample and out of sample, using a standard technical analysis system as a benchmark. Furthermore, we successfully verify the ability of the optimized trading system to accurately classify the stock price trends.

1 Introduction

This paper deals with the maximization the investor’s performances resulting from the trading of financial securities. More precisely, it proposes an algorithmic trading system based on technical analysis indicators with optimization of the signal aggregation, the trading rule definition, and the indicator setting (including the time windows selection). The parameters of the technical analysis indicator, the trading rules and the signal weights are the inputs of the system, and the fitness function to be maximized is the net capital at the end of the trading period. It is a complex global optimization problem, which we address by the Particle Swarm Optimization (PSO) metaheuristic (Kennedy and Eberhart 1995), an approximate bio-inspired numerical optimizer emulating the social behavior of animals in search of food.

Technical analysis, that is the study of market action, is widely used in the financial practice for price and trend predictions (Wilder 1978; Pring 1991; Murphy 1999; Brabazon and O’Neill 2004; Farias et al. 2017; Lahmiri 2018, Lorig et al. 2019; Hudson and Urquhart 2021). Several different technical analysis indicators are considered, each one based on different information on the financial market and providing different trading signals for buying, selling, or holding via appropriate trading rules. Technical indicators depend on one or more parameters (mainly time windows) generally assumed equal to standard parameter values in the financial practice, including typical time windows, that are time lengths much used by traders. The trading rules are functions whose inputs are the technical indicators; these functions generate signals based on market prices through the comparison with (generally standard) thresholds.

Usually, trading systems jointly consider a plurality of technical indicators, whose trading signals are aggregated through weighting, permetting a more informed decision-making (Briza and Naval 2011; Wang et al. 2014; Worasucheep et al. 2017; Corazza et al. 2017, 2021; Bakhtiyari Asl et al. 2022). We select and aggregate four standard indicators, much employed in theory and practice, that are the Exponential Moving Average (MA), the Relative Strength Index (RSI), the Moving Average Convergence/Divergence (MACD), and the Bollinger Bands (BB). The motivations for considering only four indicators, that is a relatively small set, are to show the high potential of a simple trading system appropriately optimized, that generally improves the trading performance of the standard technical analysis system based on the same indicators. In different words, our purpose is to present that greater final capital amounts compared to the benchmarks can be obtained by investors even relying on a small number of indicators appropriately optimized. Furthermore, the selection of few indicators to be included in the system avoids overfitting problems.

In research and practice, the indicators setting, the trading rule definitions and the signal aggregations are often treated through standard values and procedures, corresponding to fixed inputs into the trading systems (e.g., Wilder 1978; Murphy 1999; Farias et al. 2017; Muruganandan 2020). As recent advancement, several studies optimize the selection of the parameters values for maximizing the investor’s performance via the application of metaheuristics, such as the Particle Swarm Optimization (e.g. applied in Briza and Naval 2011; Wang et al. 2014; Worasucheep et al. 2017; Corazza et al. 2017, 2021; Bakhtiyari Asl et al. 2022; see Thakkar and Chaudhari 2021 for a review) and the Genetic Algorithms and Programming (e.g. applied in Mousavi et al. 2014; Ozturk et al. 2016; Macedo et al. 2020; Zhang and Khushi 2020), in such a way that the parameter values are not necessarily equal to the standard ones suggested by the practice or those proposed by traders based on their personal experience.

The existing studies on trading system optimization have a main limitation: they only optimize a single category of parameters at a time, only the indicators setting, only the trading rules definition, or only the signal aggregation, as described in the following list.

• Indicator settings are optimized over the signal weights decided by the trader using traditional trading rules in several studies. More precisely, Corazza et al. (2017, 2021) applied PSO for computing the optimal time windows, whereas they considered the standard trading rules and fixed the indicator weights, gaving each indicator the same importance. Macedo et al. (2020) generated optimized solutions consisting of one indicator (belonging to momentum, trend and breakout categories) with a setting for its parameters via Genetic Algoritms. Furthermore, Zhang and Khushi (2020) proposed the optimization of the parameters of technical indicators to maximize the Sharpe and Sterling ratio via Genetic Algorithm.

• Signal weights are optimized over standard indicator settings and trading rules in Worasucheep et al. (2017) and Briza and Naval (2011). They used the standard time windows and applied PSO for determining the optimal weights of the indicators. Furthermore, interestingly, Wang et al. (2014) and Bakhtiyari Asl et al. (2022) optimized the indicator weights and not the time window, but their set of indicators is composed of the same indicators computed on different windows. In particular, in Wang et al. (2014) 140 indicators are created taking different time windows of the Moving Average and the Trading Range Break-out; in Bakhtiyari Asl et al. (2022) 106 indicators are obtained computing the Relative Strength Index, the Moving Averages and the Moving Average Convergence/Divergence with several windows. In the present study, our approach is different: we include only one setting for each indicator, calculated in time windows optimized by PSO over the fitness function.

• The trading rules alone are optimized over standard parameter settings and signal weights in numerous works. Allen and Karjalainen (1999) applied Genetic Algorithm for finding technical trading rules. Neely (2003) applied Genetic Programming for building risk-adjusted optimal trading rules. Mousavi et al. (2014) proposed a dynamic portfolio trading system and optimized the rule base built on technical indices via Genetic Programming. Dai et al. (2016) analyzed the optimality of trend following rules, obtained in terms of a sequence of stopping times, aiming at wealth maximization. Ozturk et al. (2016) optimized the selection and combination of trading rules based on technical indicators using Genetic Algorithm and a greedy search heuristic.

In the current work we propose to simultaneously optimize the three categories of parameters. At the best of our knowledge, our contribution is the first one in literature with the present purpose.

Figure 1 represents the influence diagram of our trading system, from the parameters to the fitness function (net final capital) via intermediate levels, that are the indicators, the signals and the aggregated signal. More precisely, the parameters of the indicators influence the computation of the technical indicators. Each indicator generates its own signal through its trading rule based on the trading rules parameters. The signals are aggregated into an overall signal using the weights of the indicators. Finally, the net final capital depends on buying, selling, and holding decisions derived from the aggregated signal. The net final capital is maximized via PSO by appropriately optimizing the three categories of parameters. Green line are for MA, blue lines for RSI, yellow line for MACD, orange line for BB.

Fig. 1

Influence diagram (colour figure online)

The remainder of this paper is organized as follows. The next section is devoted to describe the methodology used in this work, combining mathematical and statistical tools in the areas of technical analysis and optimization via metaheuristics. The Sect. 3 presents the in-sample and out-of-sample results of our optimized trading system. Some final remarks conclude the paper.

2 Methodology

In the present section we introduce the foundation of our simple trading system based on four common technical indicators. Our purpose is to optimize the parametrization of indicators, trading rules, and signal aggregation (for a total of 23 parameters). The optimization problem is complex and the use of exact solution method could be excessively burdensome in terms of computing time. Therefore, we need to apply a metaheuristic and we choose Particle Swarm Optimization for its exploration and exploitation capabilities (Kennedy and Eberhart 1995; Olorunda and Engelbrecht 2008; Wakasa et al. 2010).

2.1 The trading system

Our trading system depends on 23 parameters, that are used for three purposes: 1. the computation of the indicators, 2. the definition of the trading rules, 3. the signal aggregation. The following Table 1 collects the parameters and describes their main features. The first column reports the explicative names of the parameters, the second column contains the symbols, the third one presents the indicators to which the parameters are referred (MA, RSI, MACD, BB). The fourth column reports the reference area of the parameters (indicator, rule, signal aggregation). Finally, the last column contains the standard values of the parameters in the literature.

The parameter set X is the set containing all the parameters,

$$\begin{aligned} X= & {} \{w_{ma}, d_{ma}, w_{rsi}, t_{rsi}^{l,en_b}, t_{rsi}^{h,en_s}, t_{rsi}^{h,ex_b}, t_{rsi}^{l,ex_s}, w_{macd}^{short}, w_{macd}^{long}, w_{macd}^{signal}, d_{macd}, \nonumber \\{} & {} w_{bb}^{ma}, w_{bb}^{std}, t_{bb}^u, t_{bb}^l, t_{bb}^{c,ex_s}, t_{bb}^{c,ex_b}, \theta _{ma}, \theta _{rsi}, \theta _{macd}, \theta _{bb}, t_{as}^b, t_{as}^s\}. \end{aligned}$$

(1)

Table 1 Parameters of the standard trading system

The trading system with the standard values of the parameters (collected in Table 1) serves as a benchmark for our optimized trading system.

2.1.1 Indicators

Our trading system considers four commonly used technical indicators that are the moving average (MA), the relative strength index (RSI), the moving average convergence/divergence (MACD), and the Bollinger bands (BB). We assume the standard definition in literature (Wilder 1978; Murphy 1999). The MA is computed over a period of $w_{ma}$ days. The RSI is computed over a period of $w_{rsi}$ days. The MACD line is the difference between the moving averages over a long period of $w_{macd}^{long}$ days and a short period of $w_{macd}^{short}$ days and the MACD signal is the moving average of the MACD line over $w_{macd}^{signal}$ days. The BB are computed using the moving average over $w_{bb}^{ma}$ days and the moving standard deviation over $w_{bb}^{std}$ days: the upper band is obtained summing $t_{bb}^u$ standard deviations to the average and the lower band is obtained subtracting $t_{bb}^l$ standard deviations to the average. We have computed these technical indicators with MATLAB software, using well-know functions included into the Financial Toolbox: tsmovavg for MA, rsindex for RSI, macd for MACD, and bolling for BB. We have appropriately modified the last two listed functions for including variable inputs (which are fixed to the standard values in the original functional version) for the purpose of our investigation.

2.1.2 Trading rules

MA The original MA trading rule compares the moving average and the price in order to produce a trading signal (Lee and Mathur 1996; Lee et al. 2001; Dunis and Miao 2004; Wang et al. 2014; Ozturk et al. 2016; Marshall et al. 2017; Worasucheep et al. 2017; Gerritsen et al. 2020):

$$\begin{aligned} s_{ma}(t)={\left\{ \begin{array}{ll} -\,1&{} \text {if}\quad ma(t) \ge P(t)\\ +\,1&{} \text {if}\quad ma(t) \le P(t).\\ \end{array}\right. } \end{aligned}$$

(2)

The rule does not admit the stay-out-of-the market case ($s_{ma}(t)=0$).

The elaborated MA trading rule considers the persistence of the condition in the original MA rule for $d_{ma}$ trading days (Hudson and Urquhart 2021):

$$\begin{aligned} s_{ma}(t)={\left\{ \begin{array}{ll} -\,1&{} \text {if}\quad \sum _{i=0}^{d_{ma}-1} sign(ma(t-i) - P(t-i))=+d_{ma}\\ +\,1&{} \text {if}\quad \sum _{i=0}^{d_{ma}-1} sign(ma(t-i) - P(t-i))=-d_{ma}\\ 0&{} \text {otherwise.} \end{array}\right. } \end{aligned}$$

(3)

In such a way, the 0 case is admitted. It is worth noting that the original rule can be obtained from the elaborated rule by setting $d_{ma}=1$.

RSI The original RSI trading rule compares the RSI with an upper line and a lower line in order to produce the signal (Rodriguez-Gonzalez et al. 2011; Nor and Wickremasinghe 2014; Gold 2015; Ozturk et al. 2016; Worasucheep et al. 2017; Gerritsen et al. 2020)

$$\begin{aligned} s_{rsi}(t)={\left\{ \begin{array}{ll} -\,1&{} \text {if}\quad rsi(t)>t_{rsi}^h \\ +\,1&{} \text {if}\quad rsi(t)<t_{rsi}^l\\ s_{rsi}(t-1) &{}\text {otherwise}\\ \end{array}\right. } \end{aligned}$$

(4)

where the standard values are $t_{rsi}^h=70$ and $t_{rsi}^l=30$. It is worth noting that the stay-out-from-the-market position ($s_{rsi}(t)=0$) is not defined in this standard trading rule.

Our generalized trading rule based on RSI duplicates the upper line $t_{rsi}^h$ into two different lines, that are $t_{rsi}^{h,ex_b}$ for exiting from buy and $t_{rsi}^{h,en_s}$ for entering in sell, and analogously duplicates the lower line $t_{rsi}^l$ into two separate line, that are $t_{rsi}^{l,en_b}$ for entering in buy and $t_{rsi}^{l,ex_s}$ for exiting from sell, such that also the stay-out-from-the-market position ($s_{rsi}(t)=0$) is defined:

$$\begin{aligned} s_{rsi}(t)={\left\{ \begin{array}{ll} -\,1&{} \text {if}\quad rsi(t)>t_{rsi}^{h,en_s} \wedge \Bigl ( s_{rsi}(t-1)=0 \vee s_{rsi}(t-1)=+1 \Bigr ) \\ -\,1&{} \text {if}\quad rsi(t)>t_{rsi}^{l,ex_s} \wedge s_{rsi}(t-1)=-1 \\ +\,1&{} \text {if}\quad rsi(t)<t_{rsi}^{l,en_b} \wedge \Bigl ( s_{rsi}(t-1)=0 \vee s_{rsi}(t-1)=-1 \Bigr ) \\ +\,1&{} \text {if}\quad rsi(t)<t_{rsi}^{h,ex_b} \wedge s_{rsi}(t-1)=+1 \\ 0 &{}\text {otherwise}.\\ \end{array}\right. } \end{aligned}$$

(5)

The original RSI rule can be obtained by setting $t_{rsi}^{h,en_s}=t_{rsi}^{h,ex_b}$ that is $t_{rsi}^h$ and setting $t_{rsi}^{l,en_b}=t_{rsi}^{l,ex_s}$ that is $t_{rsi}^l$.

MACD The original MACD trading rule compares the MACD line and the signal line in order to produce the trading signal (Ozturk et al. 2016; Vezeris et al. 2018):

$$\begin{aligned} s_{macd}(t)={\left\{ \begin{array}{ll} -\,1&{} \text {if}\quad macd(t) \le signal_{macd}(t)\\ +\,1&{} \text {if}\quad macd(t) \ge signal_{macd}(t),\\ \end{array}\right. } \end{aligned}$$

(6)

in such a way that the stay-out-of-the market position is not admitted ($s_{macd}(t)=0$).

We elaborate the MACD rule in the same fashion of MA rule, by considering the persistence of the condition for $d_{ma}$ trading days:

$$\begin{aligned} s_{macd}(t)={\left\{ \begin{array}{ll} -\,1&{} \text {if}\quad \sum _{i=0}^{d_{macd}-1} sign(macd(t-i) - signal_{macd}(t-i))=-d_{macd}\\ +\,1&{} \text {if}\quad \sum _{i=0}^{d_{macd}-1} sign(macd(t-i) - signal_{macd}(t-i))=+d_{macd}\\ 0&{} \text {otherwise.} \end{array}\right. } \end{aligned}$$

(7)

In our elaborated trading rule, the 0 case is admitted. The original rule can be obtained from the elaborated rule by setting $d_{macd}=1$.

BB The original BB trading rule we consider is the one described in Yan et al. (2017):

$$\begin{aligned} s_{bb}(t)={\left\{ \begin{array}{ll} -\,1&{} \text {if}\quad P(t)>bb^{u}(t) \wedge \Bigl ( s_{bb}(t-1)=0 \vee s_{bb}(t-1)=+1 \Bigr )\\ -\,1&{} \text {if}\quad P(t)>bb^{c}(t) \wedge s_{bb}(t-1)=-1 \\ +\,1&{} \text {if}\quad P(t)<bb^{l}(t) \wedge \Bigl ( s_{bb}(t-1)=0 \vee s_{bb}(t-1)=-1 \Bigr )\\ +\,1&{} \text {if}\quad P(t)<bb^{c}(t) \wedge s_{bb}(t-1)=+1 \\ 0 &{}\text {otherwise}\\ \end{array}\right. } \end{aligned}$$

(8)

where $s_{bb}(t)$ is the trading signal at time t, $bb^u$ is the upper band, typically 2 standard deviations above the average, $bb^l$ is the lower band, typically 2 standard deviations under the average, and $bb^c$ is the central line, typically the average line.

Our revisited BB trading rule duplicates the central line into two different lines, $bb^{c,ex_b}$ for exiting from buy and $bb^{c,ex_s}$ for exiting from sell:

$$\begin{aligned} s_{bb}(t)={\left\{ \begin{array}{ll} -\,1&{} \text {if}\quad P(t)>bb^{u}(t) \wedge \Bigl ( s_{bb}(t-1)=0 \vee s_{bb}(t-1)=+1 \Bigr ) \\ -\,1&{} \text {if}\quad P(t)>bb^{c,ex_s}(t) \wedge s_{bb}(t-1)=-1 \\ +\,1&{} \text {if}\quad P(t)<bb^{l}(t) \wedge \Bigl ( s_{bb}(t-1)=0 \vee s_{bb}(t-1)=-1 \Bigr )\\ +\,1&{} \text {if}\quad P(t)<bb^{c,ex_b}(t) \wedge s_{bb}(t-1)=+1 \\ 0 &{}\text {otherwise.}\\ \end{array}\right. } \end{aligned}$$

(9)

2.1.3 Signal aggregation

The signals deriving from the four indicators’ trading rules are aggregated into a single signal. The aggregation procedure consists of two steps: 1. the computation of the weighted average of the four signals through the indicators weights $\theta _{ma}, \theta _{rsi}, \theta _{macd}, \theta _{bb}$; 2. the conversion of the weigthed average into a buying signal, a selling signal or a staying out signal through the threshold $t_{as}^b$ and $t_{as}^s$.

The weighted average of the four signal, $\tilde{s}(t)$, is

$$\begin{aligned} \tilde{s}(t)=\theta _{ma}s_{ma}(t)+\theta _{rsi}s_{rsi}(t)+\theta _{macd}s_{macd}(t)+\theta _{bb}s_{bb}(t). \end{aligned}$$

(10)

The aggregated signal, s(t), is

$$\begin{aligned} s(t)={\left\{ \begin{array}{ll} {-\,1}&{} \text {if}\quad \tilde{s}(t){<} t_{as}^s\\ {+\,1}&{} \text {if}\quad \tilde{s}(t){>} t_{as}^b\\ 0 &{}\text {otherwise}. \end{array}\right. } \end{aligned}$$

(11)

2.2 The constrained optimization of net capital

Among the several performance measures for financial investments, we consider the fitness function $\rho$: the net capital at the end of the trading period T, which is an absolute measure of wealth. For its computation, we need to determine the daily return and the capital at the end of each trading day.

The realized return of the trading system in a given day, e(t), depends on the previous strategy, the stock price increase, and the trading fee due to a strategy change:

$$\begin{aligned} e(t)=s(t-1)\ln \Bigl (\frac{P(t)}{P(t-1)}\Bigr )-\delta |s(t)-s(t-1)|,\;t=2,\dots ,T. \end{aligned}$$

(12)

The net capital at the end of the trading day is the result of compounding the capital at the previous day with the realized return of the trading system:

$$\begin{aligned} c(t)=c(t-1) \cdot \bigl (1+e(t)\bigr ),\;t=2,\dots ,T. \end{aligned}$$

(13)

The evolution of capital through time is represented in the so-called equity line. Our attention is to the final capital amount, that is c(T), which corresponds to the last point of the equity line.

The maximization of c(T) is equivalent to the maximization of the overall return, which is the difference between the final capital and the initial capital, $c(T)-c(start)$, where start is the first day in which all the technical indicators can be computed, depending on the time windows required by their definitions.

The return on yearly basis (so called, annualized return) is:

$$\begin{aligned} \bar{e}=\Bigl (\frac{c(T)}{c(start)}\Bigr )^{\frac{252}{T-start}}-1. \end{aligned}$$

(14)

The net Sharpe ratio (Sharpe 1966) is a measure of excess return, that is the return on yearly basis minus the risk free rate $r_f$, over the standard deviation of the returns on yearly basis, $\sigma _e$, computed from $t=start+1$ until $t=T$¹:

$$\begin{aligned} SR(T)=\frac{\bar{e}-r_f}{\sigma _e}. \end{aligned}$$

(15)

Therefore, the fitness function to be maximized is the capital at the end of the trading period, $\rho =c(T)$, under several constraints related to the parameters of the system. Equivalently, the problems can be rewritten as the constrained maximization of $\rho$:

$$\begin{aligned} \begin{matrix} \displaystyle \max _{\chi \in X} \rho &{}\\ \text {s.t.}\; {\left\{ \begin{array}{ll} w_{ma},d_{ma}, w_{rsi},w_{macd}^{short},w_{macd}^{long}, w_{macd}^{signal},d_{macd}, w_{bb}^{ma},w_{bb}^{std}\in \mathbb {N}^+\\ t_{rsi}^{l,en_b}\ge 0 \\ t_{rsi}^{h,en_s}\le 100 \\ t_{rsi}^{l,en_b} \le t_{rsi}^{l,ex_s} \le t_{rsi}^{h,en_s}\\ t_{rsi}^{l,en_b} \le t_{rsi}^{h,ex_b} \le t_{rsi}^{h,en_s}\\ w_{macd}^{long}> w_{macd}^{short} \\ w_{bb}^{std}\ge 2 \\ t_{bb}^u, t_{bb}^l \ge 0\\ -t_{bb}^{l} \le t_{bb}^{c,ex_s} \le t_{bb}^{u}\\ -t_{bb}^{l} \le t_{bb}^{c,ex_b} \le t_{bb}^{u}\\ \theta _{ma},\theta _{rsi},\theta _{macd},\theta _{bb}\ge 0\\ \theta _{ma}+\theta _{rsi}+\theta _{macd}+\theta _{bb}=1 \\ t_{as}^b > t_{as}^s. \\ \end{array}\right. }&\end{matrix} \end{aligned}$$

(16)

It is worth noting that the constraint system of problem (16) also describes the allowed domain of each parameter: for instance, the time windows have as their domain the set of positive naturals, the lowest threshold for RSI $t_{rsi}^{l,en_b}$ is a real number higher than or equal to 0, the highest threshold for RSI $t_{rsi}^{h,en_s}$ is a real number lower than or equal to 100, and so on.

The problem is formulated in terms of mixed-integer variables and it is nonlinear and nondifferentiable. Due to these complexities, exact solution algorithms both effective and efficient are still sought in literature and we need to use an approximate solution method, namely the PSO.

It is noteworthy to emphasize that since the late 1990s metaheuristics have been recognized as powerful and adaptable optimization solvers, although approximate, able $\ll$to attack complex optimization problems where classical heuristics and optimization methods have failed to be effective and efficient [$\ldots$] by combining intelligently different concepts for exploring and exploiting the search space [$\ldots$] in order to find efficiently near-optimal solutions.$\gg$ (Osman and Laporte 1996, pp. 513–514).

Of course, PSO is not the only bio-inspired metaheuristics able to deal with optimization problems like ours. In this regard, Hussain et al. (2019) performed a comprehensive survey of metaheuristic research in literature which consists of 1222 publications from year 1983 to 2016. According to the number of publications on the various techniques, they found out that $\ll$among other metaheuristic methods, PSO was the most attractive technique. There seems significant distinction between PSO and the rest of the methods. PSO has gained immense popularity amongst researchers due to simplicity and effectiveness in plenty of scientific and industrial applications$\gg$. In their comprehensive survey, they also tried to determine the validating techniques used to investigate performance of metaheuristics: benchmark test functions have been commonly used and some benchmark engineering design problems have also been solved by using metaheuristic methods to measure and compare performances.

Anyway, as possible alternative to PSO we can consider Genetic Algorithms (GAs), that can be considered an unquestioned benchmark in the field of evolutionary population-based metaheuristics. The advantages of PSO with respect to GAs have resulted in several works, such as Hassan et al. (2005) and Corazza et al. (2012).

Hassan et al. (2005) compared the performance, in terms of effectiveness (finding the true global optimal solution) and computational efficiency (less function evaluations), of the PSO and the GA using a set of benchmark test problems, that are the Banana (Rosenbrock) Function, the Eggcrate Function, and Golinski’s Speed Reducer, as well as two space systems design optimization problems, namely, telescope array configuration and spacecraft reliability-based design. The test problems represented a wide range of complexity, nonlinearity, and constraint levels. They implemented statistical analysis and formal hypothesis testing. The results of the statistical tests supported the hypothesis that while both PSO and GA obtain high quality solutions (with quality indices of $99\%$ or more with a $99\%$ confidence level for most test problems), the computational effort required by PSO to arrive to such high quality solutions is less than the effort required to arrive at the same high quality solutions by the GA. In particular, the results showed that the computational efficiency superiority of PSO over the GA is statically proven with a $99\%$ confidence level in 7 out of 8 test problems investigated.

Corazza et al. (2012) have applied PSO to (approximately) solve optimization financial problems, and its performances have been compared with those from the application of standard GAs. The results of the comparison in Corazza et al. (2012) have shown that the two metaheuristics are more or less equivalent, both in terms of fitness function values and of risk measure values, but the average computational time required by GAs is about one order magnitude greater than that required by PSO (see Table 3 therein). These results motivated our choice for a PSO-based approach, in this paper.

2.3 Application of Particle Swarm Optimization

PSO is an iterative metaheuristic suitable to solve global unconstrained continuous optimization problems. Differently, our problem is global constrained mixed-integer. For this reason, the next subsection describes the basics on standard PSO and the following one considers the specificities of our problem.

2.3.1 Standard PSO

PSO metaheuristic replicates the social behaviour of a swarm of animals collaborating in the search for food. Each member (particle) of the swarm explores the space with memory of its personal best position so far and communicates this information with its neighbors in the swarm. Therefore, the whole swarm tends to converge towards the best global position of its particles, that is the best of the personal best positions.

In mathematical terms, we consider the global minimization problem $\min _{\textbf{x}\in \mathbb {R}^d} f(\textbf{x})$, where $f(\textbf{x}):\mathbb {R}^d\rightarrow \mathbb {R}$ is the objective function. We apply PSO for its solution, using M particles and niter iterations. Every particle is a possible solution of the minimization problem and is initially positioned randomly with the attribution of the random position $\mathbf {x^1_j}$ and random velocity $\mathbf {v^1_j}$, which represents its initial direction of movement. At the k-th iteration of the metaheuristic, the j-th particle (with $j=1,\dots ,M$) is associated with the following three vectors:

• the position $\textbf{x}^k_j\in \mathbb {R}^d$,

• the velocity $\textbf{v}^k_j\in \mathbb {R}^d$,

• the personal best position so far $\textbf{p}^k_j\in \mathbb {R}^d$.

Let $pbest_j^k=f(\textbf{p}^k_j)$ be the value of the objective function in position $\textbf{p}^k_j$ of the j-th particle and let us to denote the best global position reached by the whole set of particles of the swarm as $\textbf{p}^k_g$.

The PSO algorithm in the widely used version with inertia weights (also assumed in this paper) consists of the following 5 steps:

• 1. Set $pbest^0_j=+\infty$ for $j=1,\dots ,M.$

• 2. Set $k=1$ and evaluate $f(\textbf{x}^1_j)$ for $j=1,\dots ,M.$

• 3. If $f(\textbf{x}^k_j)<pbest^{k-1}_j$ then set $\textbf{p}^k_j=\textbf{x}^k_j$ and $pbest^k_j=f(\textbf{x}^k_j)$. Else $\textbf{p}^k_j=\textbf{p}^{k-1}_j$ and $pbest^k_j=pbest^{k-1}_j$.

• 4. Update the position and velocity of the j-th particle with $j=1,\dots ,M$. Two main alternative versions are considered in literature, a random one (Kennedy and Eberhart 1995) and a deterministic one (Clerc 1999; Clerc and Kennedy 2002). As for the random version,

  $$\begin{aligned} {\left\{ \begin{array}{ll} &{}\textbf{v}_j^{k+1}=w^{k+1}\textbf{v}^k_j+\textbf{U}_{\phi _1}\otimes (\textbf{p}^k_j-\textbf{x}_j^k)+\textbf{U}_{\phi _2}\otimes (\textbf{p}^k_g-\textbf{x}_j^k)\\ &{} \textbf{x}_j^{k+1}=\textbf{x}_j^{k}+\textbf{v}^{k+1}_j \\ \end{array}\right. } \end{aligned}$$

  (17)

  where $\textbf{U}_{\phi _1},\textbf{U}_{\phi _2}\in \mathbb {R}^d$ and their components are uniformly randomly distributed in $[0,\phi _1]$ and $[0,\phi _2]$ respectively, and $\otimes$ is the component-wise product. As for the (alternative) deterministic version,

  $$\begin{aligned} {\left\{ \begin{array}{ll} &{}\textbf{v}_j^{k+1}=w^{k+1}\textbf{v}^k_j+{\phi _1}\cdot (\textbf{p}^k_j-\textbf{x}_j^k)+{\phi _2}\cdot (\textbf{p}^k_g-\textbf{x}_j^k)\\ &{} \textbf{x}_j^{k+1}=\textbf{x}_j^{k}+\textbf{v}^{k+1}_j. \\ \end{array}\right. } \end{aligned}$$

  (18)

• 5. Increase k by a unity and go back to step 3.

The values of $\phi _1$ and $\phi _2$ affect the attractive forces towards the personal and the swarm best positions explored so far. A typical standard value tested in literature of $\phi _1$ and $\phi _2$ is 1.49618, therefore we assume this value. The inertia weight $w^k$ affects the influence of previous velocity $\textbf{v}^k_j$ on the updated velocity and position. Two main alternative versions of the parameter $w^k$ are defined in literature, a linearly decreasing version with the number of iterations and a constant version. As for the decreasing version,

$$\begin{aligned} w^k=w_{max}+\frac{w_{min}-w_{max}}{K}k \end{aligned}$$

(19)

where standard values for $w_{max}$ and $w_{min}$ are respectively 0.9 and 0.4 and K is usually equal to the maximum number of iterations niter (Corazza et al. 2017). As for the constant version,

$$\begin{aligned} w^k=w, \quad \text {for all } k=1,\dots ,niter \end{aligned}$$

(20)

where the constant inertia weight w is usually assumed equal to 0.7298, therefore we use this value.

2.3.2 PSO with the constrained mixed-integer problem

Standard PSO is a solver for global unconstrained problems, whereas our optimization problem is a global constrained mixed-integer one. Consequently, we appropriately adapt the standard PSO for managing these specificities.

For dealing with integer variables, we follow a widespread approach in literature, that is the truncation method proposed in Laskari et al. (2002) and Parsopoulos and Vrahatis (2002) (see also Gaing 2005; dos Santos Coelho 2009; Saber and Venayagamoorthy 2009; Wu and Tsai 2011; Corazza et al. 2017), according to which $\ll$each particle of the swarm [is] truncated to the closest integer, after the determination of its new position$\gg$ (Laskari et al. 2002, p. 1584). In such a way, we implement the integer constraints $w_{ma},d_{ma}, w_{rsi},w_{macd}^{short},w_{macd}^{long}, w_{macd}^{signal},d_{macd}, w_{bb}^{ma},w_{bb}^{std}\in \mathbb {N}^+$.

For dealing with the other constraints, we reformulate our problem as an unconstrained one using the exact penalty method described in Fletcher 2000 and more recently applied in the financial context in Corazza et al. (2013, 2017). This method permits a correspondence between the optimizer of the original constrained problem and the unconstrained penalized one, in such a way that the original PSO intent, that is the solution of unconstrained optimization problems, is preserved.² The reformulated unconstrained (penalized) version of the optimization problem is the maximization of the following function $\hat{\rho }$ with penalty parameter $\epsilon$:

$$\begin{aligned} {\begin{matrix} \max _{\chi \in X} \hat{\rho }=&{}\, \rho -\frac{1}{\epsilon }\Bigl [ \max (0, -t_{rsi}^{l,en_b})+\max (0, t_{rsi}^{h,en_s}- 100)+\max (0, t_{rsi}^{l,en_b} - t_{rsi}^{l,ex_s})\\ &{}+\max (0, t_{rsi}^{l,ex_s} - t_{rsi}^{h,en_s}) +\max (0, t_{rsi}^{l,en_b} - t_{rsi}^{h,ex_b}) +\max (0, t_{rsi}^{h,ex_b} - t_{rsi}^{h,en_s}) \\ &{}+\max (0, -w_{macd}^{long}+w_{macd}^{short}) +\max (0, -w_{bb}^{std}+ 2) +\max (0,-t_{bb}^u)\\ &{}+\max (0,-t_{bb}^l)+\max (0,-t_{bb}^{l}-t_{bb}^{c,ex_s}) +\max (0, t_{bb}^{c,ex_s} - t_{bb}^{u}) \\ &{}+\max (0,-t_{bb}^{l}-t_{bb}^{c,ex_b})+\max (0, t_{bb}^{c,ex_b} - t_{bb}^{u})+\max (0,-\theta _{ma})\\ &{}+\max (0,-\theta _{rsi})+\max (0,-\theta _{macd})+\max (0,-\theta _{bb})\\ &{}+|\theta _{ma}+\theta _{rsi}+\theta _{macd}+\theta _{bb}-1|+\max (0,-t_{as}^b + t_{as}^s)\Bigr ]. \\ \end{matrix}} \end{aligned}$$

(21)

The correspondence between the solutions of the original constrained problem and of the reformulated unconstrained one results from the appropriate choices of $\epsilon$. In our study we assume $\epsilon =10^{-2}$. In Eq. (21), the constraints $w_{ma},d_{ma}, w_{rsi},w_{macd}^{short},w_{macd}^{long}, w_{macd}^{signal},d_{macd}, w_{bb}^{ma},w_{bb}^{std}\in \mathbb {N}^+$ do not appear since they are dealt with the above described truncation approach for integer variables.

3 Applications

This paper considers the optimization of a simple trading system based on four technical-analysis indicators. More precisely, the standard settings of the 23 parameters (see Table 1, fifth column) are optimized via the application of PSO metaheuristic described in Sect. 2.3. The present section compares both in-sample and out-of-sample results of our optimized system with a benchmark given by the same trading system with standard settings.

Our applications consider the Italian financial market. We focus on the closing prices of the FTSE MIB index and a set of selected stocks belonging to the FTSE MIB at the date of May 31, 2022, and traded on the market starting before January 2, 2007.³ We select five sectors, that are highly representative of the Italian economy, picking the stock with the highest capitalization in its sector. As results of this selection procedure, we apply our methodology to the following stocks belonging to the following sectors:

Banks::

Intesa Sanpaolo S.p.A.

Industrial products and services::

Atlantia S.p.A.

Insurance::

Assicurazioni Generali S.p.A.

Oil and natural gas::

Eni S.p.A.

Public services::

Enel S.p.A.

In our analysis, we use the percentage transaction cost applied by several Italian brokers $\delta =0.15\%$ and the initial capital value $C(1)=100$.

3.1 Initialization of particles

The initialization of particles is the identification of $\textbf{x}^1_j \text { for } j=1,\dots ,M$, such that the PSO procedure can start.

The PSO metaheuristic generates the initial value of the parameters according to uniform distributions in appropriate intervals. This initialization lets the parameters initially assume quite distant values from the standard ones, therefore fostering the exploration of the parameter space.

3.2 PSO settings

The PSO settings include decisions about

• the decreasing or constant nature of inertia weight (see the alternative Eqs. (19), (20) respectively);

• the random or deterministic nature of velocity for updating the particles’ positions (see the alternative Eqs. (17), (18) respectively);

• the number of particles M into the swarm;

• the number of iterations niter of the PSO metaheuristic;

• the number of iterations without improvement of the fitness function $niter_{woi}$, after which we force the PSO interruption.

As an innovative element of our study, we tested all four the different sets resulting from the combinations of options about the inertia weight and the updating velocity. Then, we chose the best performing one (results of the comparison are available upon requests). This goes further the typical applications in literature where only one set over four is considered, without justifying its preference over the others (Briza and Naval 2011 used constant inertia and random velocity; Wang et al. (2014) and Corazza et al. (2017) applied decreasing inertia and random velocity; Worasucheep et al. (2017) used decreasing inertia, while the nature of velocity is not specified).

Furthermore, we carried out investigations via grid search for determining the best hyperparametrization, namely the optimal value of the penalty parameter, number of particles, number of iterations, and number of iterations without improvement. The best performing setting in our analysis is based on decreasing inertia weight and deterministic velocity with $M=75, niter=500, niter_{woi}=5$.⁴

It is worth noting that we do not intend to seek an ad hoc hyperparametrization of the trading system for each individual stock so as to achieve the best possible fine tuning on an asset-by-asset basis. Rather, we intend to identify a single generalist hyperparametrization that is on average good for all stocks and for both in-sample and out-of-sample analysis.

The next sections describe the results of our in-sample and out-of-sample applications, both computed over the time period from January 2, 2007 to May 31, 2022. We have analyzed the annualized return $\bar{e}$, the equity line, and the net final Sharpe ratio SR(T).

3.3 In sample analysis

In this section, we present and compare the in-sample performances achieved by the trading system optimized using the metaheuristic PSO with those of the same trading system obtained using the standard values of the parameters reported in the fifth column of Table 1. It is worth noting that the performances of the latter trading system serve as the benchmark. Moreover, we remember that the metaheuristic PSO we use is stochastic due to the random initialization of particle positions and velocities. For this reason, we applied our approach to each stock 10 times (i.e., 10 iterations), and then we chose the optimal quantities of interest as those associated to the iteration with the highest fitness value (i.e., the best iteration).

Table 2 reports the in-sample performances of both trading systems: in the first column, the stock is identified, in the second column the annualized return of the standard system $\bar{e}_{st}$ is shown (that obviously does not depend on the iteration), in column 3 we display the annualized return of the optimized system, $\bar{e}_{PSO}$. In column 4 we compare the equity line of the optimized system with the equity line of the standard system: we show the percentage of days in the trading period in which the equity line of the optimized system is higher than the equity line of the standard system, denoted with the symbol $\overline{\%>}_c$. In column 5 we report the net final Sharpe ratio of the standard system $SR(T)_{st}$ and in column 6 we show the net final Sharpe ratio of the optimized system, $SR(T)_{PSO}$. In column 7, the value of the fitness function is shown. For clarity, we recall that the reported performances of the optimized system are those related to the best iteration.

Table 2 In-sample performances of the optimized and the standard trading systems for each asset

Noteworthily, the optimized system performs better, that is, realizes higher annualized returns, than the standardly set system 5 times out of 6, i.e., in 83.33% of cases. More precisely:

• The optimized system overperforms the standard one for all the five selected stocks. In particular, the annualized return of the optimized system for the five stocks is always positive, whereas the corresponding annualized return coming from the standard system is always negative. Intesa Sanpaolo is the best performing stock with the optimized system (with an annualized return equal to 16.34%), while it is the worst performing stock with the standard system (with an annualized return equal to − 5.75%). Figure 2 shows the evolution of the stock price of Intesa Sanpaolo, the trading signals generated by the optimized system, and the equity lines, for both the optimized system and the standard one;

• The optimized system underperforms the standard one only for the market index FTSE MIB (see Fig. 3); in any case, the annualized return for the optimized system is positive, equal to 0.32%, not distant from the annualized return obtained from the standard system, equal to 0.51%; furthermore, the net final Sharpe ratio of the optimized system is higher than the corresponding one of the standard system;

• In 3 out of 5 winning cases (Enel, Eni, and Intesa Sanpaolo) the equity lines coming from the optimized system are above the corresponding equity lines of the standardly set system in 50% or more of the trading days and, in any case, mainly in the second part of the trading period. This behavior could be explained by the important crises occurred during the second part of the trading period (e.g., the Covid years and the Russian-Ukrainian conflict). During this second part, the market manifested much greater volatility and price fluctuations than in the past, entailing significant opportunities for gains. Hence, the optimized trading system has seized the large profit possibilities in the second part of the trading period, even at the expense of the gains achievable in the first part.

Fig. 2

Intesa Sanpaolo is the best performing stock with the optimized trading system for the in-sample analysis. In the upper panel, the end-of-day price is represented; in the middle panel, the actions undertaken by the system are shown; in the lower panel, the gross and net equity lines produced by the optimized system (the blue and the red ones, respectively, with the gross equity line always over the net one) are compared with the net equity line obtained by the standard system (the black one) (colour figure online)

Fig. 3

FTSE MIB is the worst performing asset with the optimized trading system for the in-sample analysis. In the figure, the gross and net equity lines obtained by the optimized system (the blue and the red ones, respectively, with the gross equity line always over the net one) are compared with the net equity line obtained by the standard system (the black one) (colour figure online)

Table 3 reports the parameter values of the in-sample optimization for each asset. It is worth noting that:

• Some parameter values are highly variable across assets: for instance, in the second column, the time window for computing the moving average $w_{ma}$ ranges from 3 days for Assicurazioni Generali to 38 days for Intesa Sanpaolo and, in the thirteenth column, the time window for calculating the moving average for the BB $w_{bb}^{ma}$ varies from 7 days for Intesa Sanpaolo to 51 days for Assicurazioni Generali. These differences could signal the need for ad hoc parameterizations for the trading system of the various assets, at least for some parameters;

• Some weights for signal aggregation are negative, specifically $\theta _{bb}$ for Atlantia and Eni and $\theta _{ma}$ for Intesa Sanpaolo, even though they were constrained to belong to the interval [0, 1]. These violations should not be surprising, since we are solving a non-smooth reformulation of the original constrained optimization problem (therefore, not the original one) and, in addition, PSO metaheuristic is an approximate method that does not ensure to provide the exact solution. Rather, these constraint violations could be an indication of the inherent complexity of the original optimization problem;

• The average weights for signal aggregation over the 6 assets are: average $\theta _{ma}=0.067$, average $\theta _{rsi}=0.598$ average $\theta _{macd}=0.212$, and average $\theta _{bb}=0.123$. These values highlight how, on average, the system evaluates RSI to be the most important of all four indicators, followed by MACD, BB, and lastly MA;

• The threshold for the aggregated sell signal for entering for Assicurazioni Generali is $t_{as}^s=-2.019$. Recalling that the minimum value of the aggregated signal is $-\,1$, reached just in case of perfect concordance of the four selling signals by MA, RSI, MACD, BB, it indicates that the trading system for Assicurazioni Generali is optimized by excluding the selling decisions. In other words, the buying decisions and holding decisions are the only ones performed for Assicurazioni Generali; the inclusion of selling operations would worsen the system performance.

Table 3 Parameters of the in-sample optimized trading system for each asset

3.4 Out-of-sample analysis

For the out-of-sample analysis, we consider the same trading period as taken into account in the in sample investigation.⁵ In order to test the capability of the trading system optimized by the metaheuristic PSO, we have conducted three different out-of-sample experiments. The structure of each of these experiments is similar. First, the trading period is divided into two subperiods, that is a training period and an out-of-sample testing one, but in each experiment a different length for the testing period is considered: 1 stock-month, 2 stock-months, and 3 stock-months, respectively; in Table 4, we provide the start and end dates for each in-sample and out-of-sample subperiod. Then, the trading system is optimized using the metaheuristic PSO over the training subperiod, and subsequently applied to the out-of-sample testing subperiod.

Our out-of-sample analysis is repeated 100 times; in this manner, for each of the three experiments, we obtain 100 realizations of the quantity to be optimized, that is the final net capital (see equation 13, where $t=T$), as well as of the other quantities of interest. Finally, we calculate the average value of each of these quantities of interest over its respective sample. In doing so, we confer a degree of statistical significance to the results, at least to some extent.

Table 4 Start and end dates for each in-sample and out-of-sample subperiod

The average results are collected in Tables 5, 6 and 7. The structures of these tables are similar to that of Table 2; in particular, the second column of the tables contains the annualized returns of the standard system (which obviously are independent from the PSO repetition) and the third column contains the mean annualized returns of the optimized system, the fourth column reports the mean percentages of days in which the equity line of the optimized system is higher than the equity line of the standard system, and the last two columns show the mean values of the net final Sharpe ratios. The mean values are computed over the 100 repetitions.

Table 5 Average out-of-sample performance of the optimized trading system over 1 month for 100 repetitions for each asset

Table 6 Average out-of-sample performance of the optimized trading system over 2 months for 100 repetitions for each asset

Table 7 Average out-of-sample performance of the optimized trading system over 3 months for 100 repetitions for each asset

It is worth highlighting that:

• When considering the 1-month long out-of-sample period, the optimized system wins 5 times out of 6 compared to the standard system, that is the same performance of the in-sample analysis; the performances slowly degrade as the length of the out-of-sample period increases, since the optimized system wins 4 times with the 2-month long out-of-sample period and 3 times with the 3-month long out-of-sample period. These results tend to confirm the hypothesis that stock markets are characterized by short-term memory. Consequently, our trading system would need to be re-optimized with appropriate frequency;

• Whatever the length of the out-of-sample period, when the optimized system loses against the standard system (for Intesa Sanpaolo with the 1-month long out-of-sample period; for Intesa Sanpaolo and FTSE MIB with the 2-month long out-of-sample period; for Assicurazioni Generali, Atlantia, and Intesa Sanpaolo with the 3-month long out-of-sample period), the mean value of the annualized return obtained by the optimized system is always positive, except in one case where it is close to zero. This could indicate that the PSO metaheuristic works well in the optimization phase also in these cases, but paying for the choice of using a unique hyper-parametrization for all stocks, for both in-sample and out-of-sample analysis;

• The net equity lines coming from the optimized system are above or equal to the corresponding net equity lines of the standard system in more than 50% of the trading days for 5 assets with the 1-month long out-of-sample period and for 3 assets with the 2- and 3-month out-of-sample periods; this means that the maximization of the final net capital is rather consistent with the maximization of the capital over most of the trading period;

• In all cases where the optimized system wins over the standard system, the final net Sharpe ratio reflects well the performances as measured by the average annualized return, that is, the Sharpe ratios obtained by the optimized system are greater than the corresponding Sharpe ratios of the standard system (except for Assicurazioni Generali with the 2-month long out-of-sample period and FTSE MIB with the 3-month long out-of-sample period); therefore, the optimized trading system allows an increase in the annualized return more than proportional to the increase in the riskiness of the strategy.

3.5 Robustness checks for the out-of-sample results

In Tables 5, 6 and 7, we have presented the values of the quantities of interest, calculated as averages over their respective samples, to provide a certain degree of statistical significance to the results, at least to some extent. In this section, we delve deeper into the robustness of the out-of-sample results displayed in Sect. 3.4. Our robustness check consists of two following steps.

In the first step, for each of the three experiments we check the capability of the optimized trading system to correctly classify the trend of the price of each stock per each day of the out-of-sample subperiod. To this end, for each experiment and for each stock, we calculate the confusion matrix to verify the stock trading signal prediction against the actual stock price trend. It is noteworthy that, while the stock trading signal can be $-\,1$ or 0 or $+\,1$, the stock price trend can be only $-\,1$ or $+\,1$. Therefore, special attention has been given to the management of the trading signal 0, that is the stay-out-of-the-market position. Specifically, we have chosen to assume a prudential investor profile, so to classify the trading signal prediction 0 as concordant with the actual stock trend when the latter is $-\,1$ (case of avoided loss), and to classify the trading signal prediction 0 as concordant with the actual stock trend when the latter is $+\,1$ (case of foregone earnings).

Once the confusion matrix is computed, we evaluate it by the use of a well established set of indicators, namely:

$$\begin{aligned} {\begin{aligned}&Accuracy \; (\textrm{ACC}) = \frac{TP + TN}{N}, \\&Precision \; (\textrm{PRE}) = \frac{TP}{TP+FP}, \\&Recall \; (\textrm{REC}) = \frac{TP}{TP+FN}, \\&F1 \; score \; (\textrm{F1s}) = 2\frac{PRE \cdot REC}{PRE + REC}, \\&Specificity \; (\textrm{SPE}) = \frac{TN}{TN+FP}, \\&False \; Positive \; rate \; (\textrm{FPr}) = \frac{FP}{FP+TN}, \\&False \; Negative \; rate \; (\textrm{FNr}) = \frac{FN}{FN+TN}, \\ \end{aligned}} \end{aligned}$$

where TP and TN respectively indicate the total number of true positive and true negative trading signal predictions, N represents the total number of the same predictions, and FP and FN respectively indicate the total number of false positive and false negative trading signal predictions.

Similarly to what was done in Sect. 3.4, this robustness investigation is also repeated 100 times for each of the three experiments. In this way, for each of the latter, we obtain 100 realizations of the aforementioned confusion matrix and evaluation indicators, which we use to calculate their average values. The average values of the evaluation indicators are reported in Table 8, whose last column also shows the medium number of operations (i.e., trades) $\overline{O}$ in the testing period, alway over the 100 repetitions.

Table 8 Average out-of-sample evaluation indicators of the optimized trading system over 1 month (Experiment 1), 2 months (Experiment 2), 3 months (Experiment 3) for 100 repetition for each asset

In the second step of our robustness check, we proceed by drawing inspiration from what is sustained by Zakamulin and Giner (2020) and Zakamulin and Giner (2022), who suggest that the dynamics of stock returns can be effectively modeled through autoregressive processes. Specifically, for each experiment and for each stock returns time series, we consider an ARIMA process, and identify its orders and estimate the coefficients over the training subperiod. Then, we apply the ARIMA process estimated in this way to the out-of-sample testing subperiod to calculate, once again over 100 realizations as above, the corresponding average confusion matrix and average evaluation indicators; the average values of these indicators are displayed in Table 9. Finally, we use these values as benchmarks for the average evaluation indicators displayed in Table 8.

Table 9 Average out-of-sample evaluation indicators of the ARIMA process over 1 month (Experiment 1), 2 months (Experiment 2), 3 months (Experiment 3) for 100 repetition for each asset

It is worth mentioning that the ARIMA model identification procedure sometimes suggested us that the data generating process was a random walk. In light of this evidence, we also estimated the confusion matrix assuming that the underlying process was a pure random walk as well. The results (which are not reported here, and are available upon request) are similar to those obtained for the ARIMA models.

As main finding, it is worth noting that for each experiment and for each stock, all the values except for two of the average evaluation indicators associated with the trading system optimized by the metaheuristic PSO are sizeably better than their respective benchmarks.⁶

It is also worth noting that the average number of operations in our optimized system is always lower than the corresponding number of operations in the ARIMA-process-based system, associated with a saving of trading fees.

4 Concluding remarks

This paper proposes a new trading system based on the simultaneous optimization of signal aggregation, trading rule definition, and indicator setting, overcoming the current limitations in the literature according to which only one category of parameters of the trading system is optimized at a time. More precisely, our algorithm uses the Particle Swarm Optimization (PSO) metaheuristic for optimizing the aggregation, the trading rules and the settings of four commonly applied technical indicators, that are the (Exponential) Moving Average, the Relative Strength Index, the Moving Average Convergence/Divergence, and the Bollinger Bands over our fitness function, which is the net capital at the end of the trading period. Our refinement generally leads to superior performance over a traditional technical-analysis trading system, measured for the FTSE MIB and for a set of financial stocks belonging to the FTSE MIB on a multi-year horizon. Such dominance is also confirmed by a two-step robustness procedure that checked the capability of the optimized trading system to correctly classify the trends of the stock prices.

Future research will focus on the use of a multi-objective fitness function and multi-objective PSO for the construction of an efficient risk-return frontier; another potential investigation is the application of the feature selection capability of PSO for automatically selecting the technical indicators and offering more complex trading systems with a higher number of indicators selected from data and not from the analysts’ experience; finally, another possible future direction is the inclusion of fundamental analysis indicators alongside the technical analysis indicators for forecasting future price movements and making value-creating trading decisions.