Augmented arithmetic optimization algorithm using opposite-based learning and lévy flight distribution for global optimization and data clustering

Abstract

This paper proposes a new data clustering method using the advantages of metaheuristic (MH) optimization algorithms. A novel MH optimization algorithm, called arithmetic optimization algorithm (AOA), was proposed to address complex optimization tasks. Math operations inspire the AOA, and it showed significant performance in dealing with different optimization problems. However, the traditional AOA faces some limitations in its search process. Thus, we develop a new variant of the AOA, namely, Augmented AOA (AAOA), integrated with the opposition-based learning (OLB) and Lévy flight (LF) distribution. The main idea of applying OLB and LF is to improve the traditional AOA exploration and exploitation trends in order to find the best clusters. To evaluate the AAOA, we implemented extensive experiments using twenty-three well-known benchmark functions and eight data clustering datasets. We also evaluated the proposed AAOA with extensive comparisons to different optimization algorithms. The outcomes verified the superiority of the AAOA over the traditional AOA and several MH optimization algorithms. Overall, the applications of the LF and OLB have a significant impact on the performance of the conventional AOA.

Introduction

The wide applications of the internet, WEB, and smart devices increase the data and produce critical problems to mine the useful data (Zhou et al., 2019; Ezugwu et al., 2022). Different data mining methods have been developed to tackle these problems using several techniques, including clustering, regression, and classification (Abualigah, 2019; Abualigah et al., 2018). Some of these applications are employed in different area, such as recommendation systems (Schickel-Zuber & Faltings, 2007), text mining (Chen et al., 2020), and computer vision applications (Dhanachandra et al., 2015; Namratha & Prajwala, 2012). Data clustering has received wide attention due to its simple application by collecting items in similar groups depending on their features (Abualigah et al., 2017). This is done by minimizing the distance between these comparable items and their centers. There are two common types of data clustering methods, called portioning and hierarchy. The hierarchical approaches face certain drawbacks with large datasets due to their slow implementations, and they can be considered time-consuming methods. Therefore, partitioning methods have been adopted for data clustering due to their efficiency with large datasets (Saxena et al., 2017; Xu & Wunsch, 2005) Moreover, the most common clustering methods are K-means and fuzzy C-means (FCM). Such methods generate canters to the group items in a random manner. Thus they face major limitations, for example, convergence in the local optima (Jain, 2010; Abualigah & Diabat, 2020).

In this regard, different types of the optimization techniques are applied to control in these algorithms (Abualigah, 2020; Abualigah & Diabat, 2021), such as particle swarm optimization (PSO) (Eberhart & Kennedy, 1995), sine-cosine algorithm (SCA) (Mirjalili, 2016), genetic algorithm (GA) (Holland, 1992), atom search optimization (ASO) (Zhao et al., 2019), artificial bee colony (ABC) (Karaboga & Basturk, 2007), salp swarm algorithm (SSA) (Mirjalili et al., 2017), gravitational search algorithm (GSA) (Rashedi et al., 2009), cuckoo search algorithm (CS) (Gandomi et al., 2013), marine predators algorithm (MPA) (Faramarzi et al., 2020), Aquila Optimizer (Abualigah et al., 2021), and other optimization algorithms (Abualigah & Diabat, 2020; Abualigah et al., 2020, 2022).

The application of these algorithms improves the performance of the clustering methods; however, these algorithms also still have some drawbacks, especially in solving mechanical clustering problems (Abualigah et al., 2021, 2020). For instance, some of them cannot effectively explore the search domain in all problems, whereas other methods have a low exploitation ability (Mukhopadhyay et al., 2015; Suresh et al., 2009). Therefore, several attempts are to overcome these limitations by combining some optimization algorithms or improving their local search methods. The results of these attempts showed an excellent ability to enhance many algorithms (Ewees et al., 2017, 2018). For example, Alswaitti et al. (2018) proposed a Kernel density-based PSO method for data clustering. To overcome the shortcomings of the traditional PSO, they applied the kernel density estimation method with a bandwidth estimation technique to solve the problem of premature convergence. They evaluated the improved PSO method with eleven UCI datasets and showed significant performance compared to the traditional PSO. In Abd Elaziz et al. (2019), an automatic data clustering algorithm was proposed using a hybrid of sine-cosine algorithm SCA and ASO. The main goal of the hybrid method is to automatically find the optimal number of centroids to minimize the Compact-separated index. Thus, the sine cosine algorithm enhances the atom search optimization algorithm’s searchability to find the optimal solution. It was evaluated with different datasets and with several performance measures. Evaluation outcomes showed that the hybrid ASOSCA obtained better results than the traditional ASO and SCA and several optimization methods.

In Zabihi and Nasiri (2018), a new data clustering method was proposed using a modified version of the ABC algorithm. The main idea of the modified version, called history-driven ABC (Hd-ABC), is to enhance the exploitation capability of the traditional ABC algorithm by employing a memory mechanism. It was evaluated on nine UCI datasets and showed superior performance. Zhou et al. 2019 proposed a clustering method using both density peaks clustering and a modified version of the GSA. They evaluated the combing approach using ten datasets, and they compared it to several existing optimization algorithms and the traditional k-means algorithm. It showed significant performance with a higher level of stability. In Boushaki et al. (2018), a new variant of the CS algorithm is proposed for data clustering. The main idea is to apply boundary handling strategy and Chaos maps to enhance the global search ability of the CS. The modified CS algorithm was evaluated with six real-life datasets and compared to eight optimization methods, and it showed competitive performance.

A hybrid of MPA and PSO for automatic data clustering was proposed by Wang et al. (2020). The global searching of the MPA is improved by using the update strategy of PSO, and it showed better performance compared to the traditional MPA, traditional PSO, and other optimization algorithms. Furthermore, various modified optimization algorithms have been applied for data clusterings, such as multi-objective GA with the fuzzy c-means (FCM) (Wikaisuksakul, 2014), an enhanced version of Grey Wolf Optimizer (Tripathi et al., 2018), a new variant of harmony search algorithm (Talaei et al., 2020), and a modified version of the multi-verse optimizer (Abasi et al., 2020).

In the same context, a new MH algorithm named Arithmetic Optimization Algorithm (AOA) was developed in Abualigah et al. (2021). This algorithm emulated the function of arithmetical operators such as subtraction, addition, division, and multiplication. These operators are used to represent exploration and exploitation. According to these behaviors, AOA has been applied to solve global and engineering optimization problems. However, similar to other MH techniques, AOA still needs more improvements to balance the exploration and exploitation during the search for the optimal solution. In addition, following the NFL theorem assumed that no one algorithm can solve all optimization problems with the same performance. This motivated us to present an alternative version of AOA and apply it to real-world applications.

Motivated by the excellent performance of MH algorithms in data clustering, we developed a new clustering method based on the modified AOA in this paper. This modification depends on using Opposition-based learning (OBL) and Lévy Flight (LF) distribution to improve the ability of AOA to converge towards the optimal solution. In general, OBL is applied to enhance the exploration of AOA and LF to improve the exploitation. These two techniques have established their performance in several applications through modifying several MH methods (Elaziz et al., 2020; Elaziz & Oliva, 2018; Elaziz & Mirjalili, 2019). For example, OBL is applied to enhance the performance of the Sine-cosine algorithm (SCA) as in Elaziz et al. (2017). The brainstorm optimization is improved using OBl, and it is used as global optimization and feature selection method in Oliva and Elaziz (2020). In Ewees et al. (2018), the modified version of the grasshopper optimization algorithm based on OBL has been applied as a global optimization technique and compared with other methods. Moreover, the LF distribution has been used to enhance the performance of several MH techniques such as improved PSO and used to improve the quality of flexible job shop greening scheduling with crane transportation application (Zhou & Liao, 2020). In Yan et al. (2017), LF distribution combined with PSO and applied to solve the atomic clusters optimization problem. Salp Swarm Algorithm has been improved using LF and applied to several global optimization methods as in Zhang and Wang (2020)

Besides these behaviors of OBL and LF, an alternative modified AOA has been presented. The developed method starts by setting the initial value for solutions. Followed by computing each solution’s fitness value and finding the best solution. The next step is to adopt the current solution using AOA, OBL, and LF distribution operators. Updating the solutions is repeated until it reaches terminal conditions and returns the best solution.

In summary, our main objectives and contributions are:

• Propose an alternative global optimization and clustering technique according to the enhanced version of AOA.

• Develop the performance of AOA using the operators of OBL and LF distribution.

• Apply the developed method to global optimization problems and real-world clustering datasets.

• Compare the results of the developed method with other MH techniques.

The sections of this paper are presented as follows. Section 2 describes the background of the applied techniques. Section 3 gave the proposed AAOA clustering method and its experimental evaluation compared to other methods. Section 4 shows the 23 benchmark functions. Section 5 is the conclusion and future directions.

Background

Arithmetic optimization algorithm

The basic steps of the Arithmetic Optimization Algorithm (AOA) (Abualigah et al., 2021) are introduced in this section. In general, AOA is similar to other MH techniques, with two phases named exploration and exploitation. These two phases are emulated using the basic mathematics operators (i.e., $-, +, *$, and /).

The first step in AOA is to generate a set of N agents; each represents the solution for the tested problem. These agents represent the population X that is given as:

$$\begin{aligned} X=\begin{bmatrix} x_{1,1} &{} \cdots &{}x_{1,j} &{} x_{1,n-1} &{} x_{1,n}\\ x_{2,1} &{} \cdots &{} x_{2,j}&{} \cdots &{} x_{2,n}\\ \cdots &{} \cdots &{} \cdots &{}\cdots &{}\cdots \\ \vdots &{} \vdots &{}\vdots &{} \vdots &{} \vdots \\ x_{N-1,1} &{}\cdots &{} x_{N-1,j} &{} \cdots &{} x_{N-1,n}\\ x_{N,1} &{} \cdots &{} x_{N,j} &{}x_{N,n-1} &{} x_{N,n}\\ \end{bmatrix} \end{aligned}$$

(1)

The next step is to compute the fitness function for each agent and determine the best of them $X_b$. Then according to the value of Math Optimizer Accelerated (MOA), AOA will perform exploration or exploitation, and the value of MOA is updated as:

$$\begin{aligned} MOA(t)=Min+t \times \left( \frac{Max_{MOA}-Min_{MOA}}{M_t} \right) \end{aligned}$$

(2)

In Equation (2), t is the current iteration, $M_t$ is the total number of iterations. $Min_{MOA}$ and $Max_{MOA}$ are the minimum and maximum value of the accelerated function, respectively Zheng et al. (2022).

In the case of the AOA exploration phase, the division (D) and multiplication (M) operators are used. This process is formulated as:

$$\begin{aligned}&X_{i,j}(t+1)\nonumber \\&\quad =\left\{ \begin{matrix} X_{ij} \div (M_{OP}+\epsilon ) \times ((UB_{j}-LB_{j}) \times \mu + LB_{j} ), &{}r_2<0.5 \\ X_{ij} \times M_{OP} \times ((UB_{j}-LB_{j}) \times \mu + LB_{j} ),&{}otherwise \end{matrix}\right. \nonumber \\ \end{aligned}$$

(3)

where $X_{ij}$ is the ith position in the jth solution, $\epsilon$ refers to a small integer value, $UB_{j}$ and $LB_{j}$ denotes the lower and upper boundaries of the search space at the jth dimension, respectively. $\mu =0.5$ denotes the control function, and the Math Optimizer ($M_{OP}$) is formulated as:

$$\begin{aligned} M_{OP}(t)=1-\frac{t^{1/\alpha }}{M_t^{1/\alpha }} \end{aligned}$$

(4)

In Equation (4), $\alpha =5$ denotes the dynamic parameter which determines the precision of exploitation throughout iterations. Meanwhile, the exploitation phase of AOA is conducted using the subtracting (S) and addition operators (A) Elaziz et al. (2021). This achieved using the following formula:

$$\begin{aligned}&x_{i,j}(t+1)\nonumber \\&=\left\{ \begin{matrix} X_{ij} - M_{OP} \times ((UB_{j}-LB_{j}) \times \mu + LB_{j} ), &{}r_3<0.5 \\ X_{ij} + M_{OP} \times ((UB_{j}-LB_{j}) \times \mu + LB_{j}),&{}otherwise \end{matrix}\right. \nonumber \\ \end{aligned}$$

(5)

where $r_3$ is a random number generated inside [0,1]. After that, the updating process of agents is performed using the operators of AOA. The steps of the AOA are given in Algorithm  1.

Lévy flight distribution

In this section, Lévy flight is one of the most popular distribution approaches which follow the non-Gaussian distribution (Houssein et al., 2020; Chegini et al., 2018). After that, Equation (6) is used to update agents inside the population according to the following formula.

$$\begin{aligned}&x(t+1) = x(t) \times Levy(Dim) \end{aligned}$$

(6)

$$\begin{aligned}&Levy(Dim)=s \times \frac{u \times \sigma }{|\upsilon |^{\frac{1}{\beta }}} \end{aligned}$$

(7)

In Equation (6), $s=0.01$ denotes a constant value, u and $\upsilon$ denote random numbers between [0 1]. $\sigma$ is given using the following formula.

$$\begin{aligned} \sigma =\left( \frac{\Gamma (1+\beta ) \times sine(\frac{\pi \beta }{2} )}{\Gamma (\frac{1+\beta }{2}) \times \beta \times 2^{(\frac{\beta -1}{2})}} \right) \end{aligned}$$

(8)

where sine denotes the sine function value, and $\beta$ is a constant value fixed to 1.5.

Opposition-based learning (OLB)

The OBL strategy was proposed by Tizhoosh (2005) as a machine intelligence method. It was used in many applications as an efficient search mechanism to enhance several optimization methods (Ewees et al., 2018). The OBL works to create a new opposition solution using the current one to improve the search space.

In the OBL method, there is an opposite value ($X^O$) for a real value. X $\in$ [LB,UB] can be calculated using Equation (9).

$$\begin{aligned} X^O=UB+LB-X \end{aligned}$$

(9)

Opposite value (Ewees et al., 2018): X = ($X_1$, $X_2$ , ..., $X_n$) is a value in the search space, $X_1, X_2,..., X_D$ and $X_j$ [$UB_j,LB_j$], j $\in$ 1, 2, ..., D. This representation is applied using the following Equation (10).

$$\begin{aligned} {X^O_j}=UB_j+LB_j-X_j, \quad \quad where \quad j=1....D. \end{aligned}$$

(10)

Furthermore, in the optimization task, the two solutions ($X^O$ and X) are evaluated using the fitness functions; then, the best solution will be reserved and ignored the other.

The proposed AAOA

The general framework of the developed method, named AAOA, is given in Fig. 1. AAOA aims to enhance the ability of the AOA to balance exploration and exploitation during the process of searching for the optimal solution. To achieve this aim, the OBL approach and LF distribution are combined with the operators of traditional AOA. Each of them is applied to perform a specific task, such as OBL, to enhance the exploration ability of AOA to discover the infeasible region. Meanwhile, LF is used to improve the convergence rate towards the optimal solution and avoid the attraction to the local optima point. This integration between AOA, OBL, and LF significantly enhance the performance of AOA.

Fig. 1

Framework of developed AAOA method

The proposed AAOA algorithm begins by randomly setting the initial value of N agents (X) using the following formula.

$$\begin{aligned}&X_{ij}=rand\times (UB-LB)+LB, \nonumber \\&\quad i=1,2,...,N, j=1,2,...,D \end{aligned}$$

(11)

In Equation (11), UB and LB are the upper and lower boundaries of the search domain, respectively. D denotes the dimension of each agent $X_i$. The following process calculates the fitness value for each agent and allocates the best of them $X_b$. Followed by starting updating the agents X using the combination between AOA, OBL, and LF. This was conducted using random factor $R_f\in [0,1]$ that switches between the operators of AOA (on one side) and the competition of OBL and LF (on the second side). For example, if $R_f <0.5$, then the operators of AOA will be used to update the current solutions. Otherwise, either the OBL or LF will be used, and inside the developed method, each of those two techniques has 50% to be applied. This process can be formulated as:

$$\begin{aligned}&X_{i}(t+1)\nonumber \\&=\left\{ \begin{matrix} Use \, operators \, of \, AOA\, as\, in \, Eqs.\, (3)-(5), &{}R_f<0.5 \\ \left\{ \begin{matrix} Apply\, OBL\, as\, in \, Equation (10) &{}rand<0.5 \\ Apply\, LF\, as\, in \, Equation (6),&{}otherwise \end{matrix}\right. ,&otherwise \end{matrix}\right. \nonumber \\ \end{aligned}$$

(12)

After that, the terminal conditions are checked, and if they are not satisfied, then the updating process is repeated. Otherwise, the best solution $X_b$ is returned as an output of the developed method. The steps of AAOA are illustrated in Algorithm 2.

Table 1 Unimodal benchmark functions

Table 2 Multimodal benchmark functions

To summarize, the proposed AAOA presented begins with the generation of a random set of solutions. During the evolution-based optimization phase, the AAOA’s search criteria look for probable placements of the current-best solution. The technological advances to the next level with each solution. The AAOA employs the Arithmetic Optimization Algorithm, Levy flight distribution, and opposition-based learning approaches, according to the Algorithm 2. Each iteration will update and enhance the prospective solutions using these search approaches according to probability conditions. The three search methods avoid the local optima solution by generating high distribution solutions. Moreover, the mutual processes between them help keep the balance between the search process (exploration and exploitation).

Performance evaluation using 23 benchmark functions

In this section, the performance of the developed method is assessed using a set of the classical benchmarks.

Benchmark description

The mathematical formulation and classifications of the employed 23 mathematical functions are presented in Tables 1, 2, and 3 . The benchmark functions in Table 1 are unimodal; the reason for using this set is to evaluate the exploitation ability of the proposed optimizer, as this set of functions has only one optimal solution. For the mathematical functions listed in Table 2, they have several peaks, some local optima, and only one global optimum; thus, they are considered the best choice in evaluating the optimization algorithm’s exploration. Finally, for examining the balance between the exploration and exploitation abilities, the Fixed-dimension multimodal benchmark functions of Table 3 are considered challenging tasks. The considered dimensions, the defined search space limits, and the global values ($f_{min}$) of the mathematical functions are reported in the Table.

Table 3 Fixed-dimension multimodal benchmark functions

Table 4 Parameter values for the comparative algorithms

Experiments and results

In this section, the proposed AAOA algorithm’s ability is evaluated through two stages; the first one is handling a set of challenging CEC benchmark functions. The second stage focuses on clustering eight UCI benchmark datasets. The description of these datasets is listed in Sect. 5.2.1; each dataset has properties and characteristics which make different challenges. The proposed AAOA algorithm compared to the original AOA as well as six well-known algorithms namely the Particle swarm optimization (PSO) (Eberhart & Kennedy, 1995), Grey wolf optimizer (GWO) (Mirjalili et al., 2014), Sine cosine algorithm (SCA) (Mirjalili, 2016), Marine Predators Algorithm (MPA) (Faramarzi et al., 2020), whale optimization algorithm (WOA) (Mirjalili & Lewis, 2016), and Salp Swarm Algorithm (SSA) (Mirjalili et al., 2017). Four measures are used in the comparisons: worst, best, average, and standard deviation of the finesse values. Besides, as a statistical test, the Wilcoxon rank-sum test is applied to check if there are significant differences between AAOA and the other algorithms or not at p-value $< 0.05$.

Fig. 2

Qualitative results for the tested 13 problems

Fig. 3

Diversity plots for between the best and worst solutions on 8 benchmark functions

Fig. 4

Execution time of the AOA and the proposed AAOA for 13 benchmark functions

First experiment: global optimization

In this sector, the proposed AAOA has assessed using more popular 23 mathematical functions and of CEC2019 suite that has several specifications. The proposed AAOA has compared with a set of recent state-of-the-art techniques using numerous statistical analyses to appraise and demonstrate the AAOA’ss efficiency in handling global optimization challenges. The considered algorithms including the basic AOA, Particle Swarm Optimization (PSO) (Abualigah et al., 2018), Grey Wolf Optimizer (GWO) (Mirjalili et al., 2014), Sine Cosine Algorithm (SCA) (Mirjalili, 2016), Marine Predators Algorithm (MPA) (Faramarzi et al., 2020), Whale Optimization Algorithm (WOA) (Mirjalili & Lewis, 2016), and Salp Swarm Algorithm (SSA) (Mirjalili et al., 2017). The algorithms were implemented in the experimental results’ fairness under the same settings: population size was set to 30, and maximum iterations 500 for 30 independent times. Table 4 summarizes the parameter settings of the counterparts algorithms, which have been taken from the original papers. All the analysis and simulations have been implemented on the Windows 10 operating system with an Intel Core i5, 2.2 GHz CPU, and 16 GB of RAM. All competitors were conducted in the MATLAB 2018 platform to guarantee unbiased comparison.

Table 5 The results of the comparative methods on 23 benchmark functions (F1-F23), where the dimension is 10

Table 6 The results of the Friedman ranking test for the comparative methods overall 23 benchmark functions, where the dimension is 10

Qualitative analysis

To validate the developed AAOA technique’s performance, the convergence and the trajectory are used in Fig. 2. This Figure depicts the qualitative measures, such as the 2D plot of the function drawn in the first column, to discuss the search space’s topology. Furthermore, the solution trajectory is exposed in the second column of the Figure, while the average fitness value and convergence curves are exhibited in the third and fourth columns, respectively.

From the second column representing the trajectory of the solution, it can be observed that the solution has a high magnitude and frequency in the early iterations. At the last iterations, they have nearly vanished. This illustrates the high exploration ability of AAOA in the early iterations and good exploitation in the last iterations. AAOA has a high chance of reaching the optimal solution based on this behavior. The average fitness value overall for the solutions among the number of iterations depicted in the third column of Fig. 2 reveals the abilities of the AAOA in converging to the high qualified solutions in less number of iterations. The AAOA starts with a high average fitness value at the beginning of iterations. However, before the number of iterations reached 50, the average became small. For the fourth column in Fig. 2 it can be noticed from the convergence curve that the convergence curves are smooth in most of the studied functions while the AAOA has higher qualified solutions than the AOA.

To illustrate the exploration and exploitation abilities of the proposed AAOA variant in comparison with its basic version (AOA), the Diversity plots between the best and worst solutions using the AAOA and AOA are exposed in Fig. 3 for different eight functions. It can be observed that the AAOA can maintain the diversity between solutions better than the traditional AOA.

Fig. 5

Convergence behaviour of the comparative methods on the test functions (F1, F4, F7, and F10), where the dimension is 10

Figure 4 shows the execution time of the AOA and the proposed AAOA for 13 benchmark functions. It is clear in this figure that the execution time of the tested methods (i.e., the original AOA and the proposed AAOA) is approximately equal, despite the modifications that happened to the original method. This reflects the extent of the proposed method’s ability to achieve high results in a time compared to the original method.

Simulations and discussions of 23 benchmark functions

This section uses the worst, best, average, and standard deviation (STD) values to measure the proposed variant’s performance. Moreover, the Wilcoxon rank-sum test with significant deference of 0.05 is considered an indicator of the existing significant difference between the proposed variant and the other counterparts. The Friedman ranking test is applied to indicate the final rank of the proposed AAOA and demonstrate the ability of the AAOA to handle most of the employed benchmark functions compared with other state-of-the-art counterparts.

The data in Table 5 represent the results of the AAOA versus that of the basic AOA, PSO, GWO, SCA, MPA, WOA, and SSA. The Worst, Best, Average, and STD values by AAOA reveal the ability of the AAOA to defeat all the other algorithms in about 50 $\%$ of the considered 23 benchmarks as it displayed the least values of the computed metrics in the functions of (F: 1, 2,3, 4, 7, 9, 10, 11, 12, 20, 21) moreover, it has comparable performance in the other 50 $\%$ of the applied functions. The attained P-value values through the Wilcoxon rank-sum test with significant deference of 0.05 confirm the superiority of the AAOA in comparison with the PSO in 17 functions as the P-value is less than 0.05; therefore, the null hypothesis test is rejected (h=1 means there is a significant difference between the considered optimize; AAOA Vs. PSO). The reported P-values in cases of GWO, SCA, MPA, WOA, and SSA demonstrate the outperforming performance of the AAOA in handling about 12 functions out of the 23 ones; therefore, the null hypothesis test is rejected (h=1). For further investigation, the Friedman ranking test is applied to determine the proposed AAOA’s rank among the other counterparts while processing the 23 functions. The obtained classes are reported in Table 6. The ranks’ average values divulge the AAOA’s ability to achieve a comparable position between the recent state-of-the-art algorithms. The average rank of AAOA is 2.83, which is much smaller than the other algorithms; hence the AAOA occupied the first position as a final rank. MPA takes the second rank with an average rank of 3.3. By observing the reported data in Tables  5 and 6, one can conclude that the AAOA proves its superiority statistically in comparison with a set of recent state-of-the-art and the basic version of AOA.

Table 7 The results of the comparative methods on 13 benchmark functions (F1-F13), where the dimension is 100

Table 8 The results of the Friedman ranking test for the comparative methods overall 13 benchmark functions (Dimension =100)

The convergence curves of the proposed AAOA are depicted in Fig. 5 versus the AOA and the state-of-the-art techniques to assess the efficiency of the AAOA’s central cores (exploration and exploitation). The curves show the smooth convergence of the AAOA by achieving higher quality solutions than the PSO, GWO, SCA, WOA, and SSA that suffered from high stagnation at the local optimal solutions.

Scalability analysis

In this section, the performance of AAOA is examined with thirteen functions of Tables 1 and 2 with a high dimension of 100 to evaluate the stability of the optimizer with increasing the dimension of the handled optimization problems. The obtained worst, best, average, and STD values by the proposed variant and the other techniques (AOA, PSO, GWO, SCA, MPA, WOA, and SSA) are reported in Table 7. Moreover, the P-value and the null hypothesis test result based on the Wilcoxon rank-sum test with significant deference of 0.05 are listed in Table 7 for AAOA versus the other techniques. The reported data of the Table reveal the stability and efficiency of the proposed AAOA as it provides the optimal solutions for the six functions (F1,F2,F3,F4, F9, and F11, where the $f_{min} = 0$ (see Tables 1 and 2)). Furthermore, it has the closest results for the optimal solutions of the other functions (F5, F6, F7, F8, F10) compared with the other algorithms. The reported P-values are less than 0.05 for 85 % of the studied functions. Therefore, one can conclude the high stability and superiority of AAOA while dealing with high-dimensional problems.

The Friedman ranking test is computed in Table 8 to emphasize the superiority of the proposed AAOA. The AAOA has the first rank in eleven problems out of the studied thirteen functions; hence it is finally located in the first place in the queue of the other techniques for solving high-dimensional problems. The MPA occupies the second position, with an average rank is nearly the number achieved by AAOA. Therefore, AAOA outperforms the contemporary state-of-the-art techniques in providing high-quality solutions for high-dimensional problems.

The convergence rate of the modified variant AAOA is the primary sector studied to assess the influence of integrating the operators of OBL and LF distribution while optimizing high-dimensional problems. Therefore, the convergence curves of AAOA versus the other counterparts are displayed in Fig. 6 for the thirteen studied functions. By inspecting the exposed curves in the Figure, one can observe that the AAOA converges to the optimal solutions in the first number of iterations; meanwhile, the PSO, GWO, and SSA have stacked the local solutions regarding all the studied functions.

Fig. 6

Convergence behaviour of the comparative methods on the test functions (F1-F23), where the dimension is 100

Table 9 shows the results of the comparative methods on 13 benchmark functions (F1-F13) through 1 second, where the dimension is 10. We have chosen 1 second, which is approximately equal to 500 iterations. These comparisons are conducted to further prove the proposed AAOA in solving the given problems compared with other methods using the same execution time. It is evident in Table 9 that the proposed AAOA got better results compared to other comparative methods, which reflects its ability also when the execution is performed using the same time. According to the Welxxcon test, the proposed method exceeds almost all the comparative methods. For example, in solving F1, the proposed AAOA overcame PSO, GWO, SCA, MPA, WOA, and SSA. The statistical analysis confirmed the ability of the proposed AAOA to get better results compared to other methods. Moreover, according to the Friedman ranking test, the proposed AAOA got the first rank, followed by GWO, MPA, AOA, WOA, PSO, SCA, and SSA.

Table 9 The results of the comparative methods on 13 functions (F1-F13) through 1 second, where the dimension is 10

Performance evaluation using CEC2019 benchmark functions

Within this section, a different set of benchmark functions Is used to assess the performance of the developed AAOA algorithm. These functions are collected from the challenging functions of CEC2019, and their descriptions are given in Table 10.

Table 10 CEC2019 benchmark functions

In Table 11, the results of the proposed AAOA method are compared with other well-known optimization methods (AOA, PSO, GWO, SCA, MPA, WOA, and SSA) using advanced benchmark functions (CEC20019). These comparisons are conducted to prove further the proposed method (AAOA) in solving various optimization problems. The results clearly show that the performance of the proposed method is better than all other comparative methods. It got the first tanking compared to other methods. The MPA, however, is located in the second rank. Therefore, AAOA is the recommended variant for this benchmark suite among the other comparable optimizers. Moreover, the obtained results illustrated that the modified version can bring new best solutions for several test cases.

Table 11 The results of the comparative methods using advanced CEC2019 benchmark functions

Table 12 The results obtained by the proposed AAOA and best-published results using CEC2019

Fig. 7

Convergence behaviour of the comparative methods on the tested benchmark functions (CEC2019)

Moreover, the proposed AAOA is further evaluated using ten CEC2019 compared to the state-of-the-art methods published in the literature. The comparative methods include Fuzzy Self-Tuning PSO (FST-PSO) (Nobile et al., 2018), improved BA with variable neighborhood (VNBA) (Wang et al., 2016), novel PSO using prey-predator relationship (PP-PSO) (Zhang et al., 2018), Hybrid KHA with differential evolution (DEKH) (Wang et al., 2014), Chaotic CS (CCS) (Wang et al., 2016), and stud krill herd algorithm (SKH) (Wang et al., 2014).

The attained best, worst, STD, and the p-values based on the Wilcoxon rank-sum test with significant deference of 0.05 by AAOA and other counterparts are illustrated in Table 12. The listed data show the efficiency of the AAOA in handling nine functions out of the ten ones with minimum statistical metrics (best, worst, and STD). In contrast, it has the second position in solving CEC2019. The computed p-values confirm the superiority of the proposed AAOA and provide evidence of exiting a significant difference between the optimizers in favor of AAOA. Accordingly, the AAOA has the least average rank based on Friedman’s test, as reported in the last lines of the Table; consequently, it is ordered as the first optimizer while solving that set of benchmarks. The VNBA, however, is located in the second rank; DEKH got the third rank, PP-PSO got the fourth rank, SKH got the fifth rank, FST-PSO got the sixth rank, and CCS got the final rank.

The plotted curves of Fig. 7 depict the acceleration rates of the AAOA versus AOA, PSO, GWO, SCA, MPA, WOA, and SSA while optimizing the ten functions of CEC2019. The exhibited curves show the convergence of the AAOA to the high-quality solutions with a smooth and fast response. Meanwhile, the AOA, PSO, GWO, SCA, WOA, and SSA suffered from high stagnation in local solutions in several functions. Accordingly, the AAOA proves its efficiency not only in accuracy but also in convergence property.

Second experiment: clustering applications

Datasets description

This experiment evaluates the AAOA using eight UCI datasets, namely Cancer, CMC, Glass, Iris, Seeds, Heart, Vowels, and Water. The descriptions of these datasets are listed in Table 13.

Results and discussion

This section shows the performance of the proposed AAOA over eight datasets; the performance results and the Wilcoxon test values are listed in one table. Regarding the Cancer dataset, Table 14 shows the clustering results for the proposed AAOA and the compared algorithms. In terms of Worst measure results, we can prominent see that the proposed AAOA obtained the best results (i.e., 373.23); this result is much better than the second-rank algorithm (i.e., PSO), the PSO obtained 2007.50 followed by GWO, AOA, SCA, SSA, MPA, and WOA.

The average measure also presented these results; the AAOA was ranked first with 248.64, followed by the PSO, GWO, and SCA with 1116.60, 2812.00, and 3158.00, respectively. The worst algorithm was also the WOA. The AAOA showed its superiority in the best measure; it was ranked first, followed by the PSO, GWO, SCA, and SSA. In this measure, the original AOA was ranked last. However, it was the most stable algorithm based on the Std measure; it recorded 79.32, whereas the AAOA was ranked second with 90.71. The third and fourth stable algorithms were WOA and GWO. The PSO showed the worst stability compared to the other algorithms with 598.12. The obtained centroids by all methods are recorded in Table 15.

The results of the CMC dataset are listed in Table 16. The proposed AAOA obtained the best results based on the worst measure (i.e., 80.81) and was ranked first in this table. The second method was the PSO with 95.97, followed by the GWO with 310.76. Whereas the other methods, AOA, SCA, MPA, WOA, and SSA, showed similar performances. In terms of Average measure results, we can see that the AAOA obtained 77.60 and outperformed the second-ranked method (i.e., PSO). The GWO was ranked third. Whereas the rest methods also showed similar performances to some extent. These results were also confirmed by inspecting the products of the Best measure. In contrast, the MPA showed the most stable behavior of all methods with 0.203, followed by AOA, WOA, SCA, and SSA, whereas the AAOA showed an acceptable Std value of 2.772. The obtained centroids by all methods are recorded in Table 17.

In the Glass dataset, as shown in Table 18, the proposed AAOA method outperformed the other method in both the Worst and Average measures; it obtained 1.23 and 0.77 by the PSO with 10.79 and 6.40, respectively. The rest methods showed similar results. Regarding the Best measure, the AAOA and PSO obtained the same results (i.e., 0.000) and outperformed all other methods. In the Std measure, the most stable method was MPA, followed by SCA, AAOA, and AOA, respectively, whereas the PSO showed the worst Std result. Table 19 shows the obtained centroids by all methods.

Table 20 records the results of the Iris dataset, and Table 21 shows the centroids results for all methods. Table 20 shows that the AAOA obtained superior results in both the Worst and Average measures, followed by the PSO and GWO. Simultaneously, the AOA, SCA, MPA, WOA, and SSA showed similar results with significant deference from the AAOA. Simultaneously, the AAOA obtained 0.90 in the Best measure and was ranked second after the PSO, which obtained 0.62. In addition, the AAOA also showed good stability in this dataset and was ranked fourth after the AOA, SSA, and WOA.

Table 13 UCI benchmark datasets

Table 22 records the results of the Seeds dataset. The AAOA achieved the first rank in Worst, Average, and Best measures from this table, followed by the PSO and GWO methods. The AOA was ranked fourth in the Worst measure, but the SCA was ranked fourth in both Average and Best measures. The last rank was recorded by the WOA method. In the Std measure, all methods showed good stability, but the most stable methods were AAOA and AOA. The obtained centroids by all methods for Seeds are recorded in Table 23.

The Heart dataset also presented the obtained results in the Seeds dataset. In Table 25, the proposed AAOA achieved the first rank in all measures, followed by the PSO except for the Std measure, and the PSO showed the worst stability among all methods. In this dataset, the WOA showed better performance than the Seeds dataset and was ranked fourth in both Average and Best measures, whereas the SCA recorded the lowest performance. The obtained centroids by all methods for Heart datasets are recorded in Table 25, respectively.

Moreover, by inspecting the results of the Vowels dataset, as in Table 27, we can see that the AAOA obtained the top results in both Worst and Average measures, whereas it was ranked second in the Best measure after the PSO method with slight deference. The WOA obtained the third rank in the Worst measure and the fourth rank in the Worst and Average measures after the GOW. Although the MPA was considered the most stable method in this dataset, it showed the worst performance in the other measurements. The AAOA and the compared methods showed good stability except for the PSO and GWO methods. Table 27 shows the obtained centroids by all methods.

Table 14 The results of the comparative methods using Cancer dataset

Table 15 Determining centroid of each cluster for the Cancer dataset

Table 16 The results of the comparative methods using CMC dataset

Table 17 Determining centroid of each cluster for the CMC dataset

Table 18 The results of the comparative methods using Glass dataset

Table 19 Determining centroid of each cluster for the Glass dataset

Table 20 The results of the comparative methods using Iris dataset

Table 21 Determining centroid of each cluster for the Iris dataset

Table 22 The results of the comparative methods using Seeds dataset

Table 23 Determining centroid of each cluster for the Seeds dataset

Table 24 The results of the comparative methods using Statlog (Heart) dataset

Table 25 Determining centroid of each cluster for the Statlog (Heart) dataset

Table 26 The results of the comparative methods using Vowels dataset

Table 27 Determining centroid of each cluster for the Vowel dataset

The results of the Water dataset, as in Tables 28 and 29, also show the superiority of the proposed AAOA method in all measures, followed by the PSO and GWO, respectively, except for the Std measure, which were seventh and eighth, respectively. Whereas the worst one was the SCA method.

Table 28 The results of the comparative methods using Water dataset

Table 29 Determining centroid of each cluster for the Water dataset

Moreover, the proposed AAOA method is evaluated using two statistical analysis analyses Carrasco et al. (2020) (i.e., Wilcoxon rank-sum test and Friedman test) to check if there are significant differences between AAOA and the other algorithms or not at p-value < 0.05. The results for all datasets are listed in Tables 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, and 28. From the results of the Wilcoxon test, we can see that there are significant differences between AAOA and all compared methods in all datasets except for the PSO in Vowels dataset. The results of the Friedman test also showed the superiority of the AAOA. It achieved the first rank in all datasets, followed by PSO, GWO, and AOA, respectively, whereas the MPA obtained the last rank.

Furthermore, to summarize the performances of all methods overall datasets, the AAOA obtained the first rank in all measures, followed by the PSO, GWO, AOA, WOA, SCA, and SSA, whereas the MPA obtained the last rank (see Table 30 and Fig. 8). These results confirm that the AAOA can solve various clustering problems and get better results than the compared algorithms with good stability and low errors. According to the Friedman rank test, the proposed AAOA method got the first rank compared to other comparative methods. Followed by PSO, GWO, AOA, SCA, WOA, SSA, and MPA.

Table 30 The results of the Friedman ranking test for the comparative methods using all the used datasets

Fig. 8

Ranking of all the comparative methods using all the tested datasets

In addition, Fig. 9 shows the results of the clustering analysis as the coloring of the multiplication signs (objects) into k clusters (cycle sign). Figure 10 shows the convergence behavior of the comparative methods overall in the tested clustering datasets. This figure shows that the AAOA reached the best fitness values than other methods in all datasets except for the Glass dataset, which ranked second. The AAOA also showed an excellent updating behavior of the search domain to explore new spaces to escape from trapping in a local optimum.

Fig. 9

The results of the clustering analysis are shown as the coloring of the multiplication signs (objects) into k clusters (cycle sign)

Fig. 10

Convergence behaviour of the comparative methods using the tested clustering datasets

Conclusion and potential future works

In recent years, different metaheuristic (MH) optimization algorithms have been widely employed for solving various engineering and optimization problems. A new optimization algorithm inspired by math operations was recently proposed, namely the Arithmetic Optimization Algorithm (AOA). The exploration and exploitation trends of the AOA require more improvements to address more complex optimization tasks. To this end, in this paper, we propose an ensemble AOA by applying two search mechanisms, namely, Lévy Flight distribution opposition-based learning (OLB), to boost the search mechanism of the traditional AOA. The new variant is called AAOA, which was evaluated using different benchmark functions and datasets. To assess the AAOA as a global optimizer, twenty-three CEC2005 benchmark functions were used. Besides, we used eight UCI datasets to evaluate the AAOA as a data closeting method. We considered extensive comparisons to the well-known optimization algorithms in all evaluation experiments, such as the traditional AOA, PSO, GWO, SCA, MPA, and SSA. Experimental statistics and outcomes have confirmed the superiority of the developed AAOA over other optimization methods, including the original AOA.

According to the superior performance of the AAOA, it can be considered a promising optimization method that could be further leveraged to address different optimization tasks, such as fog computing scheduling, image segmentation, and feature selection.