Keywords

1 Introduction

Optimization is a method to search the best solution in a given circumstance. Engineers need to decide on several technological and managerial issues for design, manufacturing, construction, as well as maintenance of systems. Minimization of efforts with maximization of the desired output is the most crucial aspect of all search decisions. Thus, efforts required to be given and/or to obtain the desired output is a function of certain variables and can be optimized to achieve the optimum level of that function [1, 2].

Different applications of optimizations in the field of engineering are as follows:

  • Optimum design of civil engineering structures.

  • Minimum-weight design of structures for earthquake, and other types of random loading.

  • Design of minimum cost materials management methods.

  • Design of maximum efficiency pumps, turbines, and equipment for heat transfer.

  • Design optimization of electrical machinery and networks.

  • Planning, scheduling, and controlling of manufacturing processes.

  • Design of materials with improved performance.

  • Decreasing the manufacturing cost, etc.

During the design process, an engineering system is defined by a set of variables on which the performance of the system depends. As a common aspect, some of the variables are fixed at the beginning as preassigned parameters. The other parameters are analyzed during the design optimization process, which are called design or decision variables [1]. In many cases, these design variables had to satisfied pre-specified requirements, hence, they cannot be chosen arbitrarily. To produce an acceptable design, these restrictions have to be satisfied and they are called optimization constraints. Among the constraints, behavior or functional constraints restrict the behavior of the system. Those constraints which indicate restrictions on design variables are called geometric or side constraints. In case of application to the manufacturing industry the plant constraints, describing the limits of the plant capacity, need to be considered during optimization . Any computed design usually leads to situations, which provide feasible and acceptable path toward the solution of the problem. Naturally, several other designs of the same problem could be computed with superior or inferior performances. To fulfill the purpose of optimum outcome, the best among all the acceptable designs available needs to be chosen. Thus, a decisive factor must be selected which could compare among the several acceptable designs and the optimum design could be chosen.

This decisive factor for optimizing the design is called the objective function. The objective function needs to be formulated depending on the inherent mechanism of the problem [1]. For example, in the case of structural design problems weight of automobiles or aircraft and cost in civil structures are common objective functions for minimization. Similarly, mechanical efficiency in mechanical engineering systems design is an objective function for maximization. An optimization algorithm starts with random initial solutions and improves in iterative methods. After certain iterations, it converges to the optimum solutions for the problem [3, 4]. The algorithm makes the system attracted towards the optimum solution from the solutions existing at different states, as a self-organizing system. Such an iterative system can be developed using some mathematically described equations or rules [5].

2 Classification of Optimization Methods

Optimization techniques can be classified from various aspects. It can be divided based on the presence of constraints. If the optimization problems have one or more constrained, it is called constraints optimization , and otherwise defined as an unconstrained optimization problem. Optimization can also be classified according to the number of objectives. In a single-objective optimization , either in search of maximum or minimum optimum solution, it leads to a single optimum solution. If there is more than one objective having conflict between them, the situation calls for multiobjective optimization . Here, a set of optimum non-dominated solutions evolve, which is represented in the form of Pareto front [6]. If the number of objectives is more than three, it is also called a many-objective optimization . The optimization problems can also be classified based on the nature of the objective functions as well as the constraints. The problem is called a linear or nonlinear problem based on the objective function and the constraints being linear or nonlinear. If the objective function is polynomial, it is called a geometric problem. If the constraints are integrated with the objective function and cannot be separated, then the problem is called a non-separable programming problem. When it comes to time constraints, optimization can be distinguished into two main types, online and offline optimization . If the job needs to be solved within a few seconds or milliseconds, it is called online optimization [6]. In such cases, the focus of the algorithm is speed. Robot localization, job schedule updating, transport process, and search engine are an example of such optimization .

These examples indicate that the online optimization has to be carried out in a repetitive manner to cater different orders arriving continuously to the system without significant weighting. In cases where time is not important as users are ready to wait for the optimal or close-to-optimal results, offline optimization is acceptable [6]. In case of offline optimization , the process is carried out only once. Here the optimization strategies are more important and have to be decided before the starting of the process. Again, deterministic and heuristic algorithms are two classes of the optimization algorithm.

In the case of deterministic algorithms, an obvious relation between the attributes of the system exists. It becomes hard to solve a problem deterministically when the relation between the system parameter and the objective/fitness of the system are complicated or obscure.

A heuristic algorithm [7, 8] gathers information about the system and fitness of random solution is tested and decision is taken for generating the next solution. Thus, these methods are dependent on the nature of the problem. On the other hand, a metaheuristic technique combines the heuristics and the objective function efficiently without depending on the structure of the problem [9, 10]. In this way, a metaheuristic algorithm makes the heuristic methods applicable to a wide range of problems.

High-quality solution for combinatorial optimization problems can be found by metaheuristics in a reasonable time [11].

A single-run algorithm like constructive methods or iterative improvement generate a limited number of solutions and even stop at local optima [12, 13]. If a function is difficult from the mathematical point of view when the context is discontinuous, not differentiable, or having multiple optima, a deterministic or classical optimization algorithm may converge to solutions in the local optima or from a small part of the search space. Thus, the problem with classical optimization algorithms is that generally, it is impossible to find whether the generated solution is situated in a local or global optima space. Similarly, it can also be stated that whether the provided solution is the result of the whole search space or only a part of it. This issue is more important for a multimodal problem, where multiple optima exist [14,15,16,17].

The concepts of metaheuristic optimization techniques are getting more and more accepted in real-life applications due to their robustness and capability to deal with complex design problems as discussed above. The metaheuristic techniques can be broadly divided into three groups. The first is the evolutionary computation group, consisting of genetic algorithm , genetic programming, evolutionary strategy, differential evolution , etc. The second group can be named swarm intelligence group consisting of particle swarm optimization , ant colony optimization , etc. The techniques within these two groups can also be termed as bioinspired optimization techniques. The third group consists of physical-process-inspired optimization techniques, and simulated annealing is the most common optimization tool in this group. The groups are shown in Fig. 1. Later in this chapter, some of these techniques will be discussed.

Fig. 1
figure 1

Major classes of metaheuristic global optimization algorithms with important variants

Full size image

3 Evolutionary Computation

Evolutionary computation (EC) uses principles inspired by the natural evolution of species and being used nowadays for solving many complex engineering problems. Four main types of this genre of algorithm are genetic algorithms [18], evolution strategies [19, 20], differential evolution [21], and genetic programming [22]. The common part of these four algorithms is that all four algorithms are populations based and use operators inspired by the concept of natural evolution. The selection operator chooses the individuals who will survive and create the next generation. If the objective function provided better fitness of an individual, then the probability of selection of the individuals is higher. The selected individuals are combined to form offspring through an operator called crossover, which merges parts of two selected individuals to generate in new individuals. The mutation operator introduces random modification within one individual to incorporate diversity of the solutions. These algorithms differ in representations of the individuals and the method of implementing the operators. For example, the mutation operator is more emphasized in the case of evolutionary strategies and genetic programming algorithm.

Compared to other methods, the most significant advantage of EC is that little knowledge about the system is required to solve the problem, and thus these can be applied to a wide range of optimization problems. Even if the fitness function is not continuous, not differentiable and highly complex then also EC can solve the problem efficiently [23]. Additional information or knowledge about the system can also be incorporated in the optimization framework as constraints to improve the performance of the algorithm and to achieve better solutions.

As the different types of EC algorithms are available, users can easily choose the methods suited for a given problem depending on the structured of the system to be optimized. These methods can easily be used for multi-objective optimization problems [24]. EC provides a described heuristic estimation of optimum solution through a process based on certain operators [25,26,27,28,29,30]. The schema of the EC Algorithm is given in the form of a flowchart in Fig. 2.

Fig. 2
figure 2

The scheme of an evolutionary computing

Full size image

All the evolutionary computing algorithms follow the same general methods with certain variations. The structure of a solution varied from one algorithm to other. In the case of Genetic Algorithms (GA), the solutions are represented in the form of a stream. In the case of Evolution Strategies (ES), it is a real-valued vector, and in the case of Genetic Programming (GP) it is represented as a tree. This difference in the structures of candidate solution makes the different types of EC algorithms applicable for different types of optimization problems. Genetic algorithms (GA) are natural selection and natural genetics inspired optimization algorithms. It can be used for a wide range of problems with high complexity in the objective function. To solve practical engineering optimization problems, the idea of using a population of solutions was evolved during the 1950 and 1960s. John Holland of the University of Michigan initiated the idea of sexual reproduction of solutions to EA [31]. This initiation of the new method of the optimization considered makes Holland considered to be the father of Evolutionary Algorithm . His book, written in 1975 [31] particularly gives an idea about his creative thinker approach. De Jong, in one of his papers entitled Genetic Algorithms emphasized that GA is just NOT Function Optimizers [32] and has more potential than a mere robust method for optimizing and engineering system.

Though there are certain differences in EC algorithms, the basic principles of assessing the performance of the solution and then keeping the better performing solutions for further processing and make the worst performers perish remains the same for all cases [33]. Population-based incremental learning (PBIL) [34], a type of EC algorithm has similarities to ant colony optimization which is discussed later. PBIL has a generating vector which is nothing but a vector and probabilities. Binary strings representing the solutions generated randomly has the ith bit having a probability value corresponding to the generating vector. When the solutions in the population are evaluated through the fitness function the probability values in the generating vector is updated is depending on the quality of the performance of the solutions. In case of ant colony optimization , pheromone trail values are similarly updated depending on the performance.

4 Swarm Optimization

In 1995 by Eberhart and Kennedy developed Particle Swarm Optimization (PSO), which is an optimization method inspired by the behaviors of birds flocking or fish schooling [35]. Later several variations of PSO have emerged for improving the convergence speed and solution quality. Basic PSO is found to be more appropriate for processing simple as well as static optimization problems. The PSO imitate behavior of such animal societies those find their food without the existence of any leader in the group or swarm. In such cases, the members of the group try to follow other members of the group depending on the closeness of the member from the source of food. This can only be possible due to the presence of communication system among the members of the group. The member who is in a better condition should have the facility to inform the other members of the flocks regarding his present position which can be followed by the other members. This situation is repeated until the food source could be reached. This basic concept of animal behavior is used by PSO for finding optimal solutions. Each particle of the swarm of a particle in particle optimization represents potential solutions.

To improve the convergence speed and solution quality in PSO several variations have been developed, viz., inertia weight, velocity clamping, synchronous and asynchronous updates, constriction coefficient [36]. PSO algorithm [5] adjusts the trajectories of the agents during its search for the optimum solutions in the search space. Each agent or particle compares its current position with its own best location during its random movement. As it reaches a location better than its previous best location, it updates the best location. In this way, the particles search the global best location during the random search. Ant colony optimization (ACO) is another global optimization technique, which falls within the swarm optimization group. Ant colonies are generally formed by social insect societies. Due to the highly structured social organization, ant colonies can accomplish complex tasks. The behavior of real ant acts as the source of the information for this novel algorithm, where the principle of self-organizing of ants in a highly coordinated manner is exploited. In case of SEO populations of artificial ants collaborate between themselves to solve optimization problems. Different ant algorithms have evolved depending on various aspects of ant behavior which include division of labor, foraging, cooperative transport, and brood sorting. In all the above activities, ants’ coordinates among themselves through indirect communication methods called stigmergy. In case of foraging a chemical named pheromone, a deposited on the ground by the ants to inform the other ants about its path of movements and directing them to follow the same path. This stigmergic communication system has been proved by the biologists as the methods to achieve the self-organization. One of the most flourishing examples of several ant algorithms is ACO. The communication through pheromone which detected by the antenna of ants make the ants follows the path where a larger amount of pheromone is present. This behavior pursuit by real ants can be considered as a heuristic method. The initial explorations of the ants in the neighboring area occur in a complete random manner without the presence of pheromone. But the addition of pheromone incorporates a certain amount of control over the random search. During the random travel deposition by the pheromone by the ants provide some feedback to the other ants and search methods becomes direction control to a certain extent. As pheromone is evaporating in nature an unused path disappeared with time [37].

Dorigo and Gambardela [38] proposed an improved algorithm, where the three main differences with the previous and system are transition rule, local updating, global updating. The transition rule is made to generate a relation between the explorations and priority of the problem. The local updating methods update the pheromone deposition of the ants during the local search whereas global updating is done for the ants with superior position. Figure 3 shows a general scheme of the Ant Colony Swarm algorithm.

Fig. 3
figure 3

Structure of ACO algorithm

Full size image

5 Simulated Annealing

Simulated Annealing (SA) mimics a physical process called ‘annealing’, which has a close relation with thermodynamics, and a method to control the microstructure and properties of metals and alloys through a process of heating and cooling [39]. In this process, the temperature is reduced slowly so that the material can achieve an internal structure or arrangements of atoms in the lowest energy configuration or equilibrium condition. In simulated annealing the objective function replaces the energy of the material being annealed. In case of a minimization problem, lower solution will have lower energy. The optimization method is made of a random move for hill-climbing. Every move gets a probability as it related to the energy or the function value using a parameter similar to Boltzmann constant. Thus, the probability value has an analogy of temperature in thermodynamics and the quality of the solution can be compared with the energy of the system. At higher probability, more uphill moves are possible. The overall probability starts high and is gradually decreased. In the process, optimum solution is reached by one or more moves.

6 Concluding Remarks

The metaheuristic algorithms have greater flexibility to handle problems with complex objective functions and constraints. This reason has made these techniques more applicable for the industrial situations. In such complex real-life situations, these techniques are being used quite successfully to solve the industrial problems related to design, quality control, productivity, and cost.