Definition

Heuristic designates a computational procedure that determines an optimal solution by iteratively trying to improve a candidate solution with regard to a given measure of quality. Heuristics make few or no assumptions about the problem being optimized and can search large spaces of candidate solutions toward finding optimal or near-optimal solutions at a reasonable computational cost without being able to guarantee either feasibility or optimality, or even in many cases to state how close to optimality a particular feasible solution is. Heuristics implement some form of stochastic search optimization, such as evolution programming, evolution strategy, genetic algorithms, genetic programming, and differential evolution (Michalewicz 1996; Reeves 1995; Sharda et al. 2003; Zhilinskas and Žilinskas 2008). Other methods having a similar meaning as heuristic are derivative-free, direct search, and black-box optimization techniques.

Characteristics

Heuristic optimization algorithms are developed in all kinds of forms variant from simple “trial and error” to complicated algorithms as evolutionary algorithms. The methods are easy to understand and easy to implement and use. The mathematical formulation of the problem is flexible. Heuristic optimization can be applied to iteratively solve continuous/integer problem. They are usually applied when there is no known algorithm that guarantees for finding the optimal solution in efficient computational cost (i.e., time or memory space) or when a “near-optimal” solution is good enough in practical use.

The common advantages of heuristic optimization algorithms are as follows:

  1. 1.

    Fast: They can find a “near-optimal” solution in a short time.

  2. 2.

    Small: They can work in a relatively small memory space.

The common disadvantages of heuristic optimization algorithms are as follows:

  1. 1.

    Not absolute optimal solution: Heuristic algorithms cannot guarantee to find the optimal solution.

  2. 2.

    Uncertainty: The time required for finding a “near-optimal” solution can be large in an unlucky case.

Cross-References

Evolution Programming

Genetic Algorithms

Particle Swarm Optimization (PSO)

Simulated Annealing

Tabu Search