Summary
EDAs have been shown to perform well on a wide variety of single-objective optimization problems, for binary and real-valued variables. In this chapter we look into the extension of the EDA paradigm to multi-objective optimization. To this end, we focus the chapter around the introduction of a simple, but effective, EDA for multi-objective optimization: the naive \( \mathbb{M} \) ID\( \mathbb{E} \)AA (mixture-based multi-objective iterated density-estimation evolutionary algorithm). The probabilistic model in this specific algorithm is a mixture distribution. Each component in the mixture is a univariate factorization. As will be shown in this chapter, mixture distributions allow for wide-spread exploration of a multi-objective front, whereas most operators focus on a specific part of the multi-objective front. This wide-spread exploration aids the important preservation of diversity in multi-objective optimization. To further improve and maintain the diversity that is obtained by the mixture distribution, a specialized diversity preserving selection operator is used in the naive \( \mathbb{M} \) ID\( \mathbb{E} \)A. We verify the effectiveness of the naive \( \mathbb{M} \) ID\( \mathbb{E} \)A in two different problem domains and compare it with two other well-known efficient multi-objective evolutionary algorithms (MOEAs).
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
- Pareto Front
- Objective Space
- Pareto Optimal Front
- Multiobjective Evolutionary Algorithm
- Diversity Preservation
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
P. A. N. Bosman and D. Thierens. Advancing continuous ID\( \mathbb{E} \)As with mixture distributions and factorization selection metrics. In M. Pelikan and K. Sastry, editors, Proceedings of the Optimization by Building and Using Probabilistic Models OBUPM Workshop at the GECCO-2001 Genetic and Evolutionary Computation Conference, pp. 208–212. Morgan Kaufmann Publishers, 2001.
P. A. N. Bosman and D. Thierens. Exploiting gradient information in continuous iterated density estimation evolutionary algorithms. In B. Kröse, M. de Rijke, G. Schreiber, and M. van Someren, editors, Proceedings of the 13th Belgium-Netherlands Artificial Intelligence Conference BNAIC’01, pp. 69–76, 2001.
P. A. N. Bosman and D. Thierens. Multi-objective optimization with diversity preserving mixture-based iterated density estimation evolutionary algorithms. International Journal of Approximate Reasoning, 31:259–289, 2002.
P. A. N. Bosman and D. Thierens. Permutation optimization by iterated estimation of random keys marginal product factorizations. In J. J. Merelo, P. Adamidis, H.-G. Beyer, J.-J. Fernández-Villicañas, and H.-P. Schwefel, editors, Parallel Problem Solving from Nature-PPSN VII, pp. 331–340, Berlin, 2002. Springer-Verlag.
P. A. N. Bosman and D. Thierens. The balance between proximity and diversity in multiobjective evolutionary algorithms. IEEE Transactions on Evolutionary Computation, 7:174–188, 2003.
P.A.N. Bosman and D. Thierens. A thorough documentation of obtained results on realvalued continuous and combinatorial multi-objective optimization problems using diversity preserving mixture-based iterated density estimation evolutionary algorithms. Technical report UU-CS-2002-52, Institute of Information and Computing Sciences, Utrecht University, Utrecht, 2002.
W. Buntine. Operations for learning with graphical models. Journal of Artificial Intelligence Research, 2:159–225, 1994.
C. K. Chow and C. N. Liu. Approximating discrete probability distributions with dependence trees. IEEE Transactions on Information Theory, 14:462–467, 1968.
C. A. Coello Coello. A comprehensive survey of evolutionary-based multiobjective optimization techniques. Knowledge and Information Systems. An International Journal, 1(3):269–308, 1999.
A. P. Dawid and S. L. Lauritzen. Hyper Markov laws in the statistical analysis of decomposable graphical models. Annals of Statistics, 21:1272–1317, 1993.
K. Deb. Multi-objective genetic algorithms: Problem difficulties and construction of test problems. Evolutionary Computation, 7(3):205–230, 1999.
K. Deb, S. Agrawal, A. Pratab, and T. Meyarivan. A fast elitist non-dominated sorting genetic algorithm for multi-objective optimization: NSGA-II. In M. Schoenauer, K. Deb, G. Rudolph, X. Yao, E. Lutton, J. J. Merelo, and H.-P. Schwefel, editors, Parallel Problem Solving from Nature-PPSN VI, pp. 849–858. Springer, 2000.
K. Deb, A. Pratap, and T. Meyarivan. Constrained test problems for multi-objective evolutionary optimization. In E. Zitzler, K. Deb, L. Thiele, C. A. Coello Coello, and D. Corne, editors, First International Conference on Evolutionary Multi-Criterion Optimization, pp. 284–298, Berlin, 2001. Springer-Verlag.
A.P. Dempster, N.M. Laird, and D.B. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistic Society, Series B 39:1–38, 1977.
M. Ehrgott and X. Gandibleux. An annotated bibliography of multi-objective combinatorial optimization. Technical Report 62/2000, Fachbereich Mathematik, Universität Kaiserslautern, Kaiserslautern, 2000.
C. M. Fonseca and P. J. Fleming. An overview of evolutionary algorithms in multiobjective optimization. Evolutionary Computation, 3(1):1–16, 1995.
D. E. Goldberg. Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley, Reading, 1989.
G. Harik. Linkage learning via probabilistic modeling in the ECGA. IlliGAL Technical Report 99010, 1999.
G. Harik, F. Lobo, and D. E. Goldberg. The compact genetic algorithm. In Proceedings of the 1998 IEEE International Conference on Evolutionary Computation, pp. 523–528. IEEE Press, 1998.
J. A. Hartigan. Clustering Algorithms. John Wiley & Sons, Inc., 1975.
M.G. Kendall and A. Stuart. The Advanced Theory Of Statistics, Volume 2, Inference And Relationship. Charles Griffin & Company Limited, 1967.
J. Knowles and D. Corne. On metrics for comparing non-dominated sets. In Proceedings of the 2002 Congress on Evolutionary Computation CEC 2002, pp. 666–674, Piscataway, New Jersey, 2002. IEEE Press.
P. Larrañaga and J. A. Lozano. Estimation of Distribution Algorithms. A New Tool for Evolutionary Computation. Kluwer Academic Publishers, 2001.
M. Laumanns, E. Zitzler, and L. Thiele. On the effects of archiving, elitism, and density based selection in evolutionary multi-objective optimization. In E. Zitzler, K. Deb, L. Thiele, C. A. Coello Coello, and D. Corne, editors, Proceedings of the First International Conference on Evolutionary Multi-Criterion Optimization-EMO 2001, pp. 181–197. Springer-Verlag, 2001.
S. L. Lauritzen. Graphical Models. Clarendon Press, Oxford, 1996.
H. Mühlenbein and T. Mahnig. FDA-a scalable evolutionary algorithm for the optimization of additively decomposed functions. Evolutionary Computation, 7:353–376, 1999.
H. Mühlenbein and G. Paaß. From recombination of genes to the estimation of distributions I. binary parameters. In A. E. Eiben, T. Bäck, M. Schoenauer, and H.-P. Schwefel, editors, Parallel Problem Solving from Nature-PPSN V, pp. 178–187. Springer, 1998.
A. Ochoa, H. Mühlenbein, and M. Soto. A factorized distribution algorithm using single connected Bayesian networks. In M. Schoenauer et al., editor, Parallel Problem Solving from Nature-PPSN VI, pp. 787–796. Springer, 2000.
M. Pelikan and D. E. Goldberg. Escaping hierarchical traps with competent genetic algorithms. In L. Spector, E. D. Goodman, A. Wu, W. B. Langdon, H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M. H. Garzon, and E. Burke, editors, Proceedings of the GECCO-2001 Genetic and Evolutionary Computation Conference, pp. 511–518. Morgan Kaufmann, 2001.
M. Pelikan, D. E. Goldberg, and F. Lobo. A survey of optimization by building and using probabilistic models. Computational Optimization and Applications, 21(1):5–20, 2002.
M. Pelikan, D. E. Goldberg, and K. Sastry. Bayesian optimization algorithm, decision graphs and Occam’s razor. In L. Spector, E. D. Goodman, A. Wu, W. B. Langdon, H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M. H. Garzon, and E. Burke, editors, Proceedings of the GECCO-2001 Genetic and Evolutionary Computation Conference, pp. 519–526. Morgan Kaufmann, 2001.
R. Santana, A. Ochoa, and M. R. Soto. The mixture of trees factorized distribution algorithm. In L. Spector, E. D. Goodman, A. Wu, W. B. Langdon, H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M. H. Garzon, and E. Burke, editors, Proceedings of the GECCO-2001 Genetic and Evolutionary Computation Conference, pp. 543–550. Morgan Kaufmann, 2001.
M. Soto and A. Ochoa. A factorized distribution algorithm based on polytrees. In Proceedings of the 2000 Congress on Evolutionary Computation CEC00, pp. 232–237. IEEE Press, 2000.
D. A. Van Veldhuizen. Multiobjective Evolutionary Algorithms: Classifications, Analyses, and New Innovations. PhD thesis, Graduate School of Engineering of the Air Force Institute of Technology, WPAFB, Ohio, 1999.
E. Zitzler. Evolutionary Algorithms for Multiobjective Optimization: Methods and Applications. PhD thesis, Swiss Federal Institute of Technology (ETH), Zurich, Switzerland, 1999.
E. Zitzler, K. Deb, and L. Thiele. Comparison of multiobjective evolutionary algorithms: Empirical results. Evolutionary Computation, 8(2):173–195, 2000.
E. Zitzler, M. Laumanns, L. Thiele, C. M. Fonseca, and V. Grunert da Fonseca. Why quality assessment of multiobjective optimizers is difficult. In W. B. Langdon, E. Cantú-Paz, K. Mathias, R. Roy, D. Davis, R. Poli, K. Balakrishnan, V. Honavar, G. Rudolph, J. Wegener, L. Bull, M. A. Potter, A. C. Schultz, J. F. Miller, E. Burke, and N. Jonoska, editors, Proceedings of the GECCO-2002 Genetic and Evolutionary Computation Conference, pp. 666–674, San Francisco, California, 2002. Morgan Kaufmann.
E. Zitzler and L. Thiele. Multiobjective evolutionary algorithms: A comparative case study and the strength Pareto approach. IEEE Transactions on Evolutionary Computation, 3(4):257–271, 1999.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Bosman, P.A., Thierens, D. (2006). Multi-objective Optimization with the Naive \( \mathbb{M} \) ID\( \mathbb{E} \)A. In: Lozano, J.A., Larrañaga, P., Inza, I., Bengoetxea, E. (eds) Towards a New Evolutionary Computation. Studies in Fuzziness and Soft Computing, vol 192. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-32494-1_6
Download citation
DOI: https://doi.org/10.1007/3-540-32494-1_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29006-3
Online ISBN: 978-3-540-32494-2
eBook Packages: EngineeringEngineering (R0)