Abstract
Hyper-heuristics represent a novel search methodology that is motivated by the goal of automating the process of selecting or combining simpler heuristics in order to solve hard computational search problems. An extension of the original hyper-heuristic idea is to generate new heuristics which are not currently known. These approaches operate on a search space of heuristics rather than directly on a search space of solutions to the underlying problem which is the case with most meta-heuristics implementations. In the majority of hyper-heuristic studies so far, a framework is provided with a set of human designed heuristics, taken from the literature, and with good measures of performance in practice. A less well studied approach aims to generate new heuristics from a set of potential heuristic components. The purpose of this chapter is to discuss this class of hyper-heuristics, in which Genetic Programming is the most widely used methodology. A detailed discussion is presented including the steps needed to apply this technique, some representative case studies, a literature review of related work, and a discussion of relevant issues. Our aim is to convey the exciting potential of this innovative approach for automating the heuristic design process.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Asmuni, H., Burke, E.K., Garibaldi, J.M., Mccollum, B.: A novel fuzzy approach to evaluate the quality of examination timetabling. In: Proceedings of the 6th International Conference on the Practice and Theory of Automated Timetabling, pp. 82–102 (2006)
Bäck, T.: An overview of parameter control methods by self-adaption in evolutionary algorithms. Fundam. Inf. 35(1-4), 51–66 (1998)
Bader-El-Din, M.B., Poli, R.: Generating SAT local-search heuristics using a GP hyper-heuristic framework. In: Monmarché, N., Talbi, E.-G., Collet, P., Schoenauer, M., Lutton, E. (eds.) EA 2007. LNCS, vol. 4926, pp. 37–49. Springer, Heidelberg (2008)
Bai, R., Kendall, G.: An investigation of automated planograms using a simulated annealing based hyper-heuristic. In: Ibaraki, T., Nonobe, K., Yagiura, M. (eds.) Metaheuristics: Progress as Real Problem Solver. Operations Research/Computer Science Interface Serices, vol. 32, pp. 87–108. Springer, Heidelberg (2005)
Banzhaf, W., Nordin, P., Keller, R.E., Francone, F.D.: Genetic Programming – An Introduction; On the Automatic Evolution of Computer Programs and its Applications. Morgan Kaufmann, San Francisco (1998)
Battiti, R.: Reactive search: Toward self–tuning heuristics. In: Rayward-Smith, V.J., Osman, I.H., Reeves, C.R., Smith, G.D. (eds.) Modern Heuristic Search Methods, pp. 61–83. John Wiley & Sons Ltd, Chichester (1996)
Battiti, R., Brunato, M.: Reactive search: Machine learning for memory-based heuristics. In: Gonzalez, T.F. (ed.) Approximation Algorithms and Metaheuristics, ch. 21, pp. 1–17. Taylor and Francis Books/CRC Press, Washington (2007)
Birattari, M.: The problem of tuning metaheuristics as seen from a machine learning perspective. Ph.D. thesis, Universite Libre de Bruxelles (2004)
Burke, E.K., Hart, E., Kendall, G., Newall, J., Ross, P., Schulenburg, S.: Hyper-heuristics: An emerging direction in modern search technology. In: Glover, F., Kochenberger, G. (eds.) Handbook of Metaheuristics, pp. 457–474. Kluwer, Dordrecht (2003)
Burke, E.K., Hyde, M.R., Kendall, G.: Evolving bin packing heuristics with genetic programming. In: Runarsson, T.P., Beyer, H.-G., Burke, E.K., Merelo-Guervós, J.J., Whitley, L.D., Yao, X. (eds.) PPSN 2006. LNCS, vol. 4193, pp. 860–869. Springer, Heidelberg (2006)
Burke, E.K., Hyde, M.R., Kendall, G., Woodward, J.: Automatic heuristic generation with genetic programming: evolving a jack-of-all-trades or a master of one. In: Thierens, D., et al. (eds.) Proceedings of the 9th annual conference on Genetic and evolutionary computation GECCO 2007, vol. 2, pp. 1559–1565. ACM Press, London (2007)
Burke, E.K., Hyde, M.R., Kendall, G., Woodward, J.R.: The scalability of evolved on line bin packing heuristics. In: Srinivasan, D., Wang, L. (eds.) 2007 IEEE Congress on Evolutionary Computation, pp. 2530–2537. IEEE Computational Intelligence Society/IEEE Press, Singapore (2007)
Burke, E.K., Kendall, G., Soubeiga, E.: A tabu-search hyperheuristic for timetabling and rostering. J. of Heuristics 9(6), 451–470 (2003)
Burke, E.K., McCollum, B., Meisels, A., Petrovic, S., Qu, R.: A graph-based hyper-heuristic for educational timetabling problems. Eur. J. of Oper. Res. 176, 177–192 (2007)
Burke, E.K., Petrovic, S., Qu, R.: Case based heuristic selection for timetabling problems. J. of Sched. 9(2), 115–132 (2006)
Coffman Jr., E.G., Galambos, G., Martello, S., Vigo, D.: Bin packing approximation algorithms: Combinatorial analysis. In: Du, D.Z., Pardalos, P.M. (eds.) Handbook of Combinatorial Optimization, pp. 151–207. Kluwer, Dordrecht (1998)
Cowling, P., Kendall, G., Soubeiga, E.: A hyperheuristic approach to scheduling a sales summit. In: Burke, E., Erben, W. (eds.) PATAT 2000. LNCS, vol. 2079, pp. 176–190. Springer, Heidelberg (2001) (selected papers)
Dowsland, K.A., Soubeiga, E., Burke, E.K.: A simulated annealing hyper-heuristic for determining shipper sizes. Eur. J. of Oper. Res. 179(3), 759–774 (2007)
Drechsler, N., Schmiedle, F., Grosse, D., Drechsler, R.: Heuristic learning based on genetic programming. In: Miller, J., Tomassini, M., Lanzi, P.L., Ryan, C., Tetamanzi, A.G.B., Langdon, W.B. (eds.) EuroGP 2001. LNCS, vol. 2038, pp. 1–10. Springer, Heidelberg (2001)
Drechsler, R., Becker, B.: Learning heuristics by genetic algorithms. In: ASP-DAC 1995: Proceedings of the 1995 conference on Asia Pacific design automation (CD-ROM), p. 53. ACM, New York (1995)
Eiben, A.E., Hinterding, R., Michalewicz, Z.: Parameter control in Evolutionary Algorithms. IEEE Trans. on Evol. Comput. 3(2), 124–141 (1999)
Falkenauer, E., Delchambre, A.: A genetic algorithm for bin packing and line balancing. In: Proc. of the IEEE 1992 International Conference on Robotics and Automation, pp. 1186–1192 (1992)
Fang, H.L., Ross, P., Corne, D.: A promising hybrid GA/heuristic approach for open-shop scheduling problems. In: Eur. Conference on Artificial Intelligence (ECAI 2004), pp. 590–594 (1994)
Fukunaga, A.S.: Automated discovery of composite sat variable-selection heuristics. In: AAAI/IAAI, pp. 641–648 (2002)
Fukunaga, A.S.: Automated discovery of local search heuristics for satisfiability testing. Evol. Comput. 16(1), 31–61 (2008)
Gagliolo, M., Schmidhuber, J.: Learning dynamic algorithm portfolios. Ann. of Math. and Artif. Intell. 47(3-4), 295–328 (2006)
Garey, M.R., Johnson, D.S.: Computers and Intractability: A Guide to the Theory of NP-Completeness. Series of Books in the Mathematical Sciences. W. H. Freeman, New York (1979)
Geiger, C.D., Uzsoy, R., Aytug, H.: Rapid modeling and discovery of priority dispatching rules: An autonomous learning approach. J. of Sched. 9(1), 7–34 (2006)
Hart, E., Ross, P., Nelson, J.: Solving a real-world problem using an evolving heuristically driven schedule builder. Evol. Comput. 6(1), 61–80 (1998)
Hoos, H.H., Sttzle, T.: Stochastic Local Search: Foundations and Applications. Morgan Kaufmann / Elsevier (2005)
Huberman, B.A., Lukose, R.M., Hogg, T.: An economics approach to hard computational problems. Sci. 275, 51–54 (1997)
Hutter, F., Hamadi, Y., Hoos, H.H., Leyton-Brown, K.: Performance prediction and automated tuning of randomized and parametric algorithms. In: Benhamou, F. (ed.) CP 2006. LNCS, vol. 4204, pp. 213–228. Springer, Heidelberg (2006)
Hutter, F., Hoos, H.H., Stützle, T.: Automatic algorithm configuration based on local search. In: AAAI, pp. 1152–1157. AAAI Press, Menlo Park (2007)
Jakob, W.: HyGLEAM – an approach to generally applicable hybridization of evolutionary algorithms. In: Guervós, J.J.M., Adamidis, P.A., Beyer, H.-G., Fernández-Villacañas, J.-L., Schwefel, H.-P. (eds.) PPSN 2002. LNCS, vol. 2439, pp. 527–536. Springer, Heidelberg (2002)
Jakob, W.: Towards an adaptive multimeme algorithm for parameter optimisation suiting the engineers’ needs. In: Runarsson, T.P., Beyer, H.-G., Burke, E.K., Merelo-Guervós, J.J., Whitley, L.D., Yao, X. (eds.) PPSN 2006. LNCS, vol. 4193, pp. 132–141. Springer, Heidelberg (2006)
Johnson, D., Demers, A., Ullman, J., Garey, M., Graham, R.: Worst-case performance bounds for simple one-dimensional packaging algorithms. SIAM J. on Comput. 3(4), 299–325 (1974)
Keller, R.E., Poli, R.: Linear genetic programming of parsimonious metaheuristics. In: Srinivasan, D., Wang, L. (eds.) 2007 IEEE Congress on Evolutionary Computation, pp. 4508–4515. IEEE Computational Intelligence Society/IEEE Press, Singapore (2007)
Koza, J.R.: Genetic Programming: on the Programming of Computers by Means of Natural Selection. The MIT Press, Boston (1992)
Koza, J.R.: Genetic Programming II: Automatic Discovery of Reusable Programs. The MIT Press, Cambridge (1994)
Koza, J.R., Bennett III, F.H., Andre, D., Keane, M.A.: Genetic Programming III: Darwinian Invention and Problem solving. Morgan Kaufmann, San Francisco (1999)
Koza, J.R., Keane, M.A., Streeter, M.J., Mydlowec, W., Yu, J., Lanza, G.: Genetic Programming IV: Routine Human-Competitive Machine Intelligence (Genetic Programming). Springer, Heidelberg (2003)
Koza, J.R., Poli, R.: Genetic programming. In: Burke, E.K., Kendall, G. (eds.) Search Methodologies: Introductory Tutorials in Optimization and Decision Support Techniques, pp. 127–164. Springer, Boston (2005)
Krasnogor, N., Gustafson, S.: A study on the use of ’self-generation’ in memetic algorithms. Nat. Comput. 3(1), 53–76 (2004)
Krasnogor, N., Smith, J.E.: Emergence of profitable search strategies based on a simple inheritance mechanism. In: Proceedings of the 2001 Genetic and Evolutionary Computation Conference, pp. 432–439. Morgan Kaufmann, San Francisco (2001)
Mockus, J.: Application of bayesian approach to numerical methods of global and stochastic optimization. J. of Glob. Optim. 4(4), 347–366 (1994)
Nareyek, A.: Choosing search heuristics by non-stationary reinforcement learning. In: Resende, M.G.C., de Sousa, J.P. (eds.) Metaheuristics: Computer Decision-Making, ch. 9, pp. 523–544. Kluwer, Dordrecht (2003)
Ong, Y.S., Keane, A.J.: Meta-lamarckian learning in memetic algorithms. IEEE Trans. on Evol. Comput. 8, 99–110 (2004)
Ong, Y.S., Lim, M.H., Zhu, N., Wong, K.W.: Classification of adaptive memetic algorithms: a comparative study. IEEE Trans. on Syst. Man and Cybern. Part B 36(1), 141–152 (2006)
Ozcan, E., Bilgin, B., Korkmaz, E.E.: A comprehensive survey of hyperheuristics. Intell. Data Anal. 12(1), 3–23 (2008)
Poli, W.B.R., Langdon, N.F.M.: A Field Guide to Genetic Programming. Lulu Enterprises, UK (2008)
Rhee, W.T., Talagrand, M.: On line bin packing with items of random size. Math. Oper. Res. 18, 438–445 (1993)
Ropke, S., Pisinger, D.: A unified heuristic for a large class of vehicle routing problems with backhauls. Eur. J. of Oper. Res. 171(3), 750–775 (2006)
Ross, P.: Hyper-heuristics. In: Burke, E.K., Kendall, G. (eds.) Search Methodologies: Introductory Tutorials in Optimization and Decision Support Techniques, ch. 17, pp. 529–556. Springer, Heidelberg (2005)
Ross, P., Marin-Blazquez, J.G., Schulenburg, S., Hart, E.: Learning a procedure that can solve hard bin-packing problems: A new ga-based approach to hyper-heuristics. In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2003, pp. 1295–1306. Springer, Heidelberg (2003)
Seiden, S.S.: On the online bin packing problem. J. ACM 49(5), 640–671 (2002)
Smith, J.E.: Co-evolving memetic algorithms: A review and progress report. IEEE Trans. in Syst. Man and Cybern. Part B 37(1), 6–17 (2007)
Stephenson, M., O’Reilly, U., Martin, M., Amarasinghe, S.: Genetic programming applied to compiler heuristic optimisation. In: Proceedings of the Eur. Conference on Genetic Programming, pp. 245–257. Springer, Heidelberg (2003)
Terashima-Marin, H., Flores-Alvarez, E.J., Ross, P.: Hyper-heuristics and classifier systems for solving 2D-regular cutting stock problems. In: Beyer, H.G., O’Reilly, U.M. (eds.) Proceedings of Genetic and Evolutionary Computation Conference, GECCO 2005, Washington DC, USA, June 25-29, pp. 637–643. ACM, New York (2005)
Terashima-Marin, H., Ross, P., Valenzuela-Rendon, M.: Evolution of constraint satisfaction strategies in examination timetabling. In: Proc. of the Genetic and Evolutionary Computation Conf. GECCO 1999, pp. 635–642. Morgan Kaufmann, San Francisco (1999)
Thrun, S., Pratt, L.: Learning to learn: Introduction and overview. In: Thrun, S., Pratt, L. (eds.) Learning To Learn. Kluwer Academic Publishers, Dordrecht (1998)
Vazquez-Rodriguez, J.A., Petrovic, S., Salhi, A.: A combined meta-heuristic with hyper-heuristic approach to the scheduling of the hybrid flow shop with sequence dependent setup times and uniform machines. In: Proceedings of the 3rd Multidisciplinary International Scheduling Conference: Theory and Applications (MISTA 2007), pp. 506–513 (2007)
Wah, B.W., Ieumwananonthachai, A.: Teacher: A genetics-based system for learning and for generalizing heuristics. In: Yao, X. (ed.) Evol. Comput., pp. 124–170. World Scientific Publishing Co. Pte. Ltd, Singapore (1999)
Wah, B.W., Ieumwananonthachai, A., Chu, L.C., Aizawa, A.: Genetics-based learning of new heuristics: Rational scheduling of experiments and generalization. IEEE Trans. on Knowl. and Data Eng. 7(5), 763–785 (1995)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Burke, E.K., Hyde, M.R., Kendall, G., Ochoa, G., Ozcan, E., Woodward, J.R. (2009). Exploring Hyper-heuristic Methodologies with Genetic Programming. In: Mumford, C.L., Jain, L.C. (eds) Computational Intelligence. Intelligent Systems Reference Library, vol 1. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-01799-5_6
Download citation
DOI: https://doi.org/10.1007/978-3-642-01799-5_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-01798-8
Online ISBN: 978-3-642-01799-5
eBook Packages: EngineeringEngineering (R0)