Abstract
Objective functions that appear in engineering practice may come from measurements of physical systems and, more often, from computer simulations. In many cases, optimization of such objectives in a straightforward way, i.e., by applying optimization routines directly to these functions, is impractical. One reason is that simulation-based objective functions are often analytically intractable (discontinuous, non-differentiable, and inherently noisy). Also, sensitivity information is usually unavailable, or too expensive to compute. Another, and in many cases even more important, reason is the high computational cost of measurement/simulations. Simulation times of several hours, days or even weeks per objective function evaluation are not uncommon in contemporary engineering, despite the increase of available computing power. Feasible handling of these unmanageable functions can be accomplished using surrogate models: the optimization of the original objective is replaced by iterative re-optimization and updating of the analytically tractable and computationally cheap surrogate. This chapter briefly describes the basics of surrogate-based optimization, various ways of creating surrogate models, as well as several examples of surrogate-based optimization techniques.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
References
Bandler, J.W., Cheng, Q.S., Dakroury, S.A., Mohamed, A.S., Bakr, M.H., Madsen, K., Søndergaard, J.: Space mapping: the state of the art. IEEE Trans. Microwave Theory Tech. 52, 337–361 (2004)
Pironneau, O.: On optimum design in fluid mechanics. J. Fluid Mech. 64, 97–110 (1974)
Queipo, N.V., Haftka, R.T., Shyy, W., Goel, T., Vaidynathan, R., Tucker, P.K.: Surrogate-based analysis and optimization. Progress in Aerospace Sciences 41, 1–28 (2005)
Forrester, A.I.J., Keane, A.J.: Recent advances in surrogate-based optimization. Prog. Aerospace Sciences 45, 50–79 (2009)
Koziel, S., Bandler, J.W., Madsen, K.: A space mapping framework for engineering optimization: theory and implementation. IEEE Trans. Microwave Theory Tech. 54, 3721–3730 (2006)
Koziel, S., Cheng, Q.S., Bandler, J.W.: Space mapping. IEEE Microwave Magazine 9, 105–122 (2008)
Alexandrov, N.M., Lewis, R.M.: An overview of first-order model management for engineering optimization. Optimization and Engineering 2, 413–430 (2001)
Echeverria, D., Hemker, P.W.: Space mapping and defect correction. CMAM Int. Mathematical Journal Computational Methods in Applied Mathematics 5, 107–136 (2005)
Booker, A.J., Dennis, J.E., Frank, P.D., Serafini, D.B., Torczon, V., Trosset, M.W.: A rigorous framework for optimization of expensive functions by surrogates. Structural Optimization 17, 1–13 (1999)
Simpson, T.W., Peplinski, J., Koch, P.N., Allen, J.K.: Metamodels for computer-based engineering design: survey and recommendations. Engineering with Computers 17, 129–150 (2001)
Conn, A.R., Gould, N.I.M., Toint, P.L.: Trust Region Methods. MPS-SIAM Series on Optimization (2000)
Alexandrov, N.M., Dennis, J.E., Lewis, R.M., Torczon, V.: A trust region framework for managing use of approximation models in optimization. Struct. Multidisciplinary Optim. 15, 16–23 (1998)
Echeverría, D., Hemker, P.W.: Manifold mapping: a two-level optimization technique. Computing and Visualization in Science 11, 193–206 (2008)
Koziel, S., Bandler, J.W., Madsen, K.: Quality assessment of coarse models and surrogates for space mapping optimization. Optimization Eng. 9, 375–391 (2008)
Koziel, S., Bandler, J.W.: Coarse and surrogate model assessment for engineering design optimization with space mapping. In: IEEE MTT-S Int. Microwave Symp. Dig, Honolulu, HI, pp. 107–110 (2007)
Koziel, S., Bandler, J.W.: Space-mapping optimization with adaptive surrogate model. IEEE Trans. Microwave Theory Tech. 55, 541–547 (2007)
Alexandrov, N.M., Nielsen, E.J., Lewis, R.M., Anderson, W.K.: First-order model management with variable-fidelity physics applied to multi-element airfoil optimization. In: AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Design and Optimization, Long Beach, CA, AIAA Paper 2000-4886 (2000)
Wu, K.-L., Zhao, Y.-J., Wang, J., Cheng, M.K.K.: An effective dynamic coarse model for optimization design of LTCC RF circuits with aggressive space mapping. IEEE Trans. Microwave Theory Tech. 52, 393–402 (2004)
Robinson, T.D., Eldred, M.S., Willcox, K.E., Haimes, R.: Surrogate-based optimization using multifidelity models with variable parameterization and corrected space mapping. AIAA Journal 46, 2814–2822 (2008)
Søndergaard, J.: Optimization using surrogate models – by the space mapping technique. Ph.D. Thesis, Informatics and Mathematical Modelling, Technical University of Denmark, Lyngby (2003)
Kleijnen, J.P.C.: Kriging metamodeling in simulation: a review. European Journal of Operational Research 192, 707–716 (2009)
Rayas-Sanchez, J.E.: EM-based optimization of microwave circuits using artificial neural networks: the state-of-the-art. IEEE Trans. Microwave Theory Tech. 52, 420–435 (2004)
Giunta, A.A., Wojtkiewicz, S.F., Eldred, M.S.: Overview of modern design of experiments methods for computational simulations. American Institute of Aeronautics and Astronautics, paper AIAA 2003–0649 (2003)
Santner, T.J., Williams, B., Notz, W.: The Design and Analysis of Computer Experiments. Springer, Heidelberg (2003)
Koehler, J.R., Owen, A.B.: Computer experiments. In: Ghosh, S., Rao, C.R. (eds.) Handbook of Statistics, vol. 13, pp. 261–308. Elsevier Science B.V., Amsterdam (1996)
Cheng, Q.S., Koziel, S., Bandler, J.W.: Simplified space mapping approach to enhancement of microwave device models. Int. J. RF and Microwave Computer-Aided Eng. 16, 518–535 (2006)
McKay, M., Conover, W., Beckman, R.: A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21, 239–245 (1979)
Beachkofski, B., Grandhi, R.: Improved distributed hypercube sampling. American Institute of Aeronautics and Astronautics, Paper AIAA 2002–1274 (2002)
Leary, S., Bhaskar, A., Keane, A.: Optimal orthogonal-array-based latin hypercubes. Journal of Applied Statistics 30, 585–598 (2003)
Ye, K.Q.: Orthogonal column latin hypercubes and their application in computer experiments. Journal of the American Statistical Association 93, 1430–1439 (1998)
Palmer, K., Tsui, K.-L.: A minimum bias latin hypercube design. IIE Transactions 33, 793–808 (2001)
Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. The Johns Hopkins University Press, Baltimore (1996)
Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to Derivative-Free Optimization. MPS-SIAM Series on Optimization, MPS-SIAM (2009)
Wild, S.M., Regis, R.G., Shoemaker, C.A.: ORBIT: Optimization by radial basis function interpolation in trust-regions. SIAM J. Sci. Comput. 30, 3197–3219 (2008)
Journel, A.G., Huijbregts, C.J.: Mining Geostatistics. Academic Press, London (1981)
O’Hagan, A.: Curve fitting and optimal design for predictions. Journal of the Royal Statistical Society B 40, 1–42 (1978)
Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)
Jones, D., Schonlau, M., Welch, W.: Efficient global optimization of expensive black-box functions. Journal of Global Optimization 13, 455–492 (1998)
Haykin, S.: Neural Networks: A Comprehensive Foundation, 2nd edn. Prentice-Hall, Englewood Cliffs (1998)
Minsky, M.I., Papert, S.A.: Perceptrons: An Introduction to Computational Geometry. MIT Press, Cambridge (1969)
Gunn, S.R.: Support vector machines for classification and regression. Technical Report. School of Electronics and Computer Science, University of Southampton (1998)
Angiulli, G., Cacciola, M., Versaci, M.: Microwave devices and antennas modeling by support vector regression machines. IEEE Trans. Magn. 43, 1589–1592 (2007)
Smola, A.J., Schölkopf, B.: A tutorial on support vector regression. Statistics and Computing 14, 199–222 (2004)
Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, New York (1995)
Levin, D.: The approximation power of moving least-squares. Mathematics of Computation 67, 1517–1531 (1998)
Aitken, A.C.: On least squares and linear combinations of observations. Proceedings of the Royal Society of Edinburgh 55, 42–48 (1935)
Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT Press, Massachussets (2006)
Geisser, S.: Predictive Inference. Chapman and Hall, Boca Raton (1993)
Koziel, S., Cheng, Q.S., Bandler, J.W.: Implicit space mapping with adaptive selection of preassigned parameters. IET Microwaves, Antennas & Propagation 4, 361–373 (2010)
Alexandrov, N.M., Lewis, R.M., Gumbert, C.R., Green, L.L., Newman, P.A.: Approximation and model management in aerodynamic optimization with variable-fidelity models. AIAA Journal of Aircraft 38, 1093–1101 (2001)
Moré, J.J.: Recent developments in algorithms and software for trust region methods. In: Bachem, A., Grötschel, M., Korte, B. (eds.) Mathematical Programming. The State of Art, pp. 258–287. Springer, Heidelberg (1983)
Leary, S.J., Bhaskar, A., Keane, A.J.: A constraint mapping approach to the structural optimization of an expensive model using surrogates. Optimization and Engineering 2, 385–398 (2001)
Redhe, M., Nilsson, L.: Optimization of the new Saab 9-3 exposed to impact load using a space mapping technique. Structural and Multidisciplinary Optimization 27, 411–420 (2004)
Koziel, S., Bandler, J.W., Cheng, Q.S.: Robust trust-region space-mapping algorithms for microwave design optimization. IEEE Trans. Microwave Theory and Tech. 58, 2166–2174 (2010)
Koziel, S., Bandler, J.W., Cheng, Q.S.: Adaptively constrained parameter extraction for robust space mapping optimization of microwave circuits. IEEE MTT-S Int. Microwave Symp. Dig., 205–208 (2010)
Echeverría, D.: Multi-Level optimization: space mapping and manifold mapping. Ph.D. Thesis, Faculty of Science, University of Amsterdam (2007)
Koziel, S., Echeverría Ciaurri, D.: Reliable simulation-driven design optimization of microwave structures using manifold mapping. Progress in Electromagnetics Research B 26, 361–382 (2010)
Hemker, P.W., Echeverría, D.: A trust-region strategy for manifold mapping optimization. JCP Journal of Computational Physics 224, 464–475 (2007)
Echeverría, D.: Two new variants of the manifold-mapping technique. COMPEL The International Journal for Computation and Mathematics in Electrical Engineering 26, 334–344 (2007)
Kolda, T.G., Lewis, R.M., Torczon, V.: Optimization by direct search: new perspectives on some classical and modern methods. SIAM Review 45, 385–482 (2003)
Marsden, A.L., Wang, M., Dennis, J.E., Moin, P.: Optimal aeroacoustic shape design using the surrogate management framework. Optimization and Engineering 5, 235–262 (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Koziel, S., Ciaurri, D.E., Leifsson, L. (2011). Surrogate-Based Methods. In: Koziel, S., Yang, XS. (eds) Computational Optimization, Methods and Algorithms. Studies in Computational Intelligence, vol 356. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-20859-1_3
Download citation
DOI: https://doi.org/10.1007/978-3-642-20859-1_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-20858-4
Online ISBN: 978-3-642-20859-1
eBook Packages: EngineeringEngineering (R0)