Abstract
We propose an algorithm for the global optimization of expensive and noisy black box functions using a surrogate model based on radial basis functions (RBFs). A method for RBF-based approximation is introduced in order to handle noise. New points are selected to minimize the total model uncertainty weighted against the surrogate function value. The algorithm is extended to multiple objective functions by instead weighting against the distance to the surrogate Pareto front; it therefore constitutes the first algorithm for expensive, noisy and multiobjective problems in the literature. Numerical results on analytical test functions show promise in comparison to other (commercial) algorithms, as well as results from a simulation based optimization problem.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Audet C, Savard G, Zghal W (2008) Multiobjective optimization through a series of single-objective formulations. SIAM J Optim 19(1):188–210
Björkman M, Holmström K (2000) Global optimization of costly nonconvex functions. Optim Eng 1:373–397
Booker AJ, Dennis JE Jr, Frank PD, Serafini DB, Torczon V (1998) Optimization using surrogate objectives on a helicopter test example. In: Computational methods for optimal design and control, Arlington, VA, 1997. Progr systems control theory, vol 24. Birkhäuser Boston, Boston, pp 49–58
Booker AJ, Dennis J Jr, Frank PD, Serafin DB, Torczon V, Torsset MW (1999) A rigorous framework for optimization of expensive functions by surrogates. Struct Optim 17:1–13
Conn AR, Scheinberg K, Toint PL (1997) Recent progress in unconstrained nonlinear optimization without derivatives. Math Program 79(1–3):397–414. Lectures on mathematical programming (97) (Lausanne, 1997)
Das I, Dennis JE (1998) Normal-boundary intersection: a new method for generating the Pareto surface in nonlinear multicriteria optimization problems. SIAM J Optim 8(3):631–657. (Electronic)
Deb K (1999) Multi-objective genetic algorithms: problem difficulties and construction of test problems. Evol Comput 7(3):205–230
Deb K, Thiele L, Laumanns M, Zitzler E (2001) Scalable test problems for evolutionary multi-objective optimization. Technical report, Computer engineering and networks laboratory (TIK), Swiss federal institute of technology (ETH)
Dixon LCW, Szegö G (1978) The global optimization problem: an introduction. North-Holland, Amsterdam
Gutmann HM (2001) A radial basis function method for global optimization. J Glob Optim 19:201–227
Hanne T (2006) Applying multiobjective evolutionary algorithms in industrial projects. In: Küfer KH, Rommelfanger H, Tammer C, Winkler K (eds) Multicriteria decision making and fuzzy systems. Theory, methods and applications. Shaker Verlag, Aachen, pp 125–142
Huang D, Allen TT, Notz WI, Zeng N (2006) Global optimization of stochastic black-box systems via sequential kriging meta-models. J Glob Optim 34:441–466
Huyer W, Neumaier A (1999) Global optimization by multilevel coordinate search. J Glob Optim 14:331–355
Jakobsson S, Saif-Ul-Hasnain M, Rundqvist R, Edelvik F, Andersson B, Patriksson M, Ljungqvist M, Lortet D, Wallesten J (2008) Combustion engine optimization: a multiobjective approach. Optim Eng (in press)
Jones DR (2001) A taxonomy of global optimization methods based on response surfaces. J Glob Optim 21:345–383
Jones DR, Perttunen CD, Stuckman BE (1993) Lipschitzian optimization without the Lipschitz constant. J Optim Theory Appl 79:157–181
Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13:455–492
Knowles J (2006) ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems. IEEE Trans Evol Comput 10:50–66
Kohavi R (1999) A study of cross-validation and bootstrap for accuracy estimation and model selection. In: Proceedings of the 14:th international joint conference on artificial intelligence (IJCAI). Morgan Kaufmann, San Mateo, pp 1137–1145
Kursawe F (1991) A variant of evolution strategies for vector optimization. In: Schwefel HP, Männer R (eds) Parallel problem solving from nature. 1st workshop, PPSN I, vol 496. Springer, Berlin, pp 193–197
Lewis RM, Torczon V, Trosset MW (2000) Direct search methods: then and now. J Comput Appl Math 124:191–207
Messac A, Mullur AA (2008) A computationally efficient metamodeling approach for expensive multiobjective optimization. Optim Eng 9(1):37–67
Miettinen K (1999) Nonlinear multiobjective optimization. International series in operations research & management science, vol 12. Kluwer Academic, Dordrecht
Powell MJD (2006) The NEWUOA software for unconstrained optimization without derivatives. In: Large-scale nonlinear optimization. Nonconvex optim appl, vol 83. Springer, New York, pp 255–297
Regis RG, Shoemaker CA (2005) Constrained global optimization of expensive black box functions using radial basis functions. J Glob Optim 31:153–171
Regis RG, Shoemaker CA (2007) A stochastic radial basis function method for the global optimization of expensive functions. INFORMS J Comput 19(4):497–509
Rudholm J, Wojciechowski A (2007) A method for simulation based optimization using radial basis functions. Master’s thesis, Chalmers university of technology, Göteborg, www.chalmers.se/math/EN/research/research-groups/optimization/master-thesis-projects
Saif-Ul-Hasnain M (2008) Simulation based multiobjective optimization of diesel combustion engines. Master’s thesis, Chalmers university of technology
Shor NZ (1985) Minimization methods for nondifferentiable functions. Springer series in computational mathematics, vol 3. Springer, Berlin
Wendland H (2005) Scattered data approximation. Cambridge monographs on applied and computational mathematics, vol 17. Cambridge University Press, Cambridge
Ye KQ, Li W, Sudjianto A (2000) Algorithmic construction of optimal symmetric Latin hypercube designs. J Stat Plan Inference 90:149–159
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Jakobsson, S., Patriksson, M., Rudholm, J. et al. A method for simulation based optimization using radial basis functions. Optim Eng 11, 501–532 (2010). https://doi.org/10.1007/s11081-009-9087-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11081-009-9087-1