Abstract
This paper summarizes the discussion at the Approximation Methods Panel that was held at the 9thAIAA/ISSMO Symposium on Multidisciplinary Analysis & Optimization in Atlanta, GA on September 2–4, 2002. The objective of the panel was to discuss the current state-of-the-art of approximation methods and identify future research directions important to the community. The panel consisted of five representatives from industry and government: (1) Andrew J. Booker from The Boeing Company, (2) Dipankar Ghosh from Vanderplaats Research & Development, (3) Anthony A. Giunta from Sandia National Laboratories, (4) Patrick N. Koch from Engineous Software, Inc., and (5) Ren-Jye Yang from Ford Motor Company. Each panelist was asked to (i) give one or two brief examples of typical uses of approximation methods by his company, (ii) describe the current state-of-the-art of these methods used by his company, (iii) describe the current challenges in the use and adoption of approximation methods within his company, and (iv) identify future research directions in approximation methods. Several common themes arose from the discussion, including differentiating between design of experiments and design and analysis of computer experiments, visualizing experimental results and data from approximation models, capturing uncertainty with approximation methods, and handling problems with large numbers of variables. These are discussed in turn along with the future directions identified by the panelists, which emphasized educating engineers in using approximation methods.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Audet, C.; Booker, A.J.; Dennis, J.E., Jr; Frank, P.D.; Moore, D. 2000: A surrogate-model-based method for constrained optimization. In: 8th AIAA/NASA/USAF/ISSMO Symposium on Multidisciplinary Analysis and Optimization (held in Long Beach, CA), AIAA-2000-4891. Reston, VA: AIAA
Balabanov, V.; Kaufman, M.; Knill, D.L.; Golovidov, O.; Giunta, A.A.; Haftka, R.T.; Grossman, B.; Mason, W.H.; Watson, L.T. 1996: Dependence of optimal structural weight on aerodynamic shape for a high speed civil transport. In: 6th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization (held in Bellevue, WA), Vol. 1, pp. 599–612, AIAA-96-4046. Reston, VA: AIAA
Balabanov, V.; Giunta, A.A.; Golovidov, O.; Grossman, B.; Mason, W.H.; Watson, L.T. 1999: Reasonable design space approach to response surface approximation. Journal of Aircraft36(1), 308–315
Balabanov, V.; Charpentier, C.; Ghosh, D.K.; Quinn, G.; Vanderplaats, G.; Venter, G. 2002: VisualDOC: A software system for general purpose integration and design optimization. In: 9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization (held in Atlanta, GA), AIAA-2002-5513. Reston, VA: AIAA
Barthelemy, J.-F.M.; Haftka, R.T. 1993: Approximation concepts for optimum structural design – a review. Structural Optimization5, 129–144
Barton, R.R. 1998: Simulation metamodels. In: Medeiros, D.J.; Watson, E.F.; Carson, J.S.; Manivannan, M.S. (eds.) Proceedings of the 1998 Winter Simulation Conference (held in Washington, DC), pp. 167–174. IEEE
Booker, A.J. 1998: Design and analysis of computer experiments. In: 7th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis & Optimization (held in St. Louis, MO), Vol. 1, pp. 118–128, AIAA-98-4757. Reston, VA: AIAA
Booker, A.J.; Dennis, J.E., Jr.; Frank, P.D.; Serafini, D.B.; Torczon, V.; Trosset, M.W. 1999: A rigorous framework for optimization of expensive functions by surrogates. Structural Optimization17(1), 1–13
Booker, A.J. 2000a: Examples of surrogate modeling of computer simulations. In: ISSMO/NASA First Internet Conference on Approximations and Fast Reanalysis in Engineering Optimization
Booker, A.J. 2000b: Well-conditioned kriging models for optimization of computer simulations. Technical Document Series, M&CT-TECH-002, Mathematics & Computing Technology, Phantom Works, The Boeing Company, Seattle, WA
Box, G.E.P.; Draper, N.R. 1959: A basis for the selection of a response surface design. Journal of the American Statistical Association54(287), 622–654
Box, G.E.P.; Draper, N.R. 1969: Evolutionary operation: A statistical method for process management. New York: John Wiley & Sons, Inc.
Box, G.E.P.; Hunter, W.G.; Hunter, J.S. 1978: Statistics for experimenters: An introduction to design, data analysis, and model building. New York: John Wiley & Sons, Inc.
Box, G.E.P.; Draper, N.R. 1987: Empirical model building and response surfaces. New York: John Wiley & Sons, Inc.
Cressie, N. 1988: Spatial prediction and ordinary kriging. Mathematical Geology20(4), 405–421
Currin, C.; Mitchell, T.; Morris, M.; Ylvisaker, D. 1991: Bayesian prediction of deterministic functions, with applications to the design and analysis of computer experiments. Journal of the American Statistical Association86(416), 953–963
Eldred, M.S.; Giunta, A.A.; van Bloemen Waanders, B.G.; Wojtkiewicz, S.F., Jr.; Hart, W.E.; Alleva, M.P. 2002a: Dakota, a multilevel parallel object-oriented framework for design optimization, parameter estimation, uncertainty quantification, and sensitivity analysis (Version 3.0 users manual). Sandia Technical Report SAND2001-3796, Sandia National Laboratories, Albuquerque, NM
Eldred, M.S.; Giunta, A.A.; Wojtkiewicz, S.F., Jr.; Truca-no, T.G. 2002b: Formulations for surrogate-based optimization under uncertainty. In: 9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization (held in Atlanta, GA), AIAA-2002-5585. Reston, VA: AIAA
Evans, M.; Swartz, T. 2000: Approximating integrals via Monte Carlo and deterministic methods. New York: Oxford University Press
Fang, K.-T.; Wang, Y. 1994: Number-theoretic methods in statistics. New York: Chapman & Hall
Fang, K.-T.; Lin, D.K.J.; Winker, P. Zhang, Y. 2000: Uniform design: Theory and application. Technometrics42, 237–248
Farhang-Mehr, A.; Azarm, S. 2002: A sequential information-theoretic approach to design of computer experiments. In: 9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization (held in Atlanta, GA), AIAA-2002-5571. Reston, VA: AIAA
Gearhart, C.; Wang, B.P. 2001: Bayesian metrics for comparing response surface models for data with uncertainty. Structural and Multidisciplinary Optimization22(3), 198–207
Giunta, A.; Watson, L.T. 1998: A comparison of approximation modeling techniques: Polynomial versus interpolating models. In: 7th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis & Optimization (held in St. Louis, MO), Vol. 1, pp. 392–404, AIAA-98-4758. Reston, VA: AIAA
Giunta, A.A.; Dudley, J.M.; Narducci, R.; Grossman, B.; Haftka, R.T.; Mason, W.H.; Watson, L.T. 1994: Noisy aerodynamic response and smooth approximations in HSCT design. In: 5th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization (held in Panama City, FL), Vol. 2, pp. 1117–1128, AIAA-94-4376. Reston, VA: AIAA
Giunta, A.A.; Balabanov, V.; Kaufmann, M.; Burgee, S.; Grossman, B.; Haftka, R.T.; Mason, W.H.; Watson, L.T. 1996: Variable-complexity response surface design of an HSCT configuration. In: Alexandrov, N.M.; Hussaini, M.Y. (eds.) Proceedings of the ICASE/NASA Langley workshop on multidisciplinary design optimization (held in Hampton, VA), pp. 348–367. SIAM
Giunta, A.A.; Eldred, M.S.; Trucano, T.G.; Wojtkiewicz, S.F., Jr. 2002: Optimization under uncertainty methods for computational shock physics applications. In: 43rd AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference (held in Denver, CO), AIAA-2002-1642. Reston, VA: AIAA
Giunta, A.A.; Wojtkiewicz, S.F., Jr.; Eldred, M.S. 2003: Overview of modern design of experiments methods for computational simulations. In: 41st AIAA Aerospace Sciences Meeting and Exhibit (held in Reno, NV), AIAA-2003-0649. Reston, VA: AIAA
Golovidov, O.; Kodiyalam, S.; Marineau, P.; Wang, L.; Rohl, P. 1999: A flexible, object-based implementation of approximation models in an MDO framework. Design Optimization: International Journal for Product & Process Improvement1(4), 388–404
Gu, L. 2001: A comparison of polynomial based regression models in vehicle safety analysis. In: Diaz, A. (ed.) ASME Design Engineering Technical Conferences – Design Automation Conference (held in Pittsburgh, PA), Paper No. DETC2001/DAC-21063. New York: ASME
Haftka, R.; Scott, E.P.; Cruz, J.R. 1998: Optimization and experiments: A survey. Applied Mechanics Review51(7), 435–448
Hamada, M.; Wu, C.J.F. 2000: Experiments: Planning, analysis, and parameter design optimization. New York: John Wiley & Sons, Inc.
Hedayat, A.S.; Sloane, N.J.A.; Stufken, J. 1999: Orthogonal arrays: Theory and applications. New York: Springer
Jin, R.; Chen, W.; Simpson, T.W. 2001: Comparative studies of metamodeling techniques under multiple modeling criteria. Structural and Multidisciplinary Optimization23(1), 1–13
Johnson, M.E.; Moore, L.M.; Ylvisaker, D. 1990: Minimax and maximin distance designs. Journal of Statistical Planning and Inference26(2), 131–148
Kalagnanam, J.R.; Diwekar, U.M. 1997: An efficient sampling technique for off-line quality control. Technometrics39(3), 308–319
Karimi, K.J.; Booker, A.J.; Mong, A. 1996: Modeling, simulation and verification of large dc power electronics systems. In: Proceedings of the 27th Annual IEEE Power Electronics Specialists Conference (held in Baveno, Italy), Vol. 2, pp. 1731–1738. IEEE
Karimi, K.J.; Booker, A.J.; Manners, B.; Mong, A. 1997: Verification of space station secondary power system stability using design of experiment. In: Proceedings of the 32nd Intersociety Energy Conversion Engineering Conference. Vol. 1, pp. 526–531, IEEE-97356. IEEE
Kleijnen, J.P.C. 1975: A comment on Blanning’s metamodel for sensitivity analysis: The regression metamodel in simulation. Interfaces5(1), 21–23
Kleijnen, J.P.C.; Van Beers, W. 2003: Kriging for interpolation in random simulation. Journal of the Operational Research Society54, 255–262
Koch, P.N.; Simpson, T.W.; Allen, J.K.; Mistree, F. 1999: Statistical approximations for multidisciplinary optimization: The problem of size. Special Multidisciplinary Design Optimization Issue of Journal of Aircraft36(1), 275–286
Koch, P.N.; Gu, L. 2001: Addressing uncertainty using the isight probabilistic design environment. In: First Annual Probabilistic Methods Conference (held in Newport Beach, CA)
Koch, P.N.; Golovidov, O.; Wujek, B.A.; Simpson, T.W. 2002a: Facilitating probabilistic multidisciplinary design optimization through advanced approximation methods. In: 9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization (held in Atlanta, GA), AIAA-2002-5415. Reston, VA: AIAA
Koch, P.N.; Evans, J.P.; Powell, D. 2002b: Interdigitation for effective design space exploration using iSIGHT. Structural and Multidisciplinary Optimization23(2), 111–126
Koch, P.N. 2002: Probabilistic design: Optimizing for six sigma quality. In: 43rd AIAA/ASME/ASCE/AHS Structures, Structural Dynamics, and Materials Conference, 4th AIAA Non-Deterministic Approaches Forum (held in Denver, CO), AIAA-2002-1471. Reston, VA: AIAA
Koch, P.N.; Yang, R.-J.; Gu, L. 2004: Design for six sigma through robust optimization. Structural and Multidiscipilnary Optimization26(3–4), 235–248
Koehler, J.R.; Owen, A.B. 1996: Computer experiments, In: Ghosh, S.; Rao, C.R. (eds.) Handbook of statistics, pp. 261–308. New York: Elsevier Science
Koehler, J.R. 1997: Estimating the response, derivatives, and transmitted variance using computer experiments. In: 1997 Symposium on the Interface of Computing Science and Statistics (held in Houston, TX)
Leiva, J.P.; Wang, L.; Recek, S.; Watson, B. 2001: Automobile design using the genesis structural optimization program. In: Nafems Seminar: Advances in Optimization Technologies for Product Design (held in Chicago, IL)
Liu, W.; Batill, S. 2000: Gradient-enhanced neural network response surface approximations. In: 8th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis & Optimization (held in Long Beach, CA), AIAA-2000-4923. Reston, VA: AIAA
Liu, W.; Batill, S.M. 2002: Gradient-enhanced response surface approximations using kriging models. In: 9th AIAA/ISSMO Symposium and Exhibit on Multidisciplinary Analysis & Optimization (held in Atlanta, GA), AIAA-2002-5456. Reston, VA: AIAA
Martin, J.D.; Simpson, T.W. 2002: Adaptive metamodeling for undersea vehicle design optimization. In: 9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization (held in Atlanta, GA), AIAA-2002-5631. Reston, VA: AIAA
Mason, J.G.; Farquhar, B.W.; Booker, A.J.; Moody, R.J. 1992: Inlet design using a blend of experimental and computational techniques. In: Proceedings of the 18th Congress of ICAS. Vol. 1, ICAS-92-3.3.1.
McKay, M.D.; Beckman, R.J.; Conover, W.J. 1979: A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics21(2), 239–245
Meckesheimer, M.; Barton, R.R.; Simpson, T.W.; Booker, A. 2002: Computationally inexpensive metamodel assessment strategies. AIAA Journal40(10), 2053–2060
Mitchell, T.J.; Morris, M.D. 1992: Bayesian design and analysis of computer experiments: Two examples. Statistica Sinica2, 359–379
Montès, P. 1994: Smoothing noisy data by kriging with nugget effects, In: Laurent, P.J.; Le Méhauté, A.; Schumaker, L.L. (eds.) Wavelets, images and surface fitting, pp. 371–378. Wellesley, MA: A.K. Peters
Montgomery, D.C. 1997: Design and analysis of experiments (Fourth Edition). New York: John Wiley & Sons, Inc.
Morris, M.D.; Mitchell, T.J.; Ylvisaker, D. 1993: Bayesian design and analysis of computer experiments: Use of derivatives in surface prediction. Technometrics35(3), 243–255
Myers, R.H.; Montgomery, D.C. 1995:Response surface method-ology: Process and product optimization using designed experiments. New York: John Wiley & Sons, Inc.
Osio, I.G.; Amon, C.H. 1996: An engineering design methodology with multistage bayesian surrogates and optimal sampling. Research in Engineering Design8(4), 189–206
Owen, A.B. 1992: Orthogonal arrays for computer experiments, integration and visualization. Statistica Sinica2, 439–452
Pérez, V.M.; Renaud, J.E.; Watson, L.T. 2002a: Adaptive experimental design for construction of response surface approximations. AIAA Journal40(12), 2495–2503
Pérez, V.M.; Renaud, J.E.; Watson, L.T. 2002b: Interior point sequential approximate optimization methodology. In: 9th AIAA/ISSMO Symposium on Multidisciplinary Analysis & Optimization (held in Atlanta, GA), AIAA-2002-5505. Reston, VA: AIAA
Pérez, V.M.; Renaud, J.E.; Watson, L.T. 2002c: Reduced sampling for construction of quadratic response surface approximations using adaptive experimental design. In: 43rd AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference (held in Denver, CO), AIAA-2002-1587. Reston, VA: AIAA
Plackett, R.L.; Burman, J.P. 1946: The design of optimum multifactorial experiments. Biometrika33(4), 305–325
Quinn, G. 2002: Cooling system design using approximation methods. Working Paper, Vanderplaats Research & Development, Inc., Colorado Springs, CO
Rodríguez, J.F.; Pérez, V.M.; Padmanabhan, D.; Renaud, J.E. 2001: Sequential approximate optimization using variable fidelity response surface approximations. Structural and Multidisciplinary Optimization22(1), 24–34
Sacks, J.; Welch, W.J.; Mitchell, T.J.; Wynn, H.P. 1989: Design and analysis of computer experiments. Statistical Science4(4), 409–435
Sasena, M.; Papalambros, P.Y.; Goovaerts, P. 2002: Global optimization of problems with disconnecte feasible regions via surrogate modeling. In: 9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization (held in Atlanta, GA), AIAA-2002-5573. Reston, VA: AIAA
Simpson, T.W.; Mauery, T.M.; Korte, J.J.; Mistree, F. 2001a: Kriging metamodels for global approximation in simulation-based multidisciplinary design optimization. AIAA Journal39(12), 2233–2241
Simpson, T.W.; Peplinski, J.; Koch, P.N.; Allen, J.K. 2001b: Metamodels for computer-based engineering design: Survey and recommendations. Engineering with Computers17(2), 129–150
Simpson, T.W.; Lin, D.K.J.; Chen, W. 2001c: Sampling strategies for computer experiments: Design and analysis. International Journal of Reliability and Applications2(3), 209–240
Sobieszczanski-Sobieski, J.; Haftka, R.T. 1997: Multidisciplinary aerospace design optimization: Survey of recent developments. Structural Optimization14(1), 1–23
Su, J.; Renaud, J.E. 1997: Automatic differentiation in robust optimization. AIAA Journal35(6), 1072–1079
van Keulen, F.; Vervenne, K. 2002: Gradient-enhanced response surface building. In: 9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization (held in Atlanta, GA), AIAA-2002-5455. Reston, VA: AIAA
Venter, G.; Haftka, R.T.; Starnes, J.H., Jr. 1998: Construction of response surface approximations for design optimization. AIAA Journal36(12), 2242–2249
Wang, G. 2003: Adaptive response surface method using inherited latin hypercube designs. ASME Journal of Mechanical Design125(2), 210–220
Wang, G.G.; Simpson, T.W. 2002: Fuzzy clustering based hierarchical metamodeling for design optimization. In: 9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization (held in Atlanta, GA), AIAA-2002-5574. Reston, VA: AIAA
Welch, W.J. 1983: A mean squared error criterion for the design of experiments. Biometrika70(1), 205–213
Welch, W.J.; Yu, T.-K.; Kang, S.M.; Sacks, J. 1990: Computer experiments for quality control by parameter design. Journal of Quality Technology22(1), 15–22
Welch, W.J.; Buck, R.J.; Sacks, J.; Wynn, H.P.; Mitchell, T.J.; Morris, M.D. 1992: Screening, predicting, and computer experiments. Technometrics34(1), 15–25
Yang, R.J.; Gu, L.; Liaw, L.; Gearhart, C.; Tho, C.H.; Liu, X.; Wang, B.P. 2000: Approximations for safety optimization of large systems. In: Renaud, J.E. (ed.) ASME Design Engineering Technical Conferences – Design Automation Conference (held in Baltimore, MD), Paper No. DETC-2000/DAC-14245. New York: ASME
Yang, R.J.; Wang, N.; Tho, C.H.; Bobineau, J.P. 2001: Metamodeling development for vehicle frontal impact simulation. In: Diaz, A. (ed.) ASME Design Engineering Technical Conferences – Design Automation Conference (held in Pittsburgh, PA), Paper No. DETC2001/DAC-21012. New York: ASME
Yang, R.-J.; Gu, L.; Tho, C.H.; Choi, K.K.; Youn, B.D. 2002: Reliability-based multidisciplinary design optimization of a full vehicle system. In: 43rd AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference (held in Denver, CO), AIAA-2002-1758. Reston, VA: AIAA
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Simpson, T., Booker, A., Ghosh, D. et al. Approximation methods in multidisciplinary analysis and optimization: a panel discussion. Struct Multidisc Optim 27, 302–313 (2004). https://doi.org/10.1007/s00158-004-0389-9
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00158-004-0389-9