On the ensemble of metamodels with multiple regional optimized weight factors

Abstract

Metamodels are often used as surrogates for expensive high fidelity computational simulations (e.g., finite element analysis). Ensemble of metamodels (EM), which combines various types of individual metamodels in the form of a weighted average ensemble, is found to have improved accuracy over the individual metamodels used alone. Currently, there are mainly two kinds of EMs called as pointwise EM and average EM. The pointwise EM generally has better prediction accuracy than the average EM, but it is much more time-consuming than the average EM. In most cases, as a metamodel, EM is often used in the engineering design optimization which needs to invoke EM tens of thousands of times. Therefore, the average EM is still the most extensively used EM. To the authors’ best knowledge, the most accurate average EM is the EM with optimized weight factors proposed by Acar et al. However, the EM proposed by Acar et al. is often too “rigid” and may not have sufficient accuracy over some regions of the design space. In order to deal with this problem and further improve the prediction accuracy, a new EM with multiple regional optimized weight factors (EM-MROWF) is proposed in this study. In this new EM, the design space is divided into multiple subdomains each of which is assigned a set of optimized weight factors. This new EM was constructed by combining three typical individual metamodels, i.e., polynomial regression (PR), radial basis function (RBF), and Kriging (KRG). The proposed technique was evaluated by ten benchmark problems and two engineering application problems. The ten benchmark problems are typical mathematical functions for evaluating the approximation performance in previous studies. And, the two engineering application problems refer to the vehicular passive safety in the field of crashworthiness design. The study results showed that the EM-MROWF performed much better than the other existing average EMs as well as the three individual metamodels.

1 Introduction

The design of modern engineering systems often relies on high-fidelity numerical simulations (e.g. finite element analysis) that are usually computationally expensive. In the optimum design of an engineering system, the high-fidelity numerical simulations are performed many times and thus the computational cost often becomes excessive and unaffordable. To reduce the computational cost, metamodels are used in the optimization work as the surrogates of the high-fidelity numerical simulations.

During the past decades, a large number of metamodeling methods have been proposed by the researchers and are available from literature. The commonly used metmodeling methods include polynomial regression (PR) (Box et al. 1978; Myers and Montgomery 2002), radial basis function (RBF) (Dyn et al. 1986; Buhmann 2003), Kriging (KRG) (Sacks et al. 1989; Simpson et al. 2001), Gaussian process (GP) (MacKay 1998; Rasmussen and Williams 2006), neural networks (Bishop 1995; Smith 1993), support vector regression (SVR) (Clarke et al. 2005; Gunn 1997) and multivariate adaptive regression splines (MARS) (Jerome 1990; Diana et al. 2015). In the work of Hou et al. (2007), they compared different PR models with different polynomial functions in the design optimization of hexagonal thin-walled structures. They found that quadratic polynomials provided the best approximations to the specific energy absorption (SEA) and maximum peak load. Fang et al. (2005) compared PR and several RBF models for the crashworthiness optimization problems and showed that the RBF models were more suitable for modeling highly nonlinear responses than the PR models. Song et al. (2013) carried out a comparative study on four commonly used metadmodels, PR, RBF, KRG and SVR, for the design optimization of foam-filled tapered structures. They found that no single model was the best for approximating all objective functions in the considered problems. Yin et al. (2011) employed PR, RBF, KRG, MARS and SVR models to approximate the responses of SEA and peak crushing stress of aluminum honeycomb structures. It was found that the best metamodels were different for the two responses. Forrester and Keane (2009) reviewed different metamodeling methods used in surrogate-based optimization and suggested that the choice of which surrogate to use should be based on the problem size, the expected complexity, and the cost of the analyses. From the available studies, the general consensus was that no single metamodel was the most effective for all problems. Different metamodels are suitable for fitting different functions; each metamodel has its advantages and drawbacks (Forrester and Keane 2009; Queipo et al. 2005; Wang and Shan 2007).

To improve the prediction accuracy of surrogate models, a number of studies were conducted on combining multiple metamodels into a single ensemble using the weighted sum approach (Goel et al. 2007; Zerpa et al. 2005; Sanchez et al. 2008; Acar and Rais-Rohani 2009; Acar 2010; Acar 2015; Lee and Dong-Hoon 2014; Fang et al. 2017; Ferreira and Serpa 2016; Zhou and Jiang 2016; Zhou et al. 2011; Zhang et al. 2012). Since an RBF could accurately capture the nonlinear aspect of a response, and a PR model could give the overall trend, an ensemble of metamodels (EM) of both RBF and PR models may have better prediction ability than the stand-alone models. In the work of Gu et al. (2015), they established the EM of PR, RBF, KRG and SVR for the responses of an occupant protection system. The results of the study showed that the EM had better prediction than all the individual metamodels.

In an EM, the weight factors have a significant effect on the prediction accuracy and there exist a number of methods for determining the weight factors such as the simple average weights (Goel et al. 2007), prediction-sum-of-squares-based average weights (Goel et al. 2007), optimized weights (Acar and Rais-Rohani 2009) and functional weights as the location of prediction point (Zerpa et al. 2005; Sanchez et al. 2008; Lee and Dong-Hoon 2014). Based on the method of determining the weight factors, EM can be classified into two types, an average EM and a pointwise EM (Lee and Dong-Hoon 2014). An average EM has unchanged weight factors in the entire design space, while a pointwise EM has varied weight factors which change as the location of a prediction point varies. The EMs proposed by Goel et al. (2007) and Acar and Rais-Rohani (2009) belong to average EM and the EMs proposed by Zerpa et al. (2005), Sanchez et al. (2008) and Lee and Dong-Hoon (2014) are typical pointwise EMs. The weight factors of the stand-alone metamodels in an average EM are determined based on their average accuracies throughout the entire design space, while the weight factors of the stand-alone metamodels in a pointwise EM are determined based on their accuracies at each prediction point. A pointwise EM was found to be more accurate than the average EM because it had varied weight factors according to the prediction point (Lee and Dong-Hoon 2014). However, it had much higher computational cost than the average EM especially when it was used as the surrogate in the design optimization.

In most cases, EM is employed as the surrogate of high fidelity computational simulation (e.g., finite element analysis) with expensive computational cost in the engineering design optimization (Hou et al. 2007; Fang et al. 2005; Song et al. 2013; Yin et al. 2011; Forrester and Keane 2009; Queipo et al. 2005; Wang and Shan 2007). Generally, the EM is invoked tens of thousands of times in the optimization process. The computational time may not be accepted if we use pointwise EM in the practical design optimization. Thus, the average EM is relatively efficient in the practical design optimization. The average EM proposed by Acar et al. was found to be more accurate than the other existing average EMs (Acar and Rais-Rohani 2009). However, the average EM proposed by Acar et al. has only one set of weight factors across the entire design space, and this approach is often too “rigid” and may not have good prediction in some areas of the design space.

To deal with this issue and improve the accuracy of the EM proposed by Acar et al., a new EM with multiple regional optimized weight factors (EM-MROWF) was proposed and investigated in this study. In this new EM, the design space was divided into multiple subdomains, each of which had its own set of optimized weight factors. The optimized weight factors in each subdomain were determined by minimizing the error metric of the training points in that subdomain. The EM-MROWF technique was evaluated using ten benchmark problems and two engineering application problems. The results showed that the EM-MROWF had better prediction accuracy and robustness than the other two considered average EMs and the individual metamodels.

2 Ensemble of metamodels

Metamodels are widely used in simulation-based design optimization to reduce the computational cost of a large number of expensive simulations. The most commonly used ensemble method is the weighted average ensemble (Goel et al. 2007) given by

$$ {\widehat{y}}_{\mathrm{ens}}\left(\mathbf{x}\right)=\sum \limits_{i=1}^{n_{\mathrm{M}}}{w}_i{\widehat{y}}_i\left(\mathbf{x}\right) $$

(1)

where ŷ_ens is the prediction by the EM, x is the vector of design variables, n_M is the number of metamodels used in the ensemble, w _i is the weight factor of the ith basis metamodel in the ensemble, and ŷ _i is the predicted value by the ith metamodel. To have unbiased response estimation, the following equation should be satisfied by the weight factors:

$$ \sum \limits_{i=1}^{n_{\mathrm{M}}}{w}_i=1 $$

(2)

A metamodel that is deemed more accurate should be assigned a large weight factor, and the less accurate model should have less influence on the predictions. There are many possible strategies of determining weight factors (Lee and Dong-Hoon 2014).

Among the existing EM, there are mainly two types of EMs, i.e., the pointwise EM (Zerpa et al. 2005; Sanchez et al. 2008; Lee and Dong-Hoon 2014) and the average EM (Goel et al. 2007; Acar and Rais-Rohani 2009). Generally, the accuracy of the pointwise EM is much better than that of the average EM, but the pointwise EM is much more time-consuming than the average EM. As a metamodel, the EM is often used as the surrogate of the numerical simulation (e.g., finite element analysis) for the engineering design optimization. In the design optimization process, the computational time will be exaggerated if we use pointwise EM as the surrogate of the numerical simulation. In order to consider the numerical cost of the EM, we only consider the average EM in this study.

2.1 The EM proposed by Goel

Goel et al. (2007) proposed an average EM using the basis metamodels of PR, KRG and RBF. The weight factors of the EM proposed by Goel were determined as:

$$ {w}_i=\frac{w_i^{\ast }}{\sum \limits_{i=1}^M{w}_i^{\ast }} $$

(3)

$$ {w}_i^{\ast }={\left({E}_i+\alpha {E}_{avg}\right)}^{\beta },\beta <0,\alpha <1 $$

(4)

$$ {E}_{avg}=\frac{\sum \limits_{i=1}^M{E}_i}{M} $$

(5)

$$ {E}_i={GMSE}_i=\frac{1}{ndes}\sum \limits_{k=1}^{ndes}{\left[y\left({\mathbf{x}}_k\right)-{{\widehat{y}}_i}^{\left(-k\right)}\left({\mathbf{x}}_k\right)\right]}^2 $$

(6)

where M is the number of individual metamodels, ndes is the number of design points, x _k is the kth design point, y(x _k ) is the real response value at x _k , and ŷ _i ^(−k)(x _k ) is the predicted response value of the ith basis metamodel generated using nexp-1 design points without the kth point at x _k . In this study, α = 0.05 and β = −1 as the Goel et al. suggested. In the method proposed by Goel et al., the generalized mean square cross-validation error (GMSE) was used to calculate the prediction errors of the basis metamodel.

2.2 The EM proposed by Acar

An error-based minimization method (Acar and Rais-Rohani 2009; Acar 2010, 2015) is recognized as an effective method for determining the weight factors of the EM. Acar and Rais-Rohani (2009) proposed an ensemble of metamodels with optimized weight factors, in which they determined the weight factors w _i by solving an optimization problem as

$$ \left\{\begin{array}{l}\operatorname{Minimize}\ \mathrm{Error}\left({\widehat{y}}_{\mathrm{ens}}\right)\\ {}\mathrm{s}.\mathrm{t}.\kern0.5em \sum \limits_{i=1}^{n_{\mathrm{M}}}{w}_i=1\end{array}\right. $$

(7)

where Error(ŷ_ens) is the selected error metric that measures the accuracy of the ensemble-predicted response ŷ_ens. In Acar and Rais-Rohani’s study, GMSE was selected as the error metric to evaluate the accuracy of the ensemble of metamodels (Acar 2015). Then, GMSE was selected to evaluate the accuracy of the ensemble-predicted response ŷ_ens. The Error(ŷ_ens) of (7) will be written as

$$ \mathrm{Error}\left({\widehat{y}}_{\mathrm{ens}}\right)={GMSE}_{\mathrm{ens}}=\frac{1}{ndes}\sum \limits_{k=1}^{ndes}{\left[y\left({\mathbf{x}}_k\right)-{\widehat{y}}_{\mathrm{ens}}^{\left(-k\right)}\left({\mathbf{x}}_k\right)\right]}^2 $$

(8)

where ndes is the number of design points, x _k is the kth design point, y(x _k ) is the real response value at x _k , and ŷ_ens^(−k)(x _k ) is the predicted response value of the ensemble metamodel generated using nexp-1 design points at x _k .

3 New proposed ensemble of metamodels

In this study, a new EM approach was investigated by dividing the design space into multiple subdomains and use a set of optimized weight factors for each subdomain. The basic idea of the new EM approach was to allow the subdomain to have independently determined sets of weight factors so that they will not affect or be affected by those of other subdomains. With multiple regional optimized weight factors, the accuracy of the EM could be improved in each of the subdomains and thus across the entire design space.

The first step of constructing the EM with multiple regional optimized weight factors was to divide the design space into m = n₁ × n₂×⋯×n _p subdomains or regions (see Fig. 1), where n _i is the number of sections of ith variable, and p is the number of variables. In each region, there is one set of weight factors that is obtained by minimizing the GMSE on the design points in that region. The flowchart of the process to establish the EM-MROWF is shown in Fig. 2 and a detailed description is given as follows.

• STEP 1: Generate ndes design points using the optimal Latin hypercube sampling, and create ntest test points using the regular Latin hypercube sampling. The design points were used to create the individual metamodels, i.e., PR, RBF and KRG. The ensemble training points were used to calculate the weight factors of the ensemble of metamodels. The test points were used to calculate the root mean square error (RMSE) and maximum absolute relative error (MARE) of the ensemble of metamodels.

• STEP 2: Divide the design space into m = n₁ × n₂×⋯×n _p regions, as illustrated in Fig. 1.

• STEP 3: Create the individual metamodels, i.e., PR, RBF and KRG, and construct the EM based on the design points. The functions and parameters of these metamodels are given in Table 1.

• STEP 4: Calculate the error metric GMSE of the EM in each subdomain using the design points in that region.

• STEP 5: Obtain the optimized weight factors for each region by minimizing the GMSE. In this study, the sequential quadratic programming (SQP) optimizer of MATLAB, the “fmincon” function, was used to solve the optimization problem of (7). A total of m = n₁ × n₂×⋯×n _p sets of optimized weight factors were obtained for the m = n₁ × n₂×⋯×n _p regions.

• STEP 6: Obtain the weight factors for the regional boundaries by averaging the weight factors of the adjacent regions. The weight factors for the regional boundaries were determined by averaging the weight factors of the adjacent regions, which is illustrated in Fig. 3 (e.g. if the weight factors of region A and region B were (w₁₁, w₁₂, w₁₃) and (w₂₁, w₂₂, w₂₃), respectively. The weight factors of the boundary of region A and region B would be (w₁₁/2+ w₂₁/2, w₁₂/2+ w₂₂/2, w₁₃/2+ w₂₃/2).)

• STEP 7: Obtain one EM for the whole design space and calculate the RMSE and MARE of the test points. RMSE and MARE can be calculated as:

Fig. 1

Illustration of the divided regions in the design space: (a) two dimensions and (b) three dimensions

Fig. 2

The flowchart of the process of establishing the EM-MROWF

Table 1 User chosen functions and parameters of four individual metamodels (Yin et al. 2014)

Fig. 3

Illustration of the weight calculation for the adjacent regions

$$ RMSE=\sqrt{\frac{1}{ntest}\sum \limits_{i=1}^{ntest}{\left[y\left({\mathbf{x}}_i\right)-\widehat{y}\left({\mathbf{x}}_i\right)\right]}^2}, $$

(9)

$$ MARE=\max \left(\left|\frac{y\left({\mathbf{x}}_i\right)-\widehat{y}\left({\mathbf{x}}_i\right)}{y\left({\mathbf{x}}_i\right)}\right|\right), $$

(10)

where ntest is the number of test points, x _i is the ith design point, y(x _i ) is the real response value at x _i , and ŷ(x _i ) is the predicted response value of the ensemble metamodel at x _i . Generally, RMSE and MARE are used to evaluate the global and local errors of the metamodel, respectively (Jin et al. 2001).

• STEP 8: Obtain n_max^p EMs with differently divided regions for the whole design space. In this study, in order to save the computational cost, n_max was chosen as four.

• STEP 9: Find the best EM with minimum RMSE of the test points and set it as the final EM-MROWF.

4 Example problems

Ten mathematical benchmark problems and two engineering application problems were used to test the performance of the proposed EM-MROWF technique. The first engineering application was a cylindrical thin-walled column crushed by a dynamic axial load, and the second application was an airbag cushion impacted by a dummy head. For both application problems, the responses were obtained using nonlinear finite element (FE) simulations.

4.1 Benchmark problems

The ten benchmark problems (Acar 2015; Lee and Dong-Hoon 2014; Wikipedia 2015) used in this study are given as follows.

• (1) Branin-Hoo function

$$ y\left({x}_1,{x}_2\right)={\left({x}_2-\frac{5.1{x}_1^2}{4{\pi}^2}+\frac{5{x}_1}{\pi }-6\right)}^2+10\left(1-\frac{1}{8\pi}\right)\cos \left({x}_1\right)+10 $$

(11)

where x₁∈[−5, 10] and x₂∈[0, 15].

• (2) Camelback function

$$ y\left({x}_1,{x}_2\right)=\left(4-2.1{x}_1^2+\frac{x_1^4}{3}\right){x}_1^2+{x}_1{x}_2+\left(-4+4{x}_2^2\right){x}_2^2 $$

(12)

where x₁, x₂∈[−2, 2].

• (3) Goldstein-price function

$$ {\displaystyle \begin{array}{l}y\left({x}_1,{x}_2\right)=\left[1+{\left({x}_1+{x}_2+1\right)}^2\times \left(19-4{x}_1+3{x}_1^2-14{x}_2+6{x}_1{x}_2+3{x}_2^2\right)\right]\\ {}\kern4.25em \times \left[30+{\left(2{x}_1-3{x}_2\right)}^2\times \left(18-32{x}_1+12{x}_1^2+48{x}_2-36{x}_1{x}_2+27{x}_2^2\right)\right]\end{array}} $$

(13)

where x₁, x₂∈[−2, 2].

• (4) 2D multi-modal function

$$ y\left({x}_1,{x}_2\right)={x}_1{x}_2\sin \left({x}_1\right)+\left({x}_1^2/10\right)+{x}_1-1.5{x}_2 $$

(14)

where x₁, x₂∈[−2, 2].

• (5) Haupt function

$$ y\left({x}_1,{x}_2\right)={x}_1\sin \left(4{x}_1\right)+1.1{x}_2\sin \left(2{x}_2\right) $$

(15)

where x₁, x₂∈[0, 4].

• (6) Crane function

$$ y\left({x}_1,{x}_2\right)={e}^{\cos \left({x}_1-{x}_2\right)}\sin \left\{\left[{\cos}^2\left({x}_1-{x}_2\right)+{x}_1+{x}_2\right]/\left[1+{\left({x}_1-{x}_2\right)}^2\right]\right\} $$

(16)

where x₁, x₂∈[−4, 4].

• (7) Peak function

$$ y\left({x}_1,{x}_2\right)=3\left(1-{x}_1^2\right){e}^{-{x}_1^2-{\left({x}_2+1\right)}^2}-10\left(\frac{x_1}{5}-{x}_1^3-{x}_2^5\right){e}^{-{x}_1^2-{x}_2^2}-\frac{1}{3}{e}^{-{\left({x}_1+1\right)}^2-{x}_2^2} $$

(17)

where x₁, x₂∈[−3, 3].

• (8) Waving function

$$ y\left({x}_1,{x}_2\right)=2+0.01{\left({x}_2-{x}_1^2\right)}^2+{\left(1-{x}_1\right)}^2+2\left(2-{x}_2^2\right)+7\sin \left(0.5{x}_1\right)\sin \left(0.7{x}_1{x}_2\right) $$

(18)

where x₁, x₂∈[0, 5].

• (9) Rastrigin function

$$ y\left({x}_1,{x}_2\right)={x}_1^2+{x}_2^2-\cos \left(18{x}_1\right)-\cos \left(18{x}_2\right) $$

(19)

where x₁, x₂∈[−1, 1].

• (10) Rosenbrock function

$$ y\left(\mathbf{x}\right)=\sum \limits_{i=1}^{n-1}\left[100{\left({x}_{i+1}-{x}_i^2\right)}^2+{\left({x}_i-1\right)}^2\right] $$

(20)

where x _i ∈[−2, 2], i = 1, 2.

4.2 Engineering application problems

4.2.1 Thin-walled column crushing problem

Thin-walled columns are widely used as energy absorbers in engineering applications. In this study, the FE model of a cylindrical thin-walled column, as shown in Fig. 4a, was created for LS-DYNA (Hallquist 1998a, b) and used to simulate axial crushing of the column under dynamic loading conditions. As an energy absorber, the energy absorption (EA), peak crushing force (PCF), and specific energy absorption (SEA) are three important indicators for evaluating the energy absorption capacity and efficiency. The EA is calculated by

$$ \mathrm{EA}={\int}_0^dF(x) dx $$

(21)

where d is the crushing distance and F is the crushing force. The SEA is defined as (Kim 2002)

$$ \mathrm{SEA}=\frac{\mathrm{EA}}{M} $$

(22)

where M is the mass of the structure.

Fig. 4

FE model and illustration of design variables of the cylinder thin-walled column: (a) FE model and (b) illustration of design variables

The three performance indicators did not have explicit mathematical formulas and thus metamodels needed to be established using FE simulation results. The design variables of the cylinder thin-walled column were the radius of the cross section (R) and wall thickness (t), as illustrated in Fig. 4b. In this study, the ranges of the design variables were chosen as [60 mm, 100 mm] and [1 mm, 3 mm] for R and t, respectively. Using numerical simulations, the three responses, EA, PCF and SEA, were obtained for design points and test points, as given in Tables 2 and 3, respectively. The metamodels of the three responses are established in the next section.

Table 2 Responses of the design points for the thin-walled column crash problem

Table 3 Responses of the test points for the thin-walled column crash problem

4.2.2 Airbag cushion problem

Airbag is recognized as effective cushion equipment and is now widely used for the occupant protection in vehicles. In this study, the FE model of a driver’s airbag, as shown in Fig. 5, was created by LS-DYNA (Hallquist 1998a, b) and used to simulate the impact between the driver’s head and the airbag. In the driver’s airbag system, the mass of the inflation gas and the vent area have a big effect on the cushion property. In order to investigate the cushion property of the driver’s airbag, the peak acceleration a_peak of the head was employed as the indicator. The input mass of the inflation gas and the vent area were chosen as the design variables. In this work, the input mass of the inflation gas of a standard driver’s airbag (LS-PrePost Online Document 2012) was scaled by a parameter λ which changed from 0.5 to 1.5. The vent area A changed from 0 to 628 mm² (i.e. the radiuses of the two vent holes of the airbag were from 0 to 10 mm). Using FE simulations, the responses a_peak were obtained for design points and test points, as given in Tables 4 and 5, respectively. The metamodel of the response is established in the next section.

Fig. 5

FE models of the airbag cushion system: (a) before airbag deployment and (b) after airbag deployment

Table 4 Responses of the design points for the airbag cushion problem

Table 5 Responses of the test points for the airbag cushion problem

5 Results and discussion

In this section, the EMs and the individual metamodels (i.e., PR, RBF and KRG) for the benchmark problems, thin-walled column crash problem and airbag cushion problem were constructed. To evaluate to prediction performances, RMSE and MARE values were calculated using the test points.

5.1 Benchmark problems

5.1.1 Test 1: Prediction performance with various number of design points

In order to compare the prediction performance of the EMs and the individual metamodels, the normalized RMSE and MARE values of these metamodels with different number of design points for the ten benchmark problems were calculated using (9) and (10), respectively. The numbers of the design points and test points generated for these benchmark problems were summarized in Table 6. In this test, the number of design points ndes was set to 20, 30, 40, 50 and 60, the corresponding normalized RMSE results were listed in Tables 7, 8, 9, 10 and 11, respectively. And, the normalized average RMSE values for the ten benchmark problems were summarized in Table 12. All the RMSE values listed in these tables were normalized by setting the best (the lowest) value to 1. In each table, the overall performance for all 10 test problems was estimated using the average normalized RMSE value shown in the penultimate row. Generally, the smaller the average normalized RMSE, the better the overall performance of the metamodel. In addition, in each table, the standard deviations of the three kinds of the EMs were calculated and given in the last row of the table. As we know, the smaller the standard deviation, the more robust the performance of the metamodel. In addition, the normalized MARE results for the cases of ndes = 20, 30, 40, 50 and 60 were given in Tables 13, 14, 15, 16 and 17, respectively. The normalized average MARE values for the ten benchmark problems were summarized in Table 18.

Table 6 Summary of the ndes, ntest and ntotal^* for the benckmark problems

Table 7 Normalized RMSE values with ndes = 20 for Test 1

Table 8 Normalized RMSE values with ndes = 30 for Test 1

Table 9 Normalized RMSE values with ndes = 40 for Test 1

Table 10 Normalized RMSE values with ndes = 50 for Test 1

Table 11 Normalized RMSE values with ndes = 60 for Test 1

Table 12 Summarized normalized average RMSE values for all benchmark problems

Table 13 Normalized MARE values with ndes = 20 for Test 1

Table 14 Normalized MARE values with ndes = 30 for Test 1

Table 15 Normalized MARE values with ndes = 40 for Test 1

Table 16 Normalized MARE values with ndes = 50 for Test 1

Table 17 Normalized MARE values with ndes = 60 for Test 1

Table 18 Summarized normalized average MARE values for all benchmark problems

From Table 7 and Table 13, it can be found that the average value and the standard deviation of EM-MROWF for the case of ndes = 20 both were the second-smallest for the normalized RMSE and MARE, respectively. And, from Table 8 and Table 14, Table 9 and Table 15, Table 10 and Table 16, and Table 11 and Table 17, the average value and the standard deviation of EM-MROWF for the cases of ndes = 30, 40, 50 and 60 both were the smallest for the normalized RMSE and MARE. From the summarized results given in Table 12 and Table 18, it can be seen that EM-MROWF performed best in 4 cases out of 5 test cases for all the ten benchmark problems. In addition, from Table 12 and Table 18, we found that the average value and the standard deviation of EM-MROWF both were the smallest for the normalized RMSE and MARE, respectively. Generally, from the comparison results of Test 1, it can be found that the proposed EM-MROWF was the most accurate and robust metamodel for the ten benchmark problems.

It should be pointed out that the adjacent region of the EM-MROWF is not strictly continuous, but there is a transition process for two adjacent regions in this study. The weight factors of the adjacent region were set as the average values of the weight factors of the two adjacent regions. Thus, the transition of the two adjacent regions was relatively smooth. Figure 6 shows the plots of the real functions and the response surfaces of the proposed EMs for the benchmark functions. From each Figure, it can be seen that the response surface of the EM-MROWF (ndes was chosen as the value when the metamodel was accurate enough) agreed with the real function well and the transition of the adjacent regions was smooth. Thus, the adjacent region of the EM-MROWF performed well.

Fig. 6

Plots of benchmark functions: (a) Branin-Hoo, (b) Camelback, (c) Goldstein-price, (d) 2-D Multi-modal, (e) Rosenbrock, (f) Haupt, (g) Crane, (h) Peak, (i) Waving, (j) Rastrigin

5.1.2 Test 2: Prediction performance with various number of design variables

In order to compare the prediction performance of the EMs described in Sections 2 and 3 as nvar varied, normalized values of the EMs and the individual metamodels for the Rosenbrock function (20) with different nvar were calculated using (9) and (10). In this test, nvar was set to 3, 4, 5, 6, 7 and 8. The number of the design points ndes was set to 30nvar. The results of Test 2 are summarized in Table 19 and Table 20. All the RMSE and MARE values listed in Tables 19 and 20 were normalized by setting the best (the lowest) value to one. From Tables 19 and 20, it can be seen that the proposed EM-MROWF was the most accurate and robust metamodel for Test 2.

Table 19 Normalized RMSE values for Test 2

Table 20 Normalized MARE values for Test 2

5.2 Engineering application problems

Tables 21 and 22 summarizes the normalized RMSEs and MAREs of the EMs as well as the individual metamodels for the three responses (SEA, EA and PCF) of the thin-walled column crash problem and one response (the peak acceleration a_peak) of the head in the airbag cushion problem, respectively. From Table 21, EM-MROWF had the best prediction performance in 4 cases. And, from Table 22, out of the 4 cases, EM-MROWF had the best prediction performance in 3 cases. Based on the average normalized RMSE and MARE shown in the penultimate row of Tables 21 and 22, we found that the overall prediction performance of EM-MROWF was the best for the two engineering application problems. As shown in the last row of Tables 21 and 22, the standard deviation of EM-MROWF was found to be the smallest. This indicated that EM-MROWF was the most robust one among the investigated metamodels. Therefore, based on the comparison results of the two engineering application problems, it was found that the accuracy and the robustness of EM-MROWF was the best.

Table 21 Normalized RMSE values for the engineering application problems

Table 22 Normalized MARE values for the engineering application problems

In a word, it can be found that EM-MROWF was the most accurate and robust EM among the three average ensemble of metamodels and the three individual metamodels for the benchmark problems as well as the engineering application problems.

6 Conclusions

In this study, a new ensemble of metamodels (EM), i.e., EM with multiple regional optimized weight factors (EM-MROWF), was proposed and evaluated. This new EM is different from the commonly used EMs in which it divides the design space into subdomains and assigns each subdomain with a set of optimized weight factors. In each of the divided region, a set of optimized weight factors is determined by minimizing the GMSE in that region. Ten benchmark problems and two engineering application problems (i.e., a thin-walled column crash problem and an airbag cushion problem) were employed to evaluate the prediction performance of the EM-MROWF.

In order to show the usefulness of the EM-MROWF, the prediction performance of EM-MROWF was compared to those of the two existing EMs and the individual metamodels. All the comparison results for the ten benchmark problems and two engineering application problems showed that EM-MROWF had the best prediction performance for both accuracy and robustness. Thus, EM-MROWF was an effective metamodeling method and could be used in practical engineering.