A radial basis function-based multi-fidelity surrogate model: exploring correlation between high-fidelity and low-fidelity models

Abstract

In computational simulation, a high-fidelity (HF) model is generally more accurate than a low-fidelity (LF) model, while the latter is generally more computationally efficient than the former. To take advantages of both HF and LF models, a multi-fidelity surrogate model based on radial basis function (MFS-RBF) is developed in this paper by combining HF and LF models. To determine the scaling factor between HF and LF models, a correlation matrix is augmented by further integrating LF responses. The scaling factor and relevant basis function weights are then calculated by employing corresponding HF responses. MFS-RBF is compared with Co-Kriging model, multi-fidelity surrogate based on linear regression (LR-MFS) model, CoRBF model, and three single-fidelity surrogates. The impact of key factors, such as the cost ratio of LF to HF models and different combinations of HF and LF samples, is also investigated. The results show that (i) MFS-RBF presents a better accuracy and robustness than the three benchmark MFS models and single-fidelity surrogates in about 90% cases of this paper; (ii) MFS-RBF is less sensitive to the correlation between HF and LF models than the three MFS models; (iii) by fixing the total computational cost, the cost ratio of LF to HF models is suggested to be less than 0.2, and 10–80% of the total cost should be used for LF samples; (iv) the MFS-RBF model is able to save an average of 50 to 70% computational cost if HF and LF models are highly correlated.

1 Introduction

Design optimization relying on computational simulations, especially high-fidelity (HF) simulations, generally requires expensive computational cost (Wang and Shan 2006). To improve the computational efficiency, surrogate models have been used to replace computationally expensive simulations, which are constructed based on a small number of computational simulations. Surrogate models can be broadly classified into four types: single-fidelity, hybrid, adaptive sampling-based, and multi-fidelity surrogate models.

Single-fidelity surrogate models are the most conventional type and have been widely used in engineering applications. Popular single-fidelity surrogate models include the polynomial response surface (PRS) (Myers et al.2016), radial basis function (RBF) (Gutmann 2001; Sun et al. 2011), kriging (KRG) (Matheron 1963; Sacks et al. 1989), and support vector regression (SVR) (Smola and Schölkopf 2004). However, it has been shown that no single-fidelity surrogate model was found to be the most effective for all problems (Goel et al. 2007), and it is challenging to select the most appropriate single-fidelity surrogate prior.

Hybrid surrogate models ensemble multiple single-fidelity surrogate models by using weight coefficients. It is crucial to determine the weight coefficients for each single-fidelity surrogate model, and two types of method are generally used with constant or adaptive weights. It has been shown that hybrid surrogate models are able to take advantages of each single-fidelity surrogate model, thereby being more accurate and robust than individual surrogate models (Zerpa et al. 2005; Acar and Rais-Rohani 2009; Acar 2010; Liu et al. 2016).

Adaptive sampling-based surrogate models are proposed to enhance the accuracy by using auxiliary criteria called infilling strategies. Based on the strategy of choosing samples, infill strategies can be generally divided into two types: one-stage approach (e.g., goal seeking proposed by Jones (2001)) and two-stage approach (e.g., probability improvement (PI) proposed by Kushner (1964) and expected improvement (EI) presented by Schonlau et al. 1998). Theoretically, the accuracy of an adaptive sampling-based surrogate model could be improved gradually with the increase of infill points, but in reality, it often cannot be guaranteed as infill points can be misled if the initial surrogate significantly deviates from the true response (Jones 2001).

For the three types of surrogate models mentioned above, the observations at samples are usually obtained from HF simulations. However, it is still computationally expensive to run HF simulations despite of the advancement in computing power nowadays. To further reduce computational time and resources, multi-fidelity surrogate (MFS) models that fuse HF and low-fidelity (LF) information have been proposed by Kennedy and O’Hagan (2000) and subsequently investigated by Forrester et al. (2007), Han et al. (2013), Tyan et al. (2015), Cai et al. (2017a), etc. Kennedy and O’Hagan presented a Bayesian approach to improve surrogate modeling efficiency by fusing expensive and cheap simulations. Forrester et al. extended the KRG surrogate model to a CoKRG model, which can be used to find optimal solutions more quickly and enhance the accuracy of a surrogate model of the highest level of analysis. Han et al. modified an MFS model by using gradient information and showed that a gradient-enhanced CoKRG model was more efficient in aero-load prediction. Tyan et al. proposed a global variable-fidelity modeling method that makes it possible to build a global approximation of the scaling function using the design of experiments (DoE) and the RBF surrogate model. Cai et al. developed a multi-fidelity high-dimensional model representation method to tackle the risk of “curse of dimensionality.”

In addition to KRG-based MFS models, other surrogate-based MFS models have also been developed in recent years. For example, Zhang et al. (2018) revised a heuristic MFS model based on linear regression (LR-MFS), to minimize prediction errors of HF samples. Cai et al. (2017b) proposed an adaptive MFS model based on RBF. Durantin et al. (2017) proposed a cooperative radial basis function (CoRBF) model, and compared the CoRBF model with CoKRG. Li et al. (2017) also developed a CoRBF model and found that it performed better than other baseline MFS models. Zhou et al. (2017a) transformed a multiple-to-one–dimensional structure to a one-to-one–dimensional structure by considering a LF model as prior knowledge. Zhou et al. (2017b) also proposed a sequential multi-fidelity model which aims at addressing an appreciate combination of HF and LF samples.

The hypothesis that a HF model is more accurate and more computationally expensive, whereas a LF model is less accurate but is considerably less computationally demanding (Viana et al. 2014), is generally assumed in the development of MFS methods. Thus, one of the key challenges in developing MFS models is how to optimally combine HF and LF information and take full advantage of the high accuracy of HF models and the high computational efficiency of LF models.

In this paper, a multi-fidelity surrogate model based on radial basis function (MFS-RBF) is developed. To construct the MFS-RBF model, an integrated correlation matrix between HF and LF samples is first constructed by augmenting the classical correlation matrix of an RBF model with LF responses. An integrated weight vector that consists of a scaling factor and relevant basis function weights are then calculated by employing HF responses. In the surrogate prediction process, an integrated correlation matrix is constructed between HF samples and testing points by augmenting LF responses at testing points. Instead of calculating LF responses directly, an RBF model built based on LF samples is used to approximate true LF responses. To evaluate the performance, MFS-RBF is compared with three baseline MFS models and three single-fidelity surrogate models on three numerical problems and one engineering problem. In addition, key factors that have significant influences on the accuracy of MFS-RBF are explored, such as the correlation between HF and LF models, the cost ratio of LF to HF models, the different combinations of HF and LF samples, the relationship between HF and LF sample set, and basis functions.

The remainder of this paper is organized as follows. Section 2 describes the overall framework of the developed MFS-RBF model. Comparisons between the MFS-RBF model and three baseline MFS models are presented in Section 3. Section 4 investigates the effects of key factors on the performance of MFS-RBF and computational cost savings. Conclusions and future works are provided in Section 5.

2 The MFS-RBF methodology

The MFS-RBF model is developed by integrating HF RBF surrogate model and LF RBF surrogate model through a scaling factor that is calculated based on a correlation matrix between HF and LF samples.

2.1 RBF surrogate model

A radial basis function (RBF) surrogate interpolates multivariate points (basis function centers) by using a series of symmetric basis functions. A typical form of RBF is expressed as:

$$ \widehat{y}(x)=\sum \limits_{i=1}^n{\lambda}_i\phi \left(r\left(x,{x}_i\right)\right) $$

(1)

where $\widehat{y}(x)$ is the prediction of a true response function, λ_i is the weighed coefficient for the i-th basis function, n is the number of samples, r(x, x_i) denotes the Euclidean distance between the i-th sample x_i and a testing point x, and ϕ is a basis function. Basis functions usually include Gaussian (G), multi-quadric (MQ), and inverse multi-quadric (IMQ) functions, as summarized in Table 1.

Table 1 Typical basis functions in RBF

Shape parameter σ shown in Table 1 can be determined in this paper by:

$$ \sigma ={d}_{\mathrm{max}}/\sqrt{2n} $$

(2)

where n is the number of samples, and d_max is the maximum distance between any two samples.

To solve the unknown parameters λ_i, interpolation conditions can be used.

$$ \widehat{y}\left({x}_i\right)=y\left({x}_i\right),i=1,2,...,n $$

(3)

By combining (1) and (3), a matrix form is given by:

$$ \boldsymbol{\Phi} \boldsymbol{\uplambda} =\mathbf{y} $$

(4)

where Φ is a correlation matrix, ϕ_ik = ϕ(r(x_i − x_k)), k = 1, 2, ...n.

2.2 MFS-RBF model

A typical form of an MFS model can be expressed as (5).

$$ {\mathbf{y}}_{\mathbf{H}}\left(\mathbf{x}\right)=\rho {\mathbf{y}}_{\mathbf{L}}\left(\mathbf{x}\right)+\mathbf{d}\left(\mathbf{x}\right) $$

(5)

where x is a set of samples, y_H(x) and y_L(x) represent the responses of HF and LF models, respectively, ρ denotes a scaling factor, and d(x) is a discrepancy function between the HF and LF models. The construction of the MFS-RBF model consists of the following two steps.

2.2.1 Step 1: scaling factor determination

The samples of HF and LF models are represented as ${\mathbf{x}}_{\mathbf{H}}=\left\{{x}_h^1,{x}_h^2,...,{x}_h^n\right\}$ and ${\mathbf{x}}_{\mathbf{L}}=\left\{{x}_l^1,{x}_l^2,...,{x}_l^p\right\}$ (x_H ⊂ x_L), respectively. The corresponding responses of HF and LF models are denoted as ${\mathbf{y}}_{\mathbf{H}}=\left\{{y}_h^1,{y}_h^2,...{y}_h^n\right\}$ and ${\mathbf{y}}_{\mathbf{L}}=\left\{{y}_l^1,{y}_l^2,...{y}_l^p\right\}$, respectively. The HF and LF sample sets are denoted as (x_H, y_H) and (x_L, y_L), which include n and p samples, respectively. Equation (5) can be transformed to

$$ {\mathbf{y}}_{\mathbf{H}}\left({\mathbf{x}}_{\mathbf{H}}\right)=\rho {\mathbf{y}}_{\mathbf{L}}\left({\mathbf{x}}_{\mathbf{H}}\right)+\mathbf{R}\left({\mathbf{x}}_{\mathbf{H}},{\mathbf{x}}_{\mathbf{H}}\right)\boldsymbol{\upomega} $$

(6)

where y_H(x_H) and y_L(x_H), respectively, represent the responses of HF and LF models at samples x_H, ρ is a scaling factor, R(x_H, x_H) is a correlation matrix between x_H and x_H, and ω is a vector of coefficients.

In the MFS-RBF model, the traditional correlation matrix R of an RBF model can be augmented as an integrated correlation matrix C. Thereby, (6) can be written in a matrix form as

$$ {\mathbf{y}}_{\mathbf{H}}=\mathbf{C}\boldsymbol{\upbeta } $$

(7)

where y_H is a vector of HF responses; the integrated correlation matrix C is augmented by combining the LF responses at the HF samples and the correlation matrix of HF samples, i.e., ${\mathbf{C}}_{n\times \left(n+1\right)}=\left[\begin{array}{cccc}{y}_{lh}^1& R\left({x}_h^1,{x}_h^1\right)& ...& R\left({x}_h^1,{x}_h^n\right)\\ {}...& ...& ...& ...\\ {}{y}_{lh}^n& R\left({x}_h^n,{x}_h^1\right)& ...& R\left({x}_h^n,{x}_h^n\right)\end{array}\right]$; β is an augmented coefficient vector constituted by ρ and ω, i.e., β_{(n + 1) × 1} = [ρ ω₁ ... ω_n]^′, where ${\left[{\omega}_1\kern0.5em ...\kern0.5em {\omega}_n\right]}^{\prime }=\boldsymbol{\upomega}$.

In this paper, the rank of the matrix C is n, namely C is row full rank. According to the matrix theory (Petersen and Pedersen 2008), there exists unique least norm solution β indicating that MFS-RBF models can strictly go through HF samples. Then, the parameter vector β can be calculated by (8) which is transformed from (7), with obtained scaling factor ρ and ω. It is seen from (7) that the first element of β denotes the scaling factor ρ.

$$ \boldsymbol{\upbeta} ={\mathbf{C}}^T{\left(\mathbf{C}{\mathbf{C}}^T\right)}^{-1}\mathbf{Y} $$

(8)

2.2.2 Step 2: prediction of MFS-RBF

The MFS-RBF model is finally formulated as.

$$ {\widehat{\mathbf{y}}}_{\mathbf{mf}}\left(\mathbf{x}\right)=\rho {\widehat{\mathbf{y}}}_{\mathbf{L}}\left(\mathbf{x}\right)+\mathbf{r}\left(\mathbf{x},{\mathbf{x}}_{\mathbf{L}}\right)\boldsymbol{\upomega} $$

(9)

where ${\widehat{\mathbf{y}}}_{\mathbf{mf}}\left(\mathbf{x}\right)$ represents the vector of predictions calculated by MFS-RBF at testing points x (i.e., x = {x₁, x₂, ..., x_m}, and m is the number of testing points); ${\widehat{\mathbf{y}}}_{\mathbf{L}}\left(\mathbf{x}\right)$ represents the vector of corresponding predictions of LF models; ρ and ω can be obtained in Step 1; r(x, x_L) is the correlation matrix between testing points x and LF samples x_L. The matrix form is then formulated as

$$ {\widehat{\mathbf{y}}}_{\mathbf{mf}}=\mathbf{c}\boldsymbol{\upbeta } $$

(10)

where the integrated correlation matrix c is augmented by combining predictions of the LF model at testing points ${\widehat{\mathbf{y}}}_{\mathbf{L}}\left(\mathbf{x}\right)$ and the correlation matrix between LF samples and testing points r(x, x_L), which is expressed as

$$ \mathbf{c}=\left[\begin{array}{cccc}{\widehat{y}}_l\left({x}_1\right)& r\left({x}_1,{x}_h^1\right)& ...& r\left({x}_1,{x}_h^n\right)\\ {}...& ...& ...& ...\\ {}{\widehat{y}}_l\left({x}_m\right)& r\left({x}_m,{x}_h^1\right)& ...& r\left({x}_m,{x}_h^n\right)\end{array}\right] $$

(11)

The integrated correlation matrix C shown in (7) is augmented by adding LF responses at samples x_H. Thus, in the prediction process, to construct the augmented matrix c between HF samples and testing points, corresponding responses of the LF model y_L(x) at testing points x are simultaneously needed. In addition to run LF simulations and get the LF sample set directly, an RBF model built based on the LF sample set (x_L, y_L) is used to obtain approximate responses ${\widehat{\mathbf{y}}}_{\mathbf{L}}\left(\mathbf{x}\right)$ at testing points x.

2.3 Performance criteria

Different metrics can be used to assess the modeling accuracy of MFS models, which can be broadly classified into two types, namely global criteria such as R² (i.e., R-square, coefficient of determination) and root mean square error (RMSE), local criteria such as relative maximum absolute error (RMAE). RMSE is highly correlated with R² and cannot express the goodness of fit intuitively, because the value of RMSE depends on responses. That is, for different problems, the value of RMSE varies with the responses. RMAE cannot represent the overall performance in the design space. Therefore, R² is selected as the only criterion for the comparisons in this paper.

In this paper, R² is adopted as the sole performance criterion, since RMSE is strongly correlated with R² and RMAE cannot reflect the global performance in the design space. The R² metric is calculated as

$$ {R}^2=1-\frac{\sum_{i=1}^n{\left({y}_i-{\widehat{y}}_i\right)}^2}{\sum_{i=1}^n{\left({y}_i-\overline{y}\right)}^2} $$

(12)

where n is the number of samples; y_i and ${\widehat{y}}_i$ represent true responses and predictions at testing points, respectively; and $\overline{y}$ is the means of true responses. Essentially, R² denotes the correlation between the true model and the surrogate model, and the surrogate model is more accurate if R² is closer to one.

The Pearson correlation coefficient (PCC), also referred to as Pearson’s r, is a measure of the correlation between two random variables X and Y. In this paper, we use square of Pearson’s r which is denoted as r² to describe the correlation between HF and LF functions which is inspired by Toal (2015), as shown in (13).

$$ {r}^2={\left(\frac{\sum_{i=1}^n\left({y}_h-{\overline{y}}_h\right)\left({y}_l-{\overline{y}}_l\right)}{\sqrt{\sum_{i=1}^n{\left({y}_h-{\overline{y}}_h\right)}^2}\sqrt{\sum_{i=1}^n{\left({y}_l-{\overline{y}}_l\right)}^2}}\right)}^2 $$

(13)

where y_h and y_l denote the HF and LF responses, respectively; ${\overline{y}}_h$ and ${\overline{y}}_l$ represent the means of the HF and LF responses, respectively.

3 Numerical examples

3.1 Design of experiments

Design of experiments (DoEs) are methods to strategically generate samples from computer simulations or experiments in a domain of interest to build surrogate models. In this paper, it is assumed that the number of DoE (HF simulations) points is m × n for a single-fidelity surrogate model, where n is the dimension of the problem and m is a user-defined value. To build an MFS model, the number of HF samples is set to be k × n (k < m), and the remaining (m − k) × n HF simulations are replaced by more LF simulations. If the computational cost ratio between LF and HF models is θ, then the number of LF samples should be (m − k) × n/θ.

To evaluate the performance, the MFS-RBF model is compared with three benchmark MFS models (i.e., CoKRG (Forrester et al. 2007), LR-MFS (Zhang et al. 2018), and CoRBF (Durantin et al. 2017)) and three single-fidelity surrogate models (i.e., PRS, RBF-MQ, and KRG) through three widely used numerical test problems (i.e., the Forretal function (n = 1) from ref. Forrester et al. (2007), the Branin function (n = 2) from ref. Liu et al. (2016), and the Colville function (n = 4) from ref. Durantin et al. (2017)) and one engineering problem. It is worth noting that since the source code of the CoRBF model is not available, some specific parameters may be different from those in Durantin’s paper even we wrote the code according to this paper. In addition, genetic algorithm (GA) is used for CoKRG and CoRBF models to search the unknown parameters, and for the LR-MFS model, a first-order PRS is used to approximate the discrepancy function. To determine the number of samples for building a surrogate, the accuracies of RBF-MQ with different sample sizes are compared. The criterion of R² ≥ 0.8 is used to determine an appropriate sample size (Forrester et al. 2008). The accuracy is averaged over 50 randomly sampling sets.

In this paper, the standard Forretal, Branin, and Colville functions and the corresponding variants are considered as the HF and LF functions, respectively. In addition, the method of defining a LF function is inspired by Toal (2015), who introduced the thought of the correlation r² between HF and LF responses. For Forretal, Branin, and Colville cases, in Subsections 3.2, 3.3, and 3.4, the value of k is set as 0.8 × m, and then the numbers of HF samples for the Branin and Colville functions are 4, 8, and 32, respectively. The cost ratio θ is set to be 0.1, meaning that the computational cost of a HF model is 10 times of a LF model. Hence, the numbers of LF samples for the Branin and Colville functions are10, 20, and 80, respectively. For the engineering case in Subsection 3.4, the value of k is set as 0.5 × m, and then the numbers of HF samples are 5. The cost ratio θ is set to be 0.2, and then the number of LF samples is 25.

Among many available DoE methods, the Latin hypercube sampling (LHS) has been proved to be capable of balancing the trade-off between accuracy and robustness by generating a near-random set of samples. For all surrogate models in this paper, the MATLAB function lhsdesign is adopted to generate DoE samples. To mitigate the impact of random DoE on surrogate performance, 20 sets of DoE samples are generated randomly and the averaged results are compared for the three numerical test problems. In addition, 1000 × n randomly generated testing points are used for validation.

3.2 Test problem 1: Forretal function

In the case of Forretal function, to ensure that the correlation between HF and LF functions varies from 0 to 1, the LF function is derived from the HF function by multiplying a coefficient function of parameter A and adding a first-order term of x.

HF model:

$$ {y}_h={\left(6x-2\right)}^2\sin \left(12x-4\right) $$

(14)

LF model:

$$ {y}_l=\left(1-{A}^2-2A\right){y}_h+10\left(x-0.5\right)-5 $$

(15)

where x ∈ [0, 1], y_h is a HF model, y_l is a LF model, and the parameter A varies from 0 to 1 to reflect the degree of the correlation r².

Figure 1 compares the MFS-RBF model with single-fidelity surrogate models. Two sample sets, 4n and 5n, are generated to construct different single-fidelity surrogate models. To eliminate the effect of DoE, the accuracies of the single-fidelity surrogates are averaged over 20 randomly sampling sets. The red dashed line in Fig. 1 shows the relationship between correlation r² and the parameter A for the Branin function. It is observed that the MFS-RBF model with the minimum r² value is obtained when A = 0.47. The r² between the HF and LF models decreases as A increases from 0 to 0.47, while the r² increases as A increases from 0.47 to 1. A maximum r² is obtained when A = 1. It is seen from Fig. 1 that the tendency for the performance of MFS-RBF strongly matches the tendency of the correlation r². From Fig. 1, we can see that MFS-RBF with 4n HF samples almost outperforms all single-fidelity surrogates no matter the sample number is 4n or 5n.

Fig. 1

Comparison between MFS-RBF and single-fidelity surrogate models for the Forretal function. “prs_4” means that the PRS model is constructed using 4n HF samples, as well as “rbf_mq_4” and “krg_4”; “prs_5” means that the PRS model is constructed using 5n HF samples, as well as “rbf_mq_5” and “krg_5”

Figure 2a compares the MFS-RBF, CoKRG, LR-MFS, and CoRBF models for the Forretal function on R² and the standard deviation (Std) of R². Each value of R² at parameter A is the average of the results obtained for 20 DoEs, and the Std of R² denotes the standard deviation of the 20 values. The results show that MFS-RBF performs better than CoKRG and is similar to CoRBF. It is interesting to find that the tendency of the performance of the MFS-RBF, CoKRG, and CoRBF models as shown in Fig. 2a is consistent with the tendency of the HF/LF model correlation r² as shown in Fig. 1, while the LR-MFS model is slightly insensitive to the correlation r².

Fig. 2

Comparison of MFS models for the Forretal function. aR² and Std of R² vary with the parameter A. bR² varies with the HF/LF correlation r²

Figure 2b shows the relationship between the HF/LF correlation r² and the accuracy R². It is observed that for the MFS-RBF, CoKRG, and CoRBF models, the accuracy R² increases as the correlation between the HF and LF models r² increases. This validates the assumption made in (5) in which a HF model can be represented as a function of a LF model. In addition, it is seen that the MFS-RBF model shows larger mean of R², which performs better than CoKRG and CoRBF models in terms of prediction accuracy and robustness.

3.3 Test problem 2: Branin function

Similar to the Forretal function, the LF function in Branin case is obtained from HF function by subtracting a quadratic term of x which multiplies a coefficient function of parameter A.

HF model:

$$ {y}_h=\left({x}_2-\frac{5.1}{4{\pi}^2}{x}_1^2+\frac{5}{\pi }{x}_1-6\right)+10\left(1-\frac{1}{8\pi}\right)\cos \left({x}_1\right)+10 $$

(16)

LF model:

$$ {y}_l={y}_h-\left(0.5{A}^2+A+0.2\right){\left({x}_2-\frac{5.1}{4{\pi}^2}{x}_1^2+\frac{5}{\pi }{x}_1-6\right)}^2 $$

(17)

where x_i ∈ [−5, 10], x_i denotes the i-th variable for i = 1, 2.

Figure 3 compares the MFS-RBF model with single-fidelity surrogate models. Two sample sets, 4n and 5n, are generated to construct different single-fidelity surrogate models. It is observed that the MFS-RBF model with the minimum r² value is obtained when A = 0.57. The r² between the HF and LF models decreases as A increases from 0 to 0.57, while the r² increases as A increases from 0.57 to 1. A maximum r² is obtained when A = 0. It is seen from Fig. 3 that the tendency for the performance of MFS-RBF strongly matches the tendency of the correlation r². When the correlation r² is greater than 0.8 (i.e., when 0 ≤ A ≤ 0.37), MFS-RBF with 4n HF samples outperforms all single-fidelity surrogates no matter the sample number is 4n or 5n. Overall, MFS-RBF performs better than the three single-fidelity surrogate models with fewer samples (i.e., “prs_4,” “rbf_mq_4,” and “krg_4”) as A varies from 0 to 1.

Fig. 3

Comparison between MFS-RBF and single-fidelity surrogate models for the Branin function

Figure 4a compares the MFS-RBF, CoKRG, LR-MFS, and CoRBF models for the Branin function on the R² and Std of R². The results show that when A is less than 0.3, i.e., the correlation r² is about 0.9, MFS-RBF performs worse than CoKRG but better than LR-MFS and CoRBF; when A is greater than 0.3, MFS-RBF outperforms all the MFS models in prediction accuracy. From the results of Std of R², we can see that MFS-RBF behaviors the best in terms of robustness as A varies from 0 to 1. It is also found that the tendency of the performance of the MFS-RBF, CoKRG, and LR-MFS models as shown in Fig. 4a is consistent with the tendency of the HF/LF model correlation r² as shown in Fig. 2.

Fig. 4

Comparison of MFS models for the Branin function. aR² and Std of R² vary with the parameter A. bR² varies with the HF/LF correlation r²

Figure 4b shows the relationship between the HF/LF correlation r² and the accuracy R². It is observed that the accuracy R² increases as the correlation between the HF and LF models r² increases. This validates the assumption made in (5) in which a HF model can be represented as a function of a LF model. In addition, it is seen that the MFS-RBF model shows the largest mean of R², which outperforms the other three baseline MFS models in terms of prediction accuracy and robustness.

3.4 Test problem 3: Colville function

Similarly, for the case of the Colville function, the LF function is derived from the standard Colville function.

HF model:

$$ {\displaystyle \begin{array}{l}{y}_h=100{\left({x}_1^2-{x}_2\right)}^2+{\left({x}_1-1\right)}^2+{\left({x}_3-1\right)}^2+90\left({x}_3^2-{x}_4\right)\\ {}\kern1.5em +10.1\left({\left({x}_2-1\right)}^2+{\left({x}_4-1\right)}^2\right)+19.8\left({x}_2-1\right)\left({x}_4-1\right)\end{array}} $$

(18)

LF model:

$$ {y}_l={y}_h\left({A}^2\left({x}_1,{x}_2,{x}_3,{x}_4\right)\right)-\left(A+0.5\right)\left(5{x}_1^2+4{x}_2^2+3{x}_3^2+{x}_4^2\right) $$

(19)

where x_i ∈ [−1, 1], x_i denotes the i-th variable for i = 1, 2, 3, 4.

Figure 5 compares the MFS-RBF model with single-fidelity surrogate models. Two sample sets, 8n and 10n, are generated to construct different single-fidelity surrogate models. It is observed that the least r² occurs when A = 0.68. When A ≤ 0.68, r² monotonically decreases from 0.27 to 0. When A ≥ 0.68, r² monotonically increases from 0 to 0.9. It is seen that the tendency of the MFS-RBF performance strongly matches the tendency of HF/LF correlation r². When the correlation r² is greater than 0.8, namely A ≥ 0.95, MFS-RBF performs the best. When the correlation r² is less than 0.8, namely 0 ≤ A ≤ 0.95, MFS-RBF always performs better than most single surrogate models except “rbf_mq_10” and “krg_10”.

Fig. 5

Comparison between MFS-RBF and single-fidelity surrogate models for the Colville function. “prs_8” means that the PRS model is constructed using 8n HF samples, as well as “rbf_mq_8” and “krg_8”; “prs_10” means that the PRS model is constructed using 10n HF samples, as well as “rbf_mq_10” and “krg_10”

Figure 6a compares MFS-RBF, CoKRG, LR-MFS, and CoRBF based on R² and Std of R² for the Colville function. It is again found that the performance tendencies for all MFS models are consistent with the correlation tendency as shown in Fig. 5. In addition, MFS-RBF is relatively insensitive to the correlation r² than the other baseline MFS models and performs the best in robustness with the lowest Std of R². Figure 6b shows that the accuracy R² increases as the HF/LF correlation r² increases. Overall, the mean and Std of R² in Fig. 6 show that MFS-RBF outperforms the other three baseline MFS models in terms of robustness, and the accuracy performance of MFS-RBF is the best when the correlation is low.

Fig. 6

Comparison of MFS models for the Colville function. aR² and Std of R² vary with the parameter A. bR² varies with the HF/LF correlation r²

3.5 Engineering problem

In addition to the three numerical problems, an engineering problem, i.e., computational fluid dynamic (CFD) analysis of a pressure relief valve (PRV) as shown in Fig. 7a, is conducted to investigate the performance of the developed MFS-RBF model. The flow enters the valve from the inlet, goes through the gap between the disc and the seat, and discharges from the outlet. The opening and closing of PRV are controlled by resultant force exerted on the disc. When the fluid force (F) is bigger than the force against on the disc, the PRV opens. To obtain the fluid force (F), steady simulations based on two CFD models with different dimensions are conducted. The standard k − ε turbulent model is used; the medium is water with an initial temperature of 300 K. The three-dimensional (3-D) CFD model established using 284,412 unstructured grids (Fig. 7b) and the 2-D axisymmetric CFD model with 9646 unstructured grids (Fig. 7c) of the relief valve are considered as the HF and LF simulation models, respectively. In CFD simulations, the inlet pressure (P) is constant. The pressure of outlet is 0. The inlet pressure (P) and the opening lift (L) are selected as two design variables, with ranges of 0.1~0.4 atm and 1~4 mm, respectively (Fig. 7).

Fig. 7

Relief valve modeling. a Configuration. b 3-D CFD model. c 2-D CFD model

One HF simulation takes about 50 min, while one LF simulation takes about 10 min on a computer with a 3.4GHz processor and 8 cores CPU, and 16G RAM. Thus, the cost ratio θ of the LF model to the HF model is 0.2. To construct MFS models, each type of MFS model is constructed based on 5 (2.5n; n is the number of variables) HF samples and 25 (12.5n) LF samples, and thus the total cost of constructing MFS models is equal to that of generating 10 (5n) HF samples. Another 40 randomly generated samples are used for validation. To eliminate the effect of DoE randomness, 20 sampling sets are generated and the results are averaged.

The comparison of MFS-RBF with three baseline MFS models and three single-fidelity surrogates are shown in Fig. 8. It is seen that MFS-RBF performs the best among all the surrogates for this engineering problem, as illustrated by the largest mean of R². Comparing with the other three MFS models, the MFS-RBF model also behaves better in the robustness metric with lower Std of R².

Fig. 8

Comparison of different MFS and single-fidelity models on the mean and Std of R². The legend of “prs_2.5” means that the PRS model is constructed using 2.5n HF samples, as well as “rbf_mq_2.5” and “krg_2.5”

4 Exploring the effects of key MFS-RBF factors

In this section, we use Branin and Colville to further explore the effects of key factors on the performance of MFS-RBF. It is assumed that the total cost of generating DoE points is fixed, with a total budget of 5n and 10n HF samples for the Branin and Colville functions, respectively. All the results in this section are averaged over 20 random sampling sets to eliminate the effect of DoE.

4.1 Effect of the cost ratio of LF to HF models

It is assumed that the total cost of HF samples accounts for 4/5 of the total cost, and the remaining 1/5 is allocated to LF samples. That is, if the initial sample number is 5n (m = 5) and 10n (m = 10), 4n (k = 4) and 8n (k = 8) HF samples are used for the Branin and Colville functions, respectively. The remaining 1n (m − k = 1) and 2n (m − k = 2) HF samples are replaced by more LF samples ((m − k)n/θ). To better evaluate the sensitivity of MFS to the cost ratio θ, five cost ratios of LF to HF models are compared in this subsection, i.e., 0.25, 0.2, 0.125, 0.1, and 0.02. The detailed sampling configurations according to different cost ratios for the Branin and Colville functions are summarized in Table 2.

Table 2 Sampling configurations for Branin and Colville

Figure 9 illustrates the effect of different cost ratios on the performance of MFS-RBF for the Branin function. It is observed that when the cost ratio θ decreases, the performance of MFS-RBF becomes more sensitive to the correlation r², the accuracy of MFS-RBF is improved significantly (as shown by the mean of R²), and the robustness of MFS-RBF is slightly improved except in the case of θ = 0.25 (as shown by the Std of R²). It is seen from Fig. 9 that MFS-RBF with θ = 0.25 shows the worst performance as illustrated by the lowest R² at most time. The HF/LF correlation r² shows little influence on the performance of MFS-RBF when θ = 0.25 and θ = 0.2. Thus, the MFS-RBF model may not be suitable for the condition that the cost ratio θ ≥ 0.2.

Fig. 9

The effect of the cost ratio on the performance of MFS-RBF for the Branin function

It is seen from Fig. 9 that when the correlation r² is greater than 0.7, MFS-RBF shows the best accuracy and robustness in the case of θ = 0.02. When the correlation 0 ≤ r² ≤ 0.7, the performance of MFS-RBF with θ = 0.25 performs the best but is still poor as shown by small values of R². The variation in the robustness is partially due to that more LF samples are added in the MFS-RBF model when decreasing the cost ratio θ, thereby more LF information is blended into MFS-RBF. Thus, the accuracy of the LF model also significantly affects the MFS-RBF model. When the HF/LF correlation r² is strong, it has positive impact on the performance of MFS-RBF, while when the HF/LF correlation r² is weak, it has negative impact on the performance of MFS-RBF.

Figure 10 illustrates the effect of different cost ratios on the performance of MFS-RBF for the Colville function as A changes from 0 to 1. It is seen from Fig. 10 that when r² ≥ 0.7, the performance of MFS-RBF becomes more sensitive to the HF/LF correlation with the increase of cost ratio θ, which is similar to the results observed for the Branin function.

Fig. 10

The effect of the cost ratio on the performance of MFS-RBF for the Colville function

From Fig. 10, we observe that when the correlation r² is less than 0.7, cost ratio θ has no significant influence on the performance of MFS-RBF. When the correlation r² is greater than 0.7, the performance of MFS-RBF becomes better as the cost ratio θ decreases. In this case, the LF and HF models are more correlated. A lower cost ratio θ indicates a larger size of LF samples, and the LF model can be fitted more accurately. A relatively more accurate LF model can improve the performance of MFS-RBF when HF and LF models have strong correlation.

4.2 Effect of the different combinations of HF and LF samples

To better explore the effect of different combinations of HF and LF samples, the cost ratio θ is set to be 0.1. The total cost used to generate HF samples for single-fidelity surrogates is divided into six cases, namely “1_4,” “2_3,” “2.5_2.5,” “3_2,” “4_1,” and “4.5_0.5.” For the Branin (or Colville) function, the case of “1_4” means that the cost of running 1/5 × 5n (or 1/5 × 10n) HF samples is allocated to generate HF samples, and the cost of running 4/5 × 5n (or 4/5 × 10n) HF samples is allocated to generate LF samples, as well as other cases. The detailed sampling configurations according to different combinations of HF and LF samples for Branin and Colville are summarized in Table 3.

Table 3 Sampling configurations for Branin and Colville

Figure 11 illustrates the effect of different combinations of HF and LF samples on the performance of MFS-RBF for the Branin function. By comparing Fig. 11 with Fig. 1, it is observed that MFS-RBF with more HF samples and fewer LF samples (e.g., cases “4_1” and “4.5_0.5”) is less sensitive to the correlation r². This can be attributed to the less biased information (from the LF samples) integrated into MFS-RBF. MFS-RBF tends to be more robust when increasing the number of HF samples. Regarding the prediction performance, when the correlation r² is less than 0.7, MFS-RBF performs better with increase of the HF sample size, but always behaves worse than the single-fidelity RBF model constructed from 5n HF samples; when the correlation r² is greater than 0.7, MFS-RBF in the cases of “3_2” and “2.5_2.5” has a slightly better performance than that of in the cases of “4_1,” “2_3,” and “1_4.” In addition, the MFS-RBF model in the case of “4.5_0.5” still performs worse than the single-fidelity surrogate “rbf_mq_5.”

Fig. 11

Impact of the combination of HF and LF samples on the performance of MFS-RBF for Branin

Figure 12 illustrates the effect of different combinations of HF and LF samples on the performance of MFS-RBF for the Colville function as A varies from 0 to 1. The results of the Colville function are similar to those of the Branin function. When the correlation r² is less than 0.7, it is observed that the accuracy of MFS-RBF with more HF samples is improved but still worse than the single-fidelity surrogate model “rbf_mq_10”; when the correlation r² is greater than 0.7, the MFS-RBF with more HF samples performs worse (e.g., case “4.5_0.5”) than those with fewer HF samples (e.g., case “4_1”). By comparing Fig. 10 and Fig. 3, the MFS-RBF model becomes less sensitive to the correlation r² by adding more HF samples. The case “1_4” always performs the worst as the correlation r² increases. Thus, it is suggested that 10–80% of the total budget should be used for generating LF samples.

Fig. 12

Impact of the combination of HF and LF samples on the performance of MFS-RBF for Colville

4.3 Effect of the relationship between HF and LF samples

In the Subsections 4.1 and 4.2, it is assumed that the HF sample set is a subset of the LF sample set which is denoted by “Condition 1.” In this part, the performance of the MFS-BRF model is investigated when the HF sample set is not a subset of the LF sample set which is expressed by “Condition 2.” The relevant settings, such as sample size, correlations, and combinations in this part, are the same as those in Subsections 4.1 and 4.2. Figures 13 and 14 show the comparisons of different cost ratios and combination of HF and LF samples on the performance of MFS-RBF through the Branin and Colville functions. The blue columns express the mean and Std of R² when r² ≥ 0.7 in the case of “Condition 2,” while the red columns denote the mean and Std of R² when r² ≥ 0.7 obtained in the case of “Condition 1” in Subsections 4.1 and 4.2.

Fig. 13

Comparison of different cost ratios when r² ≥ 0.7. a Branin function. b Colville function

Fig. 14

Comparison of different combinations of HF and LF samples when r² ≥ 0.7. a Branin function. b Colville

From Figs. 13 and 14, accuracy and robustness metrics in terms of “Condition 2” are consistent with those in the case of “Condition 1” for the Branin and Colville functions. Overall, it can be concluded that HF sample set is not necessarily a subset of LF sample set. It is worth noting that the HF sample set is still a subset of the LF sample set in the rest of this paper.

4.4 Effect of the different basis functions

In this subsection, the effect of different basis functions of MFS-RBF models is explored. For the Branin (Colville) function, the total budget is the cost of 5n (10n) HF samples and the number of HF samples is 4n (8n). The cost ratio θ is set to be 0.1. Basis functions used in this part are shown in Table 1, including multi-quadric (MQ), Gaussian (G), and inverse multi-quadric (IMQ) functions. Figure 15 shows that in terms of accuracy and robustness, MFS-RBF with MQ basis function performs the best, MFS-RBF with IMQ basis function behaves the second, and MFS-RBF with G basis function is the worst. From Fig. 15b, we can see that when the basis function is G, R² has no results, which means that the accuracy of the MFS-BRF with G basis function is too poor. Overall, MFS-RBF with MQ basis function behaves the best, and MQ is suggested to be chosen as the basis function of the MFS-RBF model in this paper.

Fig. 15

Effect of different basis functions on the performance of MFS-RBF model. a Branin function. b Colville function

4.5 Computational cost

In addition to the accuracy improvement, this subsection further investigates computational cost savings from the MFS-RBF model. It is assumed that the cost ratio of LF to HF functions θ is 0.1 and the MFS-RBF model is constructed with the “4_1” combination as defined above. It is seen that MFS-RBF performs relatively better when r² ≥ 0.7, and hence the computational cost comparison is performed under large correlation r² values in this subsection. In addition, since the MFS-RBF model is an RBF-MQ-based multi-fidelity surrogate model, we compare the MFS-RBF model to a single-fidelity RBF-MQ model.

For the Branin function, the comparisons are implemented under three HF/LF correlation values, namely r² = 0.9, r² = 0.95, and r² = 1.0. In these three cases, the total computational cost of MFS-RBF models is the same, with the total budget of 5n HF models. Figure 16 and Table 4 show the computational cost of MFS-RBF and RBF-MQ under the three correlation cases. It is seen from Fig. 16 that when the correlation r² = 0.9, the R² of MFS-RBF built from 5n samples is approximately 0.86, which is similar to the RBF-MQ when using the same number of 7n samples, and then MFS-RBF saves approximately 40% computational cost. When the correlation r² = 0.95, the R² of MFS-RBF is approximately 0.87, and RBF-MQ requires more than 7n samples to attain the same accuracy. Hence, MFS-RBF saves approximately 40% computational cost. When the correlation r² = 1.0, the R² of the MFS-RBF model is about 0.90, and RBF-MQ requires 9n samples to attain the same accuracy. Hence, MFS-RBF saves approximately 80% computational cost. Overall, MFS-RBF saves an average of 50% computational cost.

Fig. 16

Computational cost of MFS-RBF and RBF-MQ under different correlation values for Branin

Table 4 Computational cost of MFS-RBF and RBF-MQ under different correlation values for Branin

For the Colville function, the comparisons are implemented under two correlation values, namely r² = 0.8 and r² = 0.9. In these two cases, the total computational cost of MFS-RBF models is the same, with the total budget of 10n HF models. Figure 17 and Table 5 show the computational cost of MFS-RBF and RBF-MQ under the two correlation cases.

Fig. 17

Computational cost of MFS-RBF and RBF-MQ under different correlation values for Colville

Table 5 Computational cost of MFS-RBF and RBF-MQ under different correlation values for Colville

From Fig. 12, we observe that when the correlation r² = 0.8, the R² of MFS-RBF built by 10n samples is about 0.85, and RBF-MQ model requires about 15n samples to attain the same accuracy. Thereby, MFS-RBF saves approximately 50% computational cost compare with single RBF surrogate. When the correlation r² = 0.9, the R² of the MFS-RBF model is about 0.88, and RBF-MQ model requires 19n samples to attain the same accuracy. Hence, the MFS-RBF model saves approximately 90% computational cost. Overall, the MFS-RBF model saves an average of 70% computational cost.

5 Conclusions

A multi-fidelity surrogate model based on RBF, called MFS-RBF, was developed in this paper. MFS-RBF integrated high-fidelity and low-fidelity models by using a scaling factor and an augmented correlation matrix. To validate the performance of MFS-RBF, three benchmark MFS models (i.e., CoKRG, LR-MFS, and CoRBF) and three widely used single-fidelity surrogates (i.e., PRS, RBF-MQ, and KRG) were selected, and three numerical test problems and one engineering problem under different correlations between HF and LF models were tested. Comparison results showed that when the correlation r² between HF and LF models is greater than 0.7, MFS-RBF generally performs better than single-fidelity surrogates. MFS-RBF also performed better than three baseline MFS models in terms of both accuracy and robustness. The results of this paper will be of great help for the MFS research community, for example, to strengthen the understanding whether a MFS model could be trusted, and to provide suggestion under what conditions a MFS model would be better than a single surrogate model.

The effects of key factors on the performance of MFS models were also investigated, including the cost ratio θ, the combination of HF and LF samples, the relationship between HF and LF samples, and basis functions. Results showed that MFS-RBF is less sensitive to the correlation between HF and LF models than the three baseline MFS models. This finding is more useful for real engineering applications, as the correlation between HF and LF models is generally unknown prior and could be weak in many cases. It was also found that with the same total computational cost, (i) the cost ratio of LF to HF is suggested to be less than 0.2, (ii) 10–80% of the total cost should be used for generating LF samples, (iii) the HF sample set is not necessarily to be a subset of LF sample set, and (iv) a MFS-RBF model with the MQ basis function performs the best. The computational cost of MFS-RBF was also investigated, and the results showed that MFS-RBF could save an average of 50 to 70% computational cost when HF and LF models are strongly correlated.

In this paper, we assumed that the cost ratio θ and the correlation r² are independent, but this is not always true in practice. In a situation such as the LF model is a coarser 3-D model and the HF model is a finer 3-D model, the cost ratio θ and the correlation are dependent. Future work will model the coupling effects between the cost ratio and the correlation to further improve MFS-RBF.