A data-driven polynomial chaos method considering correlated random variables

Abstract

Variable correlation commonly exists in practical engineering applications. However, most of the existing polynomial chaos (PC) approaches for uncertainty propagation (UP) assume that the input random variables are independent. To address variable correlation, an intrusive PC method has been developed for dynamic system, which however is not applicable to problems with black-box-type functions. Therefore, based on the existing data-driven PC method, a new non-intrusive data-driven polynomial chaos approach that can directly consider variable correlation for UP of black-box computationally expensive problems is developed in this paper. With the proposed method, the multivariate orthogonal polynomial basis corresponding to the correlated input random variables is conveniently constructed by solving the moment-matching equations based on the correlation statistical moments to consider the variable correlation. A comprehensive comparative study on several numerical examples of UP and design optimization under uncertainty with correlated input random variables is conducted to verify the effectiveness and advantage of the proposed method. The results show that the proposed method is more accurate than the existing data-driven PC method with Nataf transformation when the variable distribution is known, and it can produce accurate results with unknown variable distribution, demonstrating its effectiveness.

1 Introduction

Uncertainty propagation (UP) methods, which could be used to quantify uncertainty in system output performance based on random or noisy inputs, are of great importance for design under uncertainty especially for problems with computationally expensive simulation analysis models of black-box type (such as finite element analysis and computational fluid dynamics). A variety of probabilistic UP approaches have been developed (Lee and Chen 2009), among which the polynomial chaos (PC) technique is a rigorous one due to its strong mathematical basis. By using PC, the function with random inputs can be represented as a stochastic meta-model, based on which lower-order statistical moments as well as reliability of the function output can be derived efficiently to facilitate the implementation of design optimization under uncertainty scenarios like robust design optimization (Xiong et al. 2011a) and reliability-based design optimization (Coelho and Bouillard 2011). As it has high efficiency and accuracy for UP, the PC method has been extensively applied to engineering problems, such as the propagation of uncertainty in composite structures (Mukhopadhyay et al. 2016), the robust aerodynamic optimization of airfoil (Dodson and Parks 2015), marine vessel (Wei et al. 2018), and trajectory of the flight vehicle (Prabhakar et al. 2010).

The original PC method employs the Hermite orthogonal polynomial as the basis, which exhibits slow convergence when the input random variable follows non-Gaussian distribution (William and Meecham 1968). To address this issue, Xiu and Karniadakis extended the original PC method to a generalized one (gPC) that can directly deal with five typical types of distribution based on the Askey scheme (Xiu and Karniadakis 2003). In addition, to solve UP problems with arbitrary distribution forms and improve the applicability of the PC method, a Gram-Schmidt PC method (GS-PC) using Schmidt orthogonalization and a multi-element generalized PC method (ME-PC) were developed by Wittevee and Bijl (Witteveen and Bijl 2013) and Wan and Karniadakis (Wan and Karniadakis 2006), respectively. However, due to cost limitations, the complete probabilistic distribution of input random variable may not exist with insufficient data in practical engineering.

To solve this problem, some non-probabilistic methods of UP could be used, such as random sets (Zhang and Achari 2010), interval probabilities (Xiao et al. 2016; Liu et al. 2019), interval theory (Xia et al. 2017; Li et al. 2017), fuzzy logic (Abou 2012), etc. On the other hand, in the classical probability theory, Oladyshkin and Nowak proposed a non-intrusive data-driven PC (DD-PC) method without referring to the distribution functions, with which the univariate orthogonal polynomial basis is constructed by matching certain order of statistical moments of the input random variable based on the discrete data (Oladyshkin et al. 2011; Oladyshkin and Nowak 2012). Later, to improve the accuracy and efficiency of the DD-PC method, they applied the Bayesian method to calibrate the DD-PC model (Oladyshkin et al. 2013). To solve higher-dimensional UP problems, an enhanced DD-PC method was developed by Wang et al. through extending the Galerkin projection method to DD-PC, in which the quadrature nodes and weights required by PC coefficient calculation were obtained by solving the moment-matching equations (Wang et al. 2017). Zhou et al. improved the accuracy of DD-PC method with equilibrium sampling strategy and weighted least-square method (Guo et al. 2019). The DD-PC methods have demonstrated to have a wider scope of application compared to the existing PC approaches (gPC, GS-PC, ME-PC) that require the complete probabilistic distributions of input random variables.

However, for all the existing PC methods including gPC, GS-PC, ME-PC, and DD-PC, it is generally assumed that the input random variables are independent to each other during the construction of PC model. However, in practical engineering applications, oftentimes, the input random variables are statistically correlated, such as the material properties and fatigue properties in structural analysis (Socie 2003), and the length and width of a beam structure considering earthquake resistance (Du 2008). To deal with the correlated input random variables, the transformation methods, such as Rosenblatt transformation (Rosenblatt 1952), orthogonal transformation (Rackwitz and Flessler 1978) and Nataf transformation (Kiureghian and Liu 1986), can be employed to first transform the correlated random input variables into independent standard normal ones. UP with PC is then conducted with respect to these variables. Noh et al. made a comparative study of Rosenblatt transformation and Nataf transformation and applied them to design optimization under uncertainty of a coil spring with correlation between the inner diameter and the wire diameter (Noh et al. 2009). Song et al. applied the Nataf transformation to statistical sensitivity analysis considering correlated input random variables (Song et al. 2011). However, these transformation methods would inevitably induce errors, which may be large especially when the correlated input random variables follow complex non-Gaussian probabilistic distributions or the response function are highly nonlinear (Lebrun and Dutfoy 2009). Moreover, when the correlated input random variable does not have complete distribution function due to cost limit and exists as data, these transformation approaches that are based on variable probabilistic distribution are clearly inapplicable. In this case, the existing DD-PC methods cannot work, not to mention the PC methods that require the complete probabilistic distribution (such as gPC, GS-PC, and ME-PC). To address this issue, Paulson et al. (Paulson et al. 2017) developed an intrusive arbitrary polynomial chaos method considering correlated input random variables for UP of dynamic system, in which the orthogonal polynomial basis was constructed for correlated random variables using the Gram-Schmidt orthogonalization technique based on the statistical moments of correlated variables and Galerkin projection was employed for PC coefficient calculation (Appendix). Later, Wang et al. (Wang et al. 2019) applied this method to probabilistic load flow. However, for black-box computationally expensive problems, it is impossible to apply the intrusive PC method.

Therefore, as an improvement of the existing DD-PC method, a new non-intrusive DD-PC approach that can directly consider the correlation of input random variables during the orthogonal polynomial construction (short for DD-PC-Corr in this paper) is developed for black-box computationally expensive problems in this work. The multivariate orthogonal polynomial basis is constructed by solving the moment-matching equations based on the correlation statistical moments of input random variables conveniently, rather than resorting to the Gram-Schmidt orthogonalization process that is much more complicated. Meanwhile, a comprehensive comparative study on problems with different nonlinearity and random variable information (different distribution types and raw data) for UP and design optimization under uncertainty are conducted to fully explore the effectiveness and advantage of the proposed method. With the proposed method, the scenario of correlation statistical moment that is defined as certain order of statistical moment considering the variable correlation is introduced based on the discrete data of input random variables. Through matching zero to certain order of correlation statistical moments, the multivariate orthogonal polynomial considering the correlation of input random variables is directly constructed based on moment-matching equations. The regression technique is employed to calculate the PC coefficients. When the probabilistic distribution of correlated random input is known, no transformation is required by the proposed method, and thus, the accuracy of UP would be improved compared to all the existing PC approaches (gPC, GS-PC, ME-PC, and DD-PC). When the probabilistic distributions of correlated random input variables are unknown and only some raw data exist, all the existing PC approaches cannot work, while the proposed DD-PC-Corr method is still applicable.

The rest of this article is organized as follows. The proposed DD-PC-Corr method that can directly consider correlated input random variables is presented in Sect. 2. In Sect. 3, the proposed method is applied to several numerical examples for UP to explore its effectiveness and accuracy. In Sect. 4, the proposed method is further employed to robust design optimization of a mathematical example and reliability-based design optimization of a coil spring problem. Conclusions are drawn in Sect. 5.

2 The proposed DD-PC-Corr method

As an improvement of the existing DD-PC approach, a new non-intrusive DD-PC-Corr method is proposed in this paper to make the PC theory applicable to UP problems with correlated input random variables that do not have complete probabilistic distribution functions. For the existing DD-PC approaches, the univariate orthogonal polynomial basis corresponding to each input random variable is constructed without considering variable correlation, while for the proposed one, the multivariate orthogonal polynomial basis for the correlated input random variables is directly constructed, without referring to transformation. A step-by-step description of the proposed DD-PC-Corr method that can directly consider variable correlation is given as follows. For brevity, a function y = g(x) with a d-dimensional input random vector x = [x₁, …, x_d] is used for illustration of UP. It is assumed that x₁, …, x_z(z ≤ d) are correlated to each other, while the others (x_z + 1, …, x_d) are independent.

• Step 1: Represent the stochastic output y as a PC model of order H:

$$ y\approx \sum \limits_{i=0}^Q{b}_i{\left[P\left({x}_1,\dots, {x}_z\right)\otimes \overline{P}\left({x}_{z+1},\dots, {x}_d\right)\right]}^{(i)} $$

(1)

where b_i represents the i^th PC coefficient; ⊗ denotes the tensor product operation; Q + 1 ($Q+1=\frac{\left(d+H\right)!}{d!H!}$) is the number polynomial terms in the PC model.

In (1), P(x₁, …, x_z) and $\overline{P}\left({x}_{z+1},\dots, {x}_d\right)$ are the orthogonal polynomials corresponding to correlated input random variables x₁, …, x_z and the independent random variables x_z + 1, …, x_d, respectively. The numbers of terms for P(x₁, …, x_z) and $\overline{P}\left({x}_{z+1},\dots, {x}_d\right)$ are $\frac{\left(z+H\right)!}{z!H!}$ and $\frac{\left(d-z+H\right)!}{\left(d-z\right)!H!}$, respectively. $\overline{P}\left({x}_{z+1},\dots, {x}_d\right)$ is constructed by conducting direct tensor product on the univariate orthogonal polynomial basis as follows:

$$ P\left({x}_{z+1},\dots, {x}_d\right)=\prod \limits_{j=z+1}^d{\phi_j}^{\left({\alpha}_j\right)}\left({x}_j\right) $$

(2)

where ${\phi_j}^{\left({\alpha}_j\right)}\left({x}_j\right)$ represents the univariate orthogonal polynomial basis constructed by matching statistical moments from 0 to some order of x_j.

The construction of ${\phi_j}^{\left({\alpha}_j\right)}\left({x}_j\right)$ is exactly the same as that of the existing DD-PC methods (Oladyshkin et al. 2011; Oladyshkin and Nowak 2012), and thus is not introduced in detail here. The main contribution of this work lies in the construction of P(x₁, …, x_z) that can directly consider variable correlation. The kth ($k=0,1,\dots, \frac{\left(z+H\right)!}{z!H!}$) multivariate orthogonal polynomial P^(k)(x₁, …, x_z) is defined as

$$ {P}^{(k)}\left({x}_1,\dots, {x}_z\right)=\sum \limits_{s=0}^k{p_s}^{(k)}\underset{j=1}{\overset{z}{\Pi}}{\left({x}_j\right)}^{\left({\alpha}_j^s\right)} $$

(3)

where p_s^(k) is the polynomial coefficient to be solved, ${\alpha}_j^s$ is the power of x_j, and it clearly satisfies $0\le \sum \limits_{j=1}^z{\alpha}_j^s\le H$.

Taking z = 2 and H = 2 for example, there are 6 orthogonal polynomial terms with polynomial order no more than 3 and P^(k)(x₁, …, x_z)(k = 0, 1, …, 5) can be constructed as below:

$$ \left\{\begin{array}{l}{P}^{(0)}\left({x}_1,{x}_2\right)={p_0}^{(0)}\\ {}{P}^{(1)}\left({x}_1,{x}_2\right)={p_0}^{(1)}+{p_1}^{(1)}{x}_1\\ {}\begin{array}{l}{P}^{(2)}\left({x}_1,{x}_2\right)={p_0}^{(2)}+{p_1}^{(2)}{x}_1+{p_2}^{(2)}{x}_2\\ {}{P}^{(3)}\left({x}_1,{x}_2\right)={p_0}^{(3)}+{p_1}^{(3)}{x}_1+{p_2}^{(3)}{x}_2+{p_3}^{(3)}{x_1}^2\\ {}\begin{array}{l}{P}^{(4)}\left({x}_1,{x}_2\right)={p_0}^{(4)}+{p_1}^{(4)}{x}_1+{p_2}^{(4)}{x}_2+{p_3}^{(4)}{x_1}^2+{p_4}^{(4)}{x}_1{x}_2\\ {}{P}^{(5)}\left({x}_1,{x}_2\right)={p_0}^{(5)}+{p_1}^{(5)}{x}_1+{p_2}^{(5)}{x}_2+{p_3}^{(5)}{x_1}^2+{p_4}^{(5)}{x}_1{x}_2+{p_5}^{(5)}{x_2}^2\end{array}\end{array}\end{array}\right. $$

(4)

• Step 2: Solve polynomial coefficient p_s^(k) in (3) to generate the multivariate orthogonal polynomial basis considering variable correlation.

According to the orthogonality property of the polynomial, it can be obtained

$$ {\int}_{x_1,\dots, {x}_z\in {\Omega}_c}{P}^{(k)}\left({x}_1,\dots, {x}_z\right){P}^{(l)}\left({x}_1,\dots, {x}_z\right)\mathrm{d}\Gamma \left({x}_1,\dots, {x}_z\right)={\delta}_{kl},\forall k,l=0,1,\dots, \frac{\left(z+H\right)!}{z!H!} $$

(5)

where δ_kl is the Kronecker delta, Ω_c stands for the original correlated random variable space, and Γ(x) represents the joint cumulative distribution function (CDF) of the correlated random variables.

Given the assumption that all the polynomial coefficients in (3) ${p}_s^{(k)}\left(k\le \frac{\left(z+H\right)!}{z!H!}\right)$ are not equal to 0 and ${p}_k^{(k)}=1$ for simplicity, a new set of equations can be obtained as below by conducting inner product between P^(r)(r = 0, 1, …, k − 1) and P^(k) based on (5):

$$ \left\{\begin{array}{l}{\int}_{x_1,\dots, {x}_z\in {\varOmega}_c}{p}_0^{(0)}\left[\sum \limits_{s=0}^k{p_s}^{(k)}\overset{z}{\underset{j=1}{\varPi }}{\left({x}_j\right)}^{\left({\alpha}_j^s\right)}\right]\mathrm{d}\varGamma \left({x}_1,\dots, {x}_z\right)=0\\ {}{\int}_{x_1,\dots, {x}_z\in {\varOmega}_c}\left[\sum \limits_{s=0}^1{p_s}^{(1)}\overset{z}{\underset{j=1}{\varPi }}{\left({x}_j\right)}^{\left({\alpha}_j^s\right)}\right]\left[\sum \limits_{s=0}^k{p_s}^{(k)}\overset{z}{\underset{j=1}{\varPi }}{\left({x}_j\right)}^{\left({\alpha}_j^s\right)}\right]\mathrm{d}\varGamma \left({x}_1,\dots, {x}_z\right)=0\\ {}\begin{array}{l}{\int}_{x_1,\dots, {x}_z\in {\varOmega}_c}\left[\sum \limits_{s=0}^2{p_s}^{(1)}\overset{z}{\underset{j=1}{\varPi }}{\left({x}_j\right)}^{\left({\alpha}_j^s\right)}\right]\left[\sum \limits_{s=0}^k{p_s}^{(k)}\overset{z}{\underset{j=1}{\varPi }}{\left({x}_j\right)}^{\left({\alpha}_j^s\right)}\right]\mathrm{d}\varGamma \left({x}_1,\dots, {x}_z\right)=0\\ {}\begin{array}{l}\vdots \\ {}{\int}_{x_1,\dots, {x}_z\in {\varOmega}_c}\left[\sum \limits_{s=0}^{k-1}{p_s}^{\left(k-1\right)}\overset{z}{\underset{j=1}{\varPi }}{\left({x}_j\right)}^{\left({\alpha}_j^s\right)}\right]\left[\sum \limits_{s=0}^k{p_s}^{(k)}\overset{z}{\underset{j=1}{\varPi }}{\left({x}_j\right)}^{\left({\alpha}_j^s\right)}\right]\mathrm{d}\varGamma \left({x}_1,\dots, {x}_z\right)=0\end{array}\end{array}\end{array}\right. $$

(6)

By expanding the second equation in (6), one can obtain

$$ {\int}_{x_1,\dots, {x}_z\in {\Omega}_c}\left[{p_0}^{(1)}\underset{j=1}{\overset{z}{\Pi}}{\left({x}_j\right)}^{\left({\alpha}_j^0\right)}+{p_1}^{(1)}\underset{j=1}{\overset{z}{\Pi}}{\left({x}_j\right)}^{\left({\alpha}_j^1\right)}\right]\left[\sum \limits_{s=0}^k{p_s}^{(k)}\underset{j=1}{\overset{z}{\Pi}}{\left({x}_j\right)}^{\left({\alpha}_j^s\right)}\right]\boldsymbol{d}\Gamma \left({x}_1,\dots, {x}_z\right)=0 $$

(7)

From the first equation in (6), it can be derived that

$$ {\int}_{x_1,\dots, {x}_z\in {\Omega}_c}{p_0}^{(1)}\underset{j=1}{\overset{z}{\Pi}}{\left({x}_j\right)}^{\left({\alpha}_j^0\right)}\left[\sum \limits_{s=0}^k{p_s}^{(k)}\underset{j=1}{\overset{z}{\Pi}}{\left({x}_j\right)}^{\left({\alpha}_j^s\right)}\right]\boldsymbol{d}\Gamma \left({x}_1,\dots, {x}_z\right)=0 $$

(8)

Substituting (8) into (7), (7) can be simplified as

$$ {\int}_{x_1,\dots, {x}_z\in {\Omega}_c}{p_1}^{(1)}\underset{j=1}{\overset{z}{\Pi}}{\left({x}_j\right)}^{\left({\alpha}_j^1\right)}\left[\sum \limits_{s=0}^k{p_s}^{(k)}\underset{j=1}{\overset{z}{\Pi}}{\left({x}_j\right)}^{\left({\alpha}_j^s\right)}\right]\boldsymbol{d}\Gamma \left({x}_1,\dots, {x}_z\right)=0 $$

(9)

As has been assumed that ${p}_k^{(k)}=1$, (9) can be further transformed as

$$ {\int}_{x_1,\dots, {x}_z\in {\Omega}_c}\sum \limits_{s=0}^k{p_s}^{(k)}\underset{j=1}{\overset{z}{\Pi}}{\left({x}_j\right)}^{\left({\alpha}_j^s+{\alpha}_j^1\right)}\boldsymbol{d}\Gamma \left({x}_1,\dots, {x}_z\right)=0 $$

(10)

Similarly, by expanding the third equation in (6), one can obtain

$$ {\int}_{x_1,\dots, {x}_z\in {\Omega}_c}\left[{p_0}^{(2)}\underset{j=1}{\overset{z}{\Pi}}{\left({x}_j\right)}^{\left({\alpha}_j^0\right)}+\dots +{p_2}^{(2)}\underset{j=1}{\overset{z}{\Pi}}{\left({x}_j\right)}^{\left({\alpha}_j^1\right)}\right]\left[\sum \limits_{s=0}^k{p_s}^{(k)}\underset{j=1}{\overset{z}{\Pi}}{\left({x}_j\right)}^{\left({\alpha}_j^s\right)}\right]\boldsymbol{d}\Gamma \left({x}_1,\dots, {x}_z\right)=0 $$

(11)

Based on the first equation in (6) and (9), it can be derived that

$$ {\int}_{x_1,\dots, {x}_z\in {\Omega}_c}{p_0}^{(2)}\underset{j=1}{\overset{z}{\Pi}}{\left({x}_j\right)}^{\left({\alpha}_j^0\right)}\left[\sum \limits_{s=0}^k{p_s}^{(k)}\underset{j=1}{\overset{z}{\Pi}}{\left({x}_j\right)}^{\left({\alpha}_j^s\right)}\right]\boldsymbol{d}\Gamma \left({x}_1,\dots, {x}_z\right)=0 $$

(12)

$$ {\int}_{x_1,\dots, {x}_z\in {\Omega}_c}{p_1}^{(2)}\underset{j=1}{\overset{z}{\Pi}}{\left({x}_j\right)}^{\left({\alpha}_j^1\right)}\left[\sum \limits_{s=0}^k{p_s}^{(k)}\underset{j=1}{\overset{z}{\Pi}}{\left({x}_j\right)}^{\left({\alpha}_j^s\right)}\right]\boldsymbol{d}\Gamma \left({x}_1,\dots, {x}_z\right)=0 $$

(13)

Substituting (12) and (13) into (11), (11) can be simplified as

$$ {\int}_{x_1,\dots, {x}_z\in {\Omega}_c}{p_2}^{(2)}\underset{j=1}{\overset{z}{\Pi}}{\left({x}_j\right)}^{\left({\alpha}_j^1\right)}\left[\sum \limits_{s=0}^k{p_s}^{(k)}\underset{j=1}{\overset{z}{\Pi}}{\left({x}_j\right)}^{\left({\alpha}_j^s\right)}\right]\boldsymbol{d}\Gamma \left({x}_1,\dots, {x}_z\right)=0 $$

(14)

As ${p}_2^{(2)}=1$, (14) can be further transformed as

$$ {\int}_{x_1,\dots, {x}_z\in {\Omega}_c}\sum \limits_{s=0}^k{p_s}^{(k)}\underset{j=1}{\overset{z}{\Pi}}{\left({x}_j\right)}^{\left({\alpha}_j^s+{\alpha}_j^2\right)}\boldsymbol{d}\Gamma \left({x}_1,\dots, {x}_z\right)=0 $$

(15)

By respectively employing the same way above on the 4th, 5th,…, kth equation in .(6), one can further obtain k-3 simplified equations. Combing these simplified equations ((10), (15),…), a new set of equations can be obtained as below:

$$ \left\{\begin{array}{c}{\int}_{x_1,\dots, {x}_z\in {\varOmega}_c}\sum \limits_{s=0}^k{p_s}^{(k)}\overset{z}{\underset{j=1}{\varPi }}{\left({x}_j\right)}^{\left({\alpha}_j^s+{\alpha}_j^0\right)}\mathrm{d}\varGamma \left({x}_1,\dots, {x}_z\right)=0\\ {}{\int}_{x_1,\dots, {x}_z\in {\varOmega}_c}\sum \limits_{s=0}^k{p_s}^{(k)}\overset{z}{\underset{j=1}{\varPi }}{\left({x}_j\right)}^{\left({\alpha}_j^s+{\alpha}_j^1\right)}\mathrm{d}\varGamma \left({x}_1,\dots, {x}_z\right)=0\\ {}{\int}_{x_1,\dots, {x}_z\in {\varOmega}_c}\sum \limits_{s=0}^k{p_s}^{(k)}\overset{z}{\underset{j=1}{\varPi }}{\left({x}_j\right)}^{\left({\alpha}_j^s+{\alpha}_j^2\right)}\mathrm{d}\varGamma \left({x}_1,\dots, {x}_z\right)=0\\ {}\vdots \\ {}{\int}_{x_1,\dots, {x}_z\in {\varOmega}_c}\sum \limits_{s=0}^k{p_s}^{(k)}\overset{z}{\underset{j=1}{\varPi }}{\left({x}_j\right)}^{\left({\alpha}_j^s+{\alpha}_j^{k-1}\right)}\mathrm{d}\varGamma \left({x}_1,\dots, {x}_z\right)=0\end{array}\right. $$

(16)

Define ${\int}_{x_1,\dots, {x}_z\in {\Omega}_c}\underset{j=1}{\overset{z}{\Pi}}{\left({x}_j\right)}^{\left({\alpha}_j^a+{\alpha}_j^b\right)}\boldsymbol{d}\Gamma \left({x}_1,\dots, {x}_z\right)={\mu}_{a,b}$, and rewrite (16) in the matrix form as

$$ \left[\begin{array}{cccc}{\mu}_{0,0}& {\mu}_{1,0}& \cdots & {\mu}_{k,0}\\ {}{\mu}_{0,1}& {\mu}_{1,1}& \cdots & {\mu}_{k,1}\\ {}\vdots & \vdots & \vdots & \vdots \\ {}{\mu}_{0,k-1}& {\mu}_{1,k-1}& \cdots & {\mu}_{k,k-1}\\ {}0& 0& \cdots & 1\end{array}\right]\left[\begin{array}{c}{p}_0^{(k)}\\ {}{p}_1^{(k)}\\ {}\vdots \\ {}{p}_{k-1}^{(k)}\\ {}{p}_k^{(k)}\end{array}\right]=\left[\begin{array}{c}0\\ {}0\\ {}0\\ {}\vdots \\ {}1\end{array}\right] $$

(17)

In (17), μ_{a, b}(a = 0, 1, …, k; b = 0, 1, …, k − 1) is named as the correlation statistical moment of x₁, …, x_z in this paper, which can directly take the correlation among x₁, …, x_z into account. If the distribution information is unknown, μ_{a, b} can be easily calculated from the given discrete data of x₁, …, x_z. Equation (17) can be easily solved to obtain the unknown polynomial coefficients in (3), and the construction of multivariate orthogonal polynomial P^(k) for correlated input random variables is completed.

• Step 3: Once P(x₁, …, x_z) and $\overline{P}\left({x}_{z+1},\dots, {x}_d\right)$ are constructed, the regression (Isukapalli et al. 2000) or Galerkin projection (Xiu and Karniadakis 2002) techniques can be employed to calculate the PC coefficients b_i in the PC model.

It should be pointed out that for the Galerkin projection method, the efficient Gaussian quadrature numerical integration technique is generally employed, where the Gaussian quadrature nodes are required to be calculated. However, for the proposed method, the multivariate orthogonal polynomial for the correlated input random variables is constructed, of which the zeros (Gaussian quadrature nodes) are difficult to obtain. Therefore, other numerical integration methods are employed in replacement of Gaussian quadrature, which however requires a great amount of nodes to calculate the PC coefficients. This is generally unaffordable to problems with computational expensive simulation models. Therefore, the regression method is recommended to calculate the PC coefficients for the proposed method. During sampling, if the distributions of input random variables are known, the Latin Hypercube sampling method considering sample weights (Xiong et al. 2011b) is employed to generate sample points for regression. If only data exist for the input random variables, the input sample points are selected from the given raw data to ensure the space-filling property and spatial uniformity as far as possible.

• Step 4: Once the PC coefficients are calculated, a cheap stochastic meta-model is constructed, based on which Monte Carlo simulation (MCS) is conducted to obtain the probabilistic characteristics of output y.

For the proposed method, the main computational cost comes from Step 3, as many function calls are required. And the computational cost of Step 1 and Step 2 from the orthogonal polynomial construction is almost negligible compared to Step 3. As an improvement of the existing DD-PC method, the proposed method will be degraded into the existing DD-PC when the variable correlation is zero.

3 Numerical test for uncertainty propagation

In this section, the proposed DD-PC-Corr method is applied to four mathematical examples and one coil spring problem to verify its effectiveness and accuracy for UP with correlated input random variables. The results of MCS obtained by repeatedly calling the original response function with 10⁶ runs are employed as the benchmark to validate the effectiveness of the proposed method. The first four statistical moments (mean, standard deviation, skewness, kurtosis) and the probability of failure of the output response are calculated (short for M, Std, Ske, Kur, and Pf in this paper), of which the errors relative to MCS are calculated.

Clearly, the order of polynomial is an important tuning parameter for PC. In literature, many works have been done to determine it in a scientific way (Hampton and Doostan 2018; Sinou and Jacquelin 2015). However, the main contribution of the proposed method lies in the construction of the orthogonal polynomial basis considering variable correlation using moment-matching, and the rest of procedures such as the calculation of PC coefficients are basically the same as the existing PC approaches. Therefore, the order of the PC model is simply set as H = 3 considering the compromise between accuracy and computational cost. Meanwhile, the regression method (Isukapalli 1999) is employed to calculate the PC coefficients, in which the sample size is set as twice the number of unknown PC coefficients as is commonly done in literature (Hosder et al. 2007).

3.1 Mathematical examples

To fully investigate the effectiveness of the proposed method, four mathematical examples with varying nonlinearity and different distribution forms of input random variables are considered, which are shown in Table 1. In Table 1, N, U, and Rayl represent normal, uniform, and Rayleigh distribution, respectively. For Case 1 to Case 3, the probabilistic distribution of the input random variable is known and correlated, and the existing DD-PC method with Nataf transformation is also tested for comparison. For Case 4, the input random variables do not have probabilistic distributions and are correlated, which exist as some raw discrete data. In this case, the existing DD-PC method with Nataf transformation cannot work, and only the proposed method is tested. For all the cases, different strengths of correlation (ρ = 0, 0.5, 0.8; weak, medium and strong) are considered.

Table 1 Test functions and the distribution information

The first four statistical moments (M, Std, Ske, and Kur) and the probability of failure (Pf) of the output response for the proposed DD-PC-Corr method (denoted by Proposed), the existing DD-PC method with Nataf transformation (denoted by Existing) and MCS are shown in Figs. 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, and 16, in which the errors relative to MCS for the proposed and existing methods are shown numerically above the corresponding bars (all the errors are in the form of percentage error). From these figures, some noteworthy observations can be made.

Fig. 1

Results of Function 1 (Case 1, y > 3.5)

Fig. 2

Results of Function 1 (Case 2, y > 1.6)

Fig. 3

Results of Function 1 (Case 3, y > 2.5)

Fig. 4

Results of Function 1 (Case 4, y >4.5)

Fig. 5

Results of Function 2 (Case 1, y > 15)

Fig. 6

Results of Function 2 (Case 2, y > 40)

Fig. 7

Results of Function 2 (Case 3, y > −0.9)

Fig. 8

Results of Function 2 (Case 4, y > 40)

Fig. 9

Results of Function 3 (Case 1, y > 2)

Fig. 10

Results of Function 3 (Case 2, y > 4)

Fig. 11

Results of Function 3 (Case 3, y > 0.3)

Fig. 12

Results of Function 3 (Case 4, y > 5)

Fig. 13

Results of Function 4 (Case 1, y > 0.5)

Fig. 14

Results of Function 4 (Case 2, y > 8)

Fig. 15

Results of Function 4 (Case 3, y > 20)

Fig. 16

Results of Function 4 (Case 4, y > 10)

Firstly, for Case 1 to Case 3 with known distributions of the input random variables, the results of the proposed DD-PC-Corr method are clearly much more accurate than the existing DD-PC method with Nataf transformation for various correlation coefficients, which is owed to the direct consideration of the variable correlation in the construction of orthogonal polynomial basis. For the existing method, the Nataf transformation is employed to transform the correlated input random variables into independent standard normal ones when the input random variables are correlated, in which more or less errors would be induced to UP. Specially, for Examples 2–4, as the functions are more nonlinear than Example 1 (linear), the errors of the existing method are clearly larger. For Example 1, as the function is linear, the results produced by the proposed method are almost the same to those of MCS.

Secondly, compared to Case 1, the errors of Case 2 and Case 3 for the existing method are generally larger and the advantage of proposed method in accuracy is more obvious, especially for the errors of skewness and kurtosis. The interpretation is that uniform and Rayleigh distributions are considered in Case 2 and Case 3, and larger errors are induced by the Nataf transformation compared to those of Case 1 with normal distribution. Moreover, for the estimation of skewness and kurtosis with PC theory, as the nonlinearity is higher, larger errors are induced by the Nataf transformation compared to those of mean and variance.

Thirdly, in Case 4 that the distribution of input random variable is unknown, the proposed method considering variable correlation can produce accurate results that are very close to MCS, while the existing DD-PC method with Nataf transformation even cannot work.

All these results demonstrate the effectiveness and advantage of the proposed method.

To study the convergence property, the relative errors (%) of the statistical moments and P_f with different PC orders obtained by the proposed DD-PC-Corr method and the existing DD-PC method with Nataf transformation are illustrated in Figs. 17, 18, 19, and 20. For space limit, only the results of Function 2 with large correlation (ρ = 0.8 or strong) are shown here. It is noticed that for both methods, with the increase of the PC order, the relative errors significantly decrease, and the decline of the proposed method is clearly more rapid, exhibiting better convergence property. Meanwhile, the errors of the proposed method are evidently smaller compared to the existing approach with the same order of PC model (i.e., the same number of function calls). These results demonstrate the effectiveness and good convergence property of the proposed method.

Fig. 17

Relative errors with different PC orders (Function 2, Case 1, ρ = 0.8)

Fig. 18

Relative errors with different PC orders (Function 2, Case 2, ρ = 0.8)

Fig. 19

Relative error with different PC orders (Function 2, Case 3, ρ = 0.8)

Fig. 20

Relative error with different PC orders (Function 2, Case 4, strong)

3.2 Coil spring

A coil spring problem adopted from (Arora 2004) (see Fig. 21) is further employed to investigate the effectiveness of the proposed method. It is originally a design problem and is modified to an example for UP in this subsection. In the manufacturing process, the inner diameter D (unit, in) and the wire diameter d (unit, in) may be physically correlated, which are assumed to follow normal distributions (D ∼ N(1, 0.1²), d ∼ N(0.15, 0.01²)). The rest of the parameters are considered to be deterministic, which are explained in Table 2.

Fig. 21

Coil spring

Table 2 Physical meaning of coil spring parameters

We are concerned about the mass of the coil spring, which is calculated as follows:

$$ \mathrm{Mass}=25000\times \left(N+Q\right){\pi}^2\left(D+d\right){d}^2\rho $$

(18)

Similarly, different strength of the correlation coefficient between D and d (ρ_Dd) is considered. The first four statistical moments (M, Std, Ske, and Kur) of the output response for the proposed DD-PC-Corr method (denoted by Proposed) and MCS are shown in Fig. 22, in which the errors relative to MCS for the proposed are shown numerically above the corresponding bars (all the errors are in the form of percentage error). From Fig. 22, it is noticed that all the results calculated by the proposed DD-PC-Corr method are very close to those produced by MCS with different correlation strength, and the relative errors are basically within 1%. Generally, the relative errors of skewness and kurtosis are larger than those of mean and standard deviation, as the nonlinearity of skewness and kurtosis estimation within PC theory is higher. These results show great agreements to those obtained in Sect. 3.1, and further demonstrate the effectiveness of the proposed method.

Fig. 22

Results of UP for coil spring

4 Numerical test for design optimization under uncertainty

In this section, the proposed DD-PC-Corr method is employed to robust optimization of one mathematical example and reliability-based design optimization of a coil spring problem to further verify its effectiveness and accuracy in addressing UP with correlated input random variables. Meanwhile, optimization is also conducted with MCS for UP, of which the results are used as benchmark to validate the effectiveness of the proposed method. Once the optimization is done, MCS is conducted to obtain the confirmed results, through substituting the obtained optimal design variables by DD-PC-Corr and MCS-based design optimization into the objective and constraint functions considering uncertainties of input random variables. The order of PC model is set as H = 3.

4.1 Mathematical example

Robust design optimization on a mathematical example shown in (19) is firstly employed with the proposed DD-PC-Corr method for statistical moment calculation. Different correlation coefficients (ρ = 0, 0.5, 0.8) are considered. Table 3 shows the results of robust optimization using the proposed method and MCS, in which F^C and G^C denote the confirmed objective and constraint function values. From this table, it is found that the results of the proposed DD-PC-Corr method are very close to those produced by the MCS method for different correlation coefficients, and the relative errors are all within 5%, indicating the effectiveness and accuracy of the proposed DD-PC-Corr method. Moreover, for different correlation coefficients, the obtained optimal solutions always satisfy the constraints:

$$ {\displaystyle \begin{array}{l}\underset{\mu_{x_1},{\mu}_{x_2}}{\mathit{\operatorname{Min}}}\kern0.5em F=\frac{\sigma_f}{15}\\ {}\mathrm{s}.\mathrm{t}.\kern2em G={\mu}_g-3{\sigma}_g\ge 0\\ {}\begin{array}{l}\kern3.5em f\left({x}_1,\kern0.33em {x}_2\right)={\left({x}_1-4\right)}^3+{\left({x}_1-3\right)}^4+{\left({x}_2-5\right)}^2+10\\ {}\kern3.25em g={x}_1+{x}_2-6.45\\ {}\begin{array}{l}\kern3.25em {x}_i:N\left({\mu}_{x_i},\kern0.33em {0.4}^2\right),\kern0.33em i=1,\kern0.33em 2\\ {}\kern3.25em 1\le {\mu}_{x_1}\le 10,\kern0.33em 1\le {\mu}_{x_2}\le 10\end{array}\end{array}\end{array}} $$

(19)

Table 3 Optimal results of robust optimization

4.2 Coil spring

The reliability-based design optimization (RBDO) formulation of the coil spring problem mentioned in Sect. 3.2 is shown in (20):

$$ {\displaystyle \begin{array}{l}\underset{\mu_D,{\mu}_d}{\mathit{\operatorname{Min}}}\kern0.5em F={\mu}_{\mathrm{mass}}\\ {}s.t.\kern1.25em {P}_i\left({G}_i>0\right)\le 0.02275,i=1,2,3\\ {}\begin{array}{l} mass=25000\times \left(N+Q\right){\pi}^2\left(D+d\right){d}^2\rho \\ {}{G}_1=1-\frac{\delta }{\varDelta}\\ {}\begin{array}{l}{G}_2=\frac{\tau }{\tau_a}-1\\ {}{G}_3=1-\frac{\omega }{\omega_0}\\ {}\begin{array}{l}\delta =\frac{P}{K}=\frac{8P{\left(D+d\right)}^3N}{d^4G}\\ {}\tau =\frac{8 kP\left(D+d\right)}{\pi {d}^3}=\frac{8P\left(D+d\right)}{\pi {d}^3}\left[\frac{4\left(D+d\right)-d}{4D}+\frac{0.615d}{D+d}\right]\\ {}\begin{array}{l}\omega =\frac{d}{2\pi N{\left(D+d\right)}^2}\sqrt{\frac{G}{2\rho }}\\ {}0.8\le {\mu}_D\le \mathrm{1.5,0.1}\le {\mu}_d\le 0.3\end{array}\end{array}\end{array}\end{array}\end{array}} $$

(20)

where μ represents the mean of the variable, and P_i denotes the probability of failure of the i^th limit state function.

The optimization of the coil spring problem aims to find the design variables (the means of the inner diameter D and wire diameter d) of the coil spring, to minimize the mass subject to constraints on the deflection δ, the shear stress τ, and the vibration frequency ω of the spring. The inner diameter D and the wire diameter d are considered to be uncertain and follow normal distribution, with standard deviation as 0.1 and 0.01, respectively. In the manufacturing process, D and d may be physically correlated to each other. Ignoring such correlation may significantly impact the performance of the spring, and thus, it is necessary to consider it. In this work, different correlation coefficients ρ_Dd = [0,  0.5,  0.8] are tested. The proposed DD-PC-Corr method and MCS is employed to estimate the probability of failure and mean of mass in RBDO.

The optimal results of RBDO using the proposed DD-PC-Corr method and MCS and the confirmed results of the original design (μ_D = 1, μ_d = 0.15) are displayed in Table 4. As shown in the table, for different correlation coefficients, the proposed method can produce feasible optimal results that are very close to those of MCS, demonstrating the effectiveness of the proposed method. Meanwhile, for different correlation coefficients, with the employment of RBDO, the mean value of spring mass and the failure probability of the deflection are significantly reduced by RBDO compared with the original design, which clearly improve the reliability. For the original design, the failure probability of the deflection (P₁) is almost 100%, yielding terrible reliability.

Table 4 Optimal results for coil spring problem

5 Conclusions

In this paper, a new non-intrusive data-driven polynomial chaos (DD-PC) method that can directly consider variable correlation into the construction of PC model is developed for uncertainty propagation with black-box computational expensive function. With the proposed method, the multivariate orthogonal polynomial basis that can consider variable correlation is constructed by solving the moment-matching equations based on the correlation statistical moments of correlated input random variables. And the regression technique is employed for PC coefficient calculation considering computational efficiency. The proposed method is applied to several numerical examples with correlated input random variables for UP and design optimization under uncertainty. It is found that when the variable distribution is known, as no transformation is required for the proposed method, it is more accurate than the existing DD-PC method with Nataf transformation; when it is unknown, almost all the existing PC methods cannot work, the proposed method can still obtain accurate results that are very close to those produced by Monte Carlo simulation. The effectiveness and advantage of the proposed method are well demonstrated.

Appendix

The correlated statistical moments and the polynomial coefficients for Function 1 (Case 1, ρ = 0.8) in Sect. 3.1 are provided as below.

For this example, the dimension of correlated random variable is 2 and the order of model is set as H = 3. Therefore, there are ten two-dimensional orthogonal polynomial bases with polynomial order on more than 3. The correlated statistical moments that are in the form of matrix in (17) are shown as below:

$$ {P}^{(0)}\left({x}_1,{x}_2\right)=\left[1\right]\left(k=0\right) $$

(21)

$$ {P}^{(1)}\left({x}_1,{x}_2\right)=\left[\begin{array}{cc}{\int}_{x_1,{x}_2\in {\Omega}_c}1\boldsymbol{d}\Gamma \left({x}_1,{x}_2\right)& {\int}_{x_1,{x}_2\in {\Omega}_c}{x}_1\boldsymbol{d}\Gamma \left({x}_1,{x}_2\right)\\ {}0& 1\end{array}\right]\left(k=1\right) $$

(22)

$$ {P}^{(2)}\left({x}_1,{x}_2\right)=\left[\begin{array}{ccc}{\int}_{x_1,{x}_2\in {\varOmega}_c}1\mathrm{d}\varGamma \left({x}_1,{x}_2\right)& {\int}_{x_1,{x}_2\in {\varOmega}_c}{x}_1\mathrm{d}\varGamma \left({x}_1,{x}_2\right)& {\int}_{x_1,{x}_2\in {\varOmega}_c}{x}_2\mathrm{d}\varGamma \left({x}_1,{x}_2\right)\\ {}{\int}_{x_1,{x}_2\in {\varOmega}_c}{x}_1\mathrm{d}\varGamma \left({x}_1,{x}_2\right)& {\int}_{x_1,{x}_2\in {\varOmega}_c}{x_1}^2\mathrm{d}\varGamma \left({x}_1,{x}_2\right)& {\int}_{x_1,{x}_2\in {\varOmega}_c}{x}_1{x}_2\mathrm{d}\varGamma \left({x}_1,{x}_2\right)\\ {}0& 0& 1\end{array}\right]\left(k=2\right) $$

(23)

$$ {P}^{(9)}\left({x}_1,{x}_2\right)=\left[\begin{array}{cccc}{\int}_{x_1,{x}_2\in {\varOmega}_c}1\mathrm{d}\varGamma \left({x}_1,{x}_2\right)& {\int}_{x_1,{x}_2\in {\varOmega}_c}{x}_1\mathrm{d}\varGamma \left({x}_1,{x}_2\right)& \cdots & {\int}_{x_1,{x}_2\in {\varOmega}_c}{x_2}^3\mathrm{d}\varGamma \left({x}_1,{x}_2\right)\\ {}{\int}_{x_1,{x}_2\in {\varOmega}_c}{x}_1\mathrm{d}\varGamma \left({x}_1,{x}_2\right)& {\int}_{x_1,{x}_2\in {\varOmega}_c}{x_1}^2\mathrm{d}\varGamma \left({x}_1,{x}_2\right)& \cdots & {\int}_{x_1,{x}_2\in {\varOmega}_c}{x}_1{x_2}^3\mathrm{d}\varGamma \left({x}_1,{x}_2\right)\\ {}\vdots & \vdots & \vdots & \vdots \\ {}{\int}_{x_1,{x}_2\in {\varOmega}_c}{x}_1{x_2}^2\mathrm{d}\varGamma \left({x}_1,{x}_2\right)& {\int}_{x_1,{x}_2\in {\varOmega}_c}{x_1}^2{x_2}^2\mathrm{d}\varGamma \left({x}_1,{x}_2\right)& \cdots & {\int}_{x_1,{x}_2\in {\varOmega}_c}{x}_1{x_2}^5\mathrm{d}\varGamma \left({x}_1,{x}_2\right)\\ {}0& 0& \cdots & 1\end{array}\right]\left(k=9\right) $$

(24)

Based on the probabilistic distribution information of the correlated input random variables, one can obtain these correlated statistical moments that will be employed in the orthogonal polynomial basis construction. As the correlated statistical moments for k = 0, 1,..., 8 are all part of those for k = 9, only those for k = 9 is given as below:

$$ \left[\begin{array}{cccccccccc}1& 1.0037& 2.0047& 1.0470& 2.0432& 4.0581& 1.1307& 2.1618& 4.1988& 8.2947\\ {}1.0037& 1.0470& 2.0432& 1.1307& 2.1618& 4.1988& 1.2600& 2.3652& 4.5040& 8.7086\\ {}2.0047& 2.0432& 4.0581& 2.1618& 4.1988& 8.2947& 2.3652& 4.5040& 8.7086& 17.1153\\ {}1.0470& 1.1307& 2.1618& 1.2600& 2.3652& 4.5040& 1.4445& 2.6671& 4.9901& 9.4668\\ {}2.0432& 2.1618& 4.1988& 2.3652& 4.5040& 8.7086& 2.6671& 4.9901& 9.4668& 18.2262\\ {}4.0581& 4.1988& 8.2947& 4.5040& 8.7086& 17.1153& 4.9901& 9.4668& 18.2262& 35.6458\\ {}1.1307& 1.2600& 2.3652& 1.4445& 2.6671& 4.9901& 1.6991& 3.0911& 5.6923& 10.6171\\ {}2.1618& 2.3652& 4.5040& 2.6671& 4.9901& 9.4668& 3.0911& 5.6923& 10.6171& 20.0705\\ {}4.1988& 4.5040& 8.7086& 4.9901& 9.4668& 18.2262& 5.6923& 10.6171& 20.0705& 38.4842\end{array}\right]\left(k=9\right) $$

(25)

Correspondingly, the polynomial coefficients of the ten two-dimensional orthogonal polynomial bases are listed as below:

$$ {\displaystyle \begin{array}{l}\left[{p}_0^{(0)}\right]=\left[1\right]\\ {}\left[{p}_0^{(1)},{p}_1^{(1)}\right]=\left[1,-1.0037\right]\\ {}\begin{array}{l}\left[{p}_0^{(2)},{p}_1^{(2)},{p}_2^{(2)}\right]=\left[1,-0.7874,-1.2143\right]\\ {}\left[{p}_0^{(3)},{p}_1^{(3)},\dots {p}_3^{(3)}\right]=\left[\mathrm{1,0.0000},-\mathrm{2.0149,0.9733}\right]\\ {}\begin{array}{l}\left[{p}_0^{(4)},{p}_1^{(4)},\dots {p}_4^{(4)}\right]=\left[1,-0.7601,-1.0339,-\mathrm{0.4550,1.2818}\right]\\ {}\left[{p}_0^{(5)},{p}_1^{(5)},\dots {p}_5^{(5)}\right]=\left[1,-\mathrm{1.5269,0.5869},-\mathrm{2.4795,1.8735,1.5374}\right]\\ {}\begin{array}{l}\left[{p}_0^{(6)},{p}_1^{(6)},\dots {p}_6^{(6)}\right]=\left[\mathrm{1,0.0124},-0.0202,-2.9868,-\mathrm{0.0242,2.8980},-0.8728\right]\\ {}\left[{p}_0^{(7)},{p}_1^{(7)},\dots {p}_7^{(7)}\right]=\left[1,-\mathrm{0.7414,0.0434},-\mathrm{2.1284,0.2956,0.9175,1.9156},-1.2221\right]\\ {}\begin{array}{l}\left[{p}_0^{(8)},{p}_1^{(8)},\dots {p}_8^{(8)}\right]=\left[1,-\mathrm{1.5061,0.5835},-1.0560,-\mathrm{0.8766,1.1998,2.6053},-0.3874,-1.6164\right]\\ {}\left[{p}_0^{(9)},{p}_1^{(9)},\dots {p}_9^{(9)}\right]=\left[1,-\mathrm{2.1648,1.5190},-0.3394,-\mathrm{3.8632,5.6408},-\mathrm{2.0132,4.9097},-3.6227,-2.0516\right]\end{array}\end{array}\end{array}\end{array}\end{array}} $$

(26)