Reliability Analysis

Abstract

Assessing the reliability of a complex system with uncertainty propagation consists in estimating its probability of failure. Common sampling strategies for such tasks are notably based on Monte Carlo sampling. This kind of methods is well suited to characterize events of which associated probabilities are not too low with respect to the simulation budget. However, for critical systems such as aerospace vehicles, the reliability specifications often induce very low probability of failures (said below 10⁻⁴). In this case, Monte Carlo based methods are not efficient inducing unaffordable costs with regard to the available simulation budget. In this chapter, we review the main simulation techniques to estimate low failure probabilities with accuracy.

Assessing the reliability of a complex system with uncertainty propagation consists in estimating its probability of failure. Common sampling strategies for such tasks are notably based on Monte Carlo sampling. This kind of methods is well suited to characterize events of which associated probabilities are not too low with respect to the simulation budget. However, for critical systems such as aerospace vehicles, the reliability specifications often induce very low probability of failures (said below 10⁻⁴). In this case, Monte Carlo based methods are not efficient inducing unaffordable costs with regard to the available simulation budget. In this chapter, we review the main simulation techniques to estimate low failure probabilities with accuracy.

1 Introduction

Analyzing the reliability of a complex system often corresponds to the estimation of its failure probability. A significant number of specific methods has been proposed to estimate this probability when the system uncertainties are not considered such as fault trees or formal methods. In this case, the global system failure probability may be estimated with some cascade of conditional probabilities. If the uncertainties of the system inputs may be modeled as random variables, then the system failure probability is expressed with an integral. Within the context of complex system design and optimization under uncertainty, the constraints of the optimization problem are often formulated through a probability that the constraint exceeds a given threshold, resulting in the need to estimate a probability of failure (see Chapter 5 for more details on optimization under uncertainty). For the design of aerospace vehicles, these failure probabilities might be low with respect to the affordable simulation budget and therefore efficient reliability analysis techniques are required.

Let us assume that the considered complex system is a deterministic black-box function g(⋅) with a d-dimensional random input U with known joint PDF ϕ(⋅) of support $\mathbb {R}^d$. For the sake of clarity, we assume the system failure is only an output threshold exceedance, that is g(u) > T with T a scalar. The methods described in the following chapter are still valid even for more intricate failure model. Reliability engineering techniques aim thus at estimating the failure probability $\mathbb {P}\left (g(\mathbf {U})>T\right )$, that is (Figure 4.1):

(4.1)

Fig. 4.1

Reliability analysis for single discipline problem (QoI: quantity of interest)

with the indicator function such that if g(u) > T, 0 otherwise. Due to the critical nature of a system failure, P _f takes in general low values (as an indication, one may consider P _f < 10⁻⁴). Nevertheless, the accuracy required for the estimation of P _f is very high because the consequences of a misestimation, and particularly of an underestimation of P _f may be catastrophic.

The estimation of P _f corresponds to an integral computation. However, numerical deterministic integration methods such as Gaussian quadrature (Novak 1988) or sparse grids (Smolyak 1963; Gerstner and Griebel 2003) are not adapted in this context. They require some smoothness of the function to be integrated: it is not the case here since we integrate an indicator function . Moreover, these methods are efficient when the dimension d is small (as an indication, one should have d < 5): this is typically not the case for a complex system.

One may sort the different techniques of failure probability estimation into four main categories:

• the simulation techniques which consist of input uncertainty propagation,

• the statistical methods which enable, from a set of output samples, to estimate P _f,

• the reliability-based approaches which take advantage of geometrical considerations on the function g(⋅), sometimes combined with sampling,

• the use of surrogate models which are of interest for computational time consuming systems g(⋅).

The exhaustive analysis of the main available techniques has already been done in Morio and Balesdent (2015). Thus, for the sake of brevity, we only focus on the most well-known techniques for reliability assessment.

The relative deviation or coefficient of variation CV  of an estimator $\widehat {P}$ of P _f is given by the following ratio:

$$\displaystyle \begin{aligned} CV(\widehat{P})=\frac{\sqrt{\mathbb{V}(\widehat{P})}}{\mathbb{E}(\widehat{P})}. \end{aligned} $$

(4.2)

We will use this metrics in the following to compare the different probability estimate for a given simulation budget. The toy case considered in this chapter is the following three dimensional function g(⋅):

$$\displaystyle \begin{aligned} \begin{array}{rcl} g\left(u^{(1)},u^{(2)},u^{(3)}\right)&\displaystyle =&\displaystyle 150+\exp\left(-0.01 \left(u^{(1)}\right)^2\right)-0.02*0.01*(u^{(2)})^5 \\ &\displaystyle &\displaystyle -0.01 \exp\left(-0.1*u^{(3)}\right). \end{array} \end{aligned} $$

The random input U follows a Gaussian distribution with zero mean and covariance matrix 0.7I ₃ with I ₃ the identity matrix. The threshold T is set to 150.9 so that an estimation of P _f with a budget for a Crude Monte Carlo sampling of 10⁷ samples is 2.92 × 10⁻⁴. It is considered as the reference for this chapter to compare the alternative techniques on this toy case.

2 Simulation Methods for Reliability Analysis

2.1 Crude Monte Carlo

As the failure probability of Equation (4.1) is an expectation, the law of large numbers may be applied. Crude Monte Carlo (CMC) (Silverman 1986; Sobol 1994; Kroese et al. 2014) takes advantage of this feature to estimate the probability P _f. For that purpose, one generates M independent and identically distributed (i.i.d.) samples u ₍₁₎, …, u _(M) with PDF ϕ(⋅) and computes their outputs through the function g(⋅): $g\left ({\mathbf {u}}_{(1)}\right ),\ldots ,g\left ({\mathbf {u}}_{(M)}\right )$. The failure probability P _f is then estimated with:

(4.3)

This estimate $\widehat {P}_f^{CMC}$ converges almost surely when M → +∞ to the failure probability P _f due to the law of large numbers (Sobol 1994). The coefficient of variation of the estimator $\widehat {P}_f^{CMC}$ is given by

$$\displaystyle \begin{aligned} CV\left(\widehat{P}_f^{CMC}\right)=\frac{\sqrt{\mathbb{V}\left(\widehat{P}_f^{CMC}\right)}}{\mathbb{E}\left(\widehat{P}_f^{CMC}\right)}=\frac{1}{\sqrt{M}}\frac{\sqrt{P_f-P_f^2}}{P_f}. {} \end{aligned} $$

(4.4)

Since we consider rare event estimation, P _f may take low values and thus

$$\displaystyle \begin{aligned} \lim_{P_f \to 0}CV\left(\widehat{P}_f^{CMC}\right)= \lim_{P_f \to 0}\frac{1}{\sqrt{M P_f}} = +\infty. \end{aligned} $$

(4.5)

The coefficient of variation is consequently unbounded. To estimate a probability P _f of order 10⁻⁴ with a 10% relative deviation, at least 10⁶ samples are required which is often not affordable on real applications.

The results of probability estimation with CMC for the toy case are proposed in Table 4.1. The variability of CMC estimate decreases with rate $\sqrt {M}$. CMC samples u _(i) that lead to a failure, that is , are given in Figure 4.2. Their distribution is far from the initial Gaussian distribution. More than 10⁵ samples are required to get a CV lower than 10%. CMC is thus not adapted to rare event probability estimation since the simulation budget to get a low CV  is often not available for complex systems.

Fig. 4.2

CMC samples u _(i) that lead to

Table 4.1 Failure probability estimation with CMC for the toy case

2.2 Importance Sampling

The idea of importance sampling (IS) (Bucklew 2004) is to rewrite the integral of Equation (4.1) with an auxiliary distribution ψ(⋅) such that

(4.6)

The notation $\mathbb {E}_\psi$ means that mathematical expectation is done under the distribution ψ(⋅). This notation will only be used in this section to avoid some confusions.

The support of the distribution ψ(⋅) must contain the support of ϕ(⋅) to avoid biased estimates. One may also notice that if ψ = ϕ, IS is CMC. Then, the principle of IS is similar to CMC since the failure probability is still described as an expectation and thus, the law of large numbers may be applied. One generates M i.i.d. samples u ₍₁₎, …, u _(M) with the input PDF ψ(⋅) and computes their outputs through the function g(⋅): $g\left ({\mathbf {u}}_{(1)}\right ),\ldots ,g\left ({\mathbf {u}}_{(M)}\right )$. The failure probability P _f is then estimated with

(4.7)

where $\frac {\phi (\cdot )}{\psi (\cdot )}$ is called the likelihood ratio. As $\widehat {P}_f^{CMC}$, the estimate $\widehat {P}_f^{IS}$ converges almost surely when M → +∞ to the failure probability P _f. Nevertheless, the choice of ψ(⋅) is of high importance to reduce the variance of the estimator. A bad choice of ψ(⋅) may also increase the variance. Let us consider the variance of the IS probability estimate.

(4.8)

As variances are non-negative quantities, the optimal auxiliary density ψ _opt(⋅) is determined by cancelling the variance in Equation (4.8). It may be shown that ψ _opt(⋅) is then defined with (Bucklew 2004)

(4.9)

The optimal auxiliary density ψ _opt(⋅) depends on P _f and is thus unusable in practice. Let us notice that the CMC samples of Figure 4.2 follow ψ _opt(⋅) as this distribution may be seen as a conditional distribution. A valuable auxiliary distribution ψ(⋅) will be close to the shape of ψ _opt(⋅). The framework to determine an efficient auxiliary distribution is rather large. Unless an initial guess is possible thanks to some a priori knowledge on g, the auxiliary distribution has to be learnt iteratively for a given range of models (parametric or nonparametric) for ψ(⋅). Recent advances on this subject are the cross-entropy optimization of importance sampling parametric distribution ψ _λ(⋅) (De Boer et al. 2005) and nonparametric adaptive importance sampling (NAIS) where ψ(⋅) is modeled with kernel density estimator (Zhang 1996; Morio 2012).

NAIS has been applied to the toy case. With a budget simulation of 6000 samples, the estimated failure probability with NAIS is 2.68 × 10⁻⁴ with a CV of 6.5%. The variance reduction is consequently significant compared to CMC. All the NAIS results are gathered in Table 4.2 for different simulation budgets. Figure 4.3 also shows samples generated during some iterations of NAIS algorithms.

Fig. 4.3

Samples u _(i) at the different iterations of NAIS algorithm (black: first iteration, green: second iteration, red: third and last iteration)

Table 4.2 Failure probability estimation with IS for the toy case

NAIS is particularly suited to the toy case because the dimension d is low. When d is greater than 10, finding an efficient IS auxiliary distribution becomes more difficult even in the parametric case.

2.3 Subset Simulation

Subset simulation (SS) also called sometimes importance splitting, subset sampling or sequential Monte Carlo, aims at decomposing the failure probability P _f as a product of conditional probabilities that can be estimated with a classical CMC approach. Different variants have been then proposed in the recent years (Au and Beck 2001; Cérou et al. 2012; Botev and Kroese 2012).

Let us consider the set $\mathbf {A}=\{\mathbf {u} \in \mathbb {R}^d|g(\mathbf {u})> T\}$, the probability P _f may be rewritten with $P_f=\mathbb {P}(\mathbf {U}\in \mathbf {A})$. The principle of SS is to iteratively estimate supersets of A and then to estimate $\mathbb {P}(\mathbf {U}\in \mathbf {A})$ with conditional probabilities.

Let us define ${\mathbf {A}}_{0}=\mathbb {R}^d \supset {\mathbf {A}}_{1} \supset \ldots \supset {\mathbf {A}}_{p-1} \supset {\mathbf {A}}_{p}=\mathbf {A}$, a decreasing sequence of $\mathbb {R}^d$ subsets with smallest element A = A _p. The easiest way to get some subsets A _i is to choose a sequence of thresholds T = T _p > T _p−1 > … > T _i > … > T ₀ = −∞ and then consider that ${\mathbf {A}}_i=\{\mathbf {u} \in \mathbb {R}^{d}|g(\mathbf {u})> T_i\}$ for i = 0, …, p. This definition is indeed well adapted to threshold exceedance probability estimation. The probability P _f is then defined with Bayes’ theorem as

$$\displaystyle \begin{aligned} P_f=\mathbb{P}(\mathbf{U} \in \mathbf{A})= \prod_{i=1}^{p}{\mathbb{P}(\mathbf{U} \in {\mathbf{A}}_{i}|\mathbf{U} \in {\mathbf{A}}_{i-1})}=\prod_{i=1}^{p}{\mathbb{P}(g(\mathbf{U})> T_i|g(\mathbf{U})> T_{i-1})}, \end{aligned} $$

(4.10)

where $\mathbb {P}(\mathbf {U} \in {\mathbf {A}}_{i}|\mathbf {U} \in {\mathbf {A}}_{i-1})$ is the probability that U ∈A _i knowing that U ∈A _i−1. The probabilities $\mathbb {P}(\mathbf {U} \in {\mathbf {A}}_{i}|\mathbf {U} \in {\mathbf {A}}_{i-1})$ may then be estimated through Monte Carlo simulations with samples of the densities h ⁱ⁻¹(⋅), the distributions of U restricted to the set A _i−1. The knowledge of h ⁱ⁻¹(⋅) is barely available in general but let us assume first that it is the case here. The different stages of SS to estimate P _f are thus the following ones when all the thresholds T _i are inputs of the algorithm:

• 1. Set i = 1, and h ⁰ = ϕ.

• 2. Generate M samples ${\mathbf {u}}_{(1)}^{i},\ldots ,{\mathbf {u}}_{(M)}^{i}$ from h ⁱ⁻¹(⋅) and apply the function g(⋅) to get $g\left ( {\mathbf { u}}_{(1)}^{i}\right ),\ldots ,g\left ( {\mathbf {u}}_{(M)}^{i}\right )$.

• 3. Estimate the probability ${\mathbb {P}(\mathbf {U} \in {\mathbf {A}}_{i}|\mathbf {U} \in {\mathbf {A}}_{i-1})}$ with CMC by $\widehat {P}_i^{CMC}$,

  

• 4. Set i = i + 1 and if i < p + 1, go back to stage 2. Otherwise, estimate the probability P _f with

  $$\displaystyle \begin{aligned}\widehat{P}^{SS}=\prod_{i=1}^{p} \widehat{P}_i^{CMC}. \end{aligned}$$

CMC is suitable to estimate the conditional probabilities since they do not correspond to rare events and thus, $\widehat {P}_i^{CMC}$ may have a low variance with a reasonable simulation budget M. The variance of $\widehat {P}^{SS}$ may be then lower than a CMC direct approach when the event g(U) > T is particularly rare, that is when P _f < 10⁻⁴ (Cérou et al. 2012).

As stated before, the complete knowledge of h ⁱ⁻¹(⋅) is not available in most cases but this density is known up to a constant as

Markov Chain Monte Carlo (MCMC) is a well-known class of algorithms to sample from a density that is known up to a constant. Metropolis–Hastings algorithm or Gibbs sampling (Robert and Casella 2005) are typical MCMC methods that may be used in that case. Moreover, the difficulty of initializing these approaches is negligible here since the samples ${\mathbf {u}}_{(j)}^{i-1}$ such that are distributed exactly with h ⁱ⁻¹(⋅) and may be used to start the Markov chain. When ϕ(⋅) is Gaussian, Crank–Nicolson technique is also another method to sample with h ⁱ⁻¹(⋅).

A priori choice of thresholds is tricky since one has to control that the different conditional probabilities are not too low and may be estimated accurately with Monte Carlo. A significant variance for a given conditional probability highly impacts the whole performance of SS. In fact, an optimal choice of the sequence A _i and thus for T _i is given when P(U ∈A _i|U ∈A _i−1) = ρ, where ρ is a constant, that is when all the conditional probabilities are equal. The variance of $\widehat {P}^{SS}$ is indeed minimized in this configuration as shown in Lagnoux (2006) and Cérou et al. (2006). In practice, the values of T _i for i = 0, …, p are set in an adaptive manner to perform valuable results (Cérou et al. 2012) using ρ-quantile of samples generated with the PDF h ⁱ⁻¹(⋅).

In practice, the different stages of SS to estimate P _f are the following ones:

• 1. Set i = 1, ρ ∈ ]0, 1[ and h ⁰ = ϕ

• 2. Generate M samples ${\mathbf {u}}_{(1)}^{i},\ldots ,{\mathbf {u}}_{(M)}^{i}$ from h ⁱ⁻¹(⋅) and apply the function g(⋅) in order to have $g\left ( {\mathbf {u}}_{(1)}^{i}\right ),\ldots ,g\left ( {\mathbf {u}}_{(M)}^{i}\right )$.

• 3. Estimate the ρ-quantile $\gamma _\rho ^{i}$ of the samples $g\left ( {\mathbf {u}}_{(1)}^{i}\right ),\ldots ,g\left ( {\mathbf {u}}_{(M)}^{i}\right )$.

• 4. Generate M samples ${\mathbf {u}}_{(1)}^{i+1},\ldots ,{\mathbf {u}}_{(M)}^{i+1}$ that follows h ⁱ(⋅) with MCMC from the samples $g\left ( {\mathbf { u}}_{(1)}^{i}\right ),\ldots ,g\left ( {\mathbf {u}}_{(M)}^{i}\right )$ that are greater than $\gamma _\rho ^{i}$.

• 5. If $\gamma _\rho ^{i}<T$, set i = i + 1 and go back to stage 3. Otherwise estimate the probability with

  

SS is not really efficient on the toy case as the failure probability is over 10⁻⁴ as shown in Table 4.3. The performance of SS is of the same order as CMC in that case. Figure 4.4 shows the SS samples for different iterations when M = 500.

Fig. 4.4

Samples ${\mathbf {u}}_{(M)}^{(i)}$ at different iterations of SS algorithm (black: first iteration, green: third iteration, red: seventh and last iteration)

Table 4.3 Failure probability estimation with SS for the toy case

SS gives its best results relatively to the other methods when the failure probability is lower than 10⁻⁴ and when the dimension d is greater than 10. More than 10⁴ samples are often necessary to get a probability estimation, so that it is hard to apply SS directly on time-costly applications.

3 Statistical Approaches for Reliability Analysis

The principle of the following theories is to approximate the maximum of g(U) or its tail PDF with a parametric model. One assumes in this section that a finite set of i.i.d. samples g(u ₍₁₎), …, g(u _(M)) of the output is available, but also that one cannot generate new samples of g(u). The i.i.d. hypothesis may be relaxed for stationary samples.

3.1 Extreme Value Theory

The Fisher–Tippett–Gnedenko theorem (Embrechts and Schmidli 1994) shows that the maxima S _M of the sequence $g\left (\mathbf { u}_{(1)}\right ),\ldots ,g\left ({\mathbf {u}}_{(M)}\right )$ converges to a generalized extreme value (GEV) distribution F _ξ:

If there exists a _M and b _M, with a _M > 0 such that, for all $y \in \mathbb {R}$,

$$\displaystyle \begin{aligned}P\left(\frac{S_M-b_M}{a_M}\leq y\right)\overset{M\rightarrow\infty}{\longrightarrow} F(y),\end{aligned}$$

where F is a non-degenerate CDF, then F is a GEV distribution F _ξ. In this case, F is said to belong the maximum domain of attraction (MA) of ξ, F ∈ MA(ξ). The set of GEV distributions is composed of three distinct types, characterized by ξ = 0, ξ > 0, and ξ < 0 that correspond to the Gumbel, Fréchet, and Weibull distributions, respectively. The expression of a _M and b _M is known for usual PDF (Embrechts and Schmidli 1994). An approximation of P _f (Embrechts and Schmidli 1994) for large values M is also available:

$$\displaystyle \begin{aligned} \widehat{P}_f^{EVT} =\frac{1}{M} \left( 1+ \xi \left(\frac{T-b_M}{a_M} \right) \right)^{-\frac{1}{\xi}}. {} \end{aligned} $$

(4.11)

Equation (4.11) is often difficult to use in practice because the estimations of a _M, b _M, and ξ have a strong variance. This approach is notably considered when only samples of maxima are available. The samples of a monthly river maximum height correspond, for instance, to this situation. When it is not the case, it is required to group the samples $g\left ({\mathbf {u}}_{(1)}\right ),\ldots ,g\left ({\mathbf {u}}_{(M)}\right )$ into blocks and fit the GEV using the maximum of each block.

3.2 Peak Over Threshold Approach

Instead of grouping the samples into block maxima, peak over threshold (POT) considers the largest samples $g\left ({\mathbf {u}}_{(1)}\right ),\ldots ,g\left (\mathbf { u}_{(M)}\right )$ to estimate the probability P _f. The Pickands–Balkema–de Haan theorem (Embrechts and Schmidli 1994) links EVT and threshold exceedance:

Let us assume that the distribution function F of i.i.d. samples $g\left ({\mathbf {u}}_{(1)}\right ),\ldots ,g$ $\left ({\mathbf {u}}_{(M)}\right )$ is continuous. Set $y^*=\sup \{y,\,g(y)<1\}=\inf \{y,\,g(y)=1\}$. Then, the two following assertions are equivalent:

• 1. F ∈ MA(ξ),

• 2. there exists a positive and measurable function u↦β(u) such that

  $$\displaystyle \begin{aligned}\underset{t\mapsto y^*}{\lim}\underset{0<y<y^*-t}{\sup}\vert F^t(y)-H_{\xi,\beta(t)}(y)\vert =0,\end{aligned}$$

where $F^t(y)=\mathbb {P}(g(\mathbf {U})-t\leq y\vert g(\mathbf {U})>t)$, and H _ξ,β(t) is the CDF of a generalized Pareto distribution with shape parameter ξ and scale parameter β(t).

The expression of the generalized Pareto CDF is the following

This theorem may be applied to estimate the probability of exceedance $\mathbb {P}(g(\mathbf {U})>T)$ as the distribution of g(U) knowing g(U) > t is modeled with a parametric PDF. Indeed, the probability P _f may be rewritten as

$$\displaystyle \begin{aligned} \mathbb{P}(g(\mathbf{U})>T)=\mathbb{P}(g(\mathbf{U})>T \vert g(\mathbf{U})>t) \mathbb{P}(g(\mathbf{U})>t). \end{aligned} $$

(4.12)

for T > t. A natural estimate of $\mathbb {P}(g(\mathbf {U})>t)$ is given by a Monte Carlo estimate

(4.13)

With the Pickands–Balkema–de Haan Theorem and for large values of t, one has

$$\displaystyle \begin{aligned}\mathbb{P}(g(\mathbf{U})>T \vert g(\mathbf{U})>t)\approx 1-H_{\xi,\beta(t)}(T-t). {}\end{aligned} $$

(4.14)

The parameters (ξ, β) of the Pareto distribution H have to be estimated with the samples g(u _(i)) such that g(u _(i)) > T. The estimate of P _f with POT is then built with

(4.15)

Three parameters have to be determined in the POT probability estimate of Equation (4.15): the threshold t and the couple (ξ, β(t)). The choice of t is sensitive since it determines the samples that are used in the estimation of (ξ, β(t)). Indeed, a high threshold leads to consider only a small number of samples in the estimation of (ξ, β(t)) and thus their estimate can be then spoiled by a large variance, whereas a low threshold introduces a bias in the probability estimate (Dekkers and De Haan 1999). There are several methods to determine a valuable threshold t knowing the samples. The most well-known ones are the Hill plot and the mean excess plot (Embrechts and Schmidli 1994). These methods are nevertheless very empirical since they are based on graphical interpretation. Automatic threshold selection has also been proposed such as in Thompson et al. (2009) based on the distribution of the difference of the parameter estimates when the threshold is changed. It is often necessary in practice to compare the estimates of t given by the different methods. Once the value of t is set, the parameters (ξ, β(t)) are often estimated by maximum likelihood (Coles 2001) or more occasionally by the method of moments (Hosking and Wallis 1987). An overview of these different methods can be found in Neves and Fraga Alves (2004). The statistical POT approach for different budget M is applied on the toy case and the corresponding results are given in Table 4.4. The POT threshold is evaluated with the method of Thompson et al. (2009) and the parameters of the Pareto distribution are estimated with maximum likelihood. One notices an overestimation of the failure probability whatever the considered budget simulation.

Table 4.4 Failure probability estimation with POT for the toy case

When resampling is not possible, extreme value theory and POT are the only solutions to estimate a failure probability. In the contrary case, if resampling is possible, specific sampling methods such as IS or SS should be considered to estimate accurately a failure probability.

4 Reliability Based Approaches

4.1 First-Order Reliability Method (FORM)/Second-Order Reliability Method (SORM)

First-order and second-order reliability method (FORM/SORM) (Madsen et al. 1986; Bjerager 1991; Yan-Gang and Tetsuro 1999; Lassen and Recho 2006) is a probabilistic computational approach for structural reliability. FORM gives an analytic approximation of the failure probability when the inputs are standard normal by estimating the most probable failure point (MPP). The limit state surface is defined as the input region where g(U) = T. Then, the MPP also called design point is the point with the minimum distance from the origin to the limit state surface. This design point is determined with optimization methods. The failure probability is then estimated with the evaluation of a standard normal CDF at the design point. In that case, the limit state surface is approximated by a linear function at the design point. Thus, accuracy problems may occur when the performance function is strongly non-linear or if the most probable failure point is not unique (Sudret 2012). The second-order reliability method (SORM) (Kiureghian et al. 1987) has been established as an attempt to improve the accuracy of FORM as it approximates the limit state surface at the design point by a second-order surface. When the most probable failure point is not unique, FORM and SORM with multiple design points have been proposed in Kiureghian and Dakessian (1998). FORM/SORM method is applied in three stages to estimate P _f:

• 1. Apply a transformation V (⋅) on the input U such that R = V (U) with R a standard normal variable. Depending on the available information on the PDF of U, several transformations may be considered (Nataf 1962; Hasofer and Lind 1974; Pei-Ling and Kiureghian 1991; Rosenblatt 1952; Lebrun and Dutfoy 2009a,b).

• 2. Evaluate the most probable failure point β such that

  $$\displaystyle \begin{aligned} \begin{array}{rcl} &\displaystyle \boldsymbol{\beta}=\underset{\mathbf{R}}{\operatorname{argmin}} \mid\mid \mathbf{R} \mid\mid \end{array} \end{aligned} $$

  (4.16)
  $$\displaystyle \begin{aligned} \begin{array}{rcl} &\displaystyle \;\;\;\;\;\;\;\;\;\;\;\;\;\; \;\;\;\;\;\;\;\text{s.t. } T-g(V^{-1}(\mathbf{R}))=0, \end{array} \end{aligned} $$

  (4.17)

  where ||.|| is the Euclidian norm. The constraint T − g(V ⁻¹(R)) = 0 defines the limit of failure space for the variable R. The parameter β is the design point and ||β|| is the reliability index. Several algorithms have been proposed to solve this optimization procedure as described in Hasofer and Lind (1974), Pei-Ling and Kiureghian (1991), Rackwitz and Flessler (1978), and Dietlevsen and Madsen (1996).

• 3. Estimate the failure probability with in the case of FORM:

  $$\displaystyle \begin{aligned} \widehat{P}_f^{FORM}=\Phi(-||\boldsymbol{\beta}||), \end{aligned} $$

  (4.18)

  where Φ is the CDF of standard normal variable. In the case of SORM, the failure probability is given by Breitung (1984)

  $$\displaystyle \begin{aligned} \widehat{P}_f^{SORM}=\Phi(-||\boldsymbol{\beta}||) \prod_{i=1}^{d-1} (1-\boldsymbol{\beta} \kappa_i)^{-\frac{1}{2}} , \end{aligned} $$

  (4.19)

  where κ _i denotes the principal curvature of T − ϕ(V ⁻¹(R)) at the minimum distance point β. The term κ _i is defined with

  $$\displaystyle \begin{aligned} \kappa_i= \left.\frac{\partial^2\left(T-\phi\left(V^{-1}(\mathbf{R})\right)\right)}{\partial^2 R^{(i)}}\right|{}_{\mathbf{R}=\boldsymbol{\beta}}, \end{aligned} $$

  (4.20)

with R ⁽ⁱ⁾, i = 1, …, d, a component of the vector R. These methods do not need a large simulation budget to obtain a valuable result. Nevertheless, the different assumptions require that one has to be careful when one applies FORM/SORM to a realistic case of function g(⋅). There is also no control of the error in FORM/ SORM. However, it is possible from the FORM/SORM design point to propose an importance sampling auxiliary density and then to sample with it to estimate the rare event probability.

Standard FORM, FORM with multiple design points, and SORM have been applied to the toy case and give the results summarized in Table 4.5. The standard FORM design point is estimated to β = [3.07, 0, −5.53 × 10⁻²] with 1096 samples as shown in Figure 4.5. The corresponding $\widehat {P}_f^{FORM}$ does not lead to an accurate estimation as the limit state surface is non-linear and the design point is not unique in this toy case. FORM with multiple design points finds a second design point located at [−3.10, 0.09, −3.22] that doubles the probability given by FORM but still underestimates the true failure probability.

Fig. 4.5

Samples generated during FORM optimization and the design point in green

Table 4.5 Failure probability estimation with FORM/SORM for the toy case

FORM/SORM are very interesting in general to get a rough approximation of the failure probability with limited simulation budgets, but as no control of the error is available, it is often better to combine FORM/SORM with IS for instance.

4.2 Directional Sampling

The principle of directional sampling (DS) (Bjerager 1988), also known as directional simulation, is to aim at the limit-state function and then estimate the failure probability as the mean of the failure probabilities over all the directions. DS may be considered when the input U follows a Gaussian standard distribution. If it is not the case, iso-probabilistic transformations should be applied on the inputs several transformations may be considered (Nataf 1962; Hasofer and Lind 1974; Pei-Ling and Kiureghian 1991; Rosenblatt 1952; Lebrun and Dutfoy 2009a,b). DS takes advantage of the rewriting of the standard Gaussian vector U as a product of random variables

$$\displaystyle \begin{aligned}\mathbf{U}= R \mathbf{A}, \end{aligned}$$

where R ² is a chi-squared random variable with d degrees of freedom with density ϕ _R and A is a random unit vector uniformly distributed on the d-dimensional unit sphere Λ^d with density ϕ _A. R ² and A are independent random variables. The failure probability P _f and the integral of Equation (4.1) may be then expressed as follows:

The DS estimation corresponds in fact to a CMC estimation of $\mathbb {E}\left (\mathbb {P}(g(R \mathbf {A})>T|\mathbf {A}\right .$ $\left .=\mathbf {a})\right )$. In practice, a sequence of M i.i.d. random direction vectors A _j for j = 1, …, M is generated and then, one determines r _j such that g(r _jA _j) = T, by dichotomy for instance. An estimate of $\mathbb {P}(g(R \mathbf {A})>T|\mathbf {A}={\mathbf {A}}_j)$ is given by $1-F_{R^2}\left (r_j^2\right )$, where $F_{R^2}(\cdot )$ is the CDF of a chi-squared random variable with d degrees of freedom. This approximation is only valuable if there is only one intersection point between the input failure region and the chosen sampling direction. The DS probability estimate $\widehat {P}_f^{DS}$ is then obtained with

$$\displaystyle \begin{aligned}\widehat{P}_f^{DS}=\frac{1}{M}\sum_{j=1}^M\left(1-F_{R^2}\left(r_j^2\right)\right). \end{aligned}$$

DS has been applied to the toy case and all the estimated failure probabilities are given in Table 4.6 for different number of directions. Samples that have been generated when M = 100 are proposed in Figure 4.6. DS is very efficient on this example as the dimension d is quite low and the function g(⋅) is regular and differentiable.

Fig. 4.6

Samples of DS algorithm for M = 100

Table 4.6 Failure probability estimation with DS for the toy case

The tuning of the number of directions M may be complicated in complex systems. For that purpose, the choice of the directions A _j could be done adaptively instead of randomly to focus on input regions that have the highest $\mathbb {P}(g(R \mathbf {A})>T|\mathbf {A}={\mathbf {A}}_j)$ as shown in Zuniga et al. (2011) to reduce the estimate variance.

5 Use of Surrogate Models in Rare Event Probability Estimation

Being able to build an efficient surrogate model which allows to reduce the number of calls to the expensive input–output function g(⋅) while keeping a reasonable accuracy of the probability estimate is the key point. Surrogate models are particularly interesting in the case of rare event probability estimation as the surrogate model has to be accurate not over all the support of U but only in the input region where g(u) = T. The use of the exact function g(⋅) and its surrogate $\widehat {g}(\cdot )$ in the probability calculation will lead to the same result if . In other words, the surrogate model might not be representative of the exact function outside the zones of interest as it does not take part of the probability estimation. $\widehat {g}(\cdot )$ has to be a good classifier of samples for the failure prediction.

A great number of methods has been proposed and compared in recent years. Polynomial chaos expansions (PCE) have been associated with Monte Carlo sampling to estimate failure probabilities (Li and Xiu 2010). Support vector machines have also been employed to estimate the domains of failure (Basudhar et al. 2008) and been coupled to rare event estimator such as subset sampling (Bourinet et al. 2011a). Kriging has been extensively used with classical Monte Carlo estimator (Echard et al. 2011), importance sampling method (Janusevskis and Le Riche 2012; Schueremans and Van Gemert 2005; Balesdent et al. 2013; Dubourg et al. 2011), importance sampling with control variates (Cannamela et al. 2008), or subset simulation (Vazquez and Bect 2009; Li et al. 2012; Bect et al. 2012). All these approaches are mainly based on three main ingredients:

• the type of surrogate models (PCE, Kriging, SVM, etc.),

• the surrogate model refinement strategy, that is how to choose samples that are evaluated on the exact function g(⋅),

• the associated sampling strategy (CMC, IS, SS, etc.).

We will not review all the different combinations that have been proposed in the literature but focus on two well-known methods for the sake of brevity.

5.1 Subset Sampling by Support Vector Margin Algorithm for Reliability esTimation (²SMART)

The ²SMART method (Deheeger and Lemaire 2007; Bourinet et al. 2011a) is dedicated to the use of adaptive subset sampling technique (SS) and consists in defining one SVM model at each adaptive threshold involved in SS. For each intermediate threshold, a SVM model is built using a three-stage refinement approach (localization, stabilization, and convergence) which allows to accurately represent the regions corresponding to the involved thresholds. At the ith stage of SS, the main steps of ²SMART are as follows:

• 1. A first set of samples is generated to build a SVM model $\widehat {g}_{T_i}(\cdot )$ in the region corresponding to the ith level of SS and some of these samples are used to determine the current intermediate threshold T _i, using the ρ-quantile level of SS and the SVM model in regression,

• 2. The SVM $\widehat {g}_{T_i}(\cdot )$ is refined using a three-stage approach, in an iterative manner, inducing resampling (by MCMC) and clustering of the generated samples. For that purpose, three populations of samples of different size are generated and used to refine the current SVM model.

• 3. The last step consists in evaluating the conditional probability $\mathbb {P}(\widehat {g}_{T_i}(\mathbf {U})>T_i|\widehat {g}_{T_i}(\mathbf {U})>T_{i-1})$, corresponding to the current threshold T _i.

²SMART has been applied on the toy case. The probability estimation is 2.82 × 10⁻⁴ with a CV equal to 17% for only 800 calls to the function g(⋅). The reduction of the number of calls compared to SS is very significant. We show in Figure 4.7 the limit-state function obtained with ²SMART algorithm from open source FERUM 4.1 Matlab^® toolbox.

Fig. 4.7

Limit state samples at the last iteration of ²SMART (FERUM 4.1 toolbox)

5.2 Active Learning Reliability Method Combining Kriging and Probability Estimation Method

From the initial training set $\mathcal {U}$, the Kriging properties are valuable to determine the additional samples which have to be evaluated on g(⋅) to refine its surrogate model. Active learning (Echard 2012) determines a new sample point u to add to the training set $\mathcal {U}$ by solving the following optimization problem:

$$\displaystyle \begin{aligned} \underset{\mathbf{u}}{\max} \left[1 - \Phi\left( \frac{|T-\widehat{g}(\mathbf{u},\mathcal{U})|}{\sigma(\mathbf{u},\mathcal{U})}\right)\right], {} \end{aligned} $$

(4.21)

where Φ(⋅) is the CDF of the standard Gaussian distribution and σ is the Kriging variance. The used criterion generates a sample for which the Kriging prediction is close to the threshold (numerator of Equation (4.21)) and which presents a high prediction error (denominator of Equation (4.21)). Due to the monotonicity of the involved CDF, the optimization problem Equation (4.21) is equivalent to:

$$\displaystyle \begin{aligned} \underset{\mathbf{u}}{\min}\; \frac{|T-\widehat{g}(\mathbf{u},\mathcal{U})|}{\sigma(\mathbf{u},\mathcal{U})}. {} \end{aligned} $$

(4.22)

This criterion has been coupled with CMC (Echard et al. 2011), IS (Echard et al. 2013), SS (Echard 2012). In practice, the optimization problem is not solved, and given a sample set {u ₍₁₎, …, u _(M)} provided by CMC, IS, or SS, the new sample which will be added to the training set is determined by

$$\displaystyle \begin{aligned} \mathbf{u} = \underset{{\mathbf{u}}_{(1)},\dots,{\mathbf{u}}_{(M)}}{\text{argmin}} \left\{\frac{|T-\widehat{g}({\mathbf{u}}_1,\mathcal{U})|}{\sigma({\mathbf{u}}_{(1)},\mathcal{U})},\dots, \frac{|T-\widehat{g}({\mathbf{u}}_{(M)},\mathcal{U})|}{\sigma({\mathbf{u}}_{(M)},\mathcal{U})} \right\}. \end{aligned}$$

Active Kriging combined with CMC (10⁵ samples) has been applied on the toy case. With a limited budget simulation of 54 calls to the exact function g(⋅), the estimated failure probability is 2.68 × 10⁻⁴ with a CV of 24%. The variance reduction is consequently significant compared to CMC. We show in Figure 4.8 the CMC samples where the samples that have been really evaluated on g(⋅) are highlighted.

Fig. 4.8

Population of CMC samples in active learning Kriging (in black) and samples that have been evaluated on g(⋅) (in green)

Surrogate models are particularly adapted to rare event estimation as they offer a low variance of the probability estimate with few calls to g(⋅). Nevertheless the complexity required to set up and tune these algorithms is important and may be case dependent. Moreover, surrogate models suffer from the curse of dimensionality. Thus, it is often hard to apply them for the estimation of failure probabilities of high-dimensional complex systems.

6 Brief Overview of Reliability Analysis with Both Aleatory and Epistemic Uncertainties

Several works proposed to extend reliability analysis that is mainly developed within the probability theory framework as presented in the previous sections, in order to include alternative uncertainty description frameworks. This section presents a brief overview of reliability analysis dealing with both aleatory and epistemic uncertainties.

Among the existing approaches for reliability analysis with both aleatory variables modeled with probability theory and epistemic variables described by an alternative modeling framework (e.g., evidence theory, interval formalism, possibility theory, Pbox) two families may be distinguished. The first ones transform the epistemic uncertain variables in order to describe them with probability theory and to use adaptation of the reliability analysis techniques presented in the previous sections. The second ones manage both formalisms resulting in a nested loop for the estimation of the bounds on the probability of failure. Some approaches try to remove the nested loop to decrease the reliability analysis cost.

In reliability analysis, different techniques have been proposed in the case where aleatory input distributions with interval uncertain parameters are considered (Hurtado 2013; Balesdent et al. 2016). For instance, if the input uncertainty is described by a Gaussian distribution but the mean value is only known within an interval. Hurtado (2013) proposed to extend a technique called reliability plot to account for interval uncertainty for the aleatory distribution hyperparameters. Balesdent et al. (2016) proposed to combine Kriging, importance sampling, and a dedicated refinement strategy of the surrogate model to find the upper and the lower bounds on the probability of failure due to interval uncertainty. Within the more general context of Pboxes, the probability of failure is defined within bounds given by:

(4.23)

(4.24)

where $\min$ and $\max$ mean that the optimization would be carried out over all PDF ϕ(⋅) that satisfy the definition of the input Pbox. Schöbi and Sudret (2017) proposed a reliability analysis approach in which the input variables are modeled by Pboxes (in both cases, parametric and general). The authors developed a two-level approach using Kriging surrogate models with adaptive experimental designs at different levels of the reliability analysis. The input uncertainties are represented using convex normal membership functions.

Similarly to the concept of Most Probable Point (MPP) of failure in probability reliability analysis with the probability formalism, the concept of Most Probable Focal Element (MPFE) has been proposed by Jiang et al. (2013). Due to the discrete nature of the basic mass assignment, the MPFE is defined as a region rather than a point, which has the maximum contribution to the failure probability (Figure 4.9). Du (2008) proposed to extend FORM to the presence of evidence theory to account for both aleatory and epistemic uncertainties into a unified framework. Let us consider an input space divided between the aleatory space $\mathcal {U}$ and the epistemic space $\mathcal {X}$. Consider also a state function $\mathbf {U}\in \mathcal {U}\times \mathbf {X}\in \mathcal {X} \rightarrow g(\mathbf {U},\mathbf {X})$ and ${\mathbf {C}}_{{\mathbf {X}}_i}$ for i = {1, …, n} the focal elements of X,with n the number of focal elements. It is possible to define the belief and plausibility of the probability of failure as follows:

$$\displaystyle \begin{aligned} \begin{array}{rcl} Bel(P_f) &\displaystyle =&\displaystyle \sum_{i=1}^n m_{\mathbf{X}}({\mathbf{C}}_{{\mathbf{X}}_i})\mathbb{P}[G_{\text{max}}(\mathbf{U},\mathbf{X})>0 | \mathbf{X}\in{\mathbf{C}}_{{\mathbf{X}}_i}] \end{array} \end{aligned} $$

(4.25)

$$\displaystyle \begin{aligned} \begin{array}{rcl} Pl(P_f) &\displaystyle =&\displaystyle \sum_{i=1}^n m_{\mathbf{X}}({\mathbf{C}}_{{\mathbf{X}}_i})\mathbb{P}[G_{\text{min}}(\mathbf{U},\mathbf{X})>0 | \mathbf{X}\in{\mathbf{C}}_{{\mathbf{X}}_i}] \end{array} \end{aligned} $$

(4.26)

with G _min(U, X) and G _max(U, X) the global minimum and maximum values of g(⋅) in the subset ${\mathbf {C}}_{{\mathbf {X}}_i}$. The calculation of the belief or the plausibility measures is transformed into the calculation of the minimum or maximum probability of failure at each focal element of X. In order to compute Bel(P _f) and Pl(P _f), it is necessary to carry out interval analysis (IA) to determine G _min and G _max and probability analysis (PA) to estimate the corresponding probability of failure. Du proposed to introduce FORM-UUA to solve this nested problem. FORM is used for the PA and non-linear optimization to solve IA. To overcome the nested loop problem, FORM-UUA provides a sequential approach in which interval variables are fixed during the MPP search in the aleatory space and optimization to find the maximum and minimum of state function g(⋅) is performed with the aleatory variables fixed to MPP. The values of both evidence theory measures indicate the effect of aleatory uncertainty on a response while the gap between them reflects the effect of epistemic uncertainty on the response. Yao et al. (2013) extended FORM-UUA to reformulate the double-loop optimization problem into an equivalent single-level optimization (SLO) problem.

Fig. 4.9

MPP in probability-based reliability analysis (left) and MPFE in evidence-theory-based reliability analysis (right)

Xiao et al. (2015) proposed to transform the variables described with evidence theory into probability random variables. Support vector regression (SVR, see Chapter 3) is used to build an approximation model of the limit-state function. Based on the surrogate model, the MPP of the approximate reliability problem with only random variables is searched out. Then, the MPFE of the original problem with evidence variables is determined. Using the MPFE and the monotonicity of the limit-state function, contributions of some focal elements to Belief and Plausibility measures are estimated. Therefore, the number of focal elements involved in the calculation of extreme values of the limit-state function is reduced, facilitating the reliability analysis for mixed uncertain variables. The challenges of this approach are the proper transformation of evidence variables into random variables and the control of SVR uncertainty.

Nannapaneni and Mahadevan (2016) developed a probabilistic framework to account for both aleatory uncertainty and epistemic uncertainty in the reliability analysis. It is based on FORM, extended to include uncertain distribution parameters, distribution type, uncertain correlations, and model errors using auxiliary variables, based on the probability integral transform and the theorem of total probability. This formalism enables to avoid the nested loop of uncertainty propagation when accounting for different types of uncertainty formalism, resulting in a single-loop approach combined with MCS and FORM for reliability analysis in the presence of aleatory and epistemic uncertainties.

In the case the epistemic variables are modeled with interval instead of evidence theory, Yang et al. (2015) proposed to combine Kriging (in a classification mode instead of a regression approach), surrogate refinement and DIRECT optimization algorithm (Finkel 2003) to find the maximum and minimum values of g(⋅) for each aleatory sample and CMC to estimate the bounds on the probability of failure.

Within the framework of possibility theory, Mourelatos and Zhou (2005) proposed a reliability estimation method with insufficient data. Instead of using classical discretization methods (Akpan et al. 2002) or vertex methods (Penmetsa and Grandhi 2002) to propagate the fuzzy uncertainty, the authors proposed a hybrid global-local optimization method. The optimization approach is used to calculate the confidence level of fuzzy response. The DIRECT (divisions of rectangles) global optimizer is first used, followed by a local optimizer based on sequential quadratic programming.

Mixed aleatory and epistemic reliability analyses are more complex to set up than traditional probability estimation due to the need to combine different mathematical formalisms to account for the diverse natures of uncertainty and to mix different numerical analysis tools such as sampling, optimization, combinatorics calculations, etc.

7 Summary

In this chapter, an overview of the main different algorithms that may be applied for the estimation of rare event probabilities modeled by a threshold exceedance of an input–output function g(⋅) has been presented. Their domain of applicability may vary a lot from an algorithm to another and thus the choice of a given simulation technique depends on practical characteristics of the reliability problem such as the ability to resample, the knowledge we have on the density of g(U), the computational cost and the non-linearity of g(⋅), the dimension of U, or the complexity of the limit state surface. Figure 4.10 proposed a classification of the reliability analysis techniques presented in this chapter.

Fig. 4.10

Classification of reliability analysis techniques

In addition to the uncertainty propagation presented in Chapter 3 and the reliability analysis presented in this chapter, in order to solve single discipline optimization problem in the presence of uncertainty, it is necessary to address the formulation of the optimization problem and to appropriately choose a suitable optimization algorithm. The next chapter is focused on these key issues.