On the Use of Reduced Basis Methods to Accelerate and Stabilize the Parareal Method

Abstract

We propose a modified parallel-in-time — parareal — multi-level time integration method that, in contrast to previously proposed methods, employs a coarse solver based on a reduced model, built from the information obtained from the fine solver at each iteration. This approach is demonstrated to offer two substantial advantages: it accelerates convergence of the original parareal method for similar problems and the reduced basis stabilizes the parareal method for purely advective problems where instabilities are known to arise. When combined with empirical interpolation methods (EIM), we develop this approach to solve both linear and nonlinear problems and highlight the minimal changes required to utilize this algorithm to accelerate existing implementations. We illustrate the advantages through algorithmic design, through analysis of stability, convergence, and computational complexity, and through several numerical examples.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

7.1 7.1 Introduction

With the number of computational cores on large scale computing platforms increasing, the demands on scalability of computational methods likewise increase, due partly to an increasing imbalance between the cost of memory access, communication and arithmetic capabilities. Among other things, traditional domain decomposition methods tend to stagnate in scaling as the number of cores increases and the computational cost is overwhelmed by other tasks. This suggests a need to consider the development of computational techniques that better balance these constraints and allow for the acceleration of large scale computational challenges.

A recent development in this direction is the parareal method, introduced in [16], that provide a strategy for ‘parallel-in-time’ computations and offers the potential for an increased level of parallelism. Relying on combining a computational inexpensive but inaccurate solver with an accurate and expensive but parallel solver, the parareal method utilizes an iterative, predictor-corrector procedure that allows the expensive solver to run across many processors in parallel. Under suitable conditions, the parareal iteration converges after a small number of iterations to the serial solution [3]. During the last decade, the parareal method has been applied successfully to a number of applications (cf. [17,19]), demonstrating its potential, accuracy, and robustness.

As a central and serial component, the properties of the coarse solver can impact the efficiency and stability of the parareal algorithm, e.g., if an explicit scheme is used in both the coarse and the fine stage of the algorithm, the efficiency of the parareal algorithm is limited by the upper bound of the time step size [19]. One can naturally also consider a different temporal integration approach such as an implicit approach, although the cost of this can be considerable and often requires the development of a new solver. An attractive alternative is to use a simplified physics model as the coarse solver [2, 17, 18], thereby ignoring small scale phenomenon but potentially impacting the accuracy. The success of such an approach is typically problem specific.

While the choice of the coarse solver clearly impacts accuracy and overall efficiency, the stability of the parareal method is considerably more subtle. For parabolic and diffusion dominated problems, stability is well understood and observed in many applications [12]. However, for hyperbolic and convection dominated problems, the question of stability is considerably more complex and generally remains open [3,8, 22]. In [8], the authors propose to regularly project the solution onto an energy manifold approximated by the fine solution. The performance of this projection method was demonstrated for the linear wave equation and the nonlinear Burgers’ equation. As an alternative, the Krylov subspace parareal method builds a new coarse solver by reusing all information from the corresponding fine solver at previous iterations. The stability of this approach was demonstrated for linear problems in structural dynamics [10] and a linear 2-D acoustic-advection system [21]. However, the Krylov subspace parareal method appears to be limited to linear problems.

The approach of combining the reduced basis method [20] with the parareal method for parabolic equations was initiated in [13] in which it is demonstrated that a coarse solver based on an existing reduced model offers better accuracy and reduces the number of iterations in the examples considered. However, that work offers no discussion on the construction of the reduced model, nor was there any attempt to analyze the stability and convergence of the method.

Inspired by [13, 21], we propose a modified parareal method, referred to as the reduced basis parareal method in which the Krylov subspace is replaced by a subspace spanned by a set of reduced bases, constructed on-the-fly from the fine solver. This method inherits most advantages of the Krylov subspace parareal method and is observed to retain stability and convergence for linear wave problems. We demonstrate that this approach accelerates the convergence in situations where the original parareal already converges. However, it also overcomes several known challenges: (i) it deals with nonlinear problems by incorporating methodologies from the reduced basis methods; and (ii) the traditional coarse propagator is needed only once at the very beginning of the algorithm to generate an initial reduced basis. This allows for the time step restrictions to be relaxed as compared to the coarse solver of the original parareal method. The main difference between our method and [13] lies in the reduced approximation space and the construction of reduced bases. The reduced model, playing the role of the coarse solver, is updated for each iteration while the reduced model in [13] is built only once during an initial offline process. Among other advantages, this allows the proposed method to adapt the dimension of the reduced approximation space based on the regularity of the solution, while in [13] the reduced model remains fixed and must be developed using some other approach.

What remains of this paper is organized as follows. We first review the original parareal method in Sect. 7.2.1 and the Krylov subspace parareal method in Sect. 7.2.2. This sets the stage for Sect. 7.2.3 wherewe introduce the reduced basis parareal method and discuss different strategies to develop reduced models for problems with nonlinear terms. Section 7.3 offers some analysis of the stability, convergence, and complexity of the reduced basis parareal method and Sect. 7.4 demonstrates the feasibility and performance of the reduced basis parareal method through various linear and nonlinear numerical examples. We conclude the paper in Sect. 7.5.

7.2 7.2 Parareal Algorithms

To set the stage for the general discussion, let us first discuss the original and the Krylov subspace parareal methods in Sect. 7.2.1 and Sect. 7.2.2, respectively. We shall highlight issues related to stability and computational complexity to motivate the reduced basis parareal method, introduced in Sect. 7.2.3.

7.2.1 7.2.1 The original parareal method

Consider the following initial value problem:

$$\begin{array}{*{20}{l}} {{u_t} = L\left( u \right): = Au\left( t \right) + N\left( {u\left( t \right)} \right),\quad t \in \left( {0,T} \right],} \\ {u\left( 0 \right) = {u_0},} \end{array}$$

(7.1)

where u ∈ ℝ^N is the unknown solution, L is an operator, possibly arising from the spatial discretization of a PDE, with A being the linear part of L, and N the nonlinear part.

In the following, we denote F _δt as the accurate but expensive fine time integrator, using a constant time step size, δt. Furthermore, G _Δt is the inaccurate but fast coarse time integrator using a larger time step size, Δt. Generally, it is assumed that Δt ≫ δt.

The original parareal method is designed to solve (7.1) in a parallel-in-time fashion to accelerate the computation. First, [0,T] is decomposed into N _c coarse time intervals or elements:

$$0 = {t_0} < \ldots < {t_i} < \ldots < {t_{{N_c}}} = T,\quad {t_i}i\Delta T,\quad \Delta T = \frac{T}{{{N_c}}}.$$

(7.2)

Assume that

$$\Delta T = {N_f}\delta t,{N_f} \in \mathbb{N},$$

(7.3)

which implies that T = N _c N _f δt. Denote F _δt (u, t _i+1, t _i ) as the accurate numerical solution integrated from t _i to t _i+1 by using F _δt with the initial condition u and the constant time step size δt. Similarly for G _Δt (u, t _i+1, t _i ). Denote also u _n = F _δt (u ₀,T,0) as the numerical solution generated using only the fine integrator. With the above notation, the original parareal method is shown below in Algorithm 7.1

Algorithm 7.1

The original parareal method

Now assume that the k-th iterated approximation u ^k _n is known. The parareal approach proceeds to the k + 1-th iteration as

$$u_{n + 1}^{k + 1} = {G_{\Delta t}}\left( {u_n^{k + 1},{t_{n + 1}},{t_n}} \right) + {F_{\delta t}}\left( {u_n^k,{t_{n + 1}},{t_n}} \right) - {G_{\Delta t}}\left( {u_n^k,{t_{n + 1,}}{t_n}} \right),\quad 0 \leqslant k \leqslant {N_c} - 1.$$

(7.4)

It is easy to see that F _δt (u ^k _n , t _n+1, t _n ) can be done in parallel across all temporal elements. If we take the limit of k → ∞ and assume that the limit of {u ^k _n } exists, we obtain [16]:

$$u_{n + 1}^{k + 1} \to {u_{n + 1}} = {F_{\delta t}}\left( {{u_n},{t_{n + 1}},{t_n}.} \right)$$

(7.5)

In order to achieve a reasonable efficiency, the number of iterations, N _it , should be much smaller than N _c .

To demonstrate the performance of the original parareal method, let us consider a few numerical examples, beginning with the viscous Burgers’ equation:

$$\begin{array}{*{20}{l}} {{u_t} + \left( {\frac{{{u^2}}}{2}} \right)x = v{u_{xx}},\quad \left( {x,t} \right) \in \left( {0,2\pi } \right) \times \left( {0,T} \right],} \\ {u\left( {x,0} \right) = \sin \left( x \right),} \end{array}$$

(7.6)

where T = 2 and v = 10₋₁. A 2π-periodic boundary condition is used. The spatial discretization is a P ₁ discontinuous Galerkin method (DG) with 100 elements [15] and the time integrator is a first-order forward Euler method. We use the following parameters in the parareal integration

$${N_c} = 100,\quad {N_{it}} = 5,\quad \Delta t = {10^{ - 3}},\quad \delta t = {10^{ - 4}}.$$

(7.7)

Figure 7.1 illustrates the L _∞-error of the parareal solution at T = 2 against the number of iterations. Notice that for this nonlinear problem the algorithm converges after only four iterations, illustrating the potential for an expected acceleration in a parallel environment.

Fig. 7.1

The L _∞-error at T = 2 against the number of iterations of the 1-D Burgers’ equation using the original parareal method

As a second example, we consider the Kuramoto-Sivashinsky equation [25]:

$$\begin{array}{*{20}{l}} {\frac{{\partial u}}{{\partial t}} = \left( {\frac{{{u^2}}}{2}} \right)x - {u_{xx}} - {u_{xxxx}},\quad \left( {x,t} \right) \in \left( { - 8,8} \right) \times \left( {0,T} \right],} \\ {u\left( {x,0} \right) = \exp \left( { - {x^2}} \right)} \end{array}$$

(7.8)

with final time T = 40 and periodic boundary conditions.

As a spatial discretization we use a Fourier collocation method with 128 points [14] and an IMEX scheme [1] as a time integrator, treating the linear terms implicitly and the nonlinear term explicitly. The parameters in the parareal method are taken as

$${N_c} = 100,\quad {N_{it}} = 5,\quad \Delta t = {10^{ - 2}},\quad \delta t = {10^{ - 4}}.$$

(7.9)

Figure 7.2 (left) shows the time evolution of the chaotic solution to the Kuramoto-Sivashinsky equation with a Gaussian initial condition. In Fig. 7.2 (right), we show the L _∞-error at T = 40 against the number of iterations. In this case, we take the solution computed by the fine solver as the exact solution. It is clear that the parareal solution converges, albeit at a slower rate. It should also be noted that Δt/δt = 100, indicating the potential for a substantial acceleration.

Fig. 7.2

The time evolution of the solution (left) and the L _∞-error at T = 40 against the number of iterations (right) of the 1-D Kuramoto-Sivashinsky equation using the original parareal method

As a last and less encouraging example, we consider the 1-D advection equation

$$\begin{array}{*{20}{l}} {{u_t} + a{u_x} = 0\quad \left( {x,t} \right) \in \left( {0,2\pi } \right) \times \left( {0,T} \right],} \\ {u\left( {x,0} \right) = \exp \left( {\sin \left( {x - at} \right)} \right),} \end{array}$$

(7.10)

with a final time T = 10, a = 2π and a 2π-periodic boundary condition. We use a DG method of order 32 and 2 elements in space [15], a singly diagonal implicit fourth-order Runge-Kutta scheme in time (a five-stage fourth-order scheme, cf. S54b in [23]), and the parareal parameters:

$${N_c} = 100,\quad {N_{it}} = 27,\Delta t = 5 \times {10^{ - 2}},\quad \delta t = {10^{ - 4}}.$$

(7.11)

Figure 7.3 shows the L _∞-error at T = 10 against the number of iterations. The instability of the original parareal method is apparent, as has also been observed by others [3, 8, 22].

Fig. 7.3

The L∞-error at T = 10 against the number of iterations of the 1-D linear advection equation using the original parareal method

7.2.2 7.2.2 The Krylov Subspace Parareal Method

We notice in Algorithm 7.1 that only $\{ u_{{f_{i + 1}}}^k\} _{i = 0}^{{N_c} - 1}$ is used in the advancement of the solution to k + 1. To fix the stability issue, [10] proposed to improve the coarse solver by reusing information computed at all previous iterations and applied this idea to linear hyperbolic problems in structural dynamics. Recently, a similar idea was successfully applied to linear hyperbolic systems [21].

The basic idea of the Krylov subspace parareal method is to project u ^k+1 _i onto a subspace spanned by all numerical solutions integrated by the fine solver at previous iterations. Denote the subspace as

$${S^k}: = span\left\{ {{u_f}_i^j,1 \leqslant i \leqslant {N_c},1 \leqslant j \leqslant k} \right\}.$$

(7.12)

The corresponding orthogonal basis set {s ₁,…,s _r } is constructed through a QR factorization.

Denote ℙ^k as the L ₂-orthogonal projection onto S ^k. The previous coarse solver G _Δt is now replaced by K _Δt as:

$${K_{\Delta t}}\left( {u,{t_{i + 1}},{t_i}} \right) = {G_{\Delta t}}\left( {\left( {\mathbb{I} - {\mathbb{P}^k}} \right)u,{t_{i + 1}},{t_i}} \right) + {F_{\delta t}}\left( {{\mathbb{P}^k}u,{t_{i + 1}},{t_i}} \right).$$

(7.13)

For a linear problem, F _δt (ℙ^k u,t _i+1, t _i ) can be computed efficiently as

$${F_{\delta t}}\left( {{\mathbb{P}^k}u,{t_{i + 1}},{t_i}} \right) = {F_{\delta t}}(\sum\limits_{j = 1}^{{N_c}k} {{C_j}sj} ,{t_{i + 1}},{t_i}) = \sum\limits_{j = 1}^{{N_c}k} {{C_j}{F_{\delta t}}\left( {sj,{t_{i + 1}},{t_i}} \right)} ,$$

(7.14)

where F _δt (s _j , t _i+1, t _i ) are computed and stored once the s _j ’s are available. Since this approach essentially produces an approximation to the fine solver, the new coarse solver is expected to be more accurate than the old coarse solver. It was shown in [11] that as the dimension of S ^k increases, ${\mathbb{P}^k} \to \mathbb{I}$ and K _Δt → F _δt , thus achieving convergence. The algorithm outline is presented in Algorithm 7.2.

Algorithm 7.2

The Krylov subspace parareal method

To demonstrate the performance of the Krylov subspace parareal method, we use it to solve the linear advection equation, (7.10). In Fig. 7.4 (left) we show the L _∞-error at T = 10 against the number of iterations. It is clear that the Krylov subspace parareal method stabilizes the parareal solver for this problem.

Fig. 7.4

The L _∞-error at T = 10 against the number of iterations (left), and the number of bases (right) for solving the 1-D linear advection equation using the Krylov subspace parareal method

Two observations are worth making. First, the Krylov subspace parareal method needs to store all the values of S ^k and F(S ^k). As k increases, this induces a memory requirement scaling O(kN _c N) and this may be become a bottleneck as illustrated in Fig. 7.4 (right). Furthermore, the efficiency of the coarse solver depends critically on the assumption of linearity of the operator and it is not clear how to extend this framework to nonlinear problems. These constraints appear to limit the practicality of the method.

7.2.3 7.2.3 The reduced basis parareal method

Let us first recall a few properties of reduced basis methods that will subsequently serve as key elements of the proposed reduced basis parareal method.

7.2.3.1 7.2.3.1 Reduced Basis Methods

We are generally interested in solving the nonlinear ODE (7.1). As a system, the dimensionality of the problem can be very large, e.g., if the problem originates from a method-of-lines discretization of a nonlinear PDE, so to achieve a high accuracy, requiring a high number of degrees of freedom, N, and it is tempting to seek to identify an approximate model to enhance the computational efficiency without significantly impacting the accuracy.

A general representation of a reduced model in matrix-form is

$$u\left( t \right) \approx Vr\tilde u\left( t \right),$$

(7.15)

where the r columns of the matrix V _r represent a linear space - the reduced basis - and ũ(t) ∈ ℝ^r are the coefficients of the reduced model. Projecting the ODE system (7.1) onto V _r , we recover the reduced system:

$$V_r^T{V_r}\frac{{d\tilde u\left( t \right)}}{{dt}} = V_r^TA{V_r}\tilde u\left( t \right) + V_r^TN\left( {{V_r}\tilde u\left( t \right)} \right).$$

(7.16)

Assuming that V _r is orthonormal, this simplifies as

$$\frac{{d\tilde u\left( t \right)}}{{dt}} = V_r^TA{V_r}\tilde u\left( t \right) + V_r^TN\left( {{V_r}\tilde u\left( t \right)} \right).$$

(7.17)

One is now left with specifying how to choose a good subspace, V _r , to adequately represent the dynamic behavior of the solution and develop a strategy for how to recover the coefficients for the reduced model in an efficient manner. There are several ways to address this question, most often based on the construction of V _r through snapshots of the solution.

Proper orthogonal decomposition. The proper orthogonal decomposition (POD) [5, 6] is perhaps the most widely used approach to generate a reduced basis from a collection of snapshots. In this case, we assume we have a collection of N _s snapshots

$$U = \left[ {{u_1}, \ldots ,u{N_s}} \right],$$

(7.18)

where each u _i is a vector of length N; this N can be large as it reflects the number of degrees of freedom in system. The POD basis, denoted by {φ _i } ^r₁ ∈ ℝ^N, is chosen as the orthonormal vectors that solve the minimization problem:

$$\begin{array}{*{20}{l}} {\mathop {\min }\limits_{{\phi _i} \in \mathbb{R}N} \sum\limits_j^{{N_s}} {\left\| {uj - \sum\limits_{i = 1}^r {\left( {u_j^T{\phi _i}} \right){\phi _i}} } \right\|_2^2} } \\ {subject\,to\,\phi _i^T{\phi _j} = {\delta _{ij}} = \left\{ {\begin{array}{*{20}{l}} {1,i = j,} \\ {0,\;otherwise.} \end{array}} \right.} \end{array}$$

(7.19)

The solution to this minimization problem is found through the singular value decomposition (SVD) of U:

$$U = V\sum {W^T},$$

(7.20)

where V ∈ ℝ^N×r and $W \in {\mathbb{R}^{{N_s} \times r}}$ are the left and right singular vectors, respectively, and V is the sought after basis. The entries of the diagonal matrix ∑ provides a measure of the relative energy of each of the orthogonal vectors in the basis.

Once the basis is available, we can increase the computational efficiency for solving (7.17) by precomputing V ^T _r AV _r of size r × r. However, the computational complexity of the nonlinear term remains dependent on N and, hence, potentially costly.

Discrete Empirical Interpolation. To address this, [7] proposed an approach, originating in previous work on empirical interpolation methods [4] but limited to the case of an existing discrete basis set. In this approach N(V _{r ũ(t))}) is represented by Ñ(t) ∈ ℝ^N which is subsequently approximated as

$$N\left( {{V_r}\tilde u\left( t \right)} \right) \approx \tilde N\left( t \right) \approx {V_p}c\left( t \right).$$

(7.21)

Here V _p = [v ₁,…,v _m ] is an orthogonal POD basis set based on snapshots of N(t). To recover c(t), we seek a solution to an overdetermined system. However, rather than employing an expensive least square method, we extract m equations from the original set of snapshots. Denote

$$P = \left[ {{e_{p1}}, \ldots ,{e_{{p_m}}}} \right] \in {\mathbb{R}^{N \times m}},$$

(7.22)

where ${e_{{p_1}}} = {[0, \ldots ,0,1,0, \ldots ,0]^T} \in {\mathbb{R}^N}$ (1 only appears on the p ₁-th position of the vector). If P ^T V _p is nonsingular, c(t) can be uniquely determined by

$${P^T}N(t) = {P^T}{V_P}c(t),$$

resulting in a final approximation of Ñ(t) as

$$\tilde N(t) \approx {V_P}{({P^T}{V_P})^{ - 1}}{P^T}N(t).$$

The interpolation index p _i is selected iteratively by minimizing the largest magnitude of the residual r = u _k — V _p,k c. The procedure, sometimes referred to as discrete empirical interpolation, is outlined in Algorithm 7.3.

Algorithm 7.3

Empirical interpolation with a given discrete basis set

With the above approximation, we can now express the reduced system as

$$\frac{{d\tilde u\left( t \right)}}{{dt}} = V_r^TA{V_r}\tilde u\left( t \right) + V_r^T{V_p}{\left( {{P^T}{V_p}} \right)^{ - 1}}N\left( {{P^T}{V_r}\tilde u\left( t \right)} \right).$$

(7.23)

Full Empirical Interpolation. Pursuing the above approach further, one is left wondering if we can use a basis other than the computational expensive POD basis, and whether we can choose the interpolation position based on other guidelines. Addressing these questions leads us to propose a full empirical interpolation method.

It is well-known that the original empirical interpolation method is commonly used to separate the dependence of parameters and spatial variables [4], and that the method chooses ‘optimal’ interpolation points in a certain sense. We propose to consider time as a parameter, and use the empirical interpolation to construct the reduced bases V _E,k of u and the reduced bases V _pE,k of the nonlinear term, i.e.,

$$u\left( t \right) \approx {V_{E,k}}c\left( t \right),\quad \tilde N\left( t \right) \approx {V_{pE,k}}c\left( t \right).$$

(7.24)

The resulting reduced model can be written as

$$\frac{{d\tilde u\left( t \right)}}{{dt}} = {V_{E,k}}^TA{V_{E,k}}\tilde u\left( t \right) + {V_{E,k}}^T{V_{pE,k}}{\left( {{P^T}{V_{pE,k}}} \right)^{ - 1}}N\left( {{P^T}{V_{E,k}}\tilde u\left( t \right)} \right).$$

(7.25)

The essential difference between the models based on discrete empirical interpolation and the full empirical interpolation approach is found in the way in which one constructs the reduced basis set. In the former case, the importance of the basis elements is guided by the SVD and the relative size of the singular values, resulting in a potentially substantial cost. The latter case is based on the interpolation error and the basis in constructed in a full greedy fashion. A detailed comparative study of the performance between the two approaches is ongoing and will be presented in a forthcoming paper.

7.2.3.2 7.2.3.2 The Reduced Basis Parareal Method

Let us now introduce the new reduced basis parareal method. Our first observation is that the first term in (7.13) can be dropped under the assumption that the projection error vanishes asymptotically. Hence, for linear problems, we can replace K _Δt by ${\hat K_{\Delta t}}$ as

$${\hat K_{\Delta t}}\left( {u,{t_{i + 1}},{t_i}} \right) = {F_{\delta t}}\left( {{\mathbb{P}^k}u,{t_{i + 1}},{t_i}} \right) = \sum\limits_{j = 1}^{{N_c}k} {{C_j}{F_{\delta t}}\left( {s\,j,{t_{i + 1}},{t_i}} \right).}$$

(7.26)

This is essentially an approximation to the fine time integrator with an admissible truncation error. Keeping in mind that F _δt is an expensive operation, we seek to reduce the dimension of S ^k to achieve a better efficiency. If the solution to the ODE is sufficiently regular, it is reasonable to seek an r-dimensional subspace, S ^k _r (the reduced basis space), of the original space S ^k. Now redefine ℙ ^k _r to be the orthogonal projection from u onto S ^k _r . Then (7.26) becomes

$${\hat K_{\Delta t}}\left( {u,{t_{i + 1}},{t_i}} \right) = {F_{\delta t}}\left( {\mathbb{P}_r^ku,{t_{i + 1}},{t_i}} \right) = \sum\limits_{j = 1}^r {{C_j}{F_{\delta t}}\left( {s\,j,{t_{i + 1}},{t_i}} \right).}$$

(7.27)

which is essentially an approximation to the fine time integrator using the reduced model.

Consequently, our reduced basis parareal method for linear problems is as follows:

$$u_{n + 1}^{k + 1} = {F_{\delta t}}\left( {\mathbb{P}_r^ku_n^{k + 1},{t_{n + 1}},{t_n}} \right) + {F_\delta }_t\left( {u_n^k,{t_{n + 1}},{t_n}} \right) - {F_{\delta t}}\left( {\mathbb{P}_r^ku_n^k,{t_{n + 1}},{t_n}} \right),\quad 0 \leqslant k \leqslant {N_c} - 1.$$

(7.28)

Depending on the construction of the reduced model, we refer to it as the POD parareal method or the EIM parareal method.

Algorithm 7.4 describes the basic steps of the reduced basis parareal method for linear problems. It follows a procedure similar to Algorithm 7.2, but requires less memory for storing the bases. Notice that for linear problems, the coarse solver is needed only for initializing the algorithm. After this first step, the fine solver produces all the information needed for the reduced model, and the algorithm no longer depends on the coarse solver.

Algorithm 7.4

The reduced parareal method for a linear problem

For nonlinear problems, the relationship

$${F_{\delta t}}\left( {\mathbb{P}_r^ku,{T_{i + 1}},{t_i}} \right) = \sum\limits_{j = 1}^r {{C_j}{F_{\delta t}}\left( {{s_j},{t_{i + 1}},{t_i}} \right)}$$

(7.29)

does not generally hold, even if ℙ^k u → u. Therefore, the Krylov subspace parareal method is not applicable. Fortunately, the knowledge of the development of reduced models using empirical interpolation offers insight into dealing with nonlinear problems, as mentioned in Sect. 7.2.3.1. We construct the coarse time integrator as follows:

$${\hat K_{\Delta t}}\left( {u,{t_{i + 1}},{t_i}} \right) = F_{\delta t}^r\left( {\mathbb{P}_r^ku,{t_{i + 1}},{t_i}} \right),$$

(7.30)

where F ^r_δt is the reduced model constructed by POD or EIM as we described in the previous section. Consequently, our reduced basis parareal method for nonlinear problems becomes

$$\begin{array}{*{20}{c}} {u_{n + 1}^{k + 1} = F_{\delta t}^r\left( {\mathbb{P}_r^ku_n^{k + 1},{t_{n - 1}},{t_n}} \right) + {F_{\delta t}}\left( {u_n^k,{t_{n + 1}},{t_n}} \right) - F_{\delta t}^r\left( {\mathbb{P}_r^ku_n^k,{t_{n + 1}},{t_n}} \right),} \\ {0 \leqslant k \leqslant {N_c} - 1.} \end{array}$$

(7.31)

As long as there exists a suitable reduced model for the problem, we can evaluate ${\hat K_{\Delta t}}$ efficiently while maintaining an accuracy commensurate with the fine solver. The reduced basis parareal method for nonlinear problems is outlined in Algorithm 7.5.

Algorithm 7.5

The reduced parareal method for a nonlinear problem

7.3 7.3 Analysis of the Reduced Basis Parareal Method

In the following we provide some analysis of the reduced basis parareal method to understand its stability, convergence and overall computational complexity. Throughout, we assume that there exists a reduced model for the continuous problem.

7.3.1 7.3.1 Stability analysis

We first consider the linear case. Define the projection error:

$$g_j^k = \left\| {\left( {\mathbb{I} - \mathbb{P}_r^k} \right)u_j^k} \right\|{L_2}\left( {0,T} \right),$$

(7.32)

where r is the dimension of the reduced space. We assume a projection error

$$g_j^k \leqslant \varepsilon ,\quad \forall j,k,$$

(7.33)

and define:

$${C_{p,r}} = \frac{\varepsilon }{{\Delta T}},\quad \forall j,k.$$

(7.34)

It is reasonable to assume that the fine propagator is L ₂ stable, i.e., there exists a nonnegative constant C _F independent of the discretization parameters, such that,

$${\left\| {{F_{\delta t}}\left( {v,{t_{i + 1}},{t_i}} \right)} \right\|_{{L_2}\left( {0,T} \right)}} \leqslant \left( {1 + {C_F}\Delta T} \right){\left\| v \right\|_{{L_2}\left( {0,T} \right)}},\quad \forall v \in {L_2}\left( {0,T} \right).$$

(7.35)

Theorem 7.1 (Stability for the linear case) Under the assumption of (7.33) and (7.35), the reduced basis parareal method is stable for (7.1) with N ≡ 0, i.e., for each i and k,

$${\left\| {u_{i + 1}^{k + 1}} \right\|_{{L_2}\left( {0,T} \right)}} \leqslant {C_L}{e^{{C_F}\left( {i + 1} \right)\Delta T}},$$

(7.36)

where C_L is a constant depending only on C_p,r, C_F, and u ₀

Proof Using the triangle inequality, linearity of the operator, and assumption (7.35), we obtain

$$\begin{array}{*{20}{c}} {{{\left\| {u_{i + 1}^{k + 1}} \right\|}_{{L_2}\left( {0,T} \right)}} \leqslant {{\left\| {F\delta \left( {\mathbb{P}_r^ku_i^{k + 1},{t_{i + 1}},{t_i}} \right)} \right\|}_{L2\left( {0,T} \right)}} + \left\| {{F_{\delta t}}\left( {u_i^k,{t_{i + 1}},{t_i}} \right)} \right.} \\ {{{\left. { - {F_{\delta t}}\left( {\mathbb{P}_r^ku_i^k,{t_{i + 1}},{t_i}} \right)} \right\|}_{{L_2}\left( {0,T} \right)}}} \end{array}$$

(7.37)

$$\begin{array}{*{20}{c}} { \leqslant \left( {1 + {C_F}\Delta T} \right){{\left\| {u_i^{k + 1}} \right\|}_{{L_2}\left( {0,T} \right)}}} \\ { + \left( {1 + {C_F}\Delta T} \right){{\left\| {\mathbb{I} - \mathbb{P}_r^ku_i^k} \right\|}_{{L_2}\left( {0,T} \right)}}} \end{array}$$

(7.38)

Then, by the discrete Gronwall’s lemma [9] and (7.33), we recover

$$\begin{array}{*{20}{l}} {{{\left\| {u_{i + 1}^{k + 1}} \right\|}_{{L_2}\left( {0,T} \right)}} \leqslant {{\left( {1 + {C_F}\Delta T} \right)}_{i + 1}}} \\ {\quad \quad \quad \quad \quad \times ({{\left\| {u_0^{k + 1}} \right\|}_{{L_2}\left( {0,T} \right)}} + \Delta T{{\sum\limits_{j = 0}^i {\left( {1 + {C_F}\Delta T} \right)} }^{ - j}}{C_{p,r}})} \end{array}$$

(7.39)

$$\begin{array}{*{20}{l}} { = {{\left( {1 + {C_F}\Delta T} \right)}^{i + 1}}{{\left\| {u_0^{k + 1}} \right\|}_{{L_2}\left( {0,T} \right)}}} \\ { + \frac{1}{{{C_F}}}\left( {{{\left( {1 + {C_F}\Delta T} \right)}^{i + 1}} - 1} \right){C_{p,r}}} \end{array}$$

(7.40)

$$\leqslant {e^{{C_F}\left( {i + 1} \right)\Delta T}}{\left\| {{u_0}} \right\|_{{L_2}\left( {0,T} \right)}} + \frac{1}{{{C_F}}}\left( {{e^{{C_F}\left( {i + 1} \right)\Delta T}} - 1} \right){C_{p,r}}.$$

(7.41)

This completes the proof.

Note that if there exists an small integer M (indicating a compact reduced approximation space) such that,

$$\mathop {\lim }\limits_{r \to M} {C_{p,r}} = 0,$$

(7.42)

then we recover the same stability property as that of the fine solver:

$${\left\| {u_{i + 1}^{k + 1}} \right\|_{{L_2}(0,T)}} \leqslant {e^{{C_F}(i + 1)\Delta T}}{\left\| {{u_0}} \right\|_{{L_2}(0,T)}}.$$

For the nonlinear case, we further assume that there exists a nonnegative constant C _r , independent of the discretization parameters, such that,

$${\left\| {{F_{\delta t}}\left( {v,{t_{i + 1}},{t_i}} \right) - F_{\delta t}^r\left( {\mathbb{P}_r^kv,{t_{i + 1}},{t_i}} \right)} \right\|_{{L_2}\left( {0,T} \right)}} \leqslant \left( {1 + {C_r}\Delta T} \right)q_i^k,\quad \forall v \in {L_2}\left( {0,T} \right),$$

(7.43)

where q ^k _i is the L ₂-difference between the fine propagator and the reduced model using the same initial condition v at t _i . As before, we assume

$$q_j^k \leqslant \varepsilon ,\quad \forall j,k.$$

(7.44)

Theorem 7.2 (Stability for the nonlinear case) Under assumptions (7.35), (7.43) and (7.44), the reduced basis parareal method is stable for (7.1) in the sense that for each i and k

$$\left\| {u_{i + 1}^{k + 1}} \right\|{L_2}_{\left( {0,T} \right)} \leqslant {C_N}{e^{{C_ \star }\left( {i + 1} \right)\Delta T}},$$

(7.45)

where C _⋆ = max{C _F ,C _r } and C _N is a constant depending only on C _p,r , C _F , C _r, and u ₀.

Proof Using the triangle inequality and assumptions (7.35) and (7.43), we have

$$\begin{array}{*{20}{c}} {{{\left\| {u_{i + 1}^{k + 1}} \right\|}_{{L_2}\left( {0,T} \right)}} \leqslant {{\left\| {F_{\delta t}^r\left( {\mathbb{P}_r^ku_i^{k + 1},{t_{i + 1}},{t_i}} \right)} \right\|}_{{L_2}\left( {0,T} \right)}} + \left\| {{F_{\delta t}}\left( {u_i^k,{t_{i + 1}},{t_i}} \right)} \right.} \\ { - F_{\delta t}^r\left( {\mathbb{P}_r^ku_i^k,{t_{i + 1}},{t_i}} \right)\left\| {_{{L_2}\left( {0,T} \right)}} \right.} \end{array}$$

(7.46)

$$\leqslant \left( {1 + {C_F}\Delta T} \right){\left\| {u_i^{k + 1}} \right\|_{{L_2}\left( {0,T} \right)}} + \left( {1 + {C_r}\Delta T} \right)q_i^k.$$

(7.47)

Next, by the discrete Gronwall’s lemma and (7.44), we derive

$$\begin{array}{*{20}{l}} {{{\left\| {u_{i + 1}^{k + 1}} \right\|}_{{L_2}\left( {0,T} \right)}} \leqslant {{\left( {1 + {C_F}\Delta T} \right)}^{i + 1}}} \\ {\quad \quad \quad \quad \quad \times \left( {{{\left\| {u_0^{k + 1}} \right\|}_{{L_2}\left( {0,T} \right)}} + \Delta T\sum\limits_{j = 0}^i {{{\left( {1 + {C_r}\Delta T} \right)}^{ - j}}{C_{p,r}}} } \right)} \end{array}$$

(7.48)

$$\begin{array}{*{20}{c}} { = {{\left( {1 + {C_F}\Delta T} \right)}^{i + 1}}{{\left\| {u_0^{k + 1}} \right\|}_{{L_2}\left( {0,T} \right)}}} \\ { + \frac{1}{{{C_r}}}\left( {{{\left( {1 + {C_r}\Delta T} \right)}^{i + 1}} - 1} \right){C_{p,r}}} \end{array}$$

(7.49)

$$\leqslant {e^{{C_F}\left( {i + 1} \right)\Delta T}}{\left\| {{u_0}} \right\|_{{L_2}\left( {0,T} \right)}} + \frac{1}{{{C_r}}}\left( {{e^{{C_r}\left( {i + 1} \right)\Delta T}} - 1} \right){C_{p,r}}.$$

(7.50)

This completes the proof.

7.3.2 7.3.2 Convergence analysis

To show convergence for the linear case, we first assume that there exists a nonnegative constant C _F , such that,

$${\left\| {{F_{\delta t}}\left( {x,{t_{i + 1}},{t_i}} \right) - {F_{\delta t}}\left( {y,{t_{i + 1,}}{t_i}} \right)} \right\|_{{L_2}\left( {0,T} \right)}} \leqslant \left( {1 + {C_F}\Delta T} \right){\left\| {x - y} \right\|_{{L_2}\left( {0,T} \right)}},\quad \forall {t_i} > 0.$$

(7.51)

We define

$$w_j^k = {\left\| {\left( {\mathbb{I} - \mathbb{P}_r^k} \right){u_j}} \right\|_{{L_2}\left( {0,T} \right)}},$$

(7.52)

and assume that

$$w_j^k \leqslant \varepsilon ,\quad \forall j,k.$$

(7.53)

Theorem 7.3 (Convergence for the linear case) Under assumption (7.33), (7.42), (7.51), (7.53) and N ≡ 0 in (7.1), the reduced basis parareal solution converges to u _i+1 for each i.

Proof Using the reduced basis parareal formula and the linearity of the operator, we obtain

$$\begin{array}{*{20}{r}} {u_{i + 1}^{k + 1} - {u_{i + 1}} = {F_{\delta t}}\left( {\mathbb{P}_r^ku_i^{k + 1},{t_{i + 1}},{t_i}} \right) + {F_{\delta t}}\left( {u_i^k,{t_{i - 1}},{t_i}} \right)} \\ { - {F_{\delta t}}\left( {\mathbb{P}_r^ku_i^k,{t_{i + 1,}}{t_i}} \right) - {F_{\delta t}}\left( {{u_i},{t_{i + 1}},{t_i}} \right)} \end{array}$$

(7.54)

$$= {F_{\delta t}}\left( {\mathbb{P}_r^ku_i^{k + 1},{t_{i + 1,}}{t_i}} \right) - {F_{\delta t}}\left( {\mathbb{P}_r^k{u_i},{t_{i + 1}},{t_i}} \right)$$

(7.55)

$$+ {F_{\delta t}}\left( {u_i^k,{t_{i + 1}},{t_i}} \right) - {F_{\delta t}}\left( {\mathbb{P}_r^ku_i^k,{t_{i + 1}},{t_i}} \right)$$

(7.56)

$$+ {F_{\delta t}}\left( {\mathbb{P}_r^k{u_i},{t_{i + 1}},{t_i}} \right) - {F_{\delta t}}\left( {{u_i},{t_{i + 1}},{t_i}} \right)$$

(7.57)

By the triangular inequality and assumption (7.51), we recover

$${\left\| {u_{i + 1}^{k + 1} - {u_{i + 1}}} \right\|_{{L_2}\left( {0,T} \right)}} \leqslant \left( {1 + {C_F}\Delta T} \right){\left\| {u_i^{k + 1} - {u_i}} \right\|_{{L_2}\left( {0,T} \right)}}$$

(7.58)

$$ + \left( {1 + {C_F}\Delta T} \right){\left\| {\left( {\mathbb{I} - \mathbb{P}_r^k} \right)u_i^k} \right\|_{{L_2}\left( {0,T} \right)}}$$

(7.59)

$$+ \left( {1 + {C_F}\Delta T} \right){\left\| {\left( {\mathbb{I} - \mathbb{P}_r^k} \right){u_i}} \right\|_{{L_2}\left( {0,T} \right)}}.$$

(7.60)

Finally by the discrete Gronwall’s lemma, (7.33) and (7.53), we obtain

$$\begin{array}{*{20}{c}} {{{\left\| {u_{i + 1}^{k + 1} - {u_{i + 1}}} \right\|}_{{L_2}\left( {0,T} \right)}} \leqslant {{\left( {1 + {C_F}\Delta T} \right)}^{i + 1}}{{\left( {\left\| {u_0^{k - 1} - {u_0}} \right\|} \right.}_{{L_2}\left( {0,T} \right)}}} \\ { + \Delta T\sum\limits_{j = 0}^i {{{\left( {1 + {C_F}\Delta T} \right)}^{ - j}}{C_{p,r}}} } \end{array}$$

(7.61)

$$+ \Delta T\sum\limits_{j = 0}^i {{{\left( {1 + {C_F}\Delta T} \right)}^{ - j}}\left. {{C_{p,r}}} \right)} $$

(7.62)

$$\leqslant 2\Delta T\sum\limits_{j = 0}^i {{{\left( {1 + {C_F}\Delta T} \right)}^{ - j}}{C_{p,r}}}$$

(7.63)

$$\leqslant \frac{2}{{{C_F}}}\left( {{{\left( {1 + {C_F}\Delta T} \right)}^{i + 1}} - 1} \right){C_{p,r}}$$

(7.64)

$$\leqslant \frac{2}{{{C_F}}}\left( {{e^{{C_F}\left( {i + 1} \right)\Delta T}} - 1} \right){C_{p,r}},$$

(7.65)

which approaches zero as r increases. This completes the proof.

For the nonlinear case, we must also assume that there exists a nonnegative constant C _r , such that,

$$\begin{array}{*{20}{l}} {{{\left\| {{F_{\delta t}}\left( {u_i^k,{t_{i + 1}},{t_i}} \right) - F_{\delta t}^r\left( {\mathbb{P}_r^ku_i^k,{t_{i + 1}},{t_i}} \right)} \right\|}_{{L_2}\left( {0,T} \right)}} \leqslant \left( {1 + {C_r}\Delta T} \right)q_i^k,} \\ {\left\| {{F_{\delta t}}\left( {{u_i},{t_{i + 1}},{t_i}} \right) - F_{\delta t}^r\left( {\mathbb{P}_r^k{u_i},{t_{i + 1}},{t_i}} \right)} \right\|{L_2}\left( {0,T} \right) \leqslant \left( {1 + {C_r}\Delta T} \right)p_i^k,} \end{array}$$

(7.66)

where q ^k _i and p ^k _i represent the L ₂-difference between the fine operator and the reduced solver using the same initial condition u ^k _i and u _i . As before, we assume that

$$p_j^k \leqslant \varepsilon ,\quad \forall j,k.$$

(7.67)

Theorem 7.4 (Convergence of the nonlinear case) Under assumptions (7.42), (7.43), (7.44), (7.66) and (7.67), the reduced basis parareal solution of (7.1) converges to u _i+1 for each i.

Proof Using the reduced basis parareal formula, we obtain

$$\begin{array}{*{20}{r}} {u_{i + 1}^{k + 1} - {u_{i + 1}} = F_{\delta t}^r\left( {\mathbb{P}_r^ku_i^{k + 1},{t_{i + 1}},{t_i}} \right) + {F_{\delta t}}\left( {u_i^k,{t_{i + 1}},{t_i}} \right)} \\ { - F_{\delta t}^r\left( {\mathbb{P}_r^ku_i^k,{t_{i + 1}}{t_i}} \right) - {F_{\delta t}}\left( {{u_{i,}}{t_{i + 1}},{t_i}} \right)} \end{array}$$

(7.68)

$$\begin{array}{*{20}{c}} { = F_{\delta t}^r\left( {\mathbb{P}_r^ku_i^{k + 1},{t_{i + 1}},{t_i}} \right) - F_{\delta t}^r\left( {\mathbb{P}_r^k{u_i},{t_{i + 1}},{t_i}} \right)} \\ { + {F_{\delta t}}\left( {u_i^k,{t_{i + 1,}}{t_i}} \right) - F_{\delta t}^r\left( {\mathbb{P}_r^ku_i^k,{t_{i + 1}},{t_i}} \right)} \\ { + F_{\delta t}^r\left( {\mathbb{P}_r^k{u_i},{t_{i + 1}},{t_i}} \right) - {F_{\delta t}}\left( {{u_i},{t_{i + 1}},{t_i}} \right).} \end{array}$$

(7.69)

By the triangular inequality and assumptions (7.66) and (7.66), we have

$$\begin{array}{*{20}{r}} {{{\left\| {u_{i + 1}^{k + 1} - {u_{i + 1}}} \right\|}_{{L_2}\left( {0,T} \right)}} \leqslant \left( {1 + {C_F}\Delta T} \right)\left\| {u_i^{k + 1} - {u_i}} \right\|{L_2}\left( {0,T} \right)} \\ { + \left( {1 + {C_r}\Delta T} \right)q_i^k + \left( {1 + {C_r}\Delta T} \right)p_i^k.} \end{array}$$

(7.70)

Then, by the discrete Gronwall’s lemma, (7.44) and (7.67) we recover

$${\left\| {u_{i + 1}^{k + 1} - {u_{i + 1}}} \right\|_{{L_2}\left( {0,T} \right)}} \leqslant \frac{2}{{{C_r}}}\left( {{{\left( {1 + {C_r}\Delta T} \right)}^{i + 1}} - 1} \right){C_{p,r}}$$

(7.71)

$$\leqslant \frac{2}{{{C_r}}}\left( {{e^{{C_r}\left( {i + 1} \right)\Delta T}} - 1} \right){C_{p,r}},$$

(7.72)

which approaches zero as r increases under assumption (7.42).

For the above analysis it is worth emphasizing two points:

• The accuracy of the new parareal algorithm is O(ε), since C _p,r depends on ε as a measure of the quality of the reduced model. We shall confirm this point by the numerical tests in Sect. 7.4.

• Theorem 7.3 and 7.4 indicate that if there exists a good reduced approximation space for the problem, the new parareal algorithm converges in one iteration.

7.3.3 7.3.3 Complexity Analysis

Let us finally discuss the computational complexity of the reduced basis parareal method. Recall that the dimension of the reduced space is r and that of the fine solution is N. This is assumed to be the same for the coarse and fine solvers although this may not be a requirement in general. The compression ratio is R = r/N. Following the notation of [21]: τ _QR (k), τ _RB (k) (representing τ _SVD (k), τ _EIM (k), and τ _DEIM (k) in different scenarios) reflect computing times required by the corresponding operations at the k-th iteration. τ _c and τ _f is the time required by the coarse and fine solvers, respectively. N _t = N _c N _f is the total number of time steps in one iteration with N _c being the number of the coarse time intervals and N _f the number of fine time steps on each coarse time interval. N _p is the number of processors.

In [21], the speedup is estimated as

$$S\left( {Np} \right) \approx \frac{{{N_t}{\tau _f}}}{{{N_c}{\tau _c} + {N_{it}}\left( {{N_c}{\tau _c} + {N_t}/{N_p}{\tau _f}} \right){N_{it}}{\tau _{QR}}\left( {it} \right)}}$$

(7.73)

$$= \frac{1}{{\left( {1 + {N_{it}}} \right)\left( {\frac{{{N_c}}}{{{N_t}}}\frac{{{\tau _c}}}{{{N_t}}}} \right) + \frac{{{N_{it}}{\tau _{QR}}\left( {{N_{it}}} \right)}}{{{N_i}\tau f}} + \frac{{{N_{it}}}}{{{N_p}}}}}.$$

(7.74)

In the reduced basis parareal method, τ _c = R ²τ _f , since the complexity of the computation of the right hand side of system is O(r ²). In addition, τ _QR becomes τ _SVD or τ _EIM . With this in mind, the speedup can be estimated as

$$S\left( {{N_p}} \right) = \frac{1}{{\left( {1 + {N_{it}}} \right)\left( {\frac{{{N_c}}}{{{N_t}}}{R^2}} \right) + \frac{{{N_{it}}{\tau _{RB}}\left( {{N_{it}}} \right)}}{{{N_i}{\tau _f}}} + \frac{{{N_{it}}}}{{{N_p}}}}}.$$

(7.75)

Next, we examine the first two terms in the denominators of (7.74) and (7.75).

• In the first term, τ _c /τ _f takes the role of R ². Hence, we can achieve a comparable performance, if $R \approx \sqrt {{\tau _c}/{\tau _f}}$, i.e, if the underlying PDE solution can be represented by a reduced basis set of size $O\left( {\sqrt {{\tau _c}/{\tau _f}} N} \right)$. Suppose that $\sqrt {{\tau _c}/{\tau _f}} = \sqrt {1/20} \approx 0.23$. This requires that R < 1/4, which is a reasonable compression ratio for many problems. In addition, it is possible to use a reduced basis approximation to achieve a better performance for cases where CFL conditions lead to restrictions for the coarse solver.

• For the second term, τ _SVD ≈ τ _QR ≈ O(NN ² _it N ² _c ), while τ _EIM ≈ O(r ³/2N _it N _c + rNN _it N _c ). Therefore, τ _SVD /τ _EIM ≈ O(2N _it N _c /Rr ²). As N _c increases, τ _EIM becomes smaller. In addition, EIM has very good parallel efficiency and requires less memory during the computation.

Also note that N _it would typically be different for the reduced basis parareal method and the original parareal method. If a reduced space exists, the modified algorithm usually converges within a few iterations, hence accelerating the overall convergence significantly.

7.4 7.4 Numerical Results

In the following, we demonstrate the feasibility and efficiency of the reduced basis parareal method for both linear and nonlinear problems. We generally use the solution obtained from the fine time integrator as the exact solution.

7.4.1 7.4.1 The Linear Advection Equation

We begin by considering the performance of the reduced basis parareal method and illustrate that it is stable for the 1-D linear advection equation (7.10). The spatial and temporal discretizations are the same as used in Sect. 7.2 and parameters in (7.11) are used.

In Fig. 7.5 (left), we show the L _∞-error at T = 10 against the number of iterations for the original parareal method, the POD parareal method, and the EIM parareal method. The accuracy of the fine time integrator at T = 10 is 4 × 10⁻¹³. The original parareal method is clearly unstable, while the other two remain stable. The very rapid convergence of the reduced basis parareal method reflects that the accuracy of reduced model is very high for this simple test case. As we will see for more complex nonlinear problems, this behavior does not carry over to general problems unless a high-accuracy reduced model is available.

Fig. 7.5

POD parareal method, and the EIM parareal method for the 1-D advection equation. On the left we show the L _∞-error at T = 10 against the number of iterations while the right shows the number of bases used for satisfying the tolerance of ε(10⁻¹³) in the POD and EIM parareal methods across the iterations

In Fig. 7.5 (right), we show the number of bases used to satisfy the tolerance ε in the POD parareal method and the EIM parareal method. Here ε in the POD context is defined as the relative energy in the truncated mode and in the EIM context it is the interpolation error. In both cases, the tolerance in the basis selection using POD or EIM is set to 10⁻¹³.We note that the EIM parareal method achieves higher accuracy but requires more memory to store the bases. This suggests that one can explore a tradeoff between accuracy and efficiency for a particular application.

Remark 7.1 It should be noted that if only snapshots from the previous iteration is used in the EIM basis construction, the scheme becomes unstable. However, when including all snapshots collected up to the previous iteration level, stability is restored.

Figure 7.6 (upper left) shows the convergence behavior of the EIM parareal algorithm with different tolerances (ε = 10^−k,k = 2,4,6,8,10, 12). The convergence stagnates at a certain level and instability may set in after further iterations. There are two reasons for this: 1) as ε becomes small, the reduced bases may become linear dependent, leading to a bad condition number of the related matrices that may impact stability; 2) the newly evolved reduced bases ${S_{{f_i}}}$ for the fine solution may not be within S anymore. To resolve this problem, we first perform the reorthogonalization of the reduced bases to obtain a new space $\tilde S$ and then project the newly evolved solution ${\hat K_{\Delta t}}(u_i^{k + 1},{t_{i + 1}},{t_i})$ back to $\tilde S$. In Fig. 7.6 (lower left) we show the convergence results following this approach. Most importantly, stability is restored. Furthermore, the dependence of the final accuracy on ε is clear. These results are consistent with Theorem 7.3, stating that the parareal solution converges to the serial solution integrated by the fine solver as long as the subspace S saturates in terms of accuracy. In practice, one can choose ε such that the accuracy of the parareal solution and the serial fine solution are comparable.

Fig. 7.6

The performance of the EIM parareal method for the 1-D advection equation against the tolerance used in the design of the reduced basis. On the upper left we show the L∞-error at T = 10 against the number of iterations as the tolerance ε decreases and on the upper right the number of bases used for satisfying the tolerance as ε decreases, where ε = 10^−k,k = 2,4,6,8,10, 12; On the lower left and right, we show the corresponding convergence results and the number bases with the reorthogonalization procedure of the evolved basis

7.4.2 7.4.2 The second order wave equation

To further evaluate the stability of the new parareal algorithm, we consider the second-order wave equation from [8]:

$$\begin{array}{*{20}{l}} {{u_{tt}} = {c^2}{u_{xx}},\quad \left( {x,t} \right) \in \left( {o,2\pi } \right) \times \left( {0,T} \right],} \\ {u\left( {x,0} \right) = f\left( x \right),\quad {u_t}\left( {x,0} \right) = g\left( x \right),} \end{array}$$

(7.76)

where T = 10 and c = 5 and a 2π-periodic boundary condition is used. The initial conditions are set as

$$\begin{array}{*{20}{c}} {f\left( x \right) = \sum\limits_{1 = - N}^N {{{\hat u}_l}{e^{ilx}},\quad g\left( x \right) = 0} } \\ {and} \\ {{{\hat u}_l} = \left\{ {\begin{array}{*{20}{c}} {\frac{1}{{{{\left| I \right|}^p}}},1 \ne 0,} \\ {0\quad 1 = 0.} \end{array}} \right.} \end{array}$$

(7.77)

and

$${\hat u_l} = \left\{ \begin{gathered} \tfrac{1}{{|l{|^p}}},l \ne 0, \hfill \\ 0\quad l = 0. \hfill \\ \end{gathered} \right.$$

and set p = 4. In the following we use a Fourier spectral discretization with 33 modes in space [14] and the velocity Verlet algorithm in time [24]. The following parameters are used in the parareal algorithm:

$${N_c} = 100,\quad {N_{it}} = 10,\quad \Delta t = {10^{ - 3}},\delta t = {10^{ - 4}}.$$

(7.78)

The tolerance for POD is set to 10⁻¹¹, respectively.

In Fig. 7.7 (left), we show the L _∞-error at T = 10 against the number of iterations for the original parareal method and the POD parareal method. The original parareal method is clearly unstable, while the POD parareal remains stable and converges in one iteration. This confirms our analysis: if the reduced model is accurate enough, the reduced basis parareal should converge in one iteration. In Fig. 7.5 (right), we show the number of bases needed to satisfy the tolerance ε in the POD parareal method.

Fig. 7.7

Results obtained using the original parareal method, the POD parareal method for the 1-D second order wave equation. On the left we show the L _∞-error at T = 10 against the number of iterations while the right shows the number of bases used for satisfying the tolerance of ε(10⁻¹¹) in the POD parareal method across the iterations

7.4.3 7.4.3 Nonlinear Equations

Let us also apply the reduced basis parareal method to examples with nonlinear PDEs. We recall that the Krylov based approach is not applicable in this case.

7.4.3.1 7.4.3.1 Viscous Burgers’ Equation

We first consider the viscous Burgers’ equation (7.6), with the same spatial and temporal discretization and the same parameters as in (7.7). To build the reduced basis, we set the tolerance for POD and EIM to be 10⁻¹⁵ and 10⁻¹⁰, respectively.

In Fig. 7.8 (left), we show the L _∞-error at T = 2 against the number of iterations for the original parareal method, the POD parareal method, and the EIM parareal method. Note that in this case, the RB parareal performs worse than the original parareal does. It is a result of the reduced model not adequately capturing the information of the fine solver. Recall that in the nonlinear case, we have to deal with two approximations: one for the state variables and one for the nonlinear term. For the POD parareal algorithm, we choose the number of reduced bases based on the tolerance for the state variable u; alternatively, we can choose the dimension of the reduced approximation space based on the tolerance for the nonlinear term. The latter approach shows better convergence behavior in Fig. 7.8 (left, parareal-podmodified). It is apparent that the quality of the reduced model directly impacts the convergence.

Fig. 7.8

We compare the performance of the original parareal method, the POD parareal method, the modified POD parareal and the EIM parareal method for the 1-D Burgers’ equation. On the left we show the L _∞-error at T = 2 against the number of iterations, while the right illustrates the number of bases

We emphasize that although the reduced basis parareal method converges slower than the original parareal, it is less expensive, as discussed in Sect. 7.2.3.1.

7.4.3.2 7.4.3.2 Kuramoto-Sivashinsky Equation

Next we consider the Kuramoto-Sivashinsky equation (7.8). The same spatial and temporal discretization and the same parameters as in (7.9) are used. To build the reduced basis, we set the tolerance for POD and EIM to be 10⁻¹³ and 10⁻⁸, respectively.

In Fig. 7.9 we show the L _∞-error at T = 40 against the number of iterations for the original parareal method, the POD parareal method, the modified POD parareal, and the EIM parareal method. It is clear that the reduced basis parareal method converges faster than the original parareal method. This is likely caused by the solution of the problem being smooth enough to ensure that there exists a compact reduced model. Moreover, to keep the corresponding tolerance, the number of degrees of freedom in the reduced basis parareal methods is roughly one-third that of the original parareal method.

Fig. 7.9

We compare the performance of the original parareal method, the POD parareal method, and the EIM parareal method for the 1-D Kuramoto-Sivashinsky equation. On the left we show the L _∞-error at T = 40 against the number of iterations, while the right shows the number of bases used against the number of iterations

7.4.3.3 7.4.3.3 Allan-Cahn Equation: Nonlinear Source

As a third nonlinear example we consider the 1-D Allan-Cahn equation:

$$\begin{array}{*{20}{l}} {\frac{{\partial u}}{{\partial t}} = v{u_{xx}} + u - {u^3},\quad \left( {x,t} \right) \in \left( {0,2\pi } \right) \times \left( {0,T} \right]} \\ {u\left( {x,0} \right) = 0.25\sin \left( x \right),} \end{array}$$

(7.79)

where T = 2 and v = 2,1, 10⁻¹, 10⁻². A periodic boundary condition is assumed. We use a P ₁ DG method with 100 elements in space [15] and a forward Euler scheme in time. The following parameters are used in the parareal algorithm

$${N_c} = 200,\quad {N_{it}} = 5,\quad \Delta t = 1 \times {10^{ - 4}},\quad \delta t = 5 \times {10^{ - 6}}.$$

(7.80)

We set the tolerance for POD and EIM to be 10⁻¹² and 10⁻⁸, respectively.

In Fig. 7.10 (left), we show the L _∞-error at T = 2 against the number of iterations for the POD parareal method for different values of v’s. It is clear that for larger values of v, the solution converges faster and less elements in the reduced basis is needed. This is expected since a larger v indicates a smoother and more localized solution which is presumed to allow for an efficient representation in a lower dimensional space. Similar results are obtained by an EIM based parareal approach and are not reproduced here.

Fig. 7.10

We compare the performance of the original parareal method, the POD parareal method, and the EIM parareal method for the 1-D Kuramoto-Sivashinsky equation. On the left we show the L _∞-error at T = 40 against the number of iterations, while the right shows the number of bases used against the number of iterations

7.4.3.4 7.4.3.4 KdV Equation: Nonlinear Flux

As a last example we consider the KdV equation (taken from [26]):

$$\begin{array}{*{20}{l}} {\frac{{\partial u}}{{\partial t}} = - (\frac{{{u^2}}}{2})x - v{u_{xxx}},\quad \left( {x,t} \right) \in \left( { - 1,1} \right) \times \left( {0,T} \right],} \\ {u\left( {x,0} \right) = 1.5 + 0.5\sin \left( {2\pi x} \right),} \end{array}$$

(7.81)

where T = 2 and v = 10⁻³ and we assume a periodic boundary condition. The equation conserves energy, much like the linear wave equation, but the nonlinearity induces a more complex behavior with the generation of propagating waves. In the parareal algorithm we use

$${N_c} = 100,\quad {N_{it}} = 10m,\quad \Delta t = {10^{ - 4}},\quad \delta t = {10^{ - 5}}.$$

(7.82)

We use a first order local discontinuous Galerkin method (LDG) with 100 elements in space [15, 26] and an IMEX scheme in time [1], with the linear terms treated implicitly and the nonlinear term explicitly. We set the tolerance for POD and EIM to be 10⁻¹³ and 10⁻⁸, respectively.

In Fig. 7.11 (left) we show the L _∞-error at T = 2 against the number of iterations for the original parareal method, the POD parareal method, and the EIM parareal method. While the POD parareal method does not work well in this case, the EIM parareal method shows remarkable performance, i.e., it converges much faster than the original parareal method. Note that even if the tolerance for the POD is smaller than that of the EIM, it does not guarantee that the reduced model error based on the POD approach is smaller. There are two reasons: 1) the meaning of the tolerance in the context of the POD and the EIM are different. 2) in the convergence proof of (7.71), the constants C _r ,C _p,r depend on the details of the reduced approximation and the dimension of reduced approximation space, which impact the final approximation error.

Fig. 7.16

We compare the performance of the original parareal method, the POD parareal method, and the EIM parareal method for the 1-D KdV equation. On the left we show the L _∞-error at T = 2 against the number of iterations, while the right shows the number of bases used against the number of iterations

7.5 7.5 Conclusions

In this paper, we propose an approach to produce and use a reduced basis method to replace the coarse solver in the parareal algorithm. We demonstrate that, as compared with the original parareal method, this new reduced basis parareal method has improved stability characteristics and efficiency, provided that the solution can be represented well by a reduced model. The analysis of the method is confirmed by the computational results, e.g., the accuracy of the parareal method is determined by the accuracy of the fine solver and the reduced model, used to replace the coarse solver. Unlike the Krylov subspace parareal method, this approach can be extended to include both linear problems and nonlinear problems, while requiring less storage and computing resources. The robustness and versatility of the method has been demonstrated through a number of different problems, setting the stage for the evaluation on more complex problems.

Acknowledgements The authors acknowledge partial support by OSD/AFOSR FA9550-09-1-0613 and AFOSR FA9550-12-1-0463.