The Approximation of Invariant Sets in Infinite Dimensional Dynamical Systems

Abstract

In this work we review the novel framework for the computation of finite dimensional invariant sets of infinite dimensional dynamical systems developed in [6] and [36]. By utilizing results on embedding techniques for infinite dimensional systems we extend a classical subdivision scheme [8] as well as a continuation algorithm [7] for the computation of attractors and invariant manifolds of finite dimensional systems to the infinite dimensional case. We show how to implement this approach for the analysis of delay differential equations and partial differential equations and illustrate the feasibility of our implementation by computing the attractor of the Mackey-Glass equation and the unstable manifold of the one-dimensional Kuramoto-Sivashinsky equation.

1 Introduction

In order to understand the long term behavior of complicated nonlinear dynamical systems, a promising approach is to study invariant sets such as the global attractor and invariant manifolds. To numerically approximate these sets set-oriented methods have been developed  [7,8,9, 17]. The underlying idea is to cover the set of interest by outer approximations that are generated by multilevel subdivision or continuation methods. They have been used successfully in various application areas such as molecular dynamics  [30], astrodynamics  [11] or ocean dynamics  [18].

Until recently, the applicability of the subdivision scheme and the continuation method was restricted to finite dimensional dynamical systems, i.e., ordinary differential equations or finite dimensional discrete systems. In this work we show how to extend these algorithms for the computation of attractors as well as invariant manifolds to the infinite dimensional context, e.g. delay differential equations with small delays [3, 12] and dissipative partial differential equations such as the Kuramoto-Sivashinsky equation [25, 32], the Ginzburg-Landau equation and reaction-diffusion equations [22]. For all these systems a finite dimensional so-called inertial manifold exists to which trajectories are attracted exponentially fast, e.g., [4, 15, 34].

The novel approach utilizes infinite dimensional embedding results [21, 28] that allow the reconstruction of finite dimensional invariant sets of infinite dimension dynamical systems. These results extend the results of Takens [33] and Sauer et al. [29] to the infinite dimensional context. The so–called observation map which consists of observations of the systems dynamics produces – in a generic sense – a one-to-one image of the underlying (infinite dimensional) dynamics provided the number of observations is large enough. This observation map and its inverse are then used for the construction of the core dynamical system (CDS), i.e., a continuous dynamical system on a state space of finite dimension. By construction the CDS possesses topologically the same dynamical behavior on the invariant set as the original infinite dimensional system. Then, application of the subdivision scheme and the continuation method allow us to compute the reconstructed invariant set of the CDS. The general numerical approach is in principle applicable to infinite dimensional dynamical systems described by a Lipschitz continuous operator on a Banach space. However, here we will restrict our attention to delay differential equations with constant delay and partial differential equations for the numerical realization. We note that in [35] the approach has been generalized to delay differential equations with state dependent time delay.

A detailed outline of the article is as follows. In Sect. 2 we briefly summarize the infinite dimensional embedding theory introduced in [21, 28]. In Sect. 3 we employ this embedding technique for the construction of the CDS. Then in Sect. 4 we review the adapted subdivision scheme and continuation method for infinite dimensional systems developed in [6, 36]. A numerical realization of the CDS for DDEs and PDEs is given in Sect. 5. Finally, in Sect. 6 we illustrate the efficiency of our methods for the Mackey-Glass delay differential equation and for the one-dimensional Kuramoto-Sivashinsky equation.

2 Infinite Dimensional Embedding Techniques

We consider dynamical systems of the form

$$\begin{aligned} u_{j+1} = \Phi (u_j),\quad j=0,1,\ldots , \end{aligned}$$

(1)

where $\Phi :Y\rightarrow Y$ is Lipschitz continuous on a Banach space Y. Moreover, we assume that $\Phi$ has an invariant compact set ${\mathcal {A}}$, that is

$$ \Phi ({\mathcal {A}}) = {\mathcal {A}}. $$

In order to approximate the set ${\mathcal {A}}$ or invariant subsets of ${\mathcal {A}}$ we combine classical subdivision and continuation techniques for the computation of such objects in a finite dimensional space with infinite dimensional embedding results (cf. [21, 28]). For the statement of the main result of [28] we require three particular notions: prevalence [29], upper box counting dimension and thickness exponent [21].

Definition 1

• (a) A Borel subset S of a normed linear space V is prevalent if there is a finite dimensional subspace E of V (the ‘probe space’) such that for each $v \in V,\ v+e$ belongs to S for (Lebesgue) almost every $e\in E$.

  Following a remark made in [29] we will say that “almost every” map in a function space V satisfies a certain property if the set of such maps is prevalent, even in the infinite dimensional case. Then this property will be called generic (in the sense of prevalence).

• (b) Let Y be a Banach space, and let ${\mathcal {A}}\subset Y$ be compact. For $\varepsilon >0$, denote by $N_Y({\mathcal {A}},\varepsilon )$ the minimal number of balls of radius $\varepsilon$ (in the norm of Y) necessary to cover the set ${\mathcal {A}}$. Then

  $$\begin{aligned} d({\mathcal {A}}; Y) = \limsup \limits _{\varepsilon \rightarrow 0} \frac{\log N_Y({\mathcal {A}},\varepsilon )}{-\log \varepsilon } = \limsup _{\varepsilon \rightarrow 0}\; -\log _{{\varepsilon }} N_Y({\mathcal {A}},\varepsilon ) \end{aligned}$$

  denotes the upper box counting dimension of ${\mathcal {A}}$.

• (c) Let Y be a Banach space, and let ${\mathcal {A}}\subset Y$ be compact. For $\varepsilon >0$, denote by $d_Y({\mathcal {A}}, \varepsilon )$ the minimal dimension of all finite dimensional subspaces $V\subset Y$ such that every point of ${\mathcal {A}}$ lies within distance $\varepsilon$ of V; if no such V exists, $d_Y({\mathcal {A}}, \varepsilon ) = \infty$. Then

  $$ \sigma ({\mathcal {A}}, Y) = \limsup _{\varepsilon \rightarrow 0} \;-\log _{{\varepsilon }} d_Y({\mathcal {A}}, \varepsilon ) $$

  is called the thickness exponent of ${\mathcal {A}}$ in Y.

These notions are essential in addressing the question when a delay embedding technique applied to an invariant subset $A\subset Y$ will generically work. More precisely, the results are as follows.

Theorem 1

( [21, Theorem 3.9]). Let Y be a Banach space and ${\mathcal {A}}\subset Y$ compact, with upper box counting dimension $d({\mathcal {A}}; Y) =: d$ and thickness exponent ${\sigma ({\mathcal {A}}, Y) =: \sigma }$. Let $N>2d$ be an integer, and let $\alpha \in \mathbb {R}$ with

$$\begin{aligned} 0< \alpha < \frac{ N-2d }{ N\cdot (1+\sigma ) }. \end{aligned}$$

Then for almost every (in the sense of prevalence) bounded linear map $\mathcal {L}:Y\rightarrow \mathbb {R}^N$ there is $C>0$ such that

$$ C\cdot \Vert \mathcal {L}(x-y)\Vert ^\alpha \ge \Vert x-y\Vert \quad \text {for all }x, y\in {\mathcal {A}}. $$

Note that this result implies that - if N is large enough - almost every (in the sense of prevalence) bounded linear map ${{\mathcal {L}}:Y\rightarrow \mathbb {R}^N}$ will be one-to-one on ${\mathcal {A}}$. Using this theorem, the following result concerning embedding techniques can be proven.

Theorem 2

( [28, Theorem 5.1]). Let Y be a Banach space and ${\mathcal {A}}\subset Y$ a compact, invariant set, with upper box counting dimension d, and thickness exponent $\sigma$. Choose an integer $k> 2(1+\sigma )d$ and suppose further that the set ${\mathcal {A}}_p$ of p-periodic points of $\Phi$ satisfies $d({\mathcal {A}}_p;Y) < p/(2+2\sigma )$ for $p=1,\ldots ,k$. Then for almost every (in the sense of prevalence) Lipschitz map $f:Y\rightarrow \mathbb {R}$ the observation map $D_k[f,\Phi ] : Y \rightarrow \mathbb {R}^k$ defined by

$$\begin{aligned} D_k[f,\Phi ] (u) = (f(u),f(\Phi (u)),\ldots ,f(\Phi ^{k-1}(u)))^\top \end{aligned}$$

is one-to-one on ${\mathcal {A}}$.

Remark 1

• (a) Following an observation already made in [29, Remark 2.9], we note that this result may be generalized to the case where several distinct observables are evaluated. More precisely, for almost all (in the sense of prevalence) Lipschitz maps $f_i:Y\rightarrow \mathbb {R}$, $i=1,\dots ,q$, the observation map $D_k[f_1, \ldots , f_q, \Phi ]: Y\rightarrow \mathbb {R}^k$,

  $$\begin{aligned} u \mapsto ( f_1(u), \ldots , f_1(\Phi ^{k_1-1}(u)), \ldots , f_q(u), \ldots , f_q(\Phi ^{k_q-1}(u)))^\top \end{aligned}$$

  is also one-to-one on ${\mathcal {A}}$, provided that

  $$ k =\sum _{i=1}^q k_i > 2(1+\sigma )\cdot d\ \text { and }\ {d({\mathcal {A}}_p) < p/(2+2\sigma )} \quad \forall p\le \max (k_1, \ldots , k_q).$$

• (b) If the thickness exponent $\sigma$ is unknown, a worst-case embedding dimension $k > 2(1+d)d$ can always be chosen since the thickness exponent is bounded by the (upper) box counting dimension (cf. [21]).

• (c) In [27] it is suspected that many of the attractors arising in dynamical systems defined by the evolution equations of mathematical physics have thickness exponent zero. In addition, in [16] it is shown that the thickness exponent is essentially inversely proportional to smoothness. This result does not rely on the dynamics associated with the set ${\mathcal {A}}$ or the form of the underlying equations, but only on assumptions on the smoothness of functions in ${\mathcal {A}}$. Thus, it is reasonable to assume $\sigma =0$, i.e., an embedding dimension $k>2d$ is sufficient in most cases.

3 The Core Dynamical System

In this section we show how the results in Sect. 2 lay the theoretical foundation for the construction of the so–called core dynamical system (CDS). This finite dimensional dynamical system then allows us to approximate invariant sets of an infinite dimensional dynamical system. For details on the construction and corresponding proofs we refer to [6].

Let ${\mathcal {A}}$ be a compact invariant set of an infinite dynamical systems (1) on a Banach space Y and suppose $k\in \mathbb {N}$ is large enough such that the embedding result (Theorem 1 or 2) is valid. Suppose $R:Y\rightarrow \mathbb {R}^k$ is the corresponding observation map, i.e., $R=\mathcal {L}$ or $R=D_k[f,\Phi ]$, respectively. We denote by $A_k$ the image of ${\mathcal {A}}\subset Y$ under the observation map, that is,

$$\begin{aligned} A_k = R({\mathcal {A}}). \end{aligned}$$

The core dynamical system (CDS)

$$\begin{aligned} x_{j+1} = \varphi (x_j), \quad j=0,1,2,\ldots , \end{aligned}$$

with $\varphi :\mathbb {R}^k\rightarrow \mathbb {R}^k$ is then constructed as follows: Since R is invertible as a mapping from ${\mathcal {A}}$ to $A_k$ there is a unique continuous map $\widetilde{E}:A_k \rightarrow Y$ satisfying

$$\begin{aligned} (\widetilde{E}\circ R)(u) = u\quad \forall u\in {\mathcal {A}}\quad \text { and }\quad (R\circ \widetilde{E})(x) = x\quad \forall x\in A_k. \end{aligned}$$

(2)

Thus, as a first step this allows us to define $\varphi$ solely on $A_k$ via

$$\begin{aligned} \varphi = R \circ \Phi \circ \widetilde{E}. \end{aligned}$$

For the extension of $\varphi$ to $\mathbb {R}^k$ we need to extend the map $\tilde{E}$ to a continuous map $E:\mathbb {R}^k\rightarrow Y$. By employing a generalization of the well-known Tietze extension theorem [14, I.5.3] found by Dugundji [13, Theorem 4.1] we obtain a continuous map $E:\mathbb {R}^k\rightarrow Y$ with $E_{\vert A_k} = \widetilde{E}$ and we define the CDS by

$$\begin{aligned} \varphi = R \circ \Phi \circ E, \end{aligned}$$

see Fig. 1 for an illustration. Observe that by this construction $A_k$ is an invariant set for $\varphi$, and that the dynamics of $\varphi$ on $A_k$ is topologically conjugate to that of $\Phi$ on ${\mathcal {A}}$.

Proposition 1

( [6, Propostion 1])

There is a continuous map $\varphi :\mathbb {R}^k \rightarrow \mathbb {R}^k$ satisfying

$$\begin{aligned} \varphi (R(u)) = R(\Phi (u)) \text { for all } u\in {\mathcal {A}}. \end{aligned}$$

Fig. 1.

(Figure adapted from [36]).

Definition of the CDS $\varphi$

Note that the arguments stated above only guarantee the existence of the continuous map E and provide no guideline on how to design or approximate it. In fact, the particular realization of the map E will depend on the actual application (see Sect. 5).

4 Computation of Embedded Invariant Sets

We are now in the position to approximate the embedded invariant set $A_k$ or invariant subsets such as the invariant manifold of a steady state via the core dynamical system

$$\begin{aligned} \varphi = R \circ \Phi \circ E. \end{aligned}$$

To this end, we employ the subdivision and continuation schemes as defined in [8] and [7].

4.1 A Subdivision Scheme for the Approximation of Embedded Attractors

In this section we give a brief review of the adapted subdivision scheme developed in  [6] that allows us to approximate the set $A_k$.

Let $Q \subset \mathbb {R}^k$ be a compact set and suppose $A_k\subset Q$ for simplicity. The embedded global attractor relative to Q is defined by

$$\begin{aligned} A_Q = \bigcap _{j\ge 0} \varphi ^j(Q). \end{aligned}$$

The aim is to approximate this set with a subdivision procedure. Given an initial finite collection ${\mathcal {B}}_0$ of compact subsets of $\mathbb {R}^k$ such that

$$ Q=\bigcup _{B\in {\mathcal {B}}_0} B, $$

we recursively obtain ${\mathcal {B}}_\ell$ from ${\mathcal {B}}_{\ell -1}$ for $\ell =1,2,\ldots$ in two steps such that the diameter

$$ {{\,\mathrm{diam}\,}}({\mathcal {B}}_\ell ) = \max _{B\in {\mathcal {B}}_\ell }{{\,\mathrm{diam}\,}}(B) $$

converges to zero for $\ell \rightarrow \infty$.

Remark 2

• a) A numerical implementation of Algorithm 1 is included in the software package GAIO (Global Analysis of Invariant Objects) [5, 10]. Here, the sets B constituting the collections ${\mathcal {B}}_\ell$ are realized by generalized k-dimensional rectangles (“boxes”) of the form

  $$ B(c,r) = \left\{ y \in \mathbb {R}^k: |y_i - c_i|\le r_i \text { for } i=1,\ldots ,k \right\} , $$

  where $c,r \in \mathbb {R}^k,\ r_i > 0$ for $i=1,\ldots ,k$, are the center and the radii, respectively. In each subdivision step each box of the current collection is subdivided by bisection with respect to the j-th coordinate, where j is varied cyclically. Therefore, these collections can easily be stored in a binary tree.

• b) Given a collection $\hat{\mathcal {B}}_\ell$ the selection step is realized as follows: At first $\varphi$ is evaluated for a large number of test points $x\in B'$ for each box $B' \in \hat{\mathcal {B}}_\ell$. Then a box B is kept in the collection ${\mathcal {B}}_\ell$ if there is a least one $x\in B'$ such that $\varphi (x)\in B$. We note that the binary tree structure implemented in GAIO allows a fast identification of the boxes that are not discarded.

The subdivision step results in decreasing box diameters with increasing $\ell$. In fact, by construction

$$\begin{aligned} {{\,\mathrm{diam}\,}}({\mathcal {B}}_\ell )\le \theta _{\max }^\ell {{\,\mathrm{diam}\,}}({\mathcal {B}}_0)\rightarrow 0\quad \text { for } \ell \rightarrow \infty . \end{aligned}$$

In the selection step each subset whose preimage does neither intersect itself nor any other subset in $\hat{\mathcal {B}}_\ell$ is removed. Denote by $Q_\ell$ the collection of compact subsets obtained after $\ell$ subdivision steps, that is

$$ Q_\ell =\bigcup _{B\in {\mathcal {B}}_\ell }B. $$

Since the $Q_\ell$’s define a nested sequence of compact sets, that is, $Q_{\ell +1}\subset Q_\ell$ we conclude for each m

$$\begin{aligned} Q_m = \bigcap \limits _{\ell =1}^{m} Q_\ell . \end{aligned}$$

Then by considering

$$\begin{aligned} Q_{\infty } = \bigcap \limits _{\ell =1}^{\infty } Q_\ell \end{aligned}$$

as the limit of the $Q_\ell$’s the selection step accounts for the fact that $Q_m$ approaches the relative global attractor.

Proposition 2

( [6, Proposition 2])

Suppose $A_Q$ satisfies ${\varphi ^{-1}(A_Q) \subset A_Q}$. Then

$$ A_Q = Q_\infty . $$

We note that we can, in general, not expect that $A_k = A_Q$. In fact, by construction $A_Q$ may contain several invariant sets and related heteroclinic connections. However, if ${\mathcal {A}}$ is an attracting set equality can be proven (see [6]).

4.2 A Continuation Technique for the Approximation of Embedded Unstable Manifolds

In [36] the classical continuation method of [7] was extended to the approximation of embedded unstable manifolds. In the following we state the main result of this scheme. Let us denote by

$$\begin{aligned} {\mathcal {W}}_{\Phi }^u(u^*) \subset {\mathcal {A}}\end{aligned}$$

the unstable manifold of $u^* \in {\mathcal {A}}$, where $u^*$ is a steady state solution of the infinite dimensional dynamical system $\Phi$ (cf. (1)). Furthermore, let us define the embedded unstable manifold $W^u(p)$ by

$$\begin{aligned} W^u(p) = R({\mathcal {W}}_{\Phi }^u(u^*)) \subset A_k, \end{aligned}$$

where $p = R(u^*)\in \mathbb {R}^k$ and R is the observation map introduced in Sect. 3. We now choose a compact set $Q \subset \mathbb {R}^k$ containing p and we assume for simplicity that Q is large enough so that it contains the entire closure of the embedded unstable manifold, i.e.,

$$\begin{aligned} \overline{W^u(p)} \subset Q. \end{aligned}$$

For the purpose of initializing the developed algorithm we define a partition ${\mathcal {P}}$ of Q to be a finite family of compact subsets of Q such that

$$ \bigcup _{B \in {\mathcal {P}}}B = Q\quad \text {and}\quad \text {int} B\cap \text {int} B' = \emptyset , \text { for all } B, B'\in {\mathcal {P}},~ B\ne B'. $$

We consider a nested sequence ${\mathcal {P}}_s,\ s \in \mathbb {N}$, of successively finer partitions of Q, requiring that for all $B \in {\mathcal {P}}_s$ there exist $B_1,\ldots ,B_m \in {\mathcal {P}}_{s+1}$ such that $B = \cup _i B_i$ and ${{{\,\mathrm{diam}\,}}(B_i)\le \theta {{\,\mathrm{diam}\,}}(B)}$ for some $0< \theta < 1$. A set $B \in {\mathcal {P}}_s$ is said to be of level s.

The aim of the continuation method is to approximate subsets $W_j \subset W^u(p)$ where $W_0 = W_{loc}^u(p) =R(\overline{{\mathcal {W}}_{\Phi ,loc}^u(u^*)})$ is the local embedded unstable manifold and

$$ W_{j+1} = \varphi (W_j) \quad \text { for } j = 0,1,2,\ldots $$

in two steps:

At first we approximate $W_{loc}^u(p)$ by applying Algorithm 1 on a compact neighborhood $C\subset A_k$ of p such that $p\in \text { int } C$ and $W_{loc}^u(p) \subset C$ in order to compute the relative global attractor $A_C$. In fact, if the steady state $u^*\in {\mathcal {A}}$ is hyperbolic $W_{loc}^u(p)$ is identical to $A_C$ [36, Proposition 3.1 (b)]. In the second step this covering of $W_{loc}^u(p)$ is then to globalized to obtain an approximation of the compact subsets $W_j\subset W^u(p)$ or even the entire closure $\overline{W^u(p)}$.

Remark 3

• a) Algorithm 2 is also implemented within the software package GAIO. In fact, the binary tree structure encodes a nested sequence of finer partitions of Q.

• b) Numerically the continuation step is realized as follows: At first $\varphi$ is evaluated for a large number of test points $x \in B'$ for each box $B' \in {\mathcal {C}}_j^{(\ell )}$. Then a box $B \in {\mathcal {P}}_{s+\ell }$ is added to the collection ${\mathcal {C}}_{j+1}^{(\ell )}$ if there is a least one $x \in B'$ such that $\varphi (x)\in B$.

Observe that the unions

$$ C_j^{(\ell )} = \bigcup _{B \in {\mathcal {C}}_j^{(\ell )}} B $$

form a nested sequence in $\ell$, i.e.,

$$ C_j^{(0)}\supset C_j^{(1)} \supset \ldots \supset C_j^{(\ell )} \ldots . $$

In fact, it is also a nested sequence in j, i.e.,

$$ C_0^{(\ell )}\subset C_1^{(\ell )} \ldots \subset C_j^{(\ell )} \ldots . $$

Due to the compactness of Q the continuation step of Algorthm 2 will terminate after finitely many, say $J_\ell$, steps. We denote the corresponding box covering obtained by the continuation method by

$$\begin{aligned} G_\ell = \bigcup _{j=0}^{J_\ell } C_j^{(\ell )} = C_{J_\ell }^{(\ell )}. \end{aligned}$$

In [36] it was proven that increasing $\ell$ eventually leads to convergence of $C_j^{(\ell )}$ to the subsets $W_j$ and assuming that the closure of the embedded unstable manifold $\overline{W^u(p)}$ is attractive $G_\ell$ converges to $\overline{W^u(p)}$.

Proposition 3

( [36, Proposition 5])

• (a) The sets $C_j^{(\ell )}$ are coverings of $W_j$ for all $j,\ell = 0,1,\ldots$. Moreover, for fixed j, we have

  $$ \bigcap _{\ell = 0}^\infty C_j^{(\ell )} = W_j. $$

• (b) Suppose that $\overline{W^u(p)}$ is linearly attractive, i.e., there is a $\lambda \in (0,1)$ and a neighborhood $U\supset Q \supset \overline{W^u(p)}$ such that

  $$\begin{aligned} \text {dist}\left( \varphi (y),\overline{W^u(p)}\right) \le \lambda ~\text {dist}\left( y,\overline{W^u(p)}\right) \quad \forall y\in U. \end{aligned}$$

  Then the box coverings obtained by Algorithm 2 converge to the closure of the embedded unstable manifold $\overline{W^u(p)}$. That is,

  $$ \bigcap _{\ell = 0}^\infty G_\ell =\overline{W^u(p)}. $$

5 Numerical Realization of the CDS

As discussed in the introduction, dynamical systems with infinite dimensional state space, but finite dimensional attractors arise in particular in two areas of applied mathematics, namely dissipative partial differential equations and delay differential equations with small constant delay. In this section we show how to numerically realize the CDS for both cases. From now on we assume that upper bounds for both the box counting dimension d and the thickness exponent $\sigma$ are available. This allows us to fix $k > 2(1+\sigma )d$ according to Theorem 2.

In order to numerically realize the construction of the map $\varphi = R\circ \Phi \circ E$ described in Sect. 3, we have to address three tasks: the implementation of E, of R, and of the time-T-map, denoted by $\Phi$, respectively. For the latter we will rely on standard methods for forward time integration of DDEs [1] and PDEs, e.g., a fourth-order time stepping method for the one-dimensional Kuramoto-Sivashinsky equation [24]. The map R will be realized on the basis of Theorem 2 and Remark 1 by an appropriately chosen observables. For the numerical construction of the continuous map E we will employ a bootstrapping method that re-uses results of previous computations. This way we will in particular guarantee that the identities in (2) are at least approximately satisfied.

5.1 Delay Differential Equations

We consider equations of the form

$$\begin{aligned} \dot{y}(t) = g(y(t),y(t-\tau )), \end{aligned}$$

(5)

where $y(t)\in \mathbb {R}^n$, $\tau >0$ is a constant time delay and $g:\mathbb {R}^n \times \mathbb {R}^n \rightarrow \mathbb {R}^n$ is a smooth map. Here, we will only consider the one-dimensional case, that is $n=1$, and refer to [6] for $n>1$. Following [19], we denote by $Y = C([-\tau , 0], \mathbb {R}^n)$ the (infinite dimensional) state space of the dynamical system (5). Observe that equipped with the maximum norm Y is indeed a Banach space. We set $T>0$ to be a natural fraction of $\tau$, that is

$$\begin{aligned} T = \frac{\tau }{K} \text { for } K\in \mathbb {N}. \end{aligned}$$

(6)

5.1.1 Numerical Realization of R

For the definition of R we have to specify the time span T and appropriate corresponding observables. In the case of a scalar equation we choose the observable f to be

$$ f(u) =u(-\tau ). $$

Thus, in this case the restriction R is simply given by

$$\begin{aligned} R = D_k[f,\Phi ] (u) = (u(-\tau ),\Phi (u)(-\tau ),\ldots ,\Phi ^{k-1}(u)(-\tau ))^\top . \end{aligned}$$

Observe that a natural choice for K in (6) would be $K = k-1$ for $k>1$. That is, for each evaluation of R the observable would be applied to a function $u:[-\tau ,0] \rightarrow \mathbb {R}$ at k equally distributed time steps within the interval $[-\tau ,0]$.

5.1.2 Numerical Realization of E

In the application of the subdivision scheme Algorithm 1 as well as the continuation method Algorithm 2 the CDS has to be evaluated for a set of test points (see Remark 2 and Remark 3). Thus, for the evaluation of $\varphi = R\circ \Phi \circ E$ at a test point x we need to define the image E(x), i.e., we need to generate adequate initial conditions for the forward integration of the DDE (5).

In the first selection or continuation step, when no information on ${\mathcal {A}}$ or ${\mathcal {W}}_{\Phi }^u(u^*)$, respectively, is available, we construct a piecewise linear function $u = E(z)$, where

$$\begin{aligned} u(t_i) = z_i, \end{aligned}$$

for $t_i = -\tau + i\cdot T,\ i=0,\dots ,k-1$. Observe that by this choice of E and R the condition $(R\circ E)(x) = x$ is satisfied for each test point x (see (2)). In the following steps we make use of the following observations for both schemes:

Remark 4

If a box $B\in {\mathcal {B}}_\ell$ (resp. $B\in {\mathcal {C}}_{j+1}^{(\ell )}$), then, by the subdivision (resp. selection) step, there must have been a $\hat{B}\in {\mathcal {B}}_{\ell -1}$ (resp. ${\hat{B}\in {\mathcal {C}}_{j}^{(\ell )}}$) such that ${\bar{x}=R(\Phi (E(\hat{x})))\in B}$ for at least one test point $\hat{x}\in \hat{B}$. Therefore, we can use the information from the computation of $\Phi (E(\hat{x}))$ to construct an appropriate E(x) for each test point $x\in B$ in both cases.

More concretely, in every step of the procedures, for every set $B \in {\mathcal {B}}_\ell$ (resp. $B\in {\mathcal {C}}_{j+1}^{(\ell )}$) we keep additional information about the trajectories $\Phi (E(\hat{z}))$ that were mapped into B by R in the previous step. For simplicity, we store $k_i\ge 1$ additional equally distributed function values for each interval ${(-\tau +(i-1) T,-\tau +iT)}$ for $i=1,\dots ,k-1$.

When $\varphi (B)$ is to be computed using test points from B, we first use the points in B for which additional information is available and generate the corresponding initial value functions via spline interpolation. Note that the more information we store, the smaller the error $\Vert \Phi (E(\hat{x}))-E(x)\Vert$ becomes for ${x = R(\Phi (E(\hat{x})))}$. That is, we enforce an approximation of the identity ${(E\circ R)(u) = u}$ for all $u\in {\mathcal {A}}$ (see (2)). If the additional information is available only for a few points in B, we generate new test points in B at random and construct corresponding trajectories by piecewise linear interpolation.

5.2 Partial Differential Equations

We will consider explicit differential equations of the form

$$\begin{aligned} \frac{\partial }{\partial t} u(y,t)=F(y,u),~u(y,0)=u_0(y), \end{aligned}$$

(7)

where $u:\mathbb {R}^n\times \mathbb {R}\rightarrow \mathbb {R}^n$ is in some Banach space Y and F is a (nonlinear) differential operator. We assume that the dynamical system (7) has a well-defined semiflow on Y.

5.2.1 Numerical Realization of R

In the previous section for delay differential equations R is defined by the delay coordinate map. In principle it would also be possible to observe the evolution of a partial differential equation by a delay coordinate map. However, from a computational point of view this would be impractical. The reason is that for the realization of the map ${E:\mathbb {R}^k \rightarrow Y}$ one would have to reconstruct functions (i.e., the space-dependent state) from time delay coordinates (of scalar, e.g., point-wise observations). Thus, for each point in observation space one would essentially have to store the entire corresponding function.

To overcome this problem, we will present a different approach. In what follows, we will assume that the function $u \in Y$ can be represented in terms of an orthonormal basis $\{\Psi _i\}_{i=1}^\infty$, i.e.,

$$\begin{aligned} u(y,t) = \sum _{i=1}^{\infty } x_i(t) \Psi _i(y), \end{aligned}$$

where the $\Psi _i$ are elements from a Hilbert space. Then our observation map R will be defined by projecting a function onto k coefficients $x_i$ of its Galerkin expansion. The function u can then be approximated within the (truncated) linear subspace spanned by the basis $\{\Psi _i\}_{i=1}^k$:

$$\begin{aligned} u(y,t) \approx \sum _{i=1}^{k} x_i(t) \Psi _i(y). \end{aligned}$$

For the computation of the basis, we use the well-known Proper Orthogonal Decomposition (POD), cf.  [2, 20, 31]. The reason is that for each basis size k, POD yields the optimal basis, i.e., the basis with the minimal $L^2$ projection error. In order to compute such a basis $\{\Psi _i\}_i$ we use the so-called method of snapshots (cf.  [20] for details). To this end, we construct the so–called snapshot matrix $S_M \in \mathbb {R}^{n_x\times r}$, where each column consists of the (discretized) state at equidistant time instances obtained from a single long-time integration of the underlying PDE (7). Then we perform a singular value decomposition (SVD) of the matrix $S_M$ and obtain $S_M = U\Sigma V^\top$, where $U \in \mathbb {R}^{n_x\times n_x}, \Sigma \in \mathbb {R}^{n_x\times r}$ and $V \in \mathbb {R}^{r\times r}$. The columns of U give us a discrete representation of the POD modes $\Psi _i$. Using the fact that this basis is orthogonal, we then define the observation map by choosing k different observables

$$\begin{aligned} f_i(u) = \langle u, \Psi _i \rangle = x_i \quad \text {for }i=1,\ldots ,k. \end{aligned}$$

This yields

$$\begin{aligned} R(u) = (f_1(u), \ldots , f_k(u))^\top = (x_1, \ldots , x_k)^\top . \end{aligned}$$

Observe that R is linear and bounded and hence, for k sufficiently large, Theorem 1 and Remark 1, respectively, guarantee that generically (in the sense of prevalence) R will be a one-to-one map on ${\mathcal {A}}$.

5.2.2 Numerical Realization of E

Since the state space for the CDS $\varphi$ is given by points $x \in \mathbb {R}^k$ where $x_1,\ldots ,x_k$ are the POD coefficients we simply construct initial conditions $u = E(x)$ by defining the map E as

$$\begin{aligned} E(x) = \sum _{i=1}^{k} x_i \Psi _i, \end{aligned}$$

if no additional information is available. Observe again that by this choice the condition $(R\circ E)(x)=x$ is satisfied for each test point x.

However, the linear space spanned by the first k POD modes is not invariant under the dynamics of $\Phi$ if k is not sufficiently large. Thus, $(E\circ R)(\bar{u})=\bar{u}$ with $\bar{u} = \Phi (E(x))$ will in general not be satisfied anymore. This is not acceptable since the requirement $(E\circ R)(u) = u$ for all $u\in {\mathcal {A}}$ (see (2)) is crucial in order to compute reliable coverings.

To enforce this equality at least approximately we extend the expansion and construct initial functions by

$$\begin{aligned} E(x) = \sum _{i=1}^{k} x_i\Psi _i + \sum _{l=k+1}^{S}x_l\Psi _l, \end{aligned}$$

(8)

where $S\gg k$. To address the choice of S we note that the singular values $\sigma _i$ of the snapshot matrix $S_M$, i.e. the diagonal elements of $\Sigma$, determine the amount of information that is neglected by truncating the basis $\{\Psi _i\}_i$ to size $S < r$ [31]. Thus, we choose S such that

$$\begin{aligned} \varepsilon (S) := \frac{\sum _{i=1}^S \sigma _i}{\sum _{j=1}^{r} \sigma _j} \approx 1. \end{aligned}$$

In (8) only the first k POD coefficients are given by the coordinates of points inside $B \subset \mathbb {R}^k$. Thus, it remains to discuss how to choose the POD coefficients $x_{k+1},\ldots ,x_S$. The idea is to use a heuristic strategy that utilizes statistical information. By Remark 4 we can compute the POD coefficients $\bar{x}_{k+1},\ldots , \bar{x}_S$ for all these points $\bar{x}$ by

$$\begin{aligned} \bar{x}_i = \langle \Phi (E(\hat{x})), \Psi _i \rangle , \quad i=k+1,\ldots ,S. \end{aligned}$$

Then we sample the box B with all points $\bar{x}$ for which additional information is available. However, the number of these points $\bar{x}$ might be too small, such that B is not discretized sufficiently well and we have to generate additional test points. For this, we first choose a certain number of points $\widetilde{x} \in B$ at random. Then we extend these points to elements in $\mathbb {R}^S$ as follows: We first compute componentwise the mean value $\mu _i$ and the variance $\sigma _i^2$ of all POD coefficients $\bar{x}_{i}$, for $i = k+1,\ldots ,S$. This allows us to make a Monte Carlo sampling for the additional coefficients of $\widetilde{x}_i$ for $i=k+1,\ldots ,S$, i.e., $\widetilde{x}_i \sim {\mathcal {N}}(\mu _i,\sigma _i^2)$ for $i=k+1,\ldots ,S.$ Finally, we compute initial functions of the form

$$ E(\widetilde{x}) = \sum _{i=1}^{S} \widetilde{x}_i\Psi _i. $$

By this construction we expect to generate initial functions that at least approximately satisfy the identity $(E\circ R)(u) = u$ for all $u\in {\mathcal {A}}$.

6 Numerical Results

In this section we present results of computations carried out for the Mackey-Glass delay differential equation and the Kuramoto-Sivashinsky equation, respectively.

6.1 The Mackey-Glass Equation

As in [6], we consider the well-known delay differential equation introduced by Mackey and Glass in 1977 [26], namely

$$\begin{aligned} \dot{u}(t) = \beta \frac{u(t-\tau )}{1+u(t-\tau )^\eta }-\gamma u(t), \end{aligned}$$

(9)

where we choose $\beta = 2, \gamma = 1$, $\eta = 9.65$, and $\tau = 2$. This equation is a model for blood production, where u(t) represents the concentration of blood at time t, $\dot{u}(t)$ represents production at time t and $u(t-\tau )$ is the concentration at an earlier time. Direct numerical simulations indicate that the dimension of the corresponding attracting set is approximately $d=2$. Thus, we choose the embedding dimension $k=7$, and approximate the relative global attractor $A_Q$ for $Q = [0,1.5]^7 \subset \mathbb {R}^7$.

In Fig. 2 (a) to (c), we show projections of the coverings obtained after $\ell =28,42$ and 63 subdivision steps. In order to investigate the effect of using information retained from prior integration runs in the implementation of the map E (see Sect. 5.1.2), we show in Fig. 2 (d) a projection of the coverings obtained after 63 subdivision steps with the map E using only piecewise linear functions – that is, no additional information from previous time integration is used. The results indicate that in this case the influence on the quality of the approximation of $A_k$ is only marginal.

6.2 The Kuramoto-Sivashinsky Equation

The well-known Kuramoto-Sivashinsky equation in one spatial dimension is given by

$$\begin{aligned} \begin{aligned}&u_t + 4u_{yyyy}+\mu \left[ u_{yy} + \frac{1}{2} (u_y)^2 \right] = 0, \quad 0\le y\le 2\pi ,\\&u(y,0) = u_0(y), \quad u(y+2\pi ,t) = u(y,t). \end{aligned} \end{aligned}$$

(10)

Here, the parameter is $\mu = L^2/4\pi ^2$, where L denotes the size of a typical pattern scale. As in [36] we are interested in computing the unstable manifold of the trivial unstable steady state $u^* = 0$ for $\mu =15.$

In what follows, the observation space is defined through projections onto the first k POD coefficients, and thus, $p = R(u^*) = 0 \in \mathbb {R}^k$. We compute the POD basis (cf. Sect. 5.2.1) by using the snapshot matrix obtained through a long-time integration with the initial condition

$$ u_0(y) = 10^{-4}\cdot \cos \left( y\right) \cdot \left( 1+\sin \left( y\right) \right) . $$

Fig. 2.

(a)–(c) Successively finer coverings of a relative global attractor after $\ell$ subdivision steps for the Mackey-Glass equation (9). (d) Embedding E using only piecewise linear interpolation.

Fig. 3.

(a) Direct simulation of the Kuramoto-Sivashinsky equation for $\mu = 15$. The initial value is attracted to a traveling wave solution; (b) Corresponding embedding in observation space. Here, the red dot depicts the unstable steady state. As expected the CDS possesses a limit cycle (green).

For $\mu =15$ the Kuramoto-Sivashinsky equation has two stable traveling waves (see. Fig. 3 (a)) traveling in opposite directions due to the symmetry imposed by the periodic boundary conditions. In the observation space this corresponds to two stable limit cycles that are symmetric in the first POD coefficient $a_1$ (see. Fig. 3 (b)). We assume that the dimension of the embedded unstable manifold is approximately two since different initial conditions result in trajectories in observation space that are rotations of each other about the origin. Therefore, assuming that the thickness exponent is zero, we have to choose $k\ge 5$ in order to obtain a one-to-one image of ${\mathcal {W}}_{\Phi }^u(u^*)$. To allow for a larger dimension or thickness exponent we choose the embedding dimension $k = 7$ in the following. We choose $Q = [-8,8]^7$ and initialize a fine partition ${\mathcal {P}}_s$ of Q for $s = 21, 35, 49, 63$. Next we set $T = 200$. In addition, we define a finite time grid $\{ t_0,\ldots , t_N \}$, where $t_N = T$, and add all boxes that are hit in any of these time steps $t_i$ (a similar approach has been used in [23]). In Fig. 4 (a) to (d) we illustrate successively finer box coverings of the unstable manifold as well as a transparent box covering depicting the complex internal structure of the unstable manifold. Observe that – as mentioned above – the boundary of the unstable manifold consists of two limit cycles which are symmetric in the first POD coefficient $x_1$. This is due to the fact that the Kuramoto-Sivashinsky equation with periodic boundary conditions (10) possesses O(2)-symmetry.

Fig. 4.

(a)–(d) Successively finer box-coverings of the unstable manifold for ${\mu = 15}$. (d) Transparent box covering for $s = 63$ and $\ell = 0$ depicting the internal structure of the unstable manifold.

7 Conclusion

In this work we review the contents of [6] and [36], where infinite dimensional embedding results have extended to the numerical analysis of infinite dimensional dynamical systems. To this end, a continuous dynamical system, a finite dimensional core dynamical system (CDS) is constructed to obtain a one-to-one representation of the underlying dynamics. For the numerical realization of this system we also identify suitable observables for delay differential and partial differential equations. This finite dimensional system then employed in the subdivision scheme for the computation of relative global attractors and the continuation method for the approximation of invariant manifolds feasible for infinite dimensional systems. The applicability of this novel framework is illustrated by the computation of the attractor of the Mackey-Glass delay differential equation and the unstable manifold of the one-dimensional Kuramoto-Sivashinsky equation.

The numerical effort of the methods proposed in this work essentially depends on the dimension of the object to be computed, and not on the dimension of the observation space of the CDS. However, note that for the numerical realization of the selection step 3 and the continuation step 4 we have to evaluate the CDS for each box and each test point $x \in B'$. Therefore, for each test point we also have to evaluate the underlying infinite dimensional dynamical system which may result in a prohibitively large computational effort. For this reason data-based local reduced order models can be used in order to significantly reduce the number of CDS evaluations [35].