A Symbolic-Numeric Validation Algorithm for Linear ODEs with Newton–Picard Method

Abstract

A symbolic-numeric validation algorithm is developed to compute rigorous and tight uniform error bounds for polynomial approximate solutions to linear ordinary differential equations, and in particular D-finite functions. It relies on an a posteriori validation scheme, where such an error bound is computed afterwards, independently from how the approximation was built. Contrary to Newton–Galerkin validation methods, widely used in the mathematical community of computer-assisted proofs, our algorithm does not rely on finite-dimensional truncations of differential or integral operators, but on an efficient approximation of the resolvent kernel using a Chebyshev spectral method. The result is a much better complexity of the validation process, carefully investigated throughout this article. Indeed, the approximation degree for the resolvent kernel depends linearly on the magnitude of the input equation, while the truncation order used in Newton–Galerkin may be exponential in the same quantity. Numerical experiments based on an implementation in C corroborate this complexity advantage over other a posteriori validation methods, including Newton–Galerkin.

1 Introduction

Many functions arising in scientific computing are implicitly defined by functional equations, of which differential equations are probably the most commonly encountered ones. Yet, one frequently requires to explicitly manipulate and evaluate these functions. Polynomials (in a broad sense) often provide accurate, global and smooth approximants, and are easy to manipulate on a computer. Moreover, in some applicative contexts, like computer-assisted proofs in mathematics, notably in dynamical systems, or safety-critical industrial applications (e.g., aerospace challenges, automatic vehicles or remote medicine), those computations must be accompanied with strong reliability guarantees.

1.1 Background and Related Works

A wide range of techniques in computer algebra allow for a symbolic, exact manipulation of certain classes of functions defined implicitly. A major example are the D-finite functions, that is the solutions of linear ordinary differential equations (ODEs) with polynomial coefficients [35]. Despite this seemingly restricted definition, D-finite functions account for about 60% [32] of the functions listed in Abramowitz and Stegun’s Handbook of Mathematical Functions [1]. Yet, many applications require to move from symbolic representations, based on linear differential operators [33], to numeric or symbolic-numeric ones providing quantitative information. While numerical analysis focuses on algorithms to compute approximations, error estimates are most of the time asymptotic. For example, the Chebfun library¹ in Matlab allows for powerful numerical computations with functions using polynomials obtained by truncating Chebyshev expansions, but it does not provide rigorous error bounds. On the other side, validated numerics [37] (also called rigorous or set-valued numerics) is concerned with the computation of such error bounds, encompassing both method and rounding errors. Such a challenge, and more specifically the following problem, is at the core of the present work.

Problem 1.1

Given a polynomial $y^\circ$ approximating the solution $y^*$ of the Initial Value Problem (IVP):

$$\begin{aligned} \left\{ \begin{aligned}&y^{(r)}(x) + a_{r-1}(x) y^{(r-1)}(x) + \dots + a_0(x) y(x) = h(x),&x \in [x_l,x_r],&\\&y^{(i)}(x_0) = v_i \in {{\mathbb {R}}},&0 \leqslant i < r, \quad x_0 \in [x_l,x_r]&, \end{aligned}\right. \end{aligned}$$

(1)

compute a bound $\varepsilon \geqslant \Vert y^\circ -y^*\Vert _\infty = \sup _{x \in [x_l,x_r]} |y^\circ (x) - y^*(x)|$.

The branch of validated numerics dedicated to the validation of functional problems has its origins in the early 80s with ultra-arithmetics [12, 13, 20]. By analogy with floating-point numbers and interval arithmetics [26] to rigorously approximate real numbers on computers, ultra-arithmetics proposes a quite general framework to provide similar operations on functions. Solutions are approximated using polynomials (in a broad sense), and rigorously enclosed in balls or other domains in function spaces using rigorous polynomial approximations (RPAs), typically pairs $(p,\varepsilon )$ with an approximation p and a rigorous error bound $\varepsilon$. Notable implementations of RPAs include Taylor models [5, 28] and Chebyshev models [10, 19], based on Taylor and Chebyshev expansions, respectively.

Numerous techniques have been developed over past decades to compute rigorous enclosures for solutions of ODEs [17, 21, 24, 27, 30, 41, 42] and of wider classes of equations (e.g., partial differential equations, delay equations), thereby contributing to the rise of computer-assisted proofs in dynamics [38]. More recently, several works [2, 6, 8, 25] paved the way towards an algorithmic and complexity approach to such methods, aiming at the development of general-purpose reliable and efficient libraries for symbolic-numeric computations. This article brings its own contributions to this approach. The counterpart is the restriction to more limited classes of ODEs, notably D-finite functions, that is, linear ODE (1) with polynomial or rational coefficients $a_i(x)$. However, we emphasize the fact that many of the ODE validation methods cited above proceed by linearization of the ODE after bounding the nonlinear terms. Also, analytic coefficients $a_i(x)$ in (1) can be rigorously represented with RPAs. Hence, the application scope of these algorithmic works, including this article in particular, goes far beyond D-finite functions: they provide an algorithmic core for the validation of differential equations in a broad perspective.

Table 1 Notation

1.2 Setting and Contributions

A powerful representation of D-finite functions y in computer algebra is via differential operators $\varvec{\mathrm {L}}$ for which $\varvec{\mathrm {L}}\{y\}=0$ (together with sufficiently many initial conditions), where:

$$\begin{aligned} \varvec{\mathrm {L}} = \varvec{\mathrm {\partial }}^r + a_{r-1}(\varvec{\mathrm {x}}) \varvec{\mathrm {\partial }}^{r-1} + \dots + a_1(\varvec{\mathrm {x}}) \varvec{\mathrm {\partial }}+ a_0(\varvec{\mathrm {x}}), \end{aligned}$$

with $a_i$ polynomials or rational functions, belongs to the Ore algebra of differential operators [11, 22], spanned by $\varvec{\mathrm {x}}$ ($\varvec{\mathrm {x}}\{y\} = x \mapsto x y(x)$) and $\varvec{\mathrm {\partial }}$ ($\varvec{\mathrm {\partial }}\{y\}=y'$), with the skew commutation rule $\varvec{\mathrm {\partial }}\varvec{\mathrm {x}}- \varvec{\mathrm {x}}\varvec{\mathrm {\partial }}= 1$. In the theoretical developments of this article, the $a_i$ may actually be analytic functions over the real compact interval $[x_l,x_r]$ of interest. For the algorithmic parts, they are supposed to be polynomials (with rational, floating-point or interval coefficients) for the sake of clarity, even if adapting the algorithms to RPA representations of the $a_i(x)$ is rather straightforward (see the example of Sect. 4.3). We thus propose a fully algorithmic validation method for Problem 1.1 with analytic coefficients $a_i$ representable by RPAs. In particular, iterating this construction allows for the rigorous numerical treatment of $D^n$-finite functions [18]. Also, in the case of non-homogeneous linear ODEs, the same remarks hold for the right-hand side h in (1).

Our validation method is based on an automatic reformulation of IVP (1) into an equivalent Volterra linear integral equation of the second kind [23] (simply referred to as integral equation in this article):

$$\begin{aligned} f(x) + \varvec{\mathrm {K}}\{f\}(x) = g(x), \qquad \text {where}\quad \varvec{\mathrm {K}}\{f\}(x) = \int _{x_0}^x {\mathfrak {K}}(x,t) f(t) \mathrm {d}t. \end{aligned}$$

(2)

The bivariate kernel ${\mathfrak {K}}(x,t)$ associated to the integral operator $\varvec{\mathrm {K}}$ is analytic, and even polynomial if the $a_i$ are. Therefore, representations through integral equations are equally algorithmic. Moreover, such integral equations are particularly suited for numerical resolution via Galerkin spectral methods [7, 15, 29], which compute a polynomial approximation $y^\circ$ for Problem 1.1 using truncated expansions (e.g., Taylor, Fourier, Chebyshev), by transforming functional equation (2) into a linear system on the unknown coefficients.

Several works, e.g. [17, 24, 41], proposed ODE validation methods based on a posteriori Newton-like methods in well-chosen coefficient spaces. The main idea is to approximate the (infinite-dimensional) inverse Jacobian by finite-dimensional truncations, as done by spectral methods in numerical analysis, in order to construct a contracting Newton-like operator and apply the Banach fixed-point theorem or a variant of it. Such methods, which we propose to call Newton–Galerkin validation methods, have been extensively used in the mathematical community of computer-assisted proofs. In [8], we developed a detailed analysis of a Newton–Galerkin validation algorithm in the Chebyshev basis, by enhancing this general scheme with the linear algebra algorithms of [29] for so-called almost-banded linear systems. In the present article, we propose an alternative validation algorithm, which we call Newton–Picard, to overcome the complexity shortcomings of Newton–Galerkin validation methods, acknowledged in [8]. The leitmotif is to approximate the inverse Jacobian by computing polynomial approximations of the so-called resolvent kernel associated to (1) which arises from the well-known Picard iterations. More or less related ideas have already been used in previous works, notably [2, 21, 30]. However, the algorithmic description and complexity analysis we propose here lead to a robust and efficient validation algorithm, together with a fair comparison with Newton–Galerkin. Our conclusions are the following.

• 1. We prove that Newton–Picard runs in polynomial time with respect to the magnitude of the coefficients in the input equation, while Newton–Galerkin’s complexity can be exponential in the same parameters, even, surprisingly, in the analytic case we consider here. The result is that moderate to hard instances of Problem 1.1 (e.g., exponentially increasing or highly oscillating functions) may be intractable with Newton–Galerkin, but solved within a few seconds with Newton–Picard on a modern computer.

• 2. The Newton–Picard validation method is widely independent of the underlying approximation tools, although presented in a Chebyshev approximation theory flavor in this article. Indeed, the algorithms could be parametrized by a generic type of RPAs to rigorously enclose functions without major modifications. Instead of Chebyshev models, one could use Fourier approximations (for trigonometric functions), splines (for ${{\mathcal {C}}}^k$ functions), depending on the context. Validation of solutions with singularities or on unbounded intervals also belongs to future applications of Newton–Picard method.

• 3. A challenging goal we want to address in the future is the formalization in a proof assistant, for example Coq [4], of an ODE a posteriori validation algorithm. To this purpose, Newton–Picard turns out to be a better candidate than Newton–Galerkin. Indeed, the correctness of the algorithms presented in this article requires very limited functional analysis background, contrary to Newton–Galerkin which involves more Chebyshev approximation theory. Moreover, the resolvent kernel approximation can be safely computed outside the proof assistant without compromising the final reliability.

Throughout this article, we consider an arithmetic model for the complexity, where each arithmetic operation ($+, -, \times , \div$), typically on floating-point numbers or intervals, is assumed to have a constant cost ${\mathcal {O}}(1)$. Therefore, the claimed complexity estimates do not take into account the fact that particularly stiff ODEs may require a larger floating-point bit precision to provide sufficiently accurate results.

1.3 Outline

First, Sect. 2 provides the necessary background (a posteriori Newton-like validation principle, reformulation of IVP (1) into integral Eq. (2), Chebyshev approximation theory and spectral methods) and summarizes the main lines of [8] about the Newton–Galerkin validation method. Then, Sect. 3 introduces Newton–Picard validation method, with the main algorithm NewtonPicardValid and the approximation routine ResolventKernel for the resolvent kernel approximation, together with detailed complexity results. After that, Sect. 4 presents how the method is implemented in the C library ChebValid² and illustrates its performances on practical examples, in comparison with other methods. We finally discuss the obtained results and the future perspectives of this work.

2 A Posteriori Validation and Newton–Galerkin Method

This section lays out the main prerequisites to this article: a posteriori validation paradigm using Newton-like fixed-point operators (Sect. 2.1), transformation(s) of linear IVPs (1) into integral equations (2) (Sect. 2.2), some highlights of Chebyshev approximation theory and in particular spectral methods (Sect. 2.3). Finally, Sect. 2.4 gives an overview of the principles and limitations of Newton–Galerkin validation method, thoroughly investigated in [8]. This serves as a basis for comparison with the novel Newton–Picard algorithm presented in Sect. 3.

2.1 Principle of A Posteriori Validation Using Newton-Like Fixed-Point Operators

In an a posteriori validation scheme, an approximation $x^\circ$ to the exact solution $x^*$ is first constructed by whatever numerical routine of choice, and then a rigorous error bound between $x^\circ$ and $x^*$ is computed in a totally independent second step. Some advantages of this paradigm include the possibility to use external non-trusted numerical routines for the approximation step without compromising the final reliability; the need for rigorous numerics (e.g., interval arithmetics) in the validation step only; and a rather elegant mathematical formulation that eases the verification by a formal proof assistant.

A wide range of such methods involve fixed-point based validation, in which $x^*$ is expressed as the fixed point of a well chosen equation $\varvec{\mathrm {T}}(x) = x$, with $\varvec{\mathrm {T}}$ a contracting operator. Then a rigorous error bound is obtained using a suitable fixed-point theorem, for example this formulation of the Banach fixed-point theorem [3, Thm. 2.1].

Theorem 2.1

Let $\varvec{\mathrm {T}} : X \rightarrow X$ with $(X,\Vert \cdot \Vert )$ a Banach space and $D \subseteq X$ closed, such that:

• D is $\varvec{\mathrm {T}}$-stable:    $\varvec{\mathrm {T}}(D) \subseteq D$;

• $\varvec{\mathrm {T}}$ is contracting over D: there exists $\lambda \in [0,1)$ such that:

  $$\begin{aligned} \Vert \varvec{\mathrm {T}}(x) - \varvec{\mathrm {T}}(y)\Vert \leqslant \lambda \Vert x - y\Vert , \qquad \qquad \text {for all } x, y \in D. \end{aligned}$$

Then:

• $\varvec{\mathrm {T}}$ has a unique fixed point $x^*$ in D;

• For any $x^\circ \in D$,

  $$\begin{aligned} \frac{d}{1+\lambda } \leqslant \Vert x^\circ - x^*\Vert \leqslant \frac{d}{1-\lambda }, \qquad \text {where} \quad d = \Vert \varvec{\mathrm {T}}(x^\circ ) - x^\circ \Vert . \end{aligned}$$

Most of the problems encountered in scientific computing are certainly not of this form. In this context, Newton-like validation methods (also called Krawczyk methods) reuse the ideas behind Newton-Raphson iterations in numerical analysis to transform the initial problem into an instance on which Theorem 2.1 applies. Assume that the original equation is of the form $\varvec{\mathrm {F}}(x) = 0$, with $\varvec{\mathrm {F}} : X \rightarrow Y$ a Fréchet differentiable map between Banach spaces X and Y. The exact solution $x^*$ is supposed to be simple root of $\varvec{\mathrm {F}}$, so that the Fréchet derivative $\mathrm {D}\varvec{\mathrm {F}}(x)$ is invertible in a neighborhood of $x^*$. We are given an approximation $x^\circ$ of $x^*$.

• 1. Compute a bounded injective linear operator $\varvec{\mathrm {A}}$ approximating $\mathrm {D}\varvec{\mathrm {F}}(x^\circ )^{-1}$, the inverse of the Fréchet derivative of $\varvec{\mathrm {F}}$ at $x^\circ$. Then $\varvec{\mathrm {F}}(x)=0$ is equivalent to the fixed-point equation associated to the Newton-like operator $\varvec{\mathrm {T}}$:

  $$\begin{aligned} \varvec{\mathrm {T}}(x) = x, \qquad \text {where} \quad \varvec{\mathrm {T}}(x) = x - \varvec{\mathrm {A}}(\varvec{\mathrm {F}}(x)). \end{aligned}$$

• 2. Compute rigorously a closed subset $D \subseteq X$ containing $x^\circ$ (e.g. a ball centered at $x^\circ$ with suitable radius r) and a contraction ratio $\lambda < 1$ satisfying the assumptions of Theorem 2.1.

• 3. Return the error bound $\varepsilon = \Vert \varvec{\mathrm {A}}(\varvec{\mathrm {F}}(x^\circ ))\Vert / (1-\lambda )$ as in Theorem 2.1. If $\varvec{\mathrm {F}}$ has several roots in X, also ensure that the unique fixed-point of $\varvec{\mathrm {T}}$ inside D is necessarily $x^*$.

Remark 2.2

The IVP (1) (or the equivalent integral equation (2)) we consider in this article are linear, which somewhat simplifies the scheme above. The Newton-like operator $\varvec{\mathrm {T}}$ is indeed affine, and hence it is locally contracting if and only if it is globally contracting (i.e. $D=X$), and $\lambda$ is given by the operator norm of its linear part. Moreover, the solution being unique by the Picard-Lindelöf theorem, no additional verification is required in step 3 above.

2.2 Reformulation of Linear ODE into Integral Equation

Several reasons may lead us to prefer integral equations rather than differential ones in numerical analysis. From a theoretical point of view, the differentiation operator $\varvec{\mathrm {\partial }}$ can rarely be seen as a bounded linear endomorphism in one function space, since it typically reduces the regularity. Manipulating differential equations thus requires to consider a separate function space for each iterated application of $\varvec{\mathrm {\partial }}$ (which is for example the approach of [29] using ultraspherical polynomials). On the contrary, the operation of integration increases the regularity, which allows us to work in a single function space. Moreover, the compactness property, necessary for the projection approach of spectral methods, easily follows (see Sect. 2.3). On the practical side also, integral equations often lead to better conditioned numerical problems [16].

There are several possibilities to transform a linear ODE into an equivalent integral equation, acting on the same function y or one of its iterated derivatives. In this article, we focus on two such transforms, giving integral equations acting on y and $y^{(r)}$, respectively. The obvious “duality” between them will play an important role later in Sect. 3.

2.2.1 Integral Equation on Highest-Order Derivative

This transformation, considered for example in [8], is the simplest one, acting on $f=y^{(r)}$. We here consider a differential equation $\varvec{\mathrm {L}}\{y\} = h$ with a linear differential operator $\varvec{\mathrm {L}}$ given in a “standard form”, which we call $\varvec{\mathrm {x}}-\varvec{\mathrm {\partial }}$ form:

$$\begin{aligned} \varvec{\mathrm {L}} = \varvec{\mathrm {\partial }}^r + a_{r-1}(x) \varvec{\mathrm {\partial }}^{r-1} + \dots + a_1(x) \varvec{\mathrm {\partial }}+ a_0(x). \end{aligned}$$

(3)

Proposition 2.3

The linear initial value problem:

$$\begin{aligned} \left\{ \begin{aligned} \varvec{\mathrm {L}}\{y\}(x)&= h(x),&x \in [x_l,x_r]&, \\ y^{(i)}(x_0)&= v_i,&0 \leqslant i < r&, \end{aligned}\right. \qquad \text {with} \quad \varvec{\mathrm {L}} \text { in } \varvec{\mathrm {x}}-\varvec{\mathrm {\partial }}\text { form }(3), \end{aligned}$$

is equivalent to the integral equation on $f = y^{(r)}$:

$$\begin{aligned} \begin{aligned} f(x) + \varvec{\mathrm {K}}\{f\}(x) = g(x), \qquad&\text {where}\quad \varvec{\mathrm {K}}\{f\}(x) = \int _{x_0}^x {\mathfrak {K}}(x,t) f(t) \mathrm {d}t,\\ \text {with} \quad {\mathfrak {K}}(x,t) = \sum _{i=0}^{r-1} a_i(x) \frac{(x-t)^{r-1-i}}{(r-1-i)!}, \qquad&\text {and} \qquad g(x) = h(x) - \sum _{i=0}^{r-1} a_i(x) \sum _{j=i}^{r-1} \frac{(x-x_0)^{j-i}}{(j-i)!} v_j. \end{aligned} \end{aligned}$$

Proof

By a repeated use of the integration by part formula, one has for $0 \leqslant i < r$:

$$\begin{aligned} \begin{aligned} y^{(i)}(x)&= \int _{x_0}^x \int _{x_0}^{t_1} \dots \int _{x_0}^{t_{r-1-i}} f(t_{r-i}) \mathrm {d}t_{r-1} \dots \mathrm {d}t_1 + \sum _{j=i}^{r-1} \frac{(x-x_0)^{j-i}}{(j-i)!} v_j \\&= \int _{x_0}^x \frac{(x-t)^{r-1-i}}{(r-1-i)!} f(t) \mathrm {d}t + \sum _{j=i}^{r-1} \frac{(x-x_0)^{j-i}}{(j-i)!} v_j. \end{aligned} \end{aligned}$$

Doing the substitutions in $\varvec{\mathrm {L}}\{y\}=h$, regrouping the integral terms and moving the initial condition terms to the right, one obtains the expected kernel ${\mathfrak {K}}(x,t)$ and the right-hand side g(x). $\square$

2.2.2 Integral Equation on the Same Function

Another possible reformulation, used for example in [2], is to define an integral equation directly on $f=y$. To do so, we need to have $\varvec{\mathrm {L}}$ in a $\varvec{\mathrm {\partial }}-\varvec{\mathrm {x}}$ form, with the polynomial (or analytic) coefficients $a_i(x)$ located to the right of the $\varvec{\mathrm {\partial }}$ symbols:

$$\begin{aligned} \varvec{\mathrm {L}} = (-\varvec{\mathrm {\partial }})^r + (-\varvec{\mathrm {\partial }})^{r-1} a_{r-1}(x) + \dots - \varvec{\mathrm {\partial }}a_1(x) + a_0(x). \end{aligned}$$

(4)

We systematically use $-\varvec{\mathrm {\partial }}$ instead of simply $\varvec{\mathrm {\partial }}$ in $\varvec{\mathrm {\partial }}-\varvec{\mathrm {x}}$ forms to be consistent with the adjunction $\varvec{\mathrm {L}} \mapsto \varvec{\mathrm {L^*}}$ that will play a crucial role in Sect. 3.3.

While a $\varvec{\mathrm {x}}-\varvec{\mathrm {\partial }}$ form linear ODE relates together the iterated derivatives $y^{(i)}$ where $y^{(i+1)} = \varvec{\mathrm {\partial }}\{ y^{(i)}\}$, it is more natural with a $\varvec{\mathrm {\partial }}-\varvec{\mathrm {x}}$ form to consider functions $y^{[i]}$ for $0 \leqslant i \leqslant r$, with $y^{[0]} = y$, $y^{[i+1]} = a_{r-i-1} y - \varvec{\mathrm {\partial }}\{y^{[i]}\}$, and then equating $y^{[r]}(x) = h(x)$. By doing so, we have $y^{[i]} = \varvec{\mathrm {L_{[i]}}}\{y\}$ where:

$$\begin{aligned} \varvec{\mathrm {L_{[i]}}} = (-\varvec{\mathrm {\partial }})^{i} + (-\varvec{\mathrm {\partial }})^{i-1} a_{r-1}(x) + \dots - \varvec{\mathrm {\partial }}a_{r-i+1}(x) + a_{r-i}(x), \end{aligned}$$

so that the “natural” $\varvec{\mathrm {\partial }}-\varvec{\mathrm {x}}$ initial conditions at $x_0$ are $y^{[i]}(x_0) = \varvec{\mathrm {L_{[i]}}}\{y\}(x_0) = w_i$ for $0 \leqslant i < r$.

Remark 2.4

It is clear that, as long as one can differentiate the $a_i(x)$ symbolically sufficiently many times, one easily converts between $\varvec{\mathrm {x}}-\varvec{\mathrm {\partial }}$ and $\varvec{\mathrm {\partial }}-\varvec{\mathrm {x}}$ operators $\varvec{\mathrm {L}}$ and initial conditions. Note however that for large r, this may result in a differential operator with enormous coefficients, potentially leading to numerical stability issues.

The integral equation on $f=y$ follows by merely integrating the linear ODE r times.

Proposition 2.5

The linear initial value problem:

$$\begin{aligned} \left\{ \begin{aligned}&\varvec{\mathrm {L}}\{y\}(x) = h(x),&x \in [x_l,x_r]&, \\&\varvec{\mathrm {L_{[i]}}}\{y\}(x_0) = w_i,&0 \leqslant i < r&, \end{aligned}\right. \qquad \text {with} \quad \varvec{\mathrm {L}} \text { in } \varvec{\mathrm {\partial }}-\varvec{\mathrm {x}}\text { form }(4), \end{aligned}$$

is equivalent to the integral equation on y:

$$\begin{aligned} \begin{aligned} y(x) + \varvec{\mathrm {K}}\{y\}(x) = g(x), \qquad&\text {where}\quad \varvec{\mathrm {K}}\{y\}(x) = \int _{x_0}^x {\mathfrak {K}}(x,t) y(t) \mathrm {d}t,\\ \text {with}\quad {\mathfrak {K}}(x,t) = -\sum _{i=0}^{r-1} \frac{(t-x)^{r-1-i}}{(r-1-i)!} a_i(t), \qquad&\text {and} \qquad g(x) = -\int _{x_0}^x \frac{(t-x)^{r-1}}{(r-1)!} h(t) \mathrm {d}t + \sum _{i=0}^{r-1} w_i \frac{(x_0-x)^i}{i!}. \end{aligned} \end{aligned}$$

Proof

We first prove by induction on $0 \leqslant k \leqslant r$:

$$\begin{aligned} \int _{x_0}^x \!\!\!\! \dots \!\! \int _{x_0}^{t_{k-1}}\!\!\!\!\!\! y^{[k]}(t_k) \mathrm {d}t_k \dots \mathrm {d}t_1 = (-1)^k \left( y - \int _{x_0}^x \sum _{i=r-k}^{r-1} \frac{(t-x)^{r-1-i}}{(r-1-i)!} a_i(t) y(t) \mathrm {d}t - \sum _{i=0}^{k-1} \frac{(x_0-x)^i}{i!} w_i \right) . \end{aligned}$$

(5)

This is trivially true for $k=0$. If it holds for $k < r$, since $y^{[k+1]} = a_{r-k-1} y - \varvec{\mathrm {\partial }}\{y^{[k]}\}$, then:

$$\begin{aligned} \begin{aligned} \int _{x_0}^x \!\!\!\!\! \dots \!\! \int _{x_0}^{t_k}\!\!\!\!\!\! y^{[k+1]}&(t_{k+1}) \mathrm {d}t_{k+1} \dots \mathrm {d}t_1 = \int _{x_0}^x \!\!\!\!\! \dots \!\! \int _{x_0}^{t_k}\!\!\!\!\!\! a_{r-k-1}(t) y(t) \mathrm {d}t_{k+1} \dots \mathrm {d}t_1 - \int _{x_0}^x \!\!\!\!\! \dots \!\! \int _{x_0}^{t_{k-1}}\!\!\!\!\!\! (y^{[k]}(t_k) - w_k) \mathrm {d}t_k \dots \mathrm {d}t_1 \\&= \int _{x_0}^x \frac{(x-t)^k}{k!} a_{r-k-1}(t) y(t)\mathrm {d}t - \int _{x_0}^x \!\!\!\! \dots \!\! \int _{x_0}^{t_{k-1}}\!\!\!\!\!\! y^{[k]}(t_k) \mathrm {d}t_k \dots \mathrm {d}t_1 + \frac{(x-x_0)^k}{k!} w_k \\&= (-1)^{k+1} \left( y - \int _{x_0}^x \sum _{i=r-k-1}^{r-1} \frac{(t-x)^{r-1-i}}{(r-1-i)!} a_i(t) y(t) \mathrm {d}t - \sum _{i=0}^k \frac{(x_0-x)^i}{i!} w_i \right) . \end{aligned} \end{aligned}$$

This completes the proof of (5). Taking this equality with $k=r$ allows us to rewrite:

$$\begin{aligned} (-1)^r \int _{x_0}^x \!\!\!\! \dots \!\! \int _{x_0}^{t_{k-1}}\!\!\!\!\!\! y^{[r]}(t_k) \mathrm {d}t_k \dots \mathrm {d}t_1 = (-1)^r \int _{x_0}^x \!\!\!\! \dots \!\! \int _{x_0}^{t_{k-1}}\!\!\!\!\!\! h(t_k) \mathrm {d}t_k \dots \mathrm {d}t_1, \end{aligned}$$

into the desired integral equation with expected ${\mathfrak {K}}(x,t)$ and g(x). $\square$

2.3 Chebyshev Approximation Theory and Spectral Methods

When dealing with analytic functions over a compact real interval $[x_l,x_r]$, Chebyshev approximation theory turns out to be an efficient generic tool to provide global, smooth and remarkably accurate approximants. Here we simply recall some fundamentals of this beautiful and rich theory, to which numerous introductory books are dedicated [7, 14, 36].

The Chebyshev polynomials form a graded family of univariate polynomials, commonly defined over $[-1,1]$ by the three-term recurrence:

$$\begin{aligned} T_0(x) = 1, \qquad T_1(x) = x, \qquad T_{n+2}(x) = 2 x T_{n+1}(x) - T_n(x) \quad \text {for } n \geqslant 0. \end{aligned}$$

The mantra from a complexity point of view is that manipulating polynomials in the Chebyshev basis is asymptotically equivalent to the same operations in the monomial basis. Indeed, multiplication and differentiation/integration are given by the formulas:

$$\begin{aligned} T_n(x) T_m(x)&= \frac{1}{2}(T_{n+m}(x) + T_{n-m}(x)),&\quad&\text {for } 0 \leqslant m \leqslant n, \end{aligned}$$

(6)

$$\begin{aligned} T_n(x)&= \frac{1}{2} \left( \frac{T_{n+1}(x)}{n+1} - \frac{T_{n-1}(x)}{n-1}\right) ',&\quad&\text {for } n \geqslant 2. \end{aligned}$$

(7)

Remark 2.6

Multiplication of two degree n polynomials in the Chebyshev basis can be computed in ${\mathcal {O}}(n \log n)$ floating-point arithmetic operations thanks to fast algorithms for Discrete Cosine Transform (DCT), which is the analog to the Fast Fourier Transform (FFT). However, designing a numerically stable algorithm for fast multiplication of polynomials with interval coefficients (in the monomial or Chebyshev basis) is still an open and debated topic.³ This is why in Sect. 3 we will assume a quadratic complexity when interval arithmetics is needed.

From an analysis point of view, the relation $T_n(\cos \vartheta ) = \cos (n\vartheta )$ strongly connects Chebyshev polynomials with Fourier approximation theory. In particular, $|T_n(x)| \leqslant 1$ for $-1 \leqslant x \leqslant 1$ and they form an orthogonal family w.r.t. the scalar product:

$$\begin{aligned} \langle f, g \rangle = \int _{-1}^1 \frac{f(x)g(x)}{\sqrt{1-x^2}} \mathrm {d}x = \int _0^\pi f(\cos \vartheta ) g(\cos \vartheta ) \mathrm {d}\vartheta , \end{aligned}$$

which motivates the definition of Chebyshev coefficients for a continuous function f over $[-1,1]$ by orthogonal projection:

$$\begin{aligned} {[}f]_n\ =\ \frac{\langle f, T_n \rangle }{\langle T_n, T_n \rangle } \ = \ \left\{ \begin{aligned}&\frac{1}{\pi } \langle f, 1 \rangle&\text {for } n = 0, \\&\frac{2}{\pi } \langle f, T_n \rangle&\text {for } n \geqslant 1. \end{aligned}\right. \end{aligned}$$

Remark 2.7

By an affine change of variable, the above definitions are transposed from $[-1,1]$ to any compact interval. We assume in this article that the Chebyshev polynomials $T_n$ are defined over $[x_l,x_r]$.

Chebyshev approximations are defined by truncated Chebyshev series using the truncation (or projection) operator:

The key point of Chebyshev approximation theory is that the more regular f is, the faster converges to f as $N \rightarrow \infty$. In the analytic case we consider—and contrary to Taylor approximations—convergence on $[x_l,x_r]$ always holds, at exponential rate with ratio $\rho$ determined by the location of singularities in the complex plane (Lemma 3.15 in Sect. 3 gives a quantitative estimate).

When working with infinite sequences of Chebyshev coefficients, a relevant subspace of ${{\mathcal {C}}}^0$ is the space of absolutely summable Chebyshev series [8, Sec. 2.2], which is the analog of the Wiener algebra $A({\mathbb {T}})$ for Fourier series.

Definition 2.8

The space is the space of continuous functions f over $[x_l,x_r]$ for which the quantity,

is finite. This makes a Banach space, with the property .

Obviously, Chebyshev coefficients $[f]_n$ cannot be directly computed most of the time. Turning back to the integral equation (2), a spectral method approach [7, 15] is to cast this problem into an infinite-dimensional linear system in the Chebyshev space of coefficients. More specifically, the integral operator $\varvec{\mathrm {K}}$ of polynomial kernel ${\mathfrak {K}}(x,t)$ becomes a bounded endomorphism of , acting on the vector of Chebyshev coefficients of a function f as an infinite-dimensional matrix (depicted in Fig. 1a). The multiplication (6) and integration (7) formulas imply that $\varvec{\mathrm {K}}$ has a sparse structure called almost-banded (following the terminology of [29]), that is, nonzero coefficients are located in a finite number of rows and diagonals. Moreover, $\varvec{\mathrm {K}}$ is a compact endomorphism of , where coefficients in the matrix tend to zero. Therefore, spectral methods relax this infinite-dimensional problem by projecting it onto a finite-dimensional subspace, using the truncated integral operator (see Fig. 1b). The resulting almost-banded linear system of dimension $N+1$ is eventually solved in linear time w.r.t. the approximation degree N using the QR-like algorithms of [29].

Proposition 2.9

(see [29, Sec. 5]) The solution of the integral equation (2) can be approximated by a degree N polynomial in the Chebyshev basis using a spectral method requiring ${\mathcal {O}}(N (r+s)^2)$ arithmetic operations. More precisely:

• A first algorithm AlmostBandedQRFactor computes a (structured) QR factorization of $I_{N+1} + \varvec{\mathrm {K}}^{[N]}$ in ${\mathcal {O}}(N (r+s)^2)$ arithmetic operations.

• Given this QR factorization, a second algorithm AlmostBandedBackSubs performs a back-substitution to solve:

  

  of unknown $y^\circ = \sum \limits _{n=0}^N c_n T_n$, in ${\mathcal {O}}(N (r+s))$ arithmetic operations.

Fig. 1

Almost-banded matrix representations in the Chebyshev basis of an integral operator $\varvec{\mathrm {K}}$ and its truncation $\varvec{\mathrm {K}}^{[N]}$

2.4 A Brief Summary of the Newton–Galerkin Validation Method

Problem 1.1 typically falls within the class of function space problems that can be solved using the fixed-point based a posteriori validation scheme presented in Sect. 2.1. One needs to compute a rigorous error bound between an approximation $f^\circ$ and the exact solution $f^*$ of the affine equation:

We recall the main ideas of Newton–Galerkin validation method, in the way it is presented in [8], with complexity analysis results and discussion about its shortcomings.

2.4.1 Newton–Galerkin Fixed-Point Operator and Validation Algorithm

The principle of the Newton–Galerkin validation algorithm is to construct the linear operator $\varvec{\mathrm {A}} \approx (\varvec{\mathrm {I}}+\varvec{\mathrm {K}})^{-1}$ (required by step 1 of Newton-like validation) using finite dimensional projections. Indeed, $\varvec{\mathrm {K}}^{[N]}$ converges to $\varvec{\mathrm {K}}$, so the algorithm chooses a suitable truncation order $N_{{\mathcal {G}}}$ (which is totally independent of the approximation $f^\circ$ to be validated) and computes a numerical inverse A of the $(N_{{\mathcal {G}}}+1)$-square matrix $I_{N_{{\mathcal {G}}}+1} + \varvec{\mathrm {K}}^{[N_{{\mathcal {G}}}]}$. $\varvec{\mathrm {A}}$ is defined by extending the matrix A over with identity (see Fig. 2a).

After that, in step 2, a contraction ratio $\lambda$ for the resulting Newton–Galerkin fixed-point operator $\varvec{\mathrm {T}}$ is automatically computed by rigorously bounding the operator norm (w.r.t. the -norm) of the linear part, simply denoted by $\mathrm {D}\varvec{\mathrm {T}}$:

(8)

• The first quantity is due to the fact that A is computed as a numerical inverse (either dense, or even approximated by an almost-banded matrix, see [8, sec. 4.2]). Bounding this quantity only requires straightforward operations on finite-dimensional matrices.

• The second quantity , where the two matrices of this product are depicted in Fig. 2, is a consequence of the fact that $\varvec{\mathrm {K}}^{[N_{{\mathcal {G}}}]}$ is only an approximation of $\varvec{\mathrm {K}}$. The difficulty, addressed in details in [8, Sec. 5.1], consists in rigorously computing a tight bound for the resulting infinite-dimensional tail, in reasonable complexity. Indeed, gross overestimations would force us to use a larger truncation index $N_{{\mathcal {G}}}$ to ensure a contraction ratio $\lambda < 1$, thus strongly impacting the overall efficiency.

Finally, the defect (step 3) is rigorously computed in a straightforward manner, yielding the final bound $d/(1-\lambda )$.

Theorem 2.10

(see [8, Prop. 5.3]) Let $f^\circ$ be an approximate solution for the integral equation (2). Then, given a validation truncation index $N_{{\mathcal {G}}}$, algorithms in [8], if they do not fail, realize the following⁴:

• A Newton–Galerkin fixed-point validation operator $\varvec{\mathrm {T}}$ with truncation index $N_{{\mathcal {G}}}$ is computed and bounded in

  $$\begin{aligned} {\mathcal {O}}\left( N_{{\mathcal {G}}}^2(r+s)\right) \quad \text {arithmetic operations}, \end{aligned}$$

  where r is the order of $\varvec{\mathrm {L}}$ and $s = \max _i \deg a_i$ in Problem 1.1.

• Having this Newton–Galerkin operator and if the rigorously computed contraction ratio $\lambda < 1$, then a candidate approximation $f^\circ$ (with right-hand side g) is validated in

  $$\begin{aligned} {\mathcal {O}}\left( N_{{\mathcal {G}}}(n+m+r+s) + n (r+s)^2\right) \quad \text {arithmetic operations}, \end{aligned}$$

  where $n = \deg f^\circ$ and $m = \deg g$.

Obviously, the overall efficiency of the procedure depends on how large $N_{{\mathcal {G}}}$ must be to ensure a contracting operator $\varvec{\mathrm {T}}$. This crucial question is discussed in the next section.

Fig. 2

Bounding the truncation error in Newton–Galerkin method

2.4.2 Limitations of the Newton–Galerkin Validation Method

As pointed out in [8], the Newton–Galerkin validation method, although widely used in the mathematical community of computer-assisted proofs, suffers from several limitations, especially when targeting certified implementation in a proof assistant.

The first shortcoming is the difficulty to process “harder” instances, e.g. LODEs with highly oscillating or rapidly increasing solutions, due to the value of the truncation index $N_{{\mathcal {G}}}$. Indeed, $\varvec{\mathrm {K}}^{[N_{{\mathcal {G}}}]}$ converges to $\varvec{\mathrm {K}}$ in ${\mathcal {O}}(1/N_{{\mathcal {G}}})$ only, while the -norm of $(\varvec{\mathrm {I}}+\varvec{\mathrm {K}})^{-1}$ (and hence the approximation $\varvec{\mathrm {A}}$) is exponentially bounded by the magnitude of the input LODE coefficients. This explains why the minimum $N_{{\mathcal {G}}}$ to ensure a contracting operator $\varvec{\mathrm {T}}$ can grow exponentially fast with the [8, Sec. 5.2]. Although other norms for the space of Chebyshev coefficients have been investigated, e.g. in [17, 24], it seems that none of them avoid this exponential bound.

Another limitation of Newton–Galerkin validation methods is that implementations can hardly be generic w.r.t. the approximation setting. Indeed, changing the approximation basis from $(T_n)$ to another $(\varphi _n)$ (e.g., Fourier, Legendre) or the norm implies substantial modifications in the technical formulas used to bound the operator norm of $\varvec{\mathrm {T}}$. Moreover, not all norms are suitable when working with coefficient spaces. This is for example why we use to overapproximate $\Vert \cdot \Vert _{\infty }$ here.

Finally, the need to consider infinite-dimensional matrix representations of the involved operators, as well as the rather technical formulas that are used to bound them, makes Newton–Galerkin not the best candidate for a certified implementation in a proof assistant. The Newton–Picard validation algorithm presented in the next section not only offers a far more satisfying solution to the above-mentioned issues of complexity and genericness, it is also in essence more “algebraic” and adapted to formal proof.

3 Newton–Picard Validation Algorithm

The conclusion to draw from Newton–Galerkin’s pessimistic complexity estimates is that $(\varvec{\mathrm {I}}+\varvec{\mathrm {K}})^{-1}$ is poorly approximated by finite-dimensional truncations $(\varvec{\mathrm {I}}+\varvec{\mathrm {K}}^{[N_{{\mathcal {G}}}]})^{-1}$. By contrast, numerical experiments (see for example Fig. 3c related to the Airy function example in Sect. 4.2) show that $(\varvec{\mathrm {I}}+\varvec{\mathrm {K}})^{-1}$ is “asymptotically” almost-banded, meaning that it is very well approximated by almost-banded operators of moderate band width. This phenomenon is investigated in Sect. 3.1 from the point of view of integral operators. More specifically, we prove that $(\varvec{\mathrm {I}}+\varvec{\mathrm {K}})^{-1} = \varvec{\mathrm {I}}+\varvec{\mathrm {R}}$, where $\varvec{\mathrm {R}}$ is an integral operator of kernel ${\mathfrak {R}}(x,t)$ called resolvent kernel. This elegant framework is the cornerstone of the Newton–Picard validation algorithm presented in Sect. 3.2.

Contrary to the validation method [2] which performs explicit Picard iterations to construct a contracting operator, our novel approach relies on the Chebyshev spectral method to compute a polynomial approximation of the resolvent kernel ${\mathfrak {R}}(x,t)$ (Sect. 3.3). The resulting complexity is carefully investigated in Sect. 3.4 and compared with Newton–Galerkin.

3.1 Kernel Composition, Picard Iterations and the Resolvent Kernel

The essence of Newton–Picard validation is to represent an integral operator $\varvec{\mathrm {K}}$ not as an infinite-dimensional matrix in some coefficient space, but using its kernel ${\mathfrak {K}}(x,t)$ defined over $[x_l,x_r|x_0]$:

$$\begin{aligned} \begin{aligned}&\varvec{\mathrm {K}}\{f\}(x) = \int _{x_0}^x {\mathfrak {K}}(x,t) f(t) \mathrm {d}t, \qquad f \in {{\mathcal {C}}}^0, \\&[x_l,x_r|x_0] \ =\ \{(x,t) \in {{\mathbb {R}}}^2 \quad \text {s.t.}\quad x_l \leqslant x \leqslant t \leqslant x_0 \quad \text {or} \quad x_0 \leqslant t \leqslant x \leqslant x_r\}. \end{aligned} \end{aligned}$$

(9)

The following lemma shows the bijectivity of this correspondence.

Lemma 3.1

Let $\varvec{\mathrm {K}}$ be an integral operator with a continuous kernel ${\mathfrak {K}}(x,t)$ as in (9). Then $\varvec{\mathrm {K}}\{f\} = 0$ for all $f \in {{\mathcal {C}}}^0$ if and only if ${\mathfrak {K}}(x,t) = 0$ for all $(x,t) \in [x_l, x_r | x_0]$.

Proof

The “if” part of the proof is straightforward. For the “only-if” part, let $x \in [x_l,x_r]$ and suppose, without loss of generality, that $x \geqslant x_0$. Define $f \in {{\mathcal {C}}}^0$ by $f(t) = {\mathfrak {K}}(x,t)$ for $t \in [x_0,x]$ and extend it constantwise over $[x_l,x_0]$ and $[x,x_r]$. Then:

$$\begin{aligned} \varvec{\mathrm {K}}\{f\}(x) = \int _{x_0}^x {\mathfrak {K}}(x,t) f(t) \mathrm {d}t = \int _{x_0}^x {\mathfrak {K}}(x,t)^2 \mathrm {d}t = 0, \end{aligned}$$

which implies ${\mathfrak {K}}(x,t) = 0$ for all $t \in [x_0,x]$. Therefore, ${\mathfrak {K}}(x,t)=0$ for all $(x,t) \in [x_l,x_r|x_0]$. $\square$

Another nice property of the class of integral operators is its closedness under composition, where the resulting kernel is given by a “convolution-like” operation, denoted by $*$ throughout this article.

Lemma 3.2

Let $\varvec{\mathrm {K}}$ and $\varvec{\mathrm {L}}$ be integral operators with respective continuous kernels ${\mathfrak {K}}(x,t)$ and ${\mathfrak {L}}(x,t)$. Then the composition $\varvec{\mathrm {K}} \, \varvec{\mathrm {L}}$ is again an integral operator, with kernel ${\mathfrak {K}}*{\mathfrak {L}}$ given by the associative law:

$$\begin{aligned} ({\mathfrak {K}} * {\mathfrak {L}})(x,t) = \int _t^x {\mathfrak {K}}(x,s) {\mathfrak {L}}(s,t) \mathrm {d}s, \qquad (x,t) \in [x_l,x_r|x_0]. \end{aligned}$$

In particular, if ${\mathfrak {K}}$ and ${\mathfrak {L}}$ are polynomials, then so is ${\mathfrak {K}} * {\mathfrak {L}}$, with:

$$\begin{aligned} \deg {\mathfrak {K}} * {\mathfrak {L}} \leqslant \deg {\mathfrak {K}} + \deg {\mathfrak {L}} + 1. \end{aligned}$$

Proof

The kernel composition formula simply follows from an application of Fubini’s theorem:

$$\begin{aligned} \varvec{\mathrm {K}}\{\varvec{\mathrm {L}}\{f\}\} = \int _{x_0}^x {\mathfrak {K}}(x,s) \int _{x_0}^s {\mathfrak {L}}(s,t) f(t) \mathrm {d}t \mathrm {d}s = \int _{x_0}^x \int _t^x {\mathfrak {K}}(x,s) {\mathfrak {L}}(s,t) f(t) \mathrm {d}s \mathrm {d}t = \int _{x_0}^x ({\mathfrak {K}}*{\mathfrak {L}})(x,t) f(t) \mathrm {d}t. \end{aligned}$$

Moreover, since the composition $\circ$ of operators is associative and by injectivity given by Lemma 3.1, we deduce that the operation $*$ of kernel composition is associative too. $\square$

Historically, the expression of $(\varvec{\mathrm {I}}+\varvec{\mathrm {K}})^{-1}$ involving an integral operator arose from the Picard iterations [23, 31], which build a sequence $(f_n)$ of functions converging to the solution $f^*$ of (2):

$$\begin{aligned} \left\{ \begin{aligned}&f_0 \ = \ g, \\&f_{n+1} \ = \ g - \varvec{\mathrm {K}}\{f_n\}, \quad n \geqslant 0. \end{aligned}\right. \end{aligned}$$

This indeed motivates to write $(\varvec{\mathrm {I}}+\varvec{\mathrm {K}})^{-1}$ (at least symbolically) as the Neumann series:

$$\begin{aligned} (\varvec{\mathrm {I}}+\varvec{\mathrm {K}})^{-1} = \varvec{\mathrm {I}} - \varvec{\mathrm {K}} + \varvec{\mathrm {K}}^2 - \dots + (-1)^n \varvec{\mathrm {K}}^n + \dots \end{aligned}$$

(10)

This matches with the observation that $(\varvec{\mathrm {I}}+\varvec{\mathrm {K}})^{-1}$ is “asymptotically” almost-banded, since $\varvec{\mathrm {K}}^n$ is almost-banded, of band width n times larger than $\varvec{\mathrm {K}}$. The classic Theorem 3.3 below, for which we provide a proof for the sake of completeness, makes it rigorous by establishing the convergence of this series. This gives rise to the resolvent kernel ${\mathfrak {R}}(x,t)$ that plays a central role in Newton–Picard validation method.

Theorem 3.3

Let $\varvec{\mathrm {K}}$ of continuous kernel ${\mathfrak {K}}(x,t)$ act as an endomorphism of ${{\mathcal {C}}}^0$. Then $(\varvec{\mathrm {I}} + \varvec{\mathrm {K}})^{-1} = \varvec{\mathrm {I}} + \varvec{\mathrm {R}}$, where the resolvent kernel ${\mathfrak {R}}(x,t)$ is defined using the iterated kernels ${\mathfrak {K}}^{*n}$:

$$\begin{aligned} {\mathfrak {R}} = \sum _{n=1}^\infty (-1)^n {\mathfrak {K}}^{*n}, \qquad \text {where} \quad \left\{ \begin{aligned}&{\mathfrak {K}}^{*1} = {\mathfrak {K}}, \\&{\mathfrak {K}}^{*n+1} = {\mathfrak {K}} * {\mathfrak {K}}^{*n}, \quad n \geqslant 1. \end{aligned}\right. \end{aligned}$$

(11)

We have the following estimates:

$$\begin{aligned}&|{\mathfrak {R}}(x,t)| \leqslant M e^{M |x-t|},&\text {for } (x,t) \in [x_l,x_r|x_0], \\&\Vert (\varvec{\mathrm {I}}+\varvec{\mathrm {K}})^{-1}\Vert _\infty = \Vert \varvec{\mathrm {I}}+\varvec{\mathrm {R}}\Vert _\infty \leqslant e^{MT},&\text {where } M = \sup _{(x,t) \in [x_l,x_r|x_0]} |{\mathfrak {K}}(x,t)|. \end{aligned}$$

Proof

First, we prove by induction on $n \geqslant 1$ that:

$$\begin{aligned} |{\mathfrak {K}}^{*n}(x,t)| \leqslant M^n \frac{|x-t|^{n-1}}{(n-1)!} \qquad \text {for } (x,t) \in [x_l,x_r|x_0]. \end{aligned}$$

This clearly holds for $n=1$. Suppose that this is true for n. Take w.l.o.g. $x_0 \leqslant t \leqslant x \leqslant x_r$:

$$\begin{aligned} |{\mathfrak {K}}^{*n+1}(x,t)| \leqslant \int _t^x |{\mathfrak {K}}(x,s)| |{\mathfrak {K}}^{*n}(s,t)| \mathrm {d}s \leqslant \int _t^x M^{n+1} \frac{(s-t)^{n-1}}{(n-1)!} \mathrm {d}s = M^{n+1} \frac{(x-t)^n}{n!}. \end{aligned}$$

It follows therefrom that the series (11) defining ${\mathfrak {R}}(x,t)$ converges in norm, that $\Vert \varvec{\mathrm {K}}^n\Vert _\infty \leqslant \frac{(MT)^n}{n!}$, and that the Neumann series (10) makes sense, with the expected estimates for $|{\mathfrak {R}}(x,t)|$ and $\Vert \varvec{\mathrm {I}}+\varvec{\mathrm {R}}\Vert _\infty$. $\square$

The notion of resolvent kernel is closely related to that of Green’s function associated to a linear differential operator [31]. The validation methods proposed in [21] and [30] rigorously bound the Green’s function to provide a bound on the exact inverse operator associated to $\varvec{\mathrm {L}}$. Our validation algorithm, presented in next section, proceeds differently: it approximates the resolvent kernel and rigorously bounds the operator norm of the resulting Newton-like operator $\varvec{\mathrm {T}}$. This allows for sharper bounds and a more convenient complexity analysis.

3.2 The Validation Algorithm

Algorithm NewtonPicardValid below implements the three steps of the Newton-like a posteriori validation scheme presented in Sect. 2.1, by approximating the inverse derivative $(\varvec{\mathrm {I}}+\varvec{\mathrm {K}})^{-1}$ as $\varvec{\mathrm {I}}+\varvec{\mathrm {R^\circ }}$, where $\varvec{\mathrm {R^\circ }}$ is an integral operator of polynomial kernel ${\mathfrak {R}}^\circ (x,t)$ approximating the analytic resolvent kernel ${\mathfrak {R}}(x,t)$. Similarly to the truncation index $N_{{\mathcal {G}}}$ in Newton–Galerkin validation procedure, a parameter $N_{{\mathfrak {R}}}$ (the approximation degree for ${\mathfrak {R}}^\circ (x,t)$) is passed to the validation algorithm. In general, $N_{{\mathfrak {R}}}$ is at least $r+s$, where r is the order of $\varvec{\mathrm {L}}$ and $s = \max _{0 \leqslant i < r} \deg a_i$, so that we make the assumption $r+s = {\mathcal {O}}(N_{{\mathfrak {R}}})$ in the complexity estimates.

• 1. First, the subroutine ResolventKernel (postponed in Sect. 3.3) computes degree $N_{{\mathfrak {R}}}$ polynomials $\alpha _i^\circ , \beta _i^\circ$ and sets:

  $$\begin{aligned} {\mathfrak {R}}^\circ (x,t) = \sum _{i=0}^{r-1} \alpha _i^\circ (x) \beta _i^\circ (t) \approx {\mathfrak {R}}(x,t), \qquad \deg \alpha _i^\circ , \deg \beta _i^\circ \leqslant N_{{\mathfrak {R}}}. \end{aligned}$$

  (12)

  This defines the Newton–Picard fixed-point operator $\varvec{\mathrm {T}}\{f\} = f - (\varvec{\mathrm {I}} + \varvec{\mathrm {R}}^\circ ) \{f + \varvec{\mathrm {K}}\{f\} - g\}$.

• 2. The linear part of affine operator $\varvec{\mathrm {T}}$:

  $$\begin{aligned} \varvec{\mathrm {E}} = \varvec{\mathrm {I}} - (\varvec{\mathrm {I}} + \varvec{\mathrm {R}}^\circ ) (\varvec{\mathrm {I}} + \varvec{\mathrm {K}}) = - \varvec{\mathrm {K}} - \varvec{\mathrm {R}}^\circ - \varvec{\mathrm {R}}^\circ \varvec{\mathrm {K}}, \end{aligned}$$

  is an integral operator whose polynomial kernel ${\mathfrak {E}} = - {\mathfrak {K}} - {\mathfrak {R}}^\circ - {\mathfrak {R}}^\circ * {\mathfrak {K}}$ is explicitly computed as $\sum \limits _{i,j} \varepsilon _{ij} T_i(x) T_j(t)$. This yields the bound $\lambda$:

  $$\begin{aligned} \Vert \varvec{\mathrm {E}}\Vert _\infty = \sup _{x \in I} \ \ {\text {sgn}}(x-x_0) \int _{x_0}^x |{\mathfrak {E}}(x,t)| \mathrm {d}t \leqslant T \sum _{i,j} |\varepsilon _{ij}| = \lambda . \end{aligned}$$

• 3. If $\lambda \geqslant 1$, then go back to step 1 using a larger approximation degree $N_{{\mathfrak {R}}}$. Otherwise, compute and bound the defect as d, using, for instance, the norm (or any other more refined, yet rigorous overestimation of the uniform norm):

  

  and return $d/(1-\lambda )$.

For the sake of efficiency, kernels are represented using low-rank decompositions.

Definition 3.4

A bivariate kernel ${\mathfrak {K}}(x,t)$ is said to be of rank k if there are functions $\alpha _0(x), \dots , \alpha _{k-1}(x)$ and $\beta _0(t), \dots , \beta _{k-1}(t)$ over $[x_l,x_r]$ such that:

$$\begin{aligned} {\mathfrak {K}}(x,t) = \sum _{i=0}^{k-1} \alpha _i(x) \beta _i(t), \qquad (x,t) \in [x_l,x_r|x_0]. \end{aligned}$$

The kernel ${\mathfrak {K}}(x,t)$ in Propositions 2.3 and 2.5 is clearly of rank r. Also, we shall see in the next section that the resolvent kernel ${\mathfrak {R}}(x,t)$ is of rank r too (which is not obvious from its definition using iterated kernels in Theorem 3.3). This explains why we compute a ${\mathfrak {R}}^\circ (x,t)$ of rank r to approximate ${\mathfrak {R}}(x,t)$ in (12). Therefore, elementary operations on low rank kernels (evaluation on a polynomial and composition with $*$) are necessary to implement these three steps in Algorithm NewtonPicardValid. Their complexity is investigated in the subsequent Lemmas 3.7 and 3.8.

Theorem 3.5

Let $f^\circ$ be a degree n polynomial approximation of the solution $f^*$ of the integral equation of Proposition 2.3 or 2.5, and $N_{{\mathfrak {R}}}$ the degree used for ${\mathfrak {R}}^\circ (x,t)$. If NewtonPicardValid$({\mathfrak {K}}, g, f^\circ , N_{{\mathfrak {R}}})$ does not fail, then it outputs a rigorous error bound for $\Vert f^\circ -f^*\Vert$. More precisely:

• It computes a Newton–Picard fixed-point operator $\varvec{\mathrm {T}}$ and a rigorous contraction ratio $\lambda$ in

  $$\begin{aligned} {\mathcal {O}}(N_{{\mathfrak {R}}}^2 r^2) \quad \text {arithmetic operations}. \end{aligned}$$

• Given $\varvec{\mathrm {T}}$ and $\lambda$, it computes for any right-hand side g and any approximation $f^\circ$ a rigorous error bound for $\Vert f^\circ -f^*\Vert$ in

  $$\begin{aligned} {\mathcal {O}}((N_{{\mathfrak {R}}}+ n + m) N_{{\mathfrak {R}}}r) \quad \text {arithmetic operations}, \end{aligned}$$

  where $n = \deg f^\circ$ and $m = \deg g$.

Remark 3.6

Theorem 3.5 gives a separate complexity estimate for Step 3, since Steps 1 and 2 are completely independent of $f^\circ$ and g. This means that the same Newton–Picard fixed-point validation operator $\varvec{\mathrm {T}}$ can be reused to validate another approximation $f^\circ$ in Problem 1.1, possibly with a different right-hand side h and/or different initial conditions $v_i$.

Lemma 3.7

If $\varvec{\mathrm {K}}$ is an integral operator with polynomial kernel ${\mathfrak {K}}(x,t) = \sum \nolimits _{i=0}^{k-1} \alpha _i(x) \beta _i(t)$ of rank k and total degree n, and f(x) a univariate polynomial of degree m, then $\displaystyle \varvec{\mathrm {K}}\{f\} = \int _{x_0}^x {\mathfrak {K}}(x,t) f(t) \mathrm {d}t$ is a polynomial of degree at most $n + m + 1$ that can be computed in ${\mathcal {O}}(k n (n+m))$ arithmetic operations.

Proof

For each $0 \leqslant i < k$, $\alpha _i(x) \int _{x_0}^x \beta _i(t) f(t) \mathrm {d}t$ is a univariate polynomial of degree at most $\deg \alpha _i + \deg \beta _i + \deg f + 1 \leqslant n + m + 1$, that requires ${\mathcal {O}}(\deg \beta _i \deg f)$ arithmetic operations for the inner multiplication, ${\mathcal {O}}(\deg \beta _i + \deg f)$ ones for the integration, and finally ${\mathcal {O}}(\deg \alpha _i (\deg \beta _i + \deg f))$ ones for the outer multiplication, giving a total of ${\mathcal {O}}(n (n+m))$ arithmetic operations. The claimed overall complexity follows from the iteration for each $0 \leqslant i < k$ and the additions. $\square$

Lemma 3.8

Let ${\mathfrak {K}}(x,t) = \sum \nolimits _{i=0}^{k-1} \alpha _i(x) \beta _i(t)$ and ${\mathfrak {L}}(x,t) = \sum \nolimits _{j=0}^{l-1} \gamma _j(x) \delta _j(t)$ be polynomial kernels of respective ranks k and l, and respective total degrees n and m. Then KernelComp$({\mathfrak {K}}, {\mathfrak {L}})$ computes a rank $k+l$ decomposition of ${\mathfrak {K}} * {\mathfrak {L}}$ with degree at most $n+m+1$, in ${\mathcal {O}}(kl(n+m)^2)$ arithmetic operations.

Proof

Algorithm KernelComp first computes primitives in line 1:

$$\begin{aligned} \zeta _{ij}(s) = \int \beta _i(s) \gamma _j(s) \mathrm {d}s, \qquad 0 \leqslant i< k, \quad 0 \leqslant j < l, \end{aligned}$$

in ${\mathcal {O}}(klnm)$ arithmetic operations, giving polynomials of degree at most $n+m+1$.

The computation of ${\mathfrak {M}}(x,t)$ then follows:

$$\begin{aligned} \begin{aligned} ({\mathfrak {K}} * {\mathfrak {L}})(x,t)\&= \ \int _t^x {\mathfrak {K}}(x,s) {\mathfrak {L}}(s,t) \mathrm {d}s = \sum _{i=0}^{k-1} \sum _{j=0}^{l-1} \alpha _i(x) (\zeta _{ij}(x) - \zeta _{ij}(t)) \delta _j(t) \\&= \ \sum _{j=0}^{l-1} \underbrace{ \left[ \sum _{i=0}^{k-1} \alpha _i(x) \zeta _{ij}(x)\right] }_{ \mu _{k+j}(x)} \underbrace{\delta _j(t)}_{\nu _{k+j}(t)} - \sum _{i=0}^{k-1} \underbrace{\alpha _i(x)}_{\mu _i(x)} \underbrace{\left[ \sum _{j=0}^{l-1} \zeta _{ij}(t) \delta _j(t)\right] }_{\nu _i(t)}. \end{aligned} \end{aligned}$$

More precisely, $\nu _i(t)$ (line 4) requires ${\mathcal {O}}(l(n+m)m)$ arithmetic operations, and similarly $\mu _{k+j}(x)$ (line 7) requires ${\mathcal {O}}(kn(n+m))$ ones. This concludes the proof of the claimed complexity. $\square$

Proof of Theorem 3.5

First, for step 1, rewriting ${\mathfrak {K}}(x,t)$ given in Propositions 2.3 or 2.5 as a rank r expression (line 1) requires ${\mathcal {O}}(r^2 s)$ arithmetic operations. Then, anticipating the complexity estimate for ResolventKernel given in Proposition 3.13, line 2 requires ${\mathcal {O}}(N_{{\mathfrak {R}}}(r+s)^2)$ arithmetic operations. Moving on to step 2, the kernel composition in line 3 runs in complexity ${\mathcal {O}}((N_{{\mathfrak {R}}}+ r + s)^2 r^2)$, according to Lemma 3.8. After that, the computation of a bound $\lambda$ for ${\mathfrak {E}}(x,t)$ (lines 4 and 5) runs in ${\mathcal {O}}((N_{{\mathfrak {R}}}+ r + s)^2 r)$. This justifies the overall complexity for steps 1 and 2, assuming $r+s={\mathcal {O}}(N_{{\mathfrak {R}}})$.

Concerning step 3, lines 7 and 8 computing $\delta (x)$ respectively require ${\mathcal {O}}(r (r + s) (r + s + n) + m)$ and ${\mathcal {O}}(2 N_{{\mathfrak {R}}}(2 N_{{\mathfrak {R}}}+ r + s + n + m) r)$ arithmetic operations, using Lemma 3.7. Finally, the bound computed line 9 requires $\deg \delta (x) = {\mathcal {O}}(2 N_{{\mathfrak {R}}}+ r + s + n + m)$ arithmetic operations. Thus, ${\mathcal {O}}((N_{{\mathfrak {R}}}+ n + m) N_{{\mathfrak {R}}}r)$ is a compact majorant for the complexity, assuming $r+s={\mathcal {O}}(N_{{\mathfrak {R}}})$. $\square$

Remark 3.9

The total complexity can be made quasi-linear w.r.t. $N_{{\mathfrak {R}}}$ by using a fast DCT-based algorithm for polynomial multiplication in the kernel composition (line 3), and bounding the low-rank kernel ${\mathfrak {E}}(x,t)$ (lines 4–5) using an adequate orthogonalization process (to reduce overestimation). However, as explained in Remark 2.6, we choose to keep the quadratic estimate in this article.

Remark 3.10

The use of Chebyshev approximations for ${\mathfrak {R}}^\circ (x,t)$ is not crucial to the principle of the validation algorithm. The reason of this choice is that they are asymptotically excellent candidates and allow for an elegant complexity analysis in Sect. 3.4. However, for intermediate precision, other approximants (e.g., splines) may perform better.

3.3 Resolvent Kernel Approximation

Algorithm NewtonPicardValid above requires a polynomial approximation ${\mathfrak {R}}^\circ (x,t)$ of ${\mathfrak {R}}(x,t)$. We first show in Sect. 3.3.1 that ${\mathfrak {R}}$ admits an analytic rank r expression, involving the solutions of the homogeneous linear ODE $\varvec{\mathrm {L}}\{y\}=0$, together with those of the adjoint equation $\varvec{\mathrm {L^*}}\{y\}=0$. The adjunction $\varvec{\mathrm {L}} \mapsto \varvec{\mathrm {L^*}}$, defined as:

$$\begin{aligned} \varvec{\mathrm {x^*}} = \varvec{\mathrm {x}}, \qquad \varvec{\mathrm {\partial ^*}} = -\varvec{\mathrm {\partial }}, \qquad (\varvec{\mathrm {L}}_1 \, \varvec{\mathrm {L}}_2)^{\varvec{\mathrm {*}}} = \varvec{\mathrm {L}}_2^{\varvec{\mathrm {*}}} \, \varvec{\mathrm {L}}_1^{\varvec{\mathrm {*}}}, \qquad (\varvec{\mathrm {L}}_1 + \varvec{\mathrm {L}}_2)^{\varvec{\mathrm {*}}} = \varvec{\mathrm {L}}_1^{\varvec{\mathrm {*}}} + \varvec{\mathrm {L}}_2^{\varvec{\mathrm {*}}}, \end{aligned}$$

plays a crucial role to this purpose. Clearly, the $\varvec{\mathrm {x}}-\varvec{\mathrm {\partial }}$ form differential operator in (3) and the one in $\varvec{\mathrm {\partial }}-\varvec{\mathrm {x}}$ form in (4) are mutually adjoint. For the sake of clarity, we opt for the convention to refer to the former as $\varvec{\mathrm {L}}$, and the latter as $\varvec{\mathrm {L^*}}$. We also call $\varvec{\mathrm {K}}$ and ${\mathfrak {K}}(x,t)$ the integral operator and kernel associated to $\varvec{\mathrm {L}}$ by Proposition 2.3, and similarly $\varvec{\mathrm {K^*}}$ and ${\mathfrak {K}}^*(x,t)$ for $\varvec{\mathrm {L^*}}$ in Proposition 2.5. Note the following adjunction relation on integral operators⁵:

$$\begin{aligned} {\mathfrak {K}}^*(x,t) = - {\mathfrak {K}}(t,x), \qquad (x,t) \in [x_l,x_r]^2. \end{aligned}$$

The formulas given in Proposition 3.12 below for the resolvent kernels ${\mathfrak {R}}$ and ${\mathfrak {R}}^*$ associated to ${\mathfrak {K}}$ and ${\mathfrak {K}}^*$ also satisfy ${\mathfrak {R}}^*(x,t)=-{\mathfrak {R}}(t,x)$.

Based on this, Sect. 3.3.2 presents the approximation routine ResolventKernel which numerically solves the homogeneous equations using the Chebyshev spectral method presented in Sect. 2.3. This completes the algorithmic description of the Newton–Picard validation procedure.

3.3.1 Formulas for the Resolvent Kernel

The scalar linear ODE (3) can be transformed into the equivalent r-dimensional first-order vectorial linear ODE $Y'(x) = A(x) Y(x) + H(x)$, where:

It is well known that the solution with initial conditions $v = (v_0, \dots , v_{r-1})^T$ can be expressed using an integral operator with kernel $\Phi (x)\Phi (t)^{-1}$:

$$\begin{aligned} Y(x) = \Phi (x) \left[ v + \int _{x_0}^x \Phi (t)^{-1} H(t) \mathrm {d}t\right] , \end{aligned}$$

where $\Phi (x)$ is the fundamental matrix, satisfying the matrix ODE $\Phi '(x) = A(x) \Phi (x)$ with $\Phi (x_0) = I_r$. In other words, $\Phi (x)$ contains the canonical basis of solutions for the homogeneous linear ODE $\varvec{\mathrm {L}}\{y\}=0$ and their derivatives up to order $r-1$. In particular, its columns are linearly independent for all x. Hence, $\Phi (x)$ is invertible and we have:

$$\begin{aligned} (\Phi (x)^{-1})' = - \Phi (x)^{-1} \Phi '(x) \Phi (x)^{-1} = - \Phi (x)^{-1} A(x) \Phi (x) \Phi (x)^{-1} = - \Phi (x)^{-1} A(x), \end{aligned}$$

which means that the inverse transpose $x \mapsto \Phi (x)^{-T}$ satisfies the matrix ODE $M'(x) = -A(x)^T M(x)$.

On the other hand, the scalar linear ODE $\varvec{\mathrm {L^*}}\{y\} = h$ with $\varvec{\mathrm {L^*}}$ given in $\varvec{\mathrm {\partial }}-\varvec{\mathrm {x}}$ form as in (4) is equivalent to the vectorial ODE $Y'(x) = B(x) Y(x) + H(x)$, where:

Similarly, the fundamental matrix $\Psi (x)$ is defined as the solution of $\Psi '(x) = B(x) \Psi (x)$ with $\Psi (x_0) = I_r$. A key observation is that:

(13)

leading to the following crucial orthogonality relations.

Lemma 3.11

The canonical basis $\varphi _0, \dots , \varphi _{r-1}$ for the solution space of $\varvec{\mathrm {L}}$:

$$\begin{aligned} \varvec{\mathrm {L}}\{\varphi _i\} = 0, \qquad \varphi _i^{(j)}(x_0) = \left\{ \begin{aligned}&1&\text {if } i = j \\&0&\text {otherwise}, \end{aligned}\right. \end{aligned}$$

and the dual basis $\psi _0, \dots , \psi _{r-1}$ for the solution space of the adjoint $\varvec{\mathrm {L^*}}$:

$$\begin{aligned} \varvec{\mathrm {L^*}}\{\psi _i\} = 0, \qquad \psi _i^{[j]}(x_0) = \left\{ \begin{aligned}&1&\text {if } i = j \\&0&\text {otherwise}, \end{aligned}\right. \end{aligned}$$

satisfy the orthogonality relations for $0 \leqslant j, k < r$:

$$\begin{aligned} \sum _{i=0}^{r-1} \, \varphi _i^{(j)}(x) \, \psi _{r-1-i}^{[k]}(x) \ = \ \left\{ \begin{aligned}&1&\text {if } k+j = r-1, \\&0&\text {otherwise}, \end{aligned}\right. \qquad \text {for all } x \in [x_l,x_r]. \end{aligned}$$

(14)

Proof

The matrices $\Phi (x) = (\varphi _j^{(i)}(x))_{0 \leqslant i, j < r}$ and $\Psi (x) = (\psi _j^{[i]}(x))_{0 \leqslant i, j < r}$ satisfy $\Phi (x_0) = \Psi (x_0) = I_r$, and are solutions to the matrix linear ODEs $\Phi '(x) = A(x) \Phi (x)$ and $\Psi '(x) = B(x) \Psi (x)$, respectively. From identity (13), we have that:

$$\begin{aligned} (J_r \Psi (x) J_r)' = J_r B(x) \Psi (x) J_r = -A(x)^T (J_r \Psi (x) J_r). \end{aligned}$$

Hence, $x \mapsto J_r \Psi (x) J_r$ satisfies the same ODE $M'(x) = -A(x)^T M(x)$ as $x \mapsto \Phi (x)^{-T}$, and both are equal to $I_r$ at $x_0$. It follows therefrom that $\Phi (x)^{-T} = J_r \Psi (x) J_r$, and in particular:

$$\begin{aligned} \Phi (x) J_r \Psi (x)^T J_r = I_r \qquad \text {for all } x \in [x_l,x_r], \end{aligned}$$

which unfolds as the orthogonality relations (14). $\square$

Proposition 3.12

The resolvent kernels ${\mathfrak {R}}$ and ${\mathfrak {R}}^*$ are given by:

$$\begin{aligned} {\mathfrak {R}}(x,t)&= \sum _{i=0}^{r-1} \varphi _i^{(r)}(x) \psi _{r-1-i}(t), \end{aligned}$$

(15)

$$\begin{aligned} {\mathfrak {R}}^*(x,t)&= - \sum _{i=0}^{r-1} \psi _i(x) \varphi _{r-1-i}^{(r)}(t), \end{aligned}$$

(16)

with the $\varphi _i, \psi _i$ defined in Lemma 3.11.

Proof

$\bullet$ For ${\mathfrak {R}}$, call ${\mathfrak {R}}'(x,t)$ the right-hand side of (15) and $\varvec{\mathrm {R'}}$ the integral operator defined by this kernel. Since, by Theorem 3.3, $\varvec{\mathrm {I}}+\varvec{\mathrm {K}}$ is already known to be invertible, of inverse $\varvec{\mathrm {I}}+\varvec{\mathrm {R}}$, it is sufficient to prove that $(\varvec{\mathrm {I}}+\varvec{\mathrm {K}})(\varvec{\mathrm {I}}+\varvec{\mathrm {R'}}) = \varvec{\mathrm {I}}$, that is ${\mathfrak {K}}*{\mathfrak {R}}' + {\mathfrak {R}}' + {\mathfrak {K}} = 0$. For $0 \leqslant i < r$, $\varvec{\mathrm {L}}\{\varphi _i\}=0$ implies from Proposition 2.3 by renaming $x_0$ into t:

$$\begin{aligned} \varphi _i^{(r)}(x) + \int _t^x {\mathfrak {K}}(x,s) \varphi _i^{(r)}(s) \mathrm {d}s = - \sum _{j=0}^{r-1} a_j(x) \sum _{k=j}^{r-1} \frac{(x-t)^{k-j}}{(k-j)!} \varphi _i^{(k)}(t). \end{aligned}$$

Multiplying the left-hand side by $\psi _{r-1-i}(t)$ and summing over i gives ${\mathfrak {R}}'(x,t)+({\mathfrak {K}}*{\mathfrak {R}}')(x,t)$. Doing the same operations on the right-hand side and commuting the summation symbols allows us to isolate the term $\sum \nolimits _{i=0}^{r-1} \varphi _i^{(k)}(t) \psi _{r-1-i}(t)$, which by Lemma 3.11 is nonzero (and equal to 1) if and only if $k=r-1$. We thus obtain the expected equality:

$$\begin{aligned} {\mathfrak {R}}'(x,t)+({\mathfrak {K}}*{\mathfrak {R}}')(x,t) = - \sum _{j=0}^{r-1} a_j(x) \frac{(x-t)^{r-1-j}}{(r-1-j)!} = - {\mathfrak {K}}(x,t). \end{aligned}$$

$\bullet$ For ${\mathfrak {R}}^*$, call ${\mathfrak {R}}''(x,t)$ the right-hand side of (16). Clearly, ${\mathfrak {R}}''(x,t) = -{\mathfrak {R}}(t,x)$. We have:

$$\begin{aligned}&({\mathfrak {K}}^* * {\mathfrak {R}}'')(x,t) + {\mathfrak {R}}''(x,t) + {\mathfrak {K}}^*(x,t) \\&\quad = \int _t^x {\mathfrak {K}}^*(x,s) {\mathfrak {R}}''(s,t) \mathrm {d}s + {\mathfrak {R}}''(x,t) + {\mathfrak {K}}^*(x,t) \\&\quad = \int _t^x {\mathfrak {K}}(s,x) {\mathfrak {R}}(t,s) \mathrm {d}s - {\mathfrak {R}}(t,x) - {\mathfrak {K}}(t,x) = - ({\mathfrak {R}} * {\mathfrak {K}})(t,x) - {\mathfrak {R}}(t,x) - {\mathfrak {K}}(t,x) = 0, \end{aligned}$$

since $-\varvec{\mathrm {R}} \, \varvec{\mathrm {K}} - \varvec{\mathrm {R}} - \varvec{\mathrm {K}} = \varvec{\mathrm {I}} - (\varvec{\mathrm {I}}+\varvec{\mathrm {R}})(\varvec{\mathrm {I}}+\varvec{\mathrm {K}})=0$. Hence the corresponding operator $\varvec{\mathrm {R}}''$ satisfies $(\varvec{\mathrm {I}}+\varvec{\mathrm {K^*}})(\varvec{\mathrm {I}}+\varvec{\mathrm {R''}}) = \varvec{\mathrm {I}}$, which implies $\varvec{\mathrm {R''}} = \varvec{\mathrm {R^*}}$ and hence ${\mathfrak {R}}''={\mathfrak {R}}^*$. $\square$

3.3.2 Resolvent Kernel Approximation Algorithm

A major advantage of the formulas in Proposition 3.12 from the computational point of view, is that in both $\varvec{\mathrm {x}}-\varvec{\mathrm {\partial }}$ and $\varvec{\mathrm {\partial }}-\varvec{\mathrm {x}}$ cases, one requires the $\psi _i$, solutions to an ODE in $\varvec{\mathrm {\partial }}-\varvec{\mathrm {x}}$ form, and the $\varphi _i^{(r)}$ where the $\varphi _i$ are solutions to an ODE in $\varvec{\mathrm {x}}-\varvec{\mathrm {\partial }}$ form. Hence, no conversion from $\varvec{\mathrm {x}}-\varvec{\mathrm {\partial }}$ and $\varvec{\mathrm {\partial }}-\varvec{\mathrm {x}}$ forms is needed here.

Algorithm ResolventKernel below computes degree $N_{{\mathfrak {R}}}$ polynomial approximations of these analytic functions. To do so, the Chebyshev spectral method of Sect. 2.3 is applied on the following integral equations, equivalent by Propositions 2.3 and 2.5 to the IVPs defining $\varphi _i$ and $\psi _i$ in Lemma 3.11:

$$\begin{aligned} \begin{aligned} (\varvec{\mathrm {I}} + \varvec{\mathrm {K}})\{\varphi _i^{(r)}\}&= g_i,&\qquad&\text {with} \quad g_i(x) = - \sum _{j=0}^i a_j(x) \frac{(x-x_0)^{i-j}}{(i-j)!}, \\ (\varvec{\mathrm {I}} + \varvec{\mathrm {K^*}})\{\psi _i\}&= g^*_i,&\qquad&\text {with} \quad g^*_i(x) = \frac{(x_0-x)^i}{i!}. \end{aligned} \end{aligned}$$

(17)

Proposition 3.13

Algorithm ResolventKernel computes a rank r, degree $2 N_{{\mathfrak {R}}}$ polynomial kernel ${\mathfrak {R}}^\circ (x,t)$ approximating the resolvent kernel ${\mathfrak {R}}(x,t)$ in ${\mathcal {O}}(N_{{\mathfrak {R}}}(r+s)^2)$ arithmetic operations.

Proof

First, the computation of M (line 5) and $M^*$ (line ) only requires ${\mathcal {O}}(N_{{\mathfrak {R}}}r(r+s))$ arithmetic operations, due to the specific form of ${\mathfrak {K}}(x,t)$ built in line 1 of Algorithm NewtonPicardValid. Indeed, for $0 \leqslant i < r$ and $0 \leqslant j \leqslant N_{{\mathfrak {R}}}$, the computation of $\kappa _i(x) \int _{x_0}^x T_i(t) T_j(t) \mathrm {d}t$ or $T_i(t) \int _{x_0}^x \kappa _i(t) T_j(t) \mathrm {d}t$ involves two multiplications where at most one operand has ${\mathcal {O}}(r+s)$ nonzero coefficients (${\mathcal {O}}(1)$ for the other one), and one integral where the integrand has at most ${\mathcal {O}}(r+s)$ nonzero coefficients. Hence, for $0 \leqslant j \leqslant N_{{\mathfrak {R}}}$, the nonzero coefficients of $\varvec{\mathrm {K}}\{T_j\}$ and $\varvec{\mathrm {K^*}}\{T_j\}$ (forming the horizontal and diagonal bands of M and $M^*$) are computed in ${\mathcal {O}}(r(r+s))$ arithmetic operations each.

Then, according to Proposition 2.9, the computation of the QR factorizations in lines 6 and  using Algorithm AlmostBandedQRFactor requires ${\mathcal {O}}(N_{{\mathfrak {R}}}(r+s)^2)$ arithmetic operations.

After that, for each $0 \leqslant i < r$, the computation of the right-hand side $g_i$ (lines 8–10) requires ${\mathcal {O}}(r(r+s))$ arithmetic operations, while the updating of the right-hand side $g^*_i$ of the adjoint equation (line ) needs ${\mathcal {O}}(r+s)$ ones. Finally, solving the resulting almost-banded linear systems using AlmostBandedBackSubs in lines 11 and 17 with the previously computed QR factorizations requires ${\mathcal {O}}(N_{{\mathfrak {R}}}(r+s))$ arithmetic operations. Iterating over $0 \leqslant i < r$, this yields a complexity of ${\mathcal {O}}(N_{{\mathfrak {R}}}r(r+s))$ for lines 7–12 and 16–19, assuming $r+s = {\mathcal {O}}(N_{{\mathfrak {R}}})$.

Hence, the overall complexity of ResolventKernel is ${\mathcal {O}}(N_{{\mathfrak {R}}}(r+s)^2)$. $\square$

Remark 3.14

Instead of computing a fundamental matrix for the adjoint equation $\varvec{\mathrm {L^*}}\{y\}=0$ (lines 16–19 in ResolventKernel), we can also directly approximate the inverse of the fundamental matrix of $\varvec{\mathrm {L}}\{y\}=0$ (already computed in lines 7–12). To do so, sample the fundamental matrix at $N_{{\mathfrak {R}}}+1$ Chebyshev nodes (with complexity ${\mathcal {O}}(N_{{\mathfrak {R}}}\log N_{{\mathfrak {R}}}r^2)$ using fast DCT), invert the matrix at each node (with a total of ${\mathcal {O}}(N_{{\mathfrak {R}}}r^\omega )$ arithmetic operations), and finally re-interpolate in complexity ${\mathcal {O}}(N_{{\mathfrak {R}}}\log N_{{\mathfrak {R}}}r^2)$.

3.4 Newton–Galerkin vs Newton–Picard Validation Complexity

The overall complexity of NewtonPicardValid depends on how large the degree $N_{{\mathfrak {R}}}$ of the approximate resolvent kernel ${\mathfrak {R}}^\circ (x,t)$ has to be to ensure a contracting Newton–Picard operator $\varvec{\mathrm {T}}$. This, in turn, is determined by the approximation errors $\varepsilon _i^{(N_{{\mathfrak {R}}})} = \Vert \varphi _i^{(r)\circ }-\varphi _i^{(r)}\Vert _\infty$ and $\eta _i^{(N_{{\mathfrak {R}}})} = \Vert \psi _i^\circ -\psi _i\Vert _\infty$ of the $\varphi _i^{(r)\circ }$ and $\psi _i^\circ$ computed by ResolventKernel.

The use of spectral methods is motivated by both efficiency and accuracy, and in practice the resulting error is close to the one from the mere truncation of the exact Chebyshev series. In the analytic framework we consider (where the coefficients of $\varvec{\mathrm {L}}$ are polynomials or analytic functions over a complex neighborhood of $[x_l,x_r]$, represented using RPAs), Chebyshev series enjoy far better convergence properties than the simple summability condition defining the space. Let ${{\mathcal {E}}}_\rho$ denote the Bernstein ellipse [36, Chap. 8] of parameter $\rho > 1$ rescaled over $[x_l,x_r]$, i.e. the ellipse in the complex plane of foci $x_l$ and $x_r$ and eccentricity $2/(\rho +\rho ^{-1})$. It is the typical domain of convergence of Chebyshev series, which play the same role as the convergence disks do for Taylor series. The following lemma (see e.g. [36, Thm. 8.1]) asserts the exponential decay of Chebyshev coefficients for analytic functions.

Lemma 3.15

Let f be an analytic function over ${{\mathcal {E}}}_\rho$. Then its Chebyshev coefficients $[f]_n$ satisfy:

$$\begin{aligned} |[f]_n| \leqslant \left\{ \begin{aligned}&\Vert f\Vert _{{{\mathcal {E}}}_\rho }&\text {for } n=0, \\&2 \Vert f\Vert _{{{\mathcal {E}}}_\rho } \rho ^{-n}&\text {for } n \geqslant 1, \end{aligned}\right. \end{aligned}$$

where $\Vert f\Vert _{{{\mathcal {E}}}_\rho } = \sup \limits _{z \in {{\mathcal {E}}}_\rho } |f(z)|$ is the uniform norm over ${{\mathcal {E}}}_\rho$. In particular,

Let us fix $\rho > 1$ such that all coefficients of $\varvec{\mathrm {L}}$ (and consequently the IVP solutions) are analytic over ${{\mathcal {E}}}_\rho$. Theorem 3.16 provides an estimate for $N_{{\mathfrak {R}}}$ under the simplifying assumption that exact truncated Chebyshev series of degree $N_{{\mathfrak {R}}}$ for $\varphi _i^{(r)}$ and $\psi _i$ can be computed. We discuss later why including the approximation error of the Chebyshev spectral method used in ResolventKernel does not significantly affect it.

Theorem 3.16

Under the hypothesis that ${\mathfrak {R}}(x,t)$ is approximated using exact truncated Chebyshev series for $\varphi _i^{(r)}$ and $\psi _i$:

(18)

the minimum value for $N_{{\mathfrak {R}}}$ to ensure a contracting Newton–Picard fixed-point operator is bounded by the asymptotic estimate:

$$\begin{aligned} N_{{\mathfrak {R}}}= {\mathcal {O}}\left( \frac{ \log \left( 1 + \sum \limits _{i=0}^{r-1} \Vert a_i\Vert _\infty \frac{T^{r-i}}{(r-i)!}\right) + \log \left( T \sum \limits _{i=0}^{r-1} \Vert \varphi _i^{(r)}\Vert _{{{\mathcal {E}}}_\rho } \Vert \psi _{r-1-i}\Vert _{{{\mathcal {E}}}_\rho }\right) - \log (\rho -1)}{\log \rho } \right) . \end{aligned}$$

Proof

$\bullet$ We first bound the operator norm of the linear part $\varvec{\mathrm {E}} = \varvec{\mathrm {I}} - (\varvec{\mathrm {I}} + \varvec{\mathrm {R^\circ }}) (\varvec{\mathrm {I}} + \varvec{\mathrm {K}}) = (\varvec{\mathrm {R}} - \varvec{\mathrm {R^\circ }}) (\varvec{\mathrm {I}} + \varvec{\mathrm {K}})$ of the Newton–Picard fixed-point operator $\varvec{\mathrm {T}}$ as:

$$\begin{aligned} \Vert \varvec{\mathrm {E}}\Vert _\infty \leqslant T \left( \sum _{i=0}^{r-1} (\Vert \varphi _i^{(r)}\Vert _\infty \eta _{r-1-i}^{(N_{{\mathfrak {R}}})} + \Vert \psi _{r-1-i}\Vert _\infty \varepsilon _i^{(N_{{\mathfrak {R}}})} + \varepsilon _i^{(N_{{\mathfrak {R}}})} \eta _{r-1-i}^{(N_{{\mathfrak {R}}})}\right) \left( 1 + \sum _{i=0}^{r-1} \Vert a_i\Vert _\infty \frac{T^{r-i}}{(r-i)!}\right) . \end{aligned}$$

(19)

Indeed, the definition of ${\mathfrak {K}}(x,t)$ in Proposition 2.3 or 2.5 yields:

$$\begin{aligned} \Vert \varvec{\mathrm {K}}\{f\}\Vert _\infty \leqslant \left( \sum _{i=0}^{r-1} \Vert a_i\Vert _\infty \!\! \sup _{x \in [x_l,x_r]} \!\! {\text {sgn}}(x-x_0) \int _{x_0}^x \frac{|x-t|^{r-1-i}}{(r-1-i)!} \mathrm {d}t \right) \Vert f\Vert _\infty \leqslant \left( \sum _{i=0}^{r-1} \Vert a_i\Vert _\infty \frac{T^{r-i}}{(r-1)!}\right) \Vert f\Vert _\infty , \end{aligned}$$

from which we have the expected bound for $\Vert \varvec{\mathrm {K}}\Vert _\infty$. Moreover, in the $\varvec{\mathrm {x}}-\varvec{\mathrm {\partial }}$ case:

$$\begin{aligned} \begin{aligned} {\mathfrak {R}}^\circ (x,t) - {\mathfrak {R}}(x,t)&= \sum _{i=0}^{r-1} \left( \varphi _i^{(r)}(x) (\psi _{r-1-i}^\circ -\psi _{r-1-i})(t) + (\varphi _i^{(r)\circ }-\varphi _i^{(r)})(x) \psi _{r-1-i}(t) \right. \\&\left. \qquad \quad + (\varphi _i^{(r)\circ }-\varphi _i^{(r)})(x) (\psi _{r-1-i}^\circ -\psi _{r-1-i})(t)\right) , \end{aligned} \end{aligned}$$

and a similar decomposition holds in the $\varvec{\mathrm {\partial }}-\varvec{\mathrm {x}}$ case. This gives the bound:

$$\begin{aligned} \Vert \varvec{\mathrm {R^\circ }}-\varvec{\mathrm {R}}\Vert _\infty \leqslant T \left( \sum _{i=0}^{r-1} (\Vert \varphi _i^{(r)}\Vert _\infty \eta _{r-1-i}^{(N_{{\mathfrak {R}}})} + \varepsilon _i^{(N_{{\mathfrak {R}}})} \Vert \psi _{r-1-i}\Vert _\infty + \varepsilon _i^{(N_{{\mathfrak {R}}})} \eta _{r-1-i}^{(N_{{\mathfrak {R}}})}\right) . \end{aligned}$$

$\bullet$ Instantiating $\varepsilon _i^{(N_{{\mathfrak {R}}})} = 2 \Vert \varphi _i^{(r)}\Vert _{{{\mathcal {E}}}_\rho } \rho ^{-N_{{\mathfrak {R}}}} / (\rho -1)$ and $\eta _i^{(N_{{\mathfrak {R}}})} = 2 \Vert \psi _i\Vert _{{{\mathcal {E}}}_\rho } \rho ^{-N_{{\mathfrak {R}}}} / (\rho -1)$ in (19) yields:

$$\begin{aligned} \Vert \varvec{\mathrm {E}}\Vert \leqslant 4 T \left( \frac{\rho ^{-N_{{\mathfrak {R}}}}}{\rho -1} + \frac{\rho ^{-2N_{{\mathfrak {R}}}}}{(\rho -1)^2}\right) \left( \sum _{i=0}^{r-1} \Vert \varphi _i^{(r)}\Vert _{{{\mathcal {E}}}_\rho } \Vert \psi _{r-1-i}\Vert _{{{\mathcal {E}}}_\rho } \right) \left( 1 + \sum _{i=0}^{r-1} \Vert a_i\Vert _\infty \frac{T^{r-i}}{(r-i)!}\right) . \end{aligned}$$

Taking the logarithm and reordering the terms gives the claimed asymptotic lower bound for $N_{{\mathfrak {R}}}$. $\square$

This estimate depends on the magnitude of the transition matrix associated to the linear ODE, which is consistent with the intuition that stiff ODEs with large amplitude solutions are harder to validate. We can however provide a more conservative estimate depending on the initial data only.

Corollary 3.17

Let $T_\rho = (x_r-x_l) (\rho +\rho ^{-1})/2 \geqslant T$ be the major axis of ${{\mathcal {E}}}_\rho$ and $M_\rho = \sum \nolimits _{i=0}^{r-1} \Vert a_i\Vert _{{{\mathcal {E}}}_\rho } \frac{T_\rho ^{r-1-i}}{(r-1-i)!}$. Then, under the same simplifying assumption (18), the minimum $N_{{\mathfrak {R}}}$ is given by:

$$\begin{aligned} N_{{\mathfrak {R}}}= {\mathcal {O}}\left( \frac{T_\rho M_\rho -\log (\rho -1)}{\log \rho }\right) . \end{aligned}$$

Proof

Reusing the notations of (17), the formulas $\varphi _i^{(r)} = (\varvec{\mathrm {I}} + \varvec{\mathrm {R}})\{g_i\}$ and $\psi _i = (\varvec{\mathrm {I}} + \varvec{\mathrm {R^*}})\{g^*_i\}$ extend over the Bernstein ellipse ${{\mathcal {E}}}_\rho$ in the complex plane. An adaptation of the inverse bound estimate of Theorem 3.3 yields $\Vert \varvec{\mathrm {I}} + \varvec{\mathrm {R}}\Vert _{{{\mathcal {E}}}_\rho }, \Vert \varvec{\mathrm {I}} + \varvec{\mathrm {R^*}}\Vert _{{{\mathcal {E}}}_\rho } \leqslant e^{T_\rho M_\rho }$. Also, for $0 \leqslant i < r$:

$$\begin{aligned} \Vert g_i\Vert _{{{\mathcal {E}}}_\rho } \Vert g^*_{r-1-i}\Vert _{{{\mathcal {E}}}_\rho } \leqslant \sum _{j=0}^i \Vert a_j\Vert _{{{\mathcal {E}}}_\rho } \frac{T_\rho ^{r-1-j}}{(i-j)!(r-1-i)!} \leqslant M_\rho . \end{aligned}$$

Hence, in the estimate of Theorem 3.16:

$$\begin{aligned} \log \left( T_\rho \sum \limits _{i=0}^{r-1} \Vert \varphi _i^{(r)}\Vert _{{{\mathcal {E}}}_\rho } \Vert \psi _{r-1-i}\Vert _{{{\mathcal {E}}}_\rho }\right) \leqslant \log (r T_\rho M_\rho e^{2 T_\rho M_\rho }) = {\mathcal {O}}(T_\rho M_\rho ), \end{aligned}$$

from which we obtain the claimed asymptotic estimate. $\square$

This estimate confirms the significant efficiency gap between the Newton–Picard and the Newton–Galerkin validation methods for moderate to hard instances. Both algorit0hms run in quadratic time w.r.t. the parameter $N_{{\mathcal {G}}}$ (truncation index in Newton–Galerkin) or $N_{{\mathfrak {R}}}$ (resolvent kernel approximation degree in Newton–Picard). However, contrary to the linear estimate for $N_{{\mathfrak {R}}}$ given above, $N_{{\mathcal {G}}}$ can grow exponentially fast w.r.t. the magnitude of the input ODE, as explained in Sect. 2.4.2.

This conclusion must be tempered by noticing that the quantity $M_\rho$ characterizes the magnitude of the input ODE over the Bernstein ellipse ${{\mathcal {E}}}_\rho$. In fact, $\Vert a_i\Vert _{{{\mathcal {E}}}_\rho }$ may be in theory way larger than (and consequently than $\Vert a_i\Vert _\infty$). Indeed, for $n \geqslant 0$, , while $\Vert T_n\Vert _{{{\mathcal {E}}}_\rho } = {\mathcal {O}}(\rho ^n)$. However, most of the time in practice, the $a_i$ are polynomials of reasonable degrees or analytic functions with fast decaying Chebyshev coefficients, so that this ratio still makes $N_{{\mathfrak {R}}}$ way smaller than $N_{{\mathcal {G}}}$. If not, then both Newton–Galerkin and Newton–Picard perform poorly for the mere reason that even the numerical spectral method computing a candidate Chebyshev approximation will require a very high degree, since the Chebyshev coefficients of the solution may be large and decrease slowly.

Including the Spectral Method Error. The estimates for $N_{{\mathfrak {R}}}$ given in Theorem 3.16 and Corollary 3.17 above rely on the simplifying assumption (18) that ${\mathfrak {R}}^\circ (x,t)$ is built from the exact truncated Chebyshev series for the $\varphi _i^{(r)}$ and $\psi _i$ defining the resolvent kernel ${\mathfrak {R}}(x,t)$. In practice, such truncations cannot be directly obtained, and Algorithm ResolventKernel resorts to spectral methods to compute the desired polynomial approximations $\varphi _i^{(r)\circ }$ and $\psi _i^\circ$. Therefore, we prove in Theorem 3.18 below that the same asymptotic estimate for $N_{{\mathfrak {R}}}$ holds in this more realistic setting where method errors of spectral methods are taken into account.

Chebyshev spectral methods perform roughly as well as merely truncating the Chebyshev series (see e.g. [29, Thm. 4.5]), with a convergence in ${\mathcal {O}}(\rho ^{-N})$ with N the approximation degree in the analytic case. However, the proof is asymptotic, and valid as long as $\varphi + \varvec{\mathrm {K}}^{[N]}\{\varphi \} = g$ has a solution $\varphi$ with $\deg \varphi \leqslant N$. Obviously, since $\varvec{\mathrm {I}} + \varvec{\mathrm {K}}$ is itself invertible, so is $\varvec{\mathrm {I}}+\varvec{\mathrm {K}}^{[N]}$ for N sufficiently large, e.g. if . But this quantity, already encountered in (8), precisely leads to an exponential bound for $N=N_{{\mathcal {G}}}$ in the Newton–Galerkin validation method. To avoid this phenomenon for the degree $N=N_{{\mathfrak {R}}}$ used in Newton–Picard, we propose a slightly modified procedure ResolventKernel, by replacing the linear system with a least-squares problem to approximate the exact solution $\varphi ^* = (\varvec{\mathrm {I}}+\varvec{\mathrm {K}})^{-1}\{g\}$. It is important to note that the following development is essentially theoretical and aims at justifying the linear asymptotic estimate for $N_{{\mathfrak {R}}}$ given in Corollary 3.17. In practice however, ResolventKernel is still implemented as presented in Sect. 3.3.2.

Let $m = N-r-s$, and call the $(N+1) \times (m+1)$ rectangular projection of $\varvec{\mathrm {K}}$. The key observation is that $\varvec{\mathrm {K}}^{[N,m]}\{\varphi \} = \varvec{\mathrm {K}}^{[N]}\{\varphi \} = \varvec{\mathrm {K}}\{\varphi \}$ for all $\varphi$ with $\deg \varphi \leqslant m$. To solve the following least-squares problem, we replace by an Euclidean norm, namely $\Vert h\Vert _2 = \sqrt{\sum \nolimits _{i=0}^N a_i^2}$ for $h = \sum \nolimits _{i=0}^N a_i T_i$, keeping in mind the classical norm equivalence .

$$\begin{aligned} \begin{aligned} \min _\varphi&\quad&\Vert (\varvec{\mathrm {I}}+\varvec{\mathrm {K}}^{[N,m]})\{\varphi \} - g\Vert _2, \\ \text {s.t.}&\quad&\deg \varphi \leqslant m. \end{aligned} \end{aligned}$$

(20)

Let ${{\widetilde{\varphi }}}$ be the optimal solution to (20). By comparing it with suboptimal , we get:

We therefore deduce the bound:

(21)

Theorem 3.18

With the polynomial approximations $\varphi _i^{(r)\circ }$ and $\psi _i^\circ$ computed by ResolventKernel using the modified spectral method presented above, the minimum value for $N_{{\mathfrak {R}}}$ to ensure a contracting Newton–Picard fixed-point operator is bounded by the asymptotic estimate:

$$\begin{aligned} N_{{\mathfrak {R}}}= {\mathcal {O}}\left( \frac{T_\rho M_\rho -\log (\rho -1)}{\log \rho }\right) . \end{aligned}$$

Proof

Combining the method error for the (modified) Chebyshev spectral method (21) with $\Vert (\varvec{\mathrm {I}}+\varvec{\mathrm {K}})^{-1}\Vert _\infty \leqslant e^{MT} \leqslant e^{M_\rho T_\rho }$ and , we get for $\varphi =\varphi _i^{(r)}$ or $\psi _i$ and $N=N_{{\mathfrak {R}}}$:

$$\begin{aligned} \Vert \varphi ^\circ - \varphi \Vert _\infty \leqslant e^{M_\rho T_\rho } \left( 1+\frac{2M_\rho T_\rho }{\rho -1}\right) 2 \Vert \varphi \Vert _{{{\mathcal {E}}}_\rho } \sqrt{N_{{\mathfrak {R}}}+1} \frac{\rho ^{-N_{{\mathfrak {R}}}+r+s}}{\rho -1}. \end{aligned}$$

Instantiating the error bounds $\varepsilon _i^{(N_{{\mathfrak {R}}})}$ and $\eta _i^{(N_{{\mathfrak {R}}})}$ with these values instead of the mere truncation bounds in the proof of Theorem 3.16 yields the same asymptotic estimate for $N=N_{{\mathfrak {R}}}$ in Corollary 3.17. $\square$

To conclude, we also note that this modified procedure still runs in ${\mathcal {O}}(N_{{\mathfrak {R}}}(r+s)^2)$ arithmetic operations. Let $M = I_{N+1,m+1} + \varvec{\mathrm {K}}^{[N,m]}$. The solutions ${{\widetilde{\varphi }}}$ to the least squares problem (20) are the solutions of:

$$\begin{aligned} M^T M {{\widetilde{\varphi }}} = M^T g. \end{aligned}$$

(22)

Since M is (rectangular) almost-banded, adapting the algorithms of [29] leads to a QR decomposition of M in complexity ${\mathcal {O}}(N_{{\mathfrak {R}}}(r+s)^2)$ and a solution ${{\widetilde{\varphi }}}$ of (22) in complexity ${\mathcal {O}}(N_{{\mathfrak {R}}}(r+s))$.

Remark 3.19

The overall complexity analysis in this article does not include the effect of rounding errors of floating-point arithmetics and the growth of interval width in the validation part. This could be avoided by using rational numbers instead. However, in most examples we tried, including those presented in the following section, rounding errors do not seem to play an important role. Also, the norm used in line 9 of NewtonPicardValid could be replaced by a polynomial extrema isolation technique to get sharper bounds.

4 Implementation, Examples and Discussion

We briefly present our implementation of the Newton–Picard validation algorithm presented in this paper, and draw up more detailed observations based on practical examples.

4.1 Implementation in the ChebValid library

Algorithm NewtonPicardValid presented in this article is implemented in the C library ChebValid⁶ dedicated to rigorous numerics with RPAs in the Chebyshev basis (a.k.a. Chebyshev models). This library, which relies on MPFR⁷ for floating-point arithmetics in arbitrary precision and its extension MPFI⁸ for interval arithmetics, was initially designed to implement the Newton–Galerkin validation method of [8]. It implements data types as polynomials with double floating-point numbers, arbitrary precision floating-point numbers or interval coefficients, under the name double_chebpoly_t, mpfr_chebpoly_t and mpfi_chebpoly_t. The type chebmodel_t implements Chebyshev models, that is a pair consisting of a mpfi_chebpoly_t and a floating-point bound for the remainder.

For the purpose of this article, we enriched ChebValid with low-rank integral operators, implemented as the type prefix_lrintop_t (where prefix_ = double_chebpoly_, mpfr_chebpoly_ or chebmodel_), together with relevant operations, in particular evaluation on a polynomial or RPA, composition (that is, operation $*$ on the kernels), and rigorous bound. The validation method is implemented in mpfi_chebpoly_newtonpicard/. In Sect. 4.2 below, we test it on a D-finite example, namely the Airy function of the first kind, and discuss the performances compared to other a-posteriori or self-validating methods. Also, Sect. 4.3 exemplifies the extension of NewtonPicardValid to non-polynomial coefficients, represented using RPAs (i.e., chebmodel_t), and implemented in chebmodel_newtonpicard/.

4.2 A D-finite Example: The Airy Function

The Airy function of the first kind ${{\,\mathrm{Ai}\,}}$ is a special function, notably used in different branches of physics (e.g., quantum mechanics and optics). It is a D-finite function, satisfying the following second order linear ODE:

$$\begin{aligned} y''(x) - x y(x) = 0, \end{aligned}$$

(23)

with transcendental initial conditions $v_0 = {{\,\mathrm{Ai}\,}}(0) = 3^{-2/3}/\Gamma (\frac{2}{3})$ and $v_1 = {{\,\mathrm{Ai}\,}}'(0) = -3^{-1/3}/\Gamma (\frac{1}{3})$. While ${{\,\mathrm{Ai}\,}}$ converges exponentially fast as $x \rightarrow +\infty$, namely ${{\,\mathrm{Ai}\,}}(x) \sim x^{-\frac{1}{4}} e^{-\frac{2}{3}x^{3/2}} / 2 \sqrt{\pi }$, the Airy function of the second kind ${{\,\mathrm{Bi}\,}}$, which together with ${{\,\mathrm{Ai}\,}}$ forms a basis of the solution space of (23), is exponentially divergent, with ${{\,\mathrm{Bi}\,}}(x) \sim x^{-\frac{1}{4}} e^{\frac{2}{3}x^{3/2}} / \sqrt{\pi }$ as $x \rightarrow +\infty$ (see Fig. 3a). This means that even slight perturbations of the (non-rational) initial conditions $v_0$ and $v_1$ result in exponentially growing errors when evaluating ${{\,\mathrm{Ai}\,}}(x)$ using the D-finite equation (23). This justifies the use of rigorous validation methods.

First, this IVP for ${{\,\mathrm{Ai}\,}}$ is transformed into an equivalent integral equation on $f={{\,\mathrm{Ai}\,}}''$ using Proposition 2.3:

$$\begin{aligned} f(x) + \int _0^x (x t - x^2) f(t) \mathrm {d}t = v_0 x + v_1 x^2. \end{aligned}$$

(24)

A polynomial approximation $y^\circ$ is computed using the Chebyshev spectral method described in Sect. 2.3. The magnitude of the Chebyshev coefficients of ${{\,\mathrm{Ai}\,}}$ over the symmetric interval $[-a,a] = [-15,15]$ is depicted in Fig. 3b. The observed super-linear convergence in log-scale (that is, in $o(\rho ^{-n})$ for any $\rho > 1$) is typical for entire functions. According to the plot, a Chebyshev polynomial approximation $y^\circ$ with double-precision accuracy ($\approx 10^{-16}$) needs to be of degree roughly 90. However, such a heuristic truncation, as done for example in the Chebfun library, is not a rigorous proof.

As clearly evidenced by Fig. 3c, the coefficients in the matrix representation in the Chebyshev basis of $(\varvec{\mathrm {I}}+\varvec{\mathrm {K}})^{-1}$ converge exponentially fast to 0 when moving off a few diagonals or upper rows, in accordance with the logarithmic bound for $N_{{\mathfrak {R}}}$ claimed in Theorem 3.16. On the contrary, it also appears that finite-dimensional truncations are, by far, less efficient approximations. This phenomenon is reflected by the minimum value for Newton–Galerkin’s $N_{{\mathcal {G}}}$ and Newton–Picard’s $N_{{\mathfrak {R}}}$ needed to ensure a contracting fixed-point operator. More precisely, Fig. 3d compares the evolution in function of $[x_l,x_r]=[-a,a]$ of $N_{{\mathcal {G}}}$ (Newton–Galerkin), $N_{{\mathfrak {R}}}$ (Newton–Picard), and $N_{{\mathcal {P}}}$, the number of Picard iterations needed in the algorithm of [2]. In addition, we provide the a priori conservative estimate for $N_{{\mathfrak {R}}}$ from Corollary 3.17, computed over a Bernstein ellipse with $\rho =2$ (the estimate can be made effective rather than asymptotic by using the bounds in the proof).

The first observation is that Newton–Galerkin validation method can hardly run with $a \geqslant 10$, due to the exponential blow-up of the truncation index $N_{{\mathcal {G}}}$. This indeed requires to perform rigorous computations with matrices with several billions entries... The second observation is that although $N_{{\mathfrak {R}}}$ and $N_{{\mathcal {P}}}$ should be similar in theory (which is reflected by the similar behavior of $N_{{\mathcal {P}}}$ and the a priori estimate for $N_{{\mathfrak {R}}}$), in practice $N_{{\mathcal {P}}}$ is significantly larger, because it is computed as a conservative, purely a priori estimate, contrary to $N_{{\mathfrak {R}}}$ where the actual contraction ratio of $\varvec{\mathrm {T}}$ is explicitly computed.

Fig. 3

Validation of the Airy function ${{\,\mathrm{Ai}\,}}$ over $[x_l,x_r] = [-a,a]$

Timing Comparison. To conclude the Airy example, we compare timings of computing RPAs in the Chebyshev basis using several a posteriori validation methods:

• The Newton–Picard validation algorithm NewtonPicardValid presented in this article and implemented in the C library ChebValid.

• The Newton–Galerkin validation method detailed in [8], for which the ChebValid library was initially designed.

• The Picard iteration based a posteriori validation algorithm of [2], which we implemented (without any optimization) in ChebValid.

Although the application scope of self-validating methods slightly differs (the validation method constructs its own approximant, which is no longer freely provided by the user), we also compared for the sake of completeness with two implementations compatible with multiprecision:

• The CAPD library⁹ computes rigorous Taylor forms with remainders using a higher-order Lohner method [40]. The result is a piecewise analytic representation of the solution, as opposed to uniform polynomial approximants considered in this article.

• The algorithm developed in [25] and implemented in the SageMath ore_algebra package¹⁰ combines a majorant series approach with a binary splitting algorithm to evaluate pointwise D-finite functions. The scope is even more restricted, but we fairly acknowledge the remarkable performances of this implementation.

We first note that in the category of Newton-like a posteriori validation methods, the Newton–Picard algorithm presented in this paper outperforms Newton–Galerkin and Picard iterations, which are both unable to deal with the instances where $a \geqslant 10$ in reasonable time. For the former, the main reason is that the required truncation index $N_{{\mathcal {G}}}$ becomes very large and rapidly exceeds the memory available on a standard machine. For the latter, the number of iterations remains reasonable (although larger than the degree $N_{{\mathfrak {R}}}$ used in Newton–Picard), but the iteration process using interval arithmetics suffers from numerical instability, which in the simple case $a=5$ forced us to use MPFR floating-point numbers with 20, 000 bits of precision.

The comparison with self-validating methods used in CAPD or SageMath is more delicate, since those implementations are by far more mature than our prototypes. However, the timings highlight the “cost to pay” to construct a contracting fixed-point operator as a common feature of a posteriori validation methods. For very high precision, this constant cost is amortized by the linear asymptotic complexity w.r.t. the degree of the candidate approximation. In particular, NewtonPicardValid performs better than CAPD when high precision is required. Note however that we did not manage to use CAPD with Taylor models of degree more than 70, which may explain the poorer performance of this reputed library in high precision.

Finally, when it comes to the simple pointwise rigorous evaluation of ${{\,\mathrm{Ai}\,}}$, the algorithm implemented in SageMath outperforms all the other methods. One reason for this is that it uses a binary splitting technique to evaluate the truncated Taylor series without computing explicitly the Taylor coefficients up to the truncation order. This is quite different from Problem 1.1 where we want to compute an RPA with an explicit polynomial approximation that can be manipulated. However, some techniques used in this implementation could certainly be reused to improve the existing algorithms for RPAs.

Table 2 Timings comparison for the rigorous computation of ${{\,\mathrm{Ai}\,}}$ over $[-a,a]$ with prescribed target accuracy, between Newton–Picard (N–P), Newton–Galerkin (N–G), Picard iterations (Pit) and Taylor forms in the CAPD library. Timings for the rigorous evaluation of ${{\,\mathrm{Ai}\,}}(a)$ using the SageMath ore_algebra package are also provided. “mpfr prec” denotes the MPFR precision in bits and “deg” the degree of the Chebyshev polynomial approximation used in (N-P), (N-G) and (Pit)

4.3 A Boundary Value Problem with Non-Polynomial Coefficients

As mentioned in the introduction, Algorithm NewtonPicardValid easily extends to the case of linear ODEs with non-plynomial coefficients, represented using RPAs. Indeed, one just needs to provide routines to manipulate low-rank kernels defined with univariate RPAs rather than pure polynomials, namely evaluation on another RPA, composition and rigorous bound. This is implemented in the ChebValid library, with the prefix chebmodel_ replacing mpfi_chebpoly_ in the names of the corresponding functions.

To illustrate this, let us consider the following boundary layer problem considered in [29] and reused in [8]:

$$\begin{aligned} \left\{ \begin{aligned}&y''(x) - \frac{2 x}{\varepsilon } \left( \cos x - \frac{8}{10}\right) y'(x) + \frac{1}{\varepsilon } \left( \cos x - \frac{8}{10}\right) y(x) = 0, \qquad x \in [-1,1], \\&y(-1) = y(1) = 1. \end{aligned}\right. \end{aligned}$$

(25)

This is a Boundary Value Problem (BVP) rather than an Initial Value Problem (IVP). However, since the problem is linear, this amounts to solving two IVPs on the same ODE, namely computing $y = y_0 + \lambda y_1$ with $y_0, y_1$ solutions to the IVP with initial conditions (1, 0) and (0, 1), respectively, and with $\lambda = (1-y_0(1))/y_1(1)$.

We first use Algorithm NewtonPicardValid to compute an RPA for $\cos$ over $[-1,1]$, using the differential equation $y''+y=0$. This allows us to provide RPAs of arbitrary accuracy for the two coefficients in (25). Then we call this algorithm again on the resulting IVP, overloaded with RPAs. The smaller $\varepsilon$ is, the steeper the slope of the solution y is, as it can be observed in Fig. 4a. This results in an increasing degree $N_{{\mathfrak {R}}}$ to approximate the resolvent kernel (see Fig. 4b). Note however that $N_{{\mathfrak {R}}}$ is way smaller than the truncation order $N_{{\mathcal {G}}}$ required by Newton–Galerkin validation method (see [8, Sec. 7.3]). As a consequence, we can rigorously deal with instances with really small $\varepsilon$ (up to $10^{-6}$ within a few seconds), while Newton–Galerkin is limited to $\varepsilon =0.005$ roughly.

On a last note, one remarks that $y_0$ and $y_1$ have very high amplitude compared to the BVP solution y. With the algorithm of [29], one can in fact directly numerically solve this BVP by replacing the first two rows of the almost-banded system by the corresponding equations representing the boundary conditions. An interesting future work is to extend Newton–Picard validation algorithm to BVPs, by investigating the resolvent kernel of the associated integral equation.

Fig. 4

Validation of the boundary layer example using NewtonPicardValid

4.4 High Order D-finite Equations

In order to check that the proposed method successfully handles linear ODEs of order greater than 2, 21 random D-finite equations $\varvec{\mathrm {L}}_{10}, \varvec{\mathrm {L}}_{11}, \dots , \varvec{\mathrm {L}}_{30}$ were generated, with order $r_i = i$ and degree 2 polynomial coefficients with integer coefficients uniformly chosen between $-100$ and 100, like the following one:

$$\begin{aligned} \begin{aligned} \varvec{\mathrm {L}}_{10}&= \varvec{\mathrm {\partial }}^{10} + (-93 \varvec{\mathrm {x}}^2 + 5 \varvec{\mathrm {x}}+ 73) \varvec{\mathrm {\partial }}^9 + (-51 \varvec{\mathrm {x}}^2 + 22 \varvec{\mathrm {x}}+ 74) \varvec{\mathrm {\partial }}^8 + (-8 \varvec{\mathrm {x}}^2 - 41 \varvec{\mathrm {x}}+ 41) \varvec{\mathrm {\partial }}^7 \\&\quad + (-18 \varvec{\mathrm {x}}^2 - 76 \varvec{\mathrm {x}}- 11) \varvec{\mathrm {\partial }}^6 + (79 \varvec{\mathrm {x}}^2 - 45 \varvec{\mathrm {x}}+ 70) \varvec{\mathrm {\partial }}^5 + (-11 \varvec{\mathrm {x}}^2 + 61 \varvec{\mathrm {x}}- 83) \varvec{\mathrm {\partial }}^4 \\&\quad + (-74 \varvec{\mathrm {x}}^2 - 5 \varvec{\mathrm {x}}- 84) \varvec{\mathrm {\partial }}^3 + (-\varvec{\mathrm {x}}^2 - 63 \varvec{\mathrm {x}}+ 96) \varvec{\mathrm {\partial }}^2 + (-95 \varvec{\mathrm {x}}^2 + 86 \varvec{\mathrm {x}}- 57) \varvec{\mathrm {\partial }}+ (-8 \varvec{\mathrm {x}}^2 + 21 \varvec{\mathrm {x}}- 56), \end{aligned} \end{aligned}$$

together with random integer initial conditions between $-100$ and 100.

Despite the high order and dense coefficients of the $\varvec{\mathrm {L}}_i$, Algorithm NewtonPicardValid requires quite moderate degrees $N_{{\mathfrak {R}}}$ to construct the Newton–Picard fixed-point operator (always less than 700 for those 21 randomly generated instances). Hence, we see in practice that the remarkable approximation properties of Chebyshev spectral methods are reflected in the validation procedure.

Examples of “natural” high order ODEs are hard to find in the literature and applications. Yet, they frequently appear in the context of D-finite functions, e.g. when performing closure operations like addition, multiplication or algebraic substitution on simpler functions, and their properties may be quite different from those of random equations. Consider for example the D-finite function $f(x) = \cos x + \cos 5 x^2 - \sin 3 x^4$ over [0, 2], plotted in Fig. 5a. The holexprtodiffeq command of Maple’s Gfun package [34] automatically computes an order 6, degree 37 differential equation $\varvec{\mathrm {L}}\{f\}=0$, with a 6-dimensional solution space spanned by $\{\cos x, \sin x, \cos 5x^2, \sin 5x^2, \cos 3 x^4, \sin 3 x^4\}$. Unfortunately, the leading coefficient,

$$\begin{aligned} \begin{aligned} a_6(\varvec{\mathrm {x}}) =&\ 268738560000 \varvec{\mathrm {x}}^{29} -5374771200 \varvec{\mathrm {x}}^{27} -365158969344 \varvec{\mathrm {x}}^{25} +3732480000 \varvec{\mathrm {x}}^{23} +463694284800 \varvec{\mathrm {x}}^{21} \\&-6366325248 \varvec{\mathrm {x}}^{19} -66390306048 \varvec{\mathrm {x}}^{17} +2217859200 \varvec{\mathrm {x}}^{15} +27057529296 \varvec{\mathrm {x}}^{13} -2477301120 \varvec{\mathrm {x}}^{11} \\&+3908641720 \varvec{\mathrm {x}}^9 -134485400 \varvec{\mathrm {x}}^7 +1100233 \varvec{\mathrm {x}}^5 +15403262 \varvec{\mathrm {x}}^3, \end{aligned} \end{aligned}$$

is non-constant, and in particular singular at $x=0$. To circumvent this difficulty, the Desingularize command can be used to compute a non-singular, higher order equation. The result is an operator of order 9 and degree 35, with significantly larger coefficients, which is generated in approximately 10 seconds in Maple. Our experiments on this equation show the following:

• The Chebyshev spectral method provides remarkably accurate numerical approximations of this function, despite the considerable size of the equation. Indeed, the Chebyshev coefficients (see Fig. 5b) converge at a super-linear rate, since f is entire.

• Unfortunately, NewtonPicardValid fails to validate this example in reasonable time, due to a too large degree $N_{{\mathfrak {R}}}$ needed to approximate the resolvent kernel so as to get a contracting operator. Indeed, the functions $\varphi _i$ and $\psi _i$ defining ${\mathfrak {R}}(x,t)$ are way larger than f, and cannot be accurately approximated with a moderate degree. The reason is that the desingularization step produces a much bigger and ill-conditioned equation (in this example, ). Hence, the extra 3 functions in the new 9-dimensional solution space most probably have Chebyshev coefficients of size exponential in this bound, and are therefore not even representable in MPFR.

The key lesson of this example is that while the Chebyshev spectral method can perform well on specific initial conditions associated to a well-behaved particular function of the solution space of the differential equation, the Newton–Picard validation method needs to inspect the whole solution space (of $\varvec{\mathrm {L}}$ and $\varvec{\mathrm {L^*}}$) to guarantee the maximum error “in every direction”. Still, future improvements could help in taming the contribution of the ill-conditioned but non-relevant solutions. Also, in this example and more generally for D-finite functions, handling directly singular equations is an interesting future challenge.

Fig. 5

An example of a D-finite function with high-order, ill-conditioned differential equation

4.5 Discussion and Future Work

Newton–Galerkin a posteriori validation methods have been widely used over recent years to provide successful computer-assisted proofs for the existence (and the effective approximation) of solutions to various kinds of functional equations, notably (linear or nonlinear) ODEs with initial or boundary value conditions, partial differential equations, delay-differential equations, etc. Elaborating on the simpler case of linear ODEs with initial values, we presented a new validation algorithm (which we propose to call Newton–Picard) and provided a thorough complexity analysis to explain why it performs better than other a posteriori validation algorithms in the literature. This algorithm is implemented in the prototype C library ChebValid.

This work is to be seen as part of a long term effort to combine the field of rigorous numerics towards computer-assisted proofs in mathematics with the algorithmic and complexity approach of computer algebra, in order to design efficient and fully trusted symbolic-numeric libraries for function space computations. In particular, several future goals can be identified:

• Combining the Newton–Picard fixed-point operator with the radii polynomial approach of [17] to bound nonlinear terms will allow for the efficient treatment of nonlinear ODEs, where even proving the existence of a solution on a given interval is a nontrivial problem. In particular, investigating the complexity and comparing with the results of [6] is an interesting challenge.

• Deriving formulas for the resolvent kernel in the case of boundary conditions is the key to adapting this algorithm to BVPs.

• Even in the case of D-finite functions, dealing with singular points (that is, when the leading coefficient, instead of being 1, vanishes at some point of the interval) is a difficult problem. While rigorous evaluation of D-finite functions near regular singular points has already been investigated, e.g. in [25, 39], rigorous uniform approximation (with a suitable norm) is, up to our knowledge, still an open problem. A similar question is the case of unbounded intervals.

• Computing rigorous approximants for PDE solutions is a major challenge due to the (time and space) complexity issues related to the several variable case. Hence, proposing efficient and reliable algorithms is crucial in the development of computer-assisted proofs for such problems.

• Finally, we propose to implement this validation algorithm in the Coq proof assistant in order to provide a fully reliable tool to engineers and mathematicians. More specifically, we want to integrate it into our experimental development of certified RPAs in Coq using an a posteriori (or certificate-based) approach [9]. The algebraic framework of Newton–Picard with integral operators and polynomial kernels makes this new validation algorithm a relevant choice for that task.