Descent Methods

Abstract

In Chap. 8 we study discrete and continuous descent methods for minimizing a convex (Lipschitz) function on a general Banach space. We consider a space of vector fields V such that for any point x in the Banach space, the directional derivative in the direction Vx is nonpositive. This space of vector fields is equipped with a complete metric. Each vector field generates two gradient type algorithms (discrete descent methods) and a flow which consists of the solutions of the corresponding evolution equation (continuous descent method). We show that most (in the sense of Baire category) vector fields produce algorithms for which values of the objective function tend to its infimum as t tends to infinity. Actually, we introduce the subclass of regular vector fields, show that the convergence property stated above holds for them and that a generic vector field is regular. We also show that this convergence property is stable under small perturbations of a given regular vector field.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

8.1 Discrete Descent Methods for a Convex Objective Function

Given a Lipschitzian convex function f on a Banach space X, we consider a complete metric space ${\mathcal{A}}$ of vector fields V on X with the topology of uniform convergence on bounded subsets. With each such vector field we associate two iterative processes. We introduce the class of regular vector fields $V \in{ \mathcal{A}}$ and prove (under two mild assumptions on f) that the complement of the set of regular vector fields is not only of the first category, but also σ-porous. We then show that for a locally uniformly continuous regular vector field V and a coercive function f, the values of f tend to its infimum for both processes. These results were obtained in [136].

Assume that (X,∥⋅∥) is a Banach space with norm ∥⋅∥, (X ^∗,∥⋅∥_∗) is its dual space with the norm ∥⋅∥_∗, and f:X→R ¹ is a convex continuous function which is bounded from below. Recall that for each pair of sets A,B⊂X ^∗,

$$H(A,B)=\max\Bigl\{ \sup_{x \in A} \inf_{y \in B} \|x-y\|_*, \sup_{y \in B} \inf_{x \in A}\|x-y\|_*\Bigr\} $$

is the Hausdorff distance between A and B.

For each x∈X, let

$$\partial f(x):=\bigl\{ l \in X^*: f(y)-f(x) \ge l(y-x) \hbox{ for all } y \in X \bigr\} $$

be the subdifferential of f at x. It is well known that the set ∂f(x) is nonempty and bounded (in the norm topology). Set

$$\inf(f):=\inf\bigl\{ f(x): x \in X\bigr\} . $$

Denote by ${\mathcal{A}}$ the set of all mappings V:X→X such that V is bounded on every bounded subset of X (i.e., for each K ₀>0 there is K ₁>0 such that ∥Vx∥≤K ₁ if ∥x∥≤K ₀), and for each x∈X and each l∈∂f(x), l(Vx)≤0. We denote by ${\mathcal{A}}_{c}$ the set of all continuous $V \in{ \mathcal{A}}$, by ${\mathcal{A}}_{u}$ the set of all $V \in{ \mathcal{A}}$ which are uniformly continuous on each bounded subset of X, and by ${\mathcal{A}}_{au}$ the set of all $V\in{ \mathcal{A}}$ which are uniformly continuous on the subsets

$$\bigl\{ x \in X: \|x\| \le n \hbox{ and } f(x) \ge\inf(f)+1/n\bigr\} $$

for each integer n≥1. Finally, let ${\mathcal{A}}_{auc}={\mathcal{A}}_{au} \cap{ \mathcal{A}}_{c}$.

Next we endow the set ${\mathcal{A}}$ with a metric ρ: For each $V_{1},V_{2} \in{ \mathcal{A}}$ and each integer i≥1, we first set

$$ \rho_i(V_1,V_2):=\sup\bigl\{ \|V_1x-V_2x\|: x \in X \hbox{ and } \|x\| \le i\bigr\} $$

(8.1)

and then define

$$ \rho(V_1,V_2):=\sum _{i=1}^{\infty}2^{-i}\bigl[\rho_i(V_1,V_2) \bigl(1+\rho_i(V_1,V_2) \bigr)^{-1}\bigr]. $$

(8.2)

Clearly $({\mathcal{A}},\rho)$ is a complete metric space. It is also not difficult to see that the collection of the sets

$$ E(N,\varepsilon )=\bigl\{ (V_1,V_2) \in{ \mathcal{A}} \times{ \mathcal{A}}: \|V_1x-V_2x\| \le \varepsilon , x \in X, \|x \| \le N\bigr\} , $$

(8.3)

where N,ε>0, is a base for the uniformity generated by the metric ρ. Evidently ${\mathcal{A}}_{c}$, ${\mathcal{A}}_{u}$, ${\mathcal{A}}_{au}$ and ${\mathcal{A}}_{auc}$ are closed subsets of the metric space $({\mathcal{A}},\rho)$. In the sequel we assign to all these spaces the same metric ρ.

To compute inf(f), we are going to associate with each vector field $W \in{ \mathcal{A}}$ two gradient-like iterative processes (see (8.5) and (8.7) below).

The study of steepest descent and other minimization methods is a central topic in optimization theory. See, for example, [2, 19, 44, 47, 69, 73, 103] and the references mentioned therein. Note, in particular, that the counterexample studied in Sect. 2.2 of Chap. VIII of [73] shows that, even for two-dimensional problems, the simplest choice for a descent direction, namely the normalized steepest descent direction,

$$V(x)=\operatorname{argmin}\Bigl\{ \max_{l \in\partial f(x)} \langle l,d \rangle: \|d \|=1\Bigr\} , $$

may produce sequences the functional values of which fail to converge to the infimum of f. This vector field V belongs to $\mathcal{A}$ and the Lipschitzian function f attains its infimum. The steepest descent scheme (Algorithm 1.1.7) presented in Sect. 1.1 of Chap. VIII of [73] corresponds to any of the two iterative processes we consider below.

In infinite dimensions the problem is even more difficult and less understood. Moreover, positive results usually require special assumptions on the space and the functions. However, as shown in our paper [135] (under certain assumptions on the function f), for an arbitrary Banach space X and a generic vector field $V \in \mathcal{A}$, the values of f tend to its infimum for both processes. In that paper, instead of considering a certain convergence property for a method generated by a single vector field V, we investigated it for the whole space $\mathcal{A}$ and showed that this property held for most of the vector fields in $\mathcal{A}$.

Here we introduce the class of regular vector fields $V \in \mathcal{A}$. Our first result, Theorem 8.2, shows (under the two mild assumptions A(i) and A(ii) on f stated below) that the complement of the set of regular vector fields is not only of the first category, but also σ-porous in each of the spaces $\mathcal{A}$, ${\mathcal{A}}_{c}$, ${\mathcal{A}}_{u}$, ${\mathcal{A}}_{au}$ and ${\mathcal{A}}_{auc}$. We then show (Theorem 8.3) that for any regular vector field $V \in{ \mathcal{A}}_{au}$, if the constructed sequence $\{x_{i}\}_{i=0}^{\infty} \subset X$ has a bounded subsequence (in the case of the first process) or is bounded (in the case of the second one), then the values of the function f tend to its infimum for both processes. If, in addition to A(i) and A(ii), f also satisfies the assumption A(iii), then this convergence result is valid for any regular $V \in \mathcal{A}$. Note that if the function f is coercive, then the constructed sequences will always stay bounded. Thus we see, by Theorem 8.2, that for a coercive f the set of divergent descent methods is σ-porous. Our last result, Theorem 8.4, shows that in this case we obtain not only convergence, but also stability.

Our results are established in any Banach space and for those convex functions which satisfy the following two assumptions.

A(i):

There exists a bounded (in the norm topology) set X ₀⊂X such that

$$\inf(f)=\inf\bigl\{ f(x): x \in X\bigr\} =\inf\bigl\{ f(x): x \in X_0 \bigr\} ; $$

A(ii):

for each r>0, the function f is Lipschitzian on the ball {x∈X:∥x∥≤r}.

Note that we may assume that the set X ₀ in A(i) is closed and convex. It is clear that assumption A(i) holds if lim_{∥x∥→∞} f(x)=∞.

We say that a mapping $V \in \mathcal{A}$ is regular if for any natural number n, there exists a positive number δ(n) such that for each x∈X satisfying

$$\|x\| \le n \quad\hbox{and}\quad f(x) \ge\inf(f)+1/n, $$

and each l∈∂f(x), we have

$$l(Vx) \le-\delta(n). $$

Denote by $\mathcal{F}$ the set of all regular vector fields $V \in \mathcal{A}$.

It is not difficult to verify the following property of regular vector fields. It means, in particular, that ${\mathcal{G}}={\mathcal{A}} \setminus {\mathcal{F}}$ is a face of the convex cone ${\mathcal{A}}$ in the sense that if a non-trivial convex combination of two vector fields in $\mathcal{A}$ belongs to $\mathcal{G}$, then both of them must belong to $\mathcal{G}$.

Proposition 8.1

Assume that $V_{1},V_{2} \in{ \mathcal{A}}$, V ₁ is regular, ϕ:X→[0,1], and that for each integer n≥1,

$$\inf\bigl\{ \phi(x): x \in X\ \textit{and}\ \|x\| \le n\bigr\} >0. $$

Then the mapping x→ϕ(x)V ₁ x+(1−ϕ(x))V ₂ x, x∈X, also belongs to $\mathcal{F}$.

Our first result shows that in a very strong sense most of the vector fields in $\mathcal{A}$ are regular.

Theorem 8.2

Assume that both A(i) and A(ii) hold. Then ${\mathcal{A}} \setminus{ \mathcal{F}}$ (respectively, ${\mathcal{A}}_{c} \setminus{ \mathcal{F}}$, ${\mathcal{A}}_{au} \setminus{ \mathcal{F}}$ and ${\mathcal{A}}_{auc} \setminus{ \mathcal{F}}$) is a σ-porous subset of the space ${\mathcal{A}}$ (respectively, ${\mathcal{A}}_{c}$, ${\mathcal{A}}_{au}$ and ${\mathcal{A}}_{auc}$). Moreover, if f attains its infimum, then the set ${\mathcal{A}}_{u} \setminus{ \mathcal{F}}$ is also a σ-porous subset of the space ${\mathcal{A}}_{u}$.

Now let $W \in \mathcal{A}$. We associate with W two iterative processes.

For x∈X we denote by P _W (x) the set of all

$$y \in\bigl\{ x+\alpha Wx: \alpha\in[0,1]\bigr\} $$

such that

$$ f(y)=\inf\bigl\{ f(x+\beta Wx): \beta\in[0,1]\bigr\} . $$

(8.4)

Given any initial point x ₀∈X, one can construct a sequence $\{x_{i}\}_{i=0}^{\infty} \subset X$ such that for all i=0,1,…,

$$ x_{i+1} \in P_W(x_i). $$

(8.5)

This is our first iterative process.

Next we describe the second iterative process.

Given a sequence $\mathbf{a}=\{a_{i}\}_{i=0}^{\infty} \subset(0,1]$ such that

$$ \lim_{i \to\infty} a_i=0 \quad\hbox{and}\quad \sum _{i=0}^{\infty} a_i =\infty, $$

(8.6)

we construct for each initial point x ₀∈X, a sequence $\{x_{i}\}_{i=0}^{\infty} \subset X$ according to the following rule:

$$ \begin{gathered} x_{i+1}=x_i+a_iW(x_i)\quad \hbox{if } f\bigl(x_i+a_iW(x_i) \bigr)<f(x_i), \\ x_{i+1}=x_i\quad \hbox{otherwise,} \end{gathered} $$

(8.7)

where i=0,1,….

We will also make use of the following assumption:

A(iii):

For each integer n≥1, there exists δ>0 such that for each x ₁,x ₂∈X satisfying

$$ \begin{gathered}\|x_1\|,\|x_2\| \le n,\quad\quad f(x_i) \ge \inf(f)+1/n,\quad i=1,2,\quad \hbox{and}\\ \|x_1-x_2\| \le\delta,\end{gathered} $$

the following inequality holds:

$$H\bigl(\partial f(x_1),\partial f(x_2)\bigr) \le1/n. $$

This assumption is certainly satisfied if f is differentiable and its derivative is uniformly continuous on those bounded subsets of X over which the infimum of f is larger than inf(f).

Our next result is a convergence theorem for those iterative processes associated with regular vector fields. It is of interest to note that we obtain convergence when either the regular vector field W or the subdifferential ∂f enjoy a certain uniform continuity property.

Theorem 8.3

Assume that $W \in \mathcal{A}$ is regular, A(i), A(ii) are valid and that at least one of the following conditions holds: 1. $W \in{ \mathcal{A}}_{au}$; 2. A(iii) is valid. Then the following two assertions are true:

• (i) Let the sequence $\{x_{i}\}_{i=0}^{\infty} \subset X$ satisfy (8.5) for all i=0,1,…. If

  $$\liminf_{i \to\infty}\|x_i\|<\infty, $$

  then lim _i→∞ f(x _i )=inf(f).

• (ii) Let a sequence $\mathbf{a}=\{a_{i}\}_{i=0}^{\infty} \subset(0,1]$ satisfy (8.6) and let the sequence $\{x_{i}\}_{i=0}^{\infty} \subset X$ satisfy (8.7) for all i=0,1,…. If $\{x_{i}\}_{i=0}^{\infty}$ is bounded, then

  $$\lim_{i \to\infty} f(x_i)=\inf(f). $$

Finally, we impose an additional coercivity condition on f and establish the following stability theorem. Note that this coercivity condition implies A(i).

Theorem 8.4

Assume that f(x)→∞ as ∥x∥→∞, $V \in \mathcal{A}$ is regular, A(ii) is valid and that at least one of the following conditions holds: 1. $V \in{ \mathcal{A}}_{au}$; 2. A(iii) is valid.

Let K,ε>0 be given. Then there exist a neighborhood ${\mathcal{U}}$ of V in $\mathcal{A}$ and a natural number N ₀ such that the following two assertions are true:

• (i) For each $W \in \mathcal{U}$ and each sequence $\{x_{i}\}_{i=0}^{N_{0}} \subset X$ which satisfies ∥x ₀∥≤K and (8.5) for all i=0,…,N ₀−1, the inequality $f(x_{N_{0}}) \le\inf(f)+\varepsilon$ holds.

• (ii) For each sequence of numbers $\mathbf{a}=\{a_{i}\}_{i=0}^{\infty} \subset(0,1]$ satisfying (8.6), there exists a natural number N such that for each $W \in \mathcal{U}$ and each sequence $\{x_{i}\}_{i=0}^{N} \subset X$ which satisfies ∥x ₀∥≤K and (8.7) for all i=0,…,N−1, the inequality f(x _N )≤inf(f)+ε holds.

8.2 An Auxiliary Result

Assume that $\mathcal{K}$ is a nonempty, closed and convex subset of X. We consider the topological subspace ${\mathcal{K}} \subset X$ with the relative topology. For each function $h: {\mathcal{K}} \to R^{1}$ define $\inf(h):= \inf\{h(x): x \in{ \mathcal{K}}\}$.

Proposition 8.5

Let $g: {\mathcal{K}} \to R^{1}$ be a convex, bounded from below, function which is uniformly continuous on bounded subsets of ${\mathcal{K}}$. Assume that there exists a bounded and convex set ${\mathcal{K}}_{0} \subset{ \mathcal{K}}$ such that for each $x \in {\mathcal{K}}$, there exists $y \in{ \mathcal{K}}_{0}$ for which g(y)≤g(x).

Then there exists a continuous mapping $A_{g}: {\mathcal{K}} \to{ \mathcal{K}}_{0}$ which satisfies g(A _g x)≤g(x) for all $x \in{ \mathcal{K}}$ and has the following two properties:

B(i):

For each integer n≥1, the mapping A _g is uniformly continuous on the set

$$\bigl\{ x \in{ \mathcal{K}}: \|x\| \le n\ \textit{and}\ g(x) \ge\inf(g)+1/n\bigr\} ; $$

B(ii):

if g(x)≥inf(g)+ε for some ε>0 and $x \in {\mathcal{K}}$, then

$$g(A_gx) \le g(x)-\varepsilon /2. $$

Proof

If there exists $x \in \mathcal{K}$ for which g(x)=inf(g), then there exists $x^{*} \in{ \mathcal{K}}_{0}$ for which g(x ^∗)=inf(g) and we can set A _g (y)=x ^∗ for all $y \in{ \mathcal{K}}$. Therefore we may assume that

$$\bigl\{ x \in{ \mathcal{K}}: g(x)=\inf(g)\bigr\} =\emptyset. $$

For each integer i≥0, there exists $y_{i} \in{ \mathcal{K}}_{0}$ such that

$$ g(y_i) \le\bigl(4(i+1)\bigr)^{-1}+\inf(g). $$

(8.8)

Consider now the linear segments which join y ₀,y ₁,…,y _n ,… (all contained in ${\mathcal{K}}_{0}$ by the convexity of ${\mathcal{K}}_{0}$), represented as a continuous curve $\gamma: [0,\infty) \to{ \mathcal{K}}_{0}$ and parametrized so that

$$ \gamma(t)=y_i+(t-i) (y_{i+1}-y_i) \quad\hbox{if } i \le t <i+1\ (i=0,1,2,\dots). $$

(8.9)

The curve γ is Lipschitzian because the set ${\mathcal{K}}_{0}$ is bounded. Define

$$ A_gx=\gamma\bigl(g(x)-\bigl(\inf(g) \bigr)^{-1}\bigr),\quad x \in{ \mathcal{K}}. $$

(8.10)

It is easy to see that $A_{g}x \in{ \mathcal{K}}_{0}$ for all $x \in{ \mathcal{K}}$, the mapping A _g is continuous on $\mathcal{K}$ and that it is uniformly continuous on the subsets

$$\bigl\{ x \in{ \mathcal{K}}: \|x\| \le n \hbox{ and } g(x) \ge\inf(g)+1/n\bigr\} $$

for each integer n≥1.

Assume that

$$ x \in{ \mathcal{K}}, \quad\quad \varepsilon >0 \quad\hbox{and}\quad g(x) \ge\inf(g) +\varepsilon . $$

(8.11)

There is an integer i≥0 such that

$$ g(x)-\inf(g) \in\bigl((i+1)^{-1},i^{-1}\bigr] $$

(8.12)

(here 0⁻¹=∞). Then

$$ \bigl(g(x)-\inf(g)\bigr)^{-1} \in[i,i+1) $$

(8.13)

and by (8.10), (8.9) and (8.13),

$$A_gx=\gamma\bigl(g(x)-\bigl(\inf(g)\bigr)^{-1} \bigr)=y_i+ \bigl(\bigl(g(x)-\inf(g)\bigr)^{-1}-i\bigr) (y_{i+1}-y_i). $$

It follows from this relation, (8.8), (8.11), (8.12) and the convexity of g that

$$\begin{aligned} g(A_gx) \le&\max\bigl\{ g(y_i),g(y_{i+1})\bigr\} \le\inf(g)+\bigl(4(i+1)\bigr)^{-1} \\ \le&\inf(g)+4^{-1}\bigl(g(x)-\inf(g)\bigr) =g(x) -3\cdot4^{-1} \bigl(g(x)-\inf(g)\bigr) \\ \le&g(x) -3\cdot4^{-1} \varepsilon . \end{aligned}$$

This completes the proof of Proposition 8.5. □

8.3 Proof of Theorem 8.2

We first note the following simple lemma.

Lemma 8.6

Assume that $V_{1},V_{2} \in{ \mathcal{A}}$, ϕ:X→[0,1], and that

$$Vx=\bigl(1-\phi(x)\bigr)V_1x+\phi(x)V_2x,\quad x \in X. $$

Then $V \in{ \mathcal{A}}$. If $V_{1},V_{2} \in{ \mathcal{A}}_{c}$ and ϕ is continuous on X, then $V \in{ \mathcal{A}}_{c}$. If $V_{1}, V_{2} \in{ \mathcal{A}}_{u}$ (respectively, ${\mathcal{A}}_{au}$, ${\mathcal{A}}_{auc}$) and ϕ is uniformly continuous on bounded subsets of X, then $V \in{ \mathcal{A}}_{u}$ (respectively, ${\mathcal{A}}_{au}$, ${\mathcal{A}}_{auc}$).

For each pair of integers m,n≥1, denote by Ω _mn the set of all $V \in{ \mathcal{A}}$ such that

$$ \|Vx\| \le m \quad\hbox{for all } x \in X \hbox{ satisfying } \|x\| \le n+1 $$

(8.14)

and

$$ \sup\bigl\{ l(Vx): x \in X, \|x\| \le n, f(x) \ge\inf(f)+1/n, l\in \partial f(x)\bigr\} =0. $$

(8.15)

Clearly,

$$ \bigcup_{m=1}^{\infty}\bigcup_{n=1}^{\infty} \varOmega _{mn}={\mathcal{A}} \setminus{ \mathcal{F}}. $$

(8.16)

Therefore in order to prove Theorem 8.2 it is sufficient to show that for each pair of integers m,n≥1, the set Ω _mn (respectively, $\varOmega _{mn}\cap{ \mathcal{A}}_{c}$, $\varOmega _{mn} \cap{ \mathcal{A}}_{au}$, $\varOmega _{mn} \cap{ \mathcal{A}}_{auc}$) is a porous subset of ${\mathcal{A}}$ (respectively, ${\mathcal{A}}_{c}$, ${\mathcal{A}}_{au}$, ${\mathcal{A}}_{auc}$), and if f attains its minimum, then $\varOmega _{mn}\cap{ \mathcal{A}}_{u}$ is a porous subset of ${\mathcal{A}}_{u}$.

By assumption A(i), there is a bounded and convex set X ₀⊂X with the following property:

C(i):

For each x∈X, there is x ₀∈X ₀ such that f(x ₀)≤f(x). If f attains its minimum, then X ₀ is a singleton.

By Proposition 8.5, there is a continuous mapping A _f :X→X such that

$$ A_f(X) \subset X_0,\quad\quad f(A_fx) \le f(x)\quad \hbox{for all } x \in X, $$

(8.17)

and which has the following two properties:

C(ii):

If x∈X, ε>0 and f(x)≥inf(f)+ε, then f(A _f x)≤f(x)−ε/2;

C(iii):

for any natural number n, the mapping A _f is uniformly continuous on the set

$$\bigl\{ x \in X: \|x\| \le n \hbox{ and } f(x) \ge\inf(f)+1/n\bigr\} . $$

Let m,n≥1 be integers. In the sequel we will use the piecewise linear function ϕ:R ¹→R ¹ defined by

$$ \phi(x)=1,\quad x \in[-n,n],\quad\quad \phi(x)=0,\quad |x| \ge n+1 $$

(8.18)

and

$$\phi(-n-1+t)=t,\quad t \in[0,1],\quad\quad \phi(n+t)=1-t,\quad t \in[0,1]. $$

By assumption A(ii), there is c ₀>1 such that

$$ \bigl|f(x)-f(y)\bigr| \le c_0\|x-y\| $$

(8.19)

for all x,y∈X satisfying ∥x∥,∥y∥≤n+2. Choose α∈(0,1) such that

$$ \alpha c_02^{n+2}<(2n)^{-1}2^{-1}(1- \alpha) \bigl(m+n+2+\sup\bigl\{ \|x\|: x \in X_0\bigr\} \bigr)^{-1}. $$

(8.20)

Assume that V∈Ω _mn and r∈(0,1]. Let

$$ \gamma=2^{-1}(1-\alpha)r\bigl(m+n+2+\sup\bigl\{ \|x\|: x \in X_0\bigr\} \bigr)^{-1} $$

(8.21)

and define V _γ :X→X by

$$ V_{\gamma}x=\bigl(1-\gamma\phi\bigl(\|x\|\bigr)\bigr)Vx+ \gamma\phi\bigl( \|x\|\bigr) (A_fx-x),\quad x \in X. $$

(8.22)

By Lemma 8.6, $V_{\gamma}\in{ \mathcal{A}}$ and moreover, if $V \in{ \mathcal{A}}_{c}$ (respectively, ${\mathcal{A}}_{au}$, ${\mathcal{A}}_{auc}$), then $V_{\gamma} \in {\mathcal{A}}_{c}$ (respectively, ${\mathcal{A}}_{au}$, ${\mathcal{A}}_{auc}$), and if $V \in{ \mathcal{A}}_{u}$ and f attains its minimum, then A _f is constant (see C(i)) and $V_{\gamma} \in{ \mathcal{A}}_{u}$.

Next we estimate the distance ρ(V _γ ,V). It follows from (8.22) and the definition of ϕ (see (8.18)) that V _γ x=Vx for all x∈X satisfying ∥x∥≥n+1 and

$$\rho_i(V_{\gamma},V)=\rho_{n+1}(V_{\gamma},V)\quad \hbox{for all integers } i \ge n+1. $$

Since V∈Ω _mn , the above equality, when combined with (8.2), (8.1), (8.22), (8.18) and (8.17), yields

$$\begin{aligned} \rho(V_{\gamma},V) \le&\sum_{i=1}^{\infty}2^{-i} \rho_i(V,V_{\gamma }) \le\rho_{n+1} (V,V_{\gamma}) \\ =&\sup\bigl\{ \|Vx-V_{\gamma}x\|: x \in X, \|x\| \le n+1\bigr\} \\ \le&\sup\bigl\{ \gamma\phi\bigl(\|x\|\bigr) \bigl(\|Vx\|+\|A_fx-x\|\bigr): x \in X, \|x\| \le n+1\bigr\} \\ \le&\gamma(m+1)+\gamma(n+1)+\gamma\sup\bigl\{ \|x\|: x \in X_0\bigr\} . \end{aligned}$$

(8.23)

Assume that $W \in{ \mathcal{A}}$ with

$$ \rho(W,V_{\gamma}) \le\alpha r. $$

(8.24)

By (8.24), (8.23) and (8.21),

$$ \rho(W,V) \le\alpha r+\gamma\bigl(m+n+2+\sup\bigl\{ \|x\|: x \in X_0\bigr\} \bigr)\le 2^{-1}(1+\alpha)r<r. $$

(8.25)

Assume now that

$$ x \in X,\quad\quad \|x\| \le n, \quad\quad f(x) \ge\inf(f)+1/n \quad\hbox{and}\quad l \in \partial f(x). $$

(8.26)

Inequality (8.19) implies that

$$\|l\|_*\le c_0. $$

By (8.22), (8.26), the definition of ϕ (see (8.18)) and C(ii),

$$\begin{aligned} l(V_{\gamma}x) =&l\bigl(\bigl(1-\gamma\phi\bigl(\|x\|\bigr)\bigr)Vx+ \gamma\phi \bigl(\|x\|\bigr) (A_fx-x)\bigr) \le \gamma\phi\bigl(\|x\|\bigr)l(A_fx-x) \\ =&\gamma l(A_fx-x) \le\gamma \bigl(f(A_fx)-f(x)\bigr) \le -\gamma(2n)^{-1}. \end{aligned}$$

(8.27)

It follows from (8.26) and (8.1) that

$$ \|Wx-V_{\gamma}x\| \le\rho_n(W,V_{\gamma}). $$

(8.28)

By (8.24), (8.28) and the inequality ∥l∥_∗≤c ₀, we have

$$ \begin{gathered} 2^{-n}\rho_n(W,V_{\gamma}) \bigl(1+ \rho_n(W,V_{\gamma})\bigr)^{-1} \le\rho (W,V_{\gamma}) \le\alpha r, \\ \rho_n(W,V_{\gamma}) \bigl(1+\rho_n(W,V_{\gamma}) \bigr)^{-1} \le2^n\alpha r, \\ \rho_n(W,V_{\gamma}) \bigl(1-2^n\alpha r\bigr) \le2^n \alpha r, \quad\quad\|Wx-V_{\gamma}x\| \le2^n \alpha r \bigl(1-2^n\alpha r\bigr)^{-1}, \end{gathered} $$

(8.29)

and

$$ \bigl|l(Wx)-l(V_{\gamma}x)\bigr| \le c_02^n \alpha r\bigl(1-2^n \alpha r\bigr)^{-1}. $$

(8.30)

By (8.30), (8.27), (8.21) and (8.20),

$$\begin{aligned} l(Wx) \le& l(V_{\gamma}x)+c_02^n\alpha r \bigl(1-2^n \alpha r\bigr)^{-1} \\ \le&-\gamma(2n)^{-1}+c_02^n \alpha r \bigl(1-2^n \alpha r\bigr)^{-1} \\ =& c_02^n \alpha r \bigl(1-2^n\alpha r\bigr)^{-1}\\ &{}-(2n)^{-1}2^{-1}(1-\alpha)r \bigl(m+n+2+\sup\bigl\{ \|x\|: x \in X_0\bigr\} \bigr)^{-1} \\ \le&-r\bigl[-c_02^n\alpha\cdot2+(2n)^{-1}2^{-1}(1- \alpha) \bigl(m+n+2 + \sup\bigl\{ \|x\|: x \in X_0\bigr\} \bigr) ^{-1}\bigr]\\ \le&-2rc_02^n \alpha. \end{aligned}$$

Thus

$$\bigl\{ W \in{ \mathcal{A}}: \rho(W,V_{\gamma})\le\alpha r\bigr\} \cap \varOmega _{mn}=\emptyset. $$

In view of (8.25), we can conclude that Ω _mn is porous in ${\mathcal{A}}$, $\varOmega _{mn} \cap{ \mathcal{A}}_{c}$ is porous in ${\mathcal{A}}_{c}$, $\varOmega _{mn} \cap{ \mathcal{A}}_{au}$ is porous in ${\mathcal{A}}_{au}$, $\varOmega _{mn} \cap{ \mathcal{A}}_{auc}$ is porous in ${\mathcal{A}}_{auc}$, and if f attains its minimum, then $\varOmega _{mn}\cap{ \mathcal{A}}_{u}$ is porous in ${\mathcal{A}}_{u}$. This completes the proof of Theorem 8.2.

8.4 A Basic Lemma

The following result is our key lemma.

Lemma 8.7

Assume that $V \in{ \mathcal{A}}$ is regular, A(i), A(ii) are valid and that at least one of the following conditions holds: 1. $V \in{ \mathcal{A}}_{au}$; 2. A(iii) is valid.

Let $\bar{K}$ and $\bar{\varepsilon }$ be positive. Then there exist a neighborhood ${\mathcal{U}}$ of V in ${\mathcal{A}}$ and positive numbers $\bar{\alpha}$ and γ such that for each $W \in{ \mathcal{U}}$, each x∈X satisfying

$$ \|x\| \le\bar{K},\quad\quad f(x) \ge\inf(f)+\bar{\varepsilon }, $$

(8.31)

and each $\beta\in(0, \bar{\alpha}]$,

$$ f(x)-f(x+\beta Wx) \ge\beta\gamma. $$

(8.32)

Proof

There exists $K_{0} >\bar{K}+1$ such that

$$ \|Vx\| \le K_0 \quad\hbox{if } x \in X \hbox{ and } \|x\| \le\bar{K}+2. $$

(8.33)

By Assumption A(ii), there exists a constant L ₀>4 such that

$$ \bigl|f(x_1)-f(x_2)\bigr| \le L_0 \|x_1-x_2\| $$

(8.34)

for all x ₁,x ₂∈X satisfying ∥x ₁∥,∥x ₂∥≤2K ₀+4. Since V is regular, there exists a positive number δ ₀∈(0,1) such that

$$ \xi(Vy) \le-\delta_0 $$

(8.35)

for each y∈X satisfying ∥y∥≤K ₀+4, $f(y) \ge\inf (f)+\bar{\varepsilon }/4$, and each ξ∈∂f(y). Choose δ ₁∈(0,1) such that

$$ 4\delta_1(K_0+L_0) < \delta_0. $$

(8.36)

There exists a positive number $\bar{\alpha}$ such that the following conditions hold:

$$ 8\bar{\alpha}(L_0+1) (K_0+1)<\min\{1,\bar{\varepsilon }\}; $$

(8.37)

(a) if $V \in{ \mathcal{A}}_{au}$, then for each x ₁,x ₂∈X satisfying

$$ \begin{gathered} \|x_1\|,\|x_2\| \le\bar{K}+4,\quad\quad \min\bigl\{ f(x_1),f(x_2)\bigr\} \ge\inf(f)+\bar{\varepsilon }/4, \\ \hbox{and} \quad\|x_1-x_2\| \le\bar{\alpha}(K_0+1), \end{gathered} $$

(8.38)

the following inequality is true:

$$ \|Vx_1-Vx_2\| \le\delta_1; $$

(8.39)

(b) if A(iii) is valid, then for each x ₁,x ₂∈X satisfying (8.38), the following inequality is true:

$$ H\bigl(\partial f(x_1),\partial f(x_2) \bigr)<\delta_1. $$

(8.40)

Next choose a positive number δ ₂ such that

$$ 8\delta_2(L_0+1) <\delta_1 \delta_0. $$

(8.41)

Now choose a positive number γ such that

$$ \gamma<\delta_0 /8 $$

(8.42)

and define

$$ {\mathcal{U}}:=\bigl\{ W \in{ \mathcal{A}}: \|Wx-Vx\| \le\delta_2, x \in X \hbox { and } \|x\| \le\bar{K}\bigr\} . $$

(8.43)

Assume that $W \in{ \mathcal{U}}$, x∈X satisfies (8.31), and that $\beta\in(0,\bar{\alpha}]$. We intend to show that (8.32) holds. To this end, we first note that (8.31), (8.33), (8.37), (8.43) and (8.41) yield

$$\|x+\beta Vx\| \le\bar{K}+\beta K_0 \le\bar{K}+\bar{\alpha}K_0 \le \bar{K}+1 $$

and

$$\|x+\beta Wx\| \le\delta_2 \beta+\|x+\beta Vx\| \le\bar{K}+1+\bar{\alpha}\delta_2 \le\bar{K}+2. $$

By these inequalities, the definition of L ₀ (see (8.34)) and (8.43),

$$ \bigl|f(x+\beta Vx)-f(x+\beta Wx)\bigr| \le L_0\beta\|Wx- Vx\| \le L_0\beta\delta_2. $$

(8.44)

Next we will estimate f(x)−f(x+βVx). There exist θ∈[0,β] and l∈∂f(x+θVx) such that

$$ f(x+\beta Vx)-f(x)=l(Vx)\beta. $$

(8.45)

By (8.31), (8.33) and (8.37),

$$ \begin{gathered}\|x\| \le\bar{K}, \quad\quad\|Vx\| \le K_0,\quad\quad \|\theta Vx\| \le\bar{\alpha}K_0,\quad \hbox{and} \\ \|x+\theta Vx\| \le \bar{K}+1. \end{gathered} $$

(8.46)

It follows from (8.46) and the definition of L ₀ (see (8.34)) that

$$ \|l\|_* \le L_0. $$

(8.47)

It follows from (8.46), the definition of L ₀ (see (8.34)), (8.37) and (8.31) that

$$\begin{aligned} f(x+\theta Vx) \ge& f(x)-L_0 \|\theta Vx\| \\ \ge&f(x)-L_0 \bar{\alpha}K_0 \ge f(x)-8^{-1} \bar{\varepsilon }\ge\inf (f)+\bar{\varepsilon }/2. \end{aligned}$$

(8.48)

Consider the case where $V \in{ \mathcal{A}}_{au}$. By (8.47), condition (a), (8.46), (8.31) and (8.48),

$$\begin{aligned} \beta l(Vx) \le&\beta l\bigl(V(x+\theta Vx)\bigr) +\beta\|l\|_* \bigl(\bigl\| V(x+\theta Vx)-Vx\bigr\| \bigr) \\ \le&\beta l \bigl(V(x+\theta Vx)\bigr)+\beta L_0 \bigl\| V(x+\theta Vx)-Vx\bigr\| \\ \le&\beta l\bigl(V(x+\theta Vx)\bigr)+\beta L_0 \delta_1. \end{aligned}$$

(8.49)

By (8.46), (8.48) and the definition of δ ₀ (see (8.35)),

$$l\bigl(V(x+\theta Vx)\bigr) \le-\delta_0. $$

When combined with (8.49) and (8.36), this inequality implies that

$$\beta l(Vx) \le-\beta\delta_0 +\beta L_0 \delta_1 \le-\beta\delta _0 /2. $$

By these inequalities and (8.45),

$$ f(x+\beta Vx)-f(x) \le-\beta\delta_0 /2. $$

(8.50)

Assume now that A(iii) is valid. It then follows from condition (b), (8.46), (8.31) and (8.48) that

$$H\bigl(\partial f(x), \partial f(x+\theta Vx)\bigr)<\delta_1. $$

Therefore there exists $\bar{l} \in\partial f(x)$ such that $\|\bar{l}-l\|_{*} \le\delta_{1}$. When combined with (8.45) and (8.46), this fact implies that

$$\begin{aligned} f(x+\beta Vx)-f(x) =& \beta l(Vx) \le\beta\bar{l}(Vx) +\beta\|\bar{l}-l\|_*\|Vx\| \\ \le&\beta\bar{l} (Vx)+ \beta\delta_1K_0. \end{aligned}$$

(8.51)

It follows from the definition of δ ₀ (see (8.35)) and (8.31) that $\beta\bar{l}(Vx) \le-\beta\delta_{0}$. Combining this inequality with (8.51) and (8.36), we see that

$$f(x+\beta Vx) -f(x) \le -\beta\delta_0+ \beta\delta_1 K_0 \le-\beta\delta_0 /2. $$

Thus in both cases (8.50) is true. It now follows from (8.50), (8.44), (8.41) and (8.42) that

$$\begin{aligned} f(x+\beta Wx)-f(x) \le& f(x+\beta Vx)-f(x)+ f(x+\beta Wx)-f(x+\beta Vx) \\ \le&-\beta\delta_0/2+L_0\beta\delta_2 \le-\beta \delta_0 /4 \le -\gamma\beta. \end{aligned}$$

Thus (8.32) holds. Lemma 8.7 is proved. □

8.5 Proofs of Theorems 8.3 and 8.4

Proof of Theorem 8.3

To show that assertion (i) holds, suppose that

$$ \{x_i\}_{i=0}^{\infty} \subset X,\quad\quad x_{i+1} \in P_Wx_i,\quad i=0,1,\dots , \quad\hbox{and}\quad \liminf_{i \to\infty} \|x_i\|<\infty. $$

(8.52)

We will show that

$$ \lim_{i \to\infty}f(x_i)=\inf(f). $$

(8.53)

Assume the contrary. Then there exists ε>0 such that

$$ f(x_i) \ge\inf(f)+\varepsilon , \quad i=0,1,\dots. $$

(8.54)

There exists a number S>0 and a strictly increasing sequence of natural numbers $\{i_{k}\}_{k=1}^{\infty}$ such that

$$ \|x_{i_k}\|\le S,\quad k=1,2,\dots. $$

(8.55)

By Lemma 8.7, there exist numbers α,γ∈(0,1) such that for each x∈X satisfying

$$ \|x\| \le S,\quad\quad f(x) \ge\inf(f)+\varepsilon , $$

(8.56)

and each β∈(0,α],

$$ f(x)-f(x+\beta Wx) \ge\gamma\beta. $$

(8.57)

It follows from (8.52), (8.4), (8.5), the definitions of α and γ, (8.55) and (8.54) that for each integer k≥1,

$$f(x_{i_k})-f(x_{i_k+1}) \ge f(x_{i_k})-f(x_{i_k}+ \alpha Wx_{i_k}) \ge \gamma \alpha. $$

Since this inequality holds for all integers k≥1, we conclude that

$$\lim_{n \to\infty} \bigl(f(x_0)-f(x_n)\bigr)= \infty. $$

This contradicts our assumption that f is bounded from below. Therefore (8.53) and assertion (i) are indeed true, as claimed.

We turn now to assertion (ii). Let $\mathbf{a}=\{a_{i}\}_{i=0}^{\infty} \subset(0,1]$ satisfy (8.6) and let a bounded $\{x_{i}\}_{i=0}^{\infty} \subset X$ satisfy (8.7) for all integers i≥0. We will show that (8.53) holds. Indeed, assume that (8.53) is not true. Then there exists ε>0 such that (8.54) holds. Since the sequence $\{x_{i}\}_{1=0}^{\infty}$ is bounded, there exists a number S>0 such that

$$ S>\|x_i\|,\quad i=0,1,\dots. $$

(8.58)

By Lemma 8.7, there exist numbers α,γ∈(0,1) such that for each x∈X satisfying (8.56) and each β∈(0,α], inequality (8.57) holds. Since a _i →0 as i→∞, there exists a natural number i ₀ such that

$$ a_i<\alpha\quad\hbox{for all integers } i \ge i_0. $$

(8.59)

Let i≥i ₀ be an integer. Then it follows from (8.58), (8.54), the definitions of α and γ, and (8.59) that

$$f(x_i)-f(x_i+a_iWx_i) \ge \gamma a_i,\quad x_{i+1}=x_i+a_iWx_i, $$

and

$$f(x_i)-f(x_{i+1}) \ge\gamma a_i . $$

Since $\sum_{i=0}^{\infty}a_{i}=\infty$, we conclude that

$$\lim_{n \to\infty} \bigl(f(x_0)-f(x_n)\bigr)= \infty. $$

The contradiction we have reached shows that (8.53), assertion (ii) and Theorem 8.3 itself are all true. □

Proof of Theorem 8.4

Let

$$ K_0 >\sup\bigl\{ f(x): x \in X, \|x\| \le K+1\bigr\} $$

(8.60)

and set

$$ E_0=\bigl\{ x \in X: f(x) \le K_0+1\bigr\} . $$

(8.61)

Clearly, E ₀ is bounded and closed. Choose

$$ K_1 >\sup\bigl\{ \|x\|: x \in E_0\bigr\} +1+K. $$

(8.62)

By Lemma 8.7, there exist a neighborhood $\mathcal{U}$ of V in $\mathcal{A}$ and numbers α,γ∈(0,1) such that for each $W \in{ \mathcal{U}}$, each x∈X satisfying

$$ \|x\| \le K_1,\quad\quad f(x) \ge\inf(f)+\varepsilon , $$

(8.63)

and each β∈(0,α],

$$ f(x)-f(x+\beta Wx) \ge\gamma\beta. $$

(8.64)

Now choose a natural number N ₀ which satisfies

$$ N_0>(\alpha\gamma)^{-1}\bigl(K_0+4+\bigl| \inf(f)\bigr|\bigr). $$

(8.65)

First we will show that assertion (i) is true. Assume that $W \in{ \mathcal{U}}$, $\{x_{i}\}_{i=0}^{N_{0}} \subset X$,

$$ \|x_0\| \le K,\quad\hbox{and} \quad x_{i+1} \in P_Wx_i,\quad i=0,\dots, N_0-1. $$

(8.66)

Our aim is to show that

$$ f(x_{N_0}) \le\inf(f)+\varepsilon . $$

(8.67)

Assume that (8.67) is not true. Then

$$ f(x_i)>\inf(f)+\varepsilon ,\quad i=0,\dots, N_0. $$

(8.68)

By (8.66) and (8.60)–(8.62), we also have

$$ \|x_i\| \le K_1, \quad i=0,\dots, N_0. $$

(8.69)

Let i∈{0,…,N ₀−1}. It follows from (8.69), (8.68) and the definitions of $\mathcal{U}$, α and γ (see (8.63) and (8.64)) that

$$f(x_i)-f(x_{i+1}) \ge f(x_i)-f(x_i+ \alpha Wx_i) \ge\gamma\alpha. $$

Summing up from i=0 to N ₀−1, we conclude that

$$f(x_0)-f(x_{N_0}) \ge N_0\gamma\alpha. $$

It follows from this inequality, (8.60), (8.65) and (8.66) that

$$\inf(f) \le f(x_{N_0}) \le f(x_0)-N_0\gamma \alpha \le K_0-N_0\gamma\alpha\le-4- \bigl|\inf(f)\bigr|. $$

Since we have reached a contradiction, we see that (8.67) must be true and assertion (i) is proved.

Now we will show that assertion (ii) is also valid. To this end, let a sequence $\mathbf{a}=\{a_{i}\}_{i=0}^{\infty} \subset(0,1]$ satisfy

$$ \lim_{i \to\infty} a_i=0 \quad\hbox{and}\quad \sum _{i=0}^{\infty}a_i=\infty. $$

(8.70)

Evidently, there exists a natural number N ₁ such that

$$ a_i \le\alpha\quad\hbox{for all } i \ge N_1. $$

(8.71)

Choose a natural number N>N ₁+4 such that

$$ \gamma\sum_{i=N_1}^{N-1}a_i >K_0+4+\bigl|\inf(f)\bigr|. $$

(8.72)

Now assume that $W \in{ \mathcal{U}}$, $\{x_{i}\}_{i=0}^{N} \subset X$, ∥x ₀∥≤K, and that (8.7) holds for all i=0,…,N−1. We claim that

$$ f(x_N) \le\inf(f)+\varepsilon . $$

(8.73)

Assume the contrary. Then

$$ f(x_i)>\inf(f)+\varepsilon ,\quad i=0,\dots, N. $$

(8.74)

Since ∥x ₀∥≤K, we see by (8.7) and (8.60)–(8.62) that

$$ \|x_i\| \le K_1, \quad i=0,\dots, N. $$

(8.75)

Let i∈{N ₁,…,N−1}. It follows from (8.75), (8.74), (8.71) and the definitions of α and γ (see (8.63) and (8.64)) that

$$f(x_i)-f(x_i+a_iWx_i) \ge \gamma a_i. $$

This implies that

$$f(x_{N_1})-f(x_N) \ge\gamma\sum _{i=N_1}^{N-1}a_i. $$

By this inequality, (8.7), the inequality ∥x ₀∥≤K, (8.60) and (8.72), we obtain

$$\begin{aligned} \inf(f) \le& f(x_N) \le f(x_{N_1})-\gamma\sum _{i=N_1}^{N-1}a_i \\ \le& K_0 -\gamma\sum_{i=N_1}^{N-1}a_i <-4-|\inf(f)|. \end{aligned}$$

The contradiction we have reached proves (8.73) and assertion (ii). This completes the proof of Theorem 8.4. □

8.6 Methods for a Nonconvex Objective Function

Assume that (X,∥⋅∥) is a Banach space, (X ^∗,∥⋅∥_∗) is its dual space, and f:X→R ¹ is a function which is bounded from below and Lipschitzian on bounded subsets of X. Recall that for each pair of sets A,B⊂X ^∗,

$$H(A,B)=\max \Bigl\{ \sup_{x \in A} \inf_{y \in B} \|x-y\|_*, \sup_{y \in B} \inf_{x \in A}\|x-y\|_*\Bigr\} $$

is the Hausdorff distance between A and B. For each x∈X, let

$$ f^0(x,h)=\limsup_{t \to0^+, y \to x} \bigl[f(y+th)-f(y)\bigr]/t,\quad h \in X, $$

(8.76)

be the Clarke derivative of f at the point x [41],

$$ \partial f(x)=\bigl\{ l \in X^*: f^0(x,h) \ge l(h) \hbox{ for all } h \in X\bigr\} $$

(8.77)

the Clarke subdifferential of f at x, and

$$ \varXi (x):=\inf\bigl\{ f^0(x,h): h \in X \hbox{ and } \|h \| =1\bigr\} . $$

(8.78)

It is well known that the set ∂f(x) is nonempty and bounded. It should be mentioned that the functional Ξ was introduced in [176] and used in [182] in order to study penalty methods in constrained optimization.

Set inf(f)=inf{f(x):x∈X}. Denote by ${\mathcal{A}}$ the set of all mappings V:X→X such that V is bounded on every bounded subset of X, and for each x∈X, f ⁰(x,Vx)≤0. We denote by ${\mathcal{A}}_{c}$ the set of all continuous $V \in{ \mathcal{A}}$ and by ${\mathcal{A}}_{b}$ the set of all $V \in{ \mathcal{A}}$ which are bounded on X. Finally, let ${\mathcal{A}}_{bc}={\mathcal{A}}_{b} \cap{ \mathcal{A}}_{c}$. Next we endow the set ${\mathcal{A}}$ with two metrics, ρ _s and ρ _w . To define ρ _s , we set, for each $V_{1},V_{2} \in{ \mathcal{A}}$, $\tilde{\rho}_{s}(V_{1},V_{2})=\sup\{\|V_{1}x-V_{2}x\|: x \in X\}$ and

$$ \rho_s(V_1,V_2)=\tilde{\rho}_s(V_1,V_2) \bigl(1+\tilde{\rho}_s(V_1,V_2)\bigr)^{-1}. $$

(8.79)

(Here we use the convention that ∞/∞=1.) It is clear that $({\mathcal{A}},\rho_{s})$ is a complete metric space. To define ρ _w , we set, for each $V_{1},V_{2} \in{ \mathcal{A}}$ and each integer i≥1,

$$\begin{aligned} \displaystyle\rho_i(V_1,V_2):=\sup\bigl\{ \|V_1x-V_2x\|: x \in X \hbox{ and } \|x\| \le i\bigr\} ,& \end{aligned}$$

(8.80)

$$\begin{aligned} \displaystyle\rho_w(V_1,V_2):=\sum _{i=1}^{\infty}2^{-i}\bigl[ \rho_i(V_1,V_2) \bigl(1+ \rho_i(V_1,V_2)\bigr)^{-1}\bigr].& \end{aligned}$$

(8.81)

Clearly, $({\mathcal{A}},\rho_{w})$ is a complete metric space. It is also not difficult to see that the collection of the sets

$$E(N,\varepsilon )=\bigl\{ (V_1,V_2) \in{ \mathcal{A}} \times{ \mathcal{A}}: \|V_1x-V_2x\| \le \varepsilon , x \in X, \|x\| \le N\bigr\} , $$

where N,ε>0, is a base for the uniformity generated by the metric ρ _w . It is easy to see that ρ _w (V ₁,V ₂)≤ρ _s (V ₁,V ₂) for all $V_{1}, V_{2} \in{ \mathcal{A}}$. The metric ρ _w induces on $\mathcal{A}$ a topology which is called the weak topology and the metric ρ _s induces a topology which is called the strong topology. Clearly, ${\mathcal{A}}_{c}$ is a closed subset of $\mathcal{A}$ with the weak topology while ${\mathcal{A}}_{b}$ and ${\mathcal{A}}_{bc}$ are closed subsets of $\mathcal{A}$ with the strong topology. We consider the subspaces ${\mathcal{A}}_{c}$, ${\mathcal{A}}_{b}$ and ${\mathcal{A}}_{bc}$ with the metrics ρ _s and ρ _w which induce the strong and the weak topologies, respectively.

When the function f is convex, one usually looks for a sequence $\{x_{i} \}_{i=1}^{\infty}$ which tends to a minimum point of f (if such a point exists) or at least such that lim _i→∞ f(x _i )=inf(f). If f is not necessarily convex, but X is finite-dimensional, then we expect to construct a sequence which tends to a critical point z of f, namely a point z for which 0∈∂f(z). If f is not necessarily convex and X is infinite-dimensional, then the problem is more difficult and less understood because we cannot guarantee, in general, the existence of a critical point and a convergent subsequence. To partially overcome this difficulty, we have introduced the function Ξ:X→R ¹. Evidently, a point z is a critical point of f if and only if Ξ(z)≥0. Therefore we say that z is ε-critical for a given ε>0 if Ξ(z)≥−ε. We look for sequences $\{x_{i}\}_{i=1}^{\infty}$ such that either lim inf _i→∞ Ξ(x _i )≥0 or at least lim sup _i→∞ Ξ(x _i )≥0. In the first case, given ε>0, all the points x _i , except possibly a finite number of them, are ε-critical, while in the second case this holds for a subsequence of $\{x_{i}\}_{i=1}^{\infty}$.

We show, under certain assumptions on f, that for most (in the sense of Baire’s categories) vector fields $W \in{ \mathcal{A}}$, the iterative processes defined below (see (8.84) and (8.85)) yield sequences with the desirable properties. Moreover, we show that the complement of the set of “good” vector fields is not only of the first category, but also σ-porous. These results, which were obtained in [141], are stated in this section. Their proofs are relegated to subsequent sections.

For each set E⊂X, we denote by $\operatorname{cl}(E)$ the closure of E in the norm topology. Our results hold for any Banach space and for those functions which satisfy the following two assumptions.

A(i):

For each ε>0, there exists δ∈(0,ε) such that

$$\operatorname{cl}\bigl(\bigl\{ x \in X: \varXi (x) <-\varepsilon \bigr\} \bigr) \subset\bigl\{ x \in X: \varXi (x)<-\delta\bigr\} ; $$

A(ii):

for each r>0, the function f is Lipschitzian on the ball {x∈X:∥x∥≤r}.

We say that a mapping $V \in{ \mathcal{A}}$ is regular if for any natural number n, there exists a positive number δ(n) such that for each x∈X satisfying ∥x∥≤n and Ξ(x)<−1/n, we have f ⁰(x,Vx)≤−δ(n).

This concept of regularity is a non-convex analog of the regular vector fields introduced in [136]. We denote by $\mathcal{F}$ the set of all regular vector fields $V \in \mathcal{A}$.

Theorem 8.8

Assume that both A(i) and A(ii) hold. Then ${\mathcal{A}} \setminus{ \mathcal{F}}$ (respectively, ${\mathcal{A}}_{c} \setminus {\mathcal{F}}$, ${\mathcal{A}}_{b} \setminus{ \mathcal{F}}$ and ${\mathcal{A}}_{bc} \setminus{ \mathcal{F}}$) is a σ-porous subset of the space ${\mathcal{A}}$ (respectively, ${\mathcal{A}}_{c}$, ${\mathcal{A}}_{b}$ and ${\mathcal{A}}_{bc}$) with respect to the pair (ρ _w ,ρ _s ).

Now let $W \in{ \mathcal{A}}$. We associate with W two iterative processes. For x∈X we denote by P _W (x) the set of all y∈{x+αWx:α∈[0,1]} such that

$$ f(y)=\inf\bigl\{ f(x+\beta Wx): \beta\in[0,1]\bigr\} . $$

(8.82)

Given any initial point x ₀∈X, one can construct a sequence $\{x_{i}\}_{i=0}^{\infty} \subset X$ such that for all i=0,1,…,

$$ x_{i+1} \in P_W(x_i). $$

(8.83)

This is our first iterative process. Next we describe the second iterative process. Given a sequence $\mathbf{a}=\{a_{i}\}_{i=0}^{\infty} \subset(0,1)$ such that

$$ \lim_{i \to\infty} a_i=0\quad \hbox{and} \quad\sum _{i=0}^{\infty} a_i =\infty, $$

(8.84)

we construct for each initial point x ₀∈X, a sequence $\{x_{i}\}_{i=0}^{\infty} \subset X$ according to the following rule:

$$ \begin{gathered} x_{i+1}=x_i+a_iW(x_i)\quad \hbox{if } f\bigl(x_i+a_iW(x_i) \bigr)<f(x_i), \\ x_{i+1}=x_i \quad\hbox{otherwise, where } i=0,1,\dots. \end{gathered} $$

(8.85)

In the sequel we will also make use of the following assumption:

A(iii):

For each integer n≥1, there exists δ>0 such that for each x ₁,x ₂∈X satisfying ∥x ₁∥,∥x ₂∥≤n, min{Ξ(x _i ):i=1,2}≤−1/n, and ∥x ₁−x ₂∥≤δ, the following inequality holds: H(∂f(x ₁),∂f(x ₂))≤1/n.

We denote by $\operatorname{Card}(B)$ the cardinality of a set B.

Theorem 8.9

Assume that $W \in \mathcal{A}$ is regular, and that A(i), A(ii) and A(iii) are all valid. Then the following two assertions are true:

• (i) Let the sequence $\{x_{i}\}_{i=0}^{\infty} \subset X$ satisfy (8.83) for all i=0,1,…. If $\{x_{i}\}_{i=0}^{\infty}$ is bounded, then lim inf _i→∞ Ξ(x _i )≥0.

• (ii) Let a sequence $\mathbf{a}=\{a_{i}\}_{i=0}^{\infty} \subset(0,1)$ satisfy (8.84) and let the sequence $\{x_{i}\}_{i=0}^{\infty} \subset X$ satisfy (8.85) for all i=0,1,…. If $\{x_{i}\}_{i=0}^{\infty}$ is bounded, then

  $$\limsup_{i \to\infty} \varXi (x_i)\ge0. $$

Theorem 8.10

Assume that f(x)→∞ as ∥x∥→∞, $V \in{ \mathcal{A}}$ is regular, and that A(i), A(ii) and A(iii) are all valid. Let K,ε>0 be given. Then there exist a neighborhood $\mathcal{U}$ of V in $\mathcal{A}$ with the weak topology and a natural number N ₀ such that the following two assertions are true:

• (i) For each $W \in \mathcal{U}$, each integer n≥N ₀ and each sequence $\{x_{i}\}_{i=0}^{n} \subset X$ which satisfies ∥x ₀∥≤K and (8.83) for all i=0,…,n−1, we have

  $$\operatorname{Card}\bigl\{ i \in\{0,\dots, N-1\}: \varXi (x_i) \le-\varepsilon \bigr\} \le N_0. $$

• (ii) For each sequence of numbers $\mathbf{a}=\{a_{i}\}_{i=0}^{\infty} \subset(0,1)$ satisfying (8.84), there exists a natural number N such that for each $W \in{ \mathcal{U}}$ and each sequence $\{x_{i}\}_{i=0}^{N} \subset X$ which satisfies ∥x ₀∥≤K and (8.85) for all i=0,…,N−1, we have

  $$\max\bigl\{ \varXi (x_i): i=0,\dots,N\bigr\} \ge-\varepsilon . $$

8.7 An Auxiliary Result

For each positive number λ, set

$$ E_{\lambda}:=\bigl\{ x \in X: \varXi (x) <-\lambda\bigr\} . $$

(8.86)

Proposition 8.11

Let ε>0 be given. Suppose that

$$ \operatorname{cl}(E_{\varepsilon }) \subset E_{\delta(\varepsilon )} $$

(8.87)

for some δ(ε)∈(0,ε). Then there exists a locally Lipschitzian vector field $V \in{ \mathcal{A}}_{b}$ such that f ⁰(y,Vy)<−δ(ε) for all y∈X satisfying Ξ(y)<−ε.

Proof

It easily follows from definitions (8.76) and (8.78) that E _λ is an open set for all λ>0. Let x∈E _δ(ε). Then there exist h _x ∈X such that ∥h _x ∥=1 and f ⁰(x,h _x )<−δ(ε), and (see (8.76)) an open neighborhood U _x of x in X such that

$$ f^0(y,h_x) <-\delta(\varepsilon ) \quad\hbox{for all } y \in U_x. $$

(8.88)

For x∈X∖E _δ(ε), set

$$ h_x=0 \quad\hbox{and}\quad U_x=X \setminus \operatorname{cl}(E_{\varepsilon }). $$

(8.89)

Clearly, {U _x } _x∈X is an open covering of X. Since any metric space is paracompact, there is a locally finite refinement {Q _α :α∈A} of {U _x :x∈X}, i.e., an open covering of X such that each x∈X has a neighborhood Q(x) with Q(x)∩Q _α ≠∅ only for finitely many α∈A, and such that for each α∈A, there exists x _α ∈X with Q _α ⊂U(x _α ). Let α∈A. Define μ _α :X→[0,∞) by μ _α (x)=0 if x∉Q _α and by μ _α (x)=inf{∥x−y∥:y∈∂Q _α } otherwise. (Here ∂B is the boundary of a set B⊂X.) The function μ _α is clearly Lipschitzian on all of X with Lipschitz constant 1. Let ω _α (x)=μ _α (x)(∑ _β∈A μ _β (x))⁻¹, x∈X. Since {Q _α :α∈A} is locally finite, each ω _α is well defined and locally Lipschitzian on X. Define a locally Lipschitzian, bounded mapping V:X→X by

$$ V(y):=\sum_{\alpha\in A}\omega_{\alpha}(y)h_{x_{\alpha}},\quad y \in X. $$

(8.90)

Let y∈X. There are a neighborhood Q of y in X and α ₁,…,α _n ∈A such that

$$ \{\alpha\in A: Q_{\alpha} \cap Q \neq \emptyset\}=\{\alpha _1,\dots, \alpha_n\}. $$

(8.91)

We have

$$ V(y)=\sum_{i=1}^n \omega_{\alpha_i}(y)h_{x_{\alpha_i}}, \quad\quad\sum_{i=1}^n \omega_{\alpha_i}(y)=1, $$

(8.92)

$$ f^0(y,Vy)=f^0\Biggl(y,\sum _{i=1}^n\omega_{\alpha_i}(y)h_{x_{\alpha_i}} \Biggr) \le\sum_{i=1}^n \omega_{\alpha_i}(y) f^0(y, h_{x_{\alpha_i}}). $$

(8.93)

Let i∈{1,…,n} with $\omega_{\alpha_{i}}(y)>0$. Then

$$ y \in\operatorname{supp}\{\omega_{\alpha_i}\} \subset Q_{\alpha_i} \subset U_{x_{\alpha_i}}. $$

(8.94)

If $x_{\alpha_{i}} \in X \setminus E_{\delta(\varepsilon )}$, then by (8.89), $h_{x_{\alpha_{i}}}= 0$ and $f^{0}(y,h_{x_{\alpha_{i}}})=0$. If $x_{\alpha_{i}} \in E_{\delta(\varepsilon )}$, then by (8.88) and (8.94), $f^{0}(y,h_{x_{\alpha_{i}}})<0$. Therefore $f^{0}(y,h_{x_{\alpha_{i}}}) \le0$ in both cases and f ⁰(y,Vy)≤0. Thus $V \in{ \mathcal{A}}$. Assume that y∈E _ε , i∈{1,…,n} and $\omega_{\alpha_{i}}(y)>0$. Then (8.94) holds. We assert that $x_{\alpha_{i}} \in E_{\delta(\varepsilon )}$. Assume the contrary. Then $x_{\alpha_{i}} \in X \setminus E_{\delta(\varepsilon )}$ and by (8.89), $U_{x_{\alpha_{i}}}=X \setminus \operatorname{cl}(E_{\varepsilon })$. When combined with (8.94), this implies that $y \in E_{\varepsilon } \cap U_{x_{\alpha_{i}}}=$ $E_{\varepsilon } \cap(X \setminus \operatorname{cl}(E_{\varepsilon }))$, a contradiction. Thus $x_{\alpha_{i}} \in E_{\delta(\varepsilon )}$, as asserted. By the definition of $U_{x_{\alpha_{i}}}$ (see (8.88)) and (8.94), $f^{0}(y,h_{x_{\alpha_{i}}})<-\delta(\varepsilon )$. When combined with (8.93), this implies that f ⁰(y,Vy)<−δ(ε). □

8.8 Proof of Theorem 8.8

For each pair of integers m,n≥1, denote by Ω _mn the set of all $V \in{ \mathcal{A}}$ such that

$$\begin{aligned} \displaystyle \|Vx\| \le m \quad\hbox{for all } x \in X \hbox{ satisfying } \|x\| \le n+1 \quad\hbox{and}& \end{aligned}$$

(8.95)

$$\begin{aligned} \displaystyle \sup\bigl\{ f^0(x,Vx): x \in X, \|x\| \le n, \varXi (x) <-1/n\bigr\} =0.& \end{aligned}$$

(8.96)

Clearly,

$$ \bigcup_{m=1}^{\infty}\bigcup_{n=1}^{\infty} \varOmega _{mn}={\mathcal{A}} \setminus{ \mathcal{F}}. $$

(8.97)

Therefore in order to prove Theorem 8.8 it is sufficient to show that for each pair of integers m,n≥1, the set Ω _mn (respectively, $\varOmega _{mn}\cap{ \mathcal{A}}_{c}$, $\varOmega _{mn} \cap{ \mathcal{A}}_{b}$, $\varOmega _{mn} \cap{ \mathcal{A}}_{bc}$) is a porous subset of ${\mathcal{A}}$ (respectively, ${\mathcal{A}}_{c}$, ${\mathcal{A}}_{b}$, ${\mathcal{A}}_{bc}$) with respect to the pair (ρ _w ,ρ _s ). Let m,n≥1 be integers. By Proposition 8.11, there exists a vector field $V_{*} \in \mathcal{A}$ such that (i) V _∗ is bounded on X and V _∗ is locally Lipschitzian on X; (ii) there exists δ _∗∈(0,1) such that

$$ f^0(y,V_*y) <-\delta_* \quad\hbox{for all } y \in X \hbox{ satisfying } \varXi (y)<-(4n)^{-1}. $$

(8.98)

By assumption A(ii), there is c ₀>1 such that

$$ \bigl|f(x)-f(y)\bigr| \le c_0\|x-y\| $$

(8.99)

for all x,y∈X satisfying ∥x∥,∥y∥≤n+2. Choose α∈(0,1) such that

$$ \alpha c_02^{n+2}<(2n)^{-1}2^{-1}(1- \alpha)\delta_* \bigl(m+1+\sup\bigl\{ \|V_*x\|: x \in X\bigr\} \bigr)^{-1}. $$

(8.100)

Assume that $V \in{ \mathcal{A}}$ and r∈(0,1]. There are two cases: (a) sup{∥Vx∥:x∈X,∥x∥≤n+1}≤m+1; (b) sup{∥Vx∥:x∈X,∥x∥≤n+1}>m+1. We first assume that (b) holds. Let $W \in{ \mathcal{A}}$ with ρ _w (W,V)≤2⁻ⁿ⁻⁴. Then ρ _n+1(W,V)(1+ρ _n+1(V,W))⁻¹≤8⁻¹, ρ _n+1(W,V)≤1/7, and sup{∥Wx∥:x∈X,∥x∥≤n+1}>m. Thus $\{W \in{ \mathcal{A}}: \rho_{w}(W,V) \le2^{-n-4}\}\cap \varOmega _{mn}=\emptyset$. Assume now that (a) holds. Let

$$ \gamma=2^{-1}(1-\alpha)r\bigl(m+1+\sup\bigl\{ \|V_*x\|: x \in X\bigr\} \bigr)^{-1} $$

(8.101)

and define $V_{\gamma} \in{ \mathcal{A}}$ by

$$ V_{\gamma}x=Vx+ \gamma V_*x,\quad x \in X. $$

(8.102)

If $V \in{ \mathcal{A}}_{c}$ (respectively, ${\mathcal{A}}_{b}$, ${\mathcal{A}}_{bc}$), then $V_{\gamma} \in {\mathcal{A}}_{c}$ (respectively, ${\mathcal{A}}_{b}$, ${\mathcal{A}}_{bc}$). Next we estimate the distance ρ _s (V _γ ,V). It follows from (8.102), (8.101) and (8.76) that

$$ \rho_s(V_{\gamma},V) \le\tilde{\rho}_s(V_{\gamma},V) \le\gamma\sup\bigl\{ \bigl\| V_*(x)\bigr\| : x \in X\bigr\} \le 2^{-1}(1-\alpha)r . $$

(8.103)

Assume that $W \in{ \mathcal{A}}$ with

$$ \rho_w(W,V_{\gamma}) \le\alpha r. $$

(8.104)

By (8.104) and (8.103),

$$\begin{aligned} \rho_w(W,V) \le&\rho_w(W,V_{\gamma})+ \rho_w(V_{\gamma},V) \le\alpha r+2^{-1}(1-\alpha)r \\ \le&2^{-1}(1+\alpha)r<r. \end{aligned}$$

(8.105)

Assume now that

$$ x \in X,\quad\quad \|x\| \le n,\qquad \varXi (x) <-1/n \quad\hbox{and}\quad l \in\partial f(x). $$

(8.106)

Inequality (8.99) implies that

$$ \|l\|_*\le c_0. $$

(8.107)

By (8.102), (8.98) and (8.106),

$$ l(V_{\gamma}x)=l(Vx)+\gamma l\bigl(V_*(x)\bigr) \le \gamma l(V_*x) \le\gamma f^0(x,V_*x) \le\gamma(-\delta_*). $$

(8.108)

It follows from (8.106) and (8.80) that

$$ \|Wx-V_{\gamma}x\| \le\rho_n(W,V_{\gamma}). $$

(8.109)

By (8.104) and (8.81), we have 2⁻ⁿ ρ _n (W,V _γ )(1+ρ _n (W,V _γ ))⁻¹≤ ρ _w (W,V _γ )≤αr, ρ _n (W,V _γ )(1+ρ _n (W,V _γ ))⁻¹≤2ⁿ αr, and ρ _n (W,V _γ )(1−2ⁿ αr)≤2ⁿ αr. When combined with (8.109), the last inequality implies that ∥Wx−V _γ x∥≤2ⁿ αr(1−2ⁿ αr)⁻¹, and when combined with (8.107), this implies that

$$ \bigl|l(Wx)-l(V_{\gamma}x)\bigr| \le c_02^n \alpha r\bigl(1-2^n \alpha r\bigr)^{-1}. $$

(8.110)

By (8.110), (8.108), (8.101) and (8.100),

$$\begin{aligned} l(Wx) \le& l(V_{\gamma}x)+c_02^n\alpha r \bigl(1-2^n \alpha r\bigr)^{-1} \le -\gamma \delta_*+c_02^n \alpha r\bigl(1-2^n \alpha r \bigr)^{-1} \\ =&c_02^n \alpha r \bigl(1-2^n\alpha r \bigr)^{-1} \\ &{}- \delta_*\bigl[2^{-1}(1-\alpha)r \bigl(m+1 +\sup\bigl\{ \|V_*x\|: x \in X\bigr\} \bigr)\bigr]^{-1} \\ =& -r\bigl[-c_02^n \alpha\bigl(1-2^n\alpha r\bigr)^{-1} \\ &{}+\delta_*2^{-1}(1-\alpha) \bigl(m+1+\sup\bigl\{ \|V_*x\|: x \in X\bigr\} \bigr)^{-1}\bigr]\\ \le& -2rc_02^n\alpha. \end{aligned}$$

Since l is an arbitrary element of ∂f(x), we conclude that f ⁰(x,Wx)≤−2rc ₀2ⁿ α. Thus $\{W \in{ \mathcal{A}}: \rho_{w}(W,V_{\gamma})\le\alpha r\} \cap \varOmega _{mn}=\emptyset$. Recall that in case (b), $\{W \in{ \mathcal{A}}: \rho_{w}(W,V) \le2^{-n-4}\} \cap \varOmega _{mn}=\emptyset$. Therefore Ω _mn is porous in ${\mathcal{A}}$, $\varOmega _{mn} \cap{ \mathcal{A}}_{c}$ is porous in ${\mathcal{A}}_{c}$, $\varOmega _{mn} \cap{ \mathcal{A}}_{b}$ is porous in ${\mathcal{A}}_{b}$, and $\varOmega _{mn} \cap{ \mathcal{A}}_{bc}$ is porous in ${\mathcal{A}}_{bc}$, as asserted.

8.9 A Basic Lemma for Theorems 8.9 and 8.10

Lemma 8.12

Assume that $V \in{ \mathcal{A}}$ is regular, and that A(i), A(ii) and A(iii) are all valid. Let $\bar{K}$ and $\bar{\varepsilon }$ be positive. Then there exist a neighborhood ${\mathcal{U}}$ of V in ${\mathcal{A}}$ with the weak topology and positive numbers $\bar{\alpha}$ and γ such that for each $W \in{ \mathcal{U}}$, each x∈X satisfying

$$ \|x\| \le\bar{K} \quad\textit{and}\quad \varXi (x) \le-\bar{\varepsilon }, $$

(8.111)

and each $\beta\in(0, \bar{\alpha}]$, we have

$$ f(x)-f(x+\beta Wx) \ge\beta\gamma. $$

(8.112)

Proof

There exists $K_{0} >\bar{K}+1$ such that

$$ \|Vx\| \le K_0 \quad\hbox{if } x \in X \hbox{ and } \|x\| \le\bar{K}+2. $$

(8.113)

By Assumption A(ii), there exists a constant L ₀>4 such that

$$ \bigl|f(x_1)-f(x_2)\bigr| \le L_0 \|x_1-x_2\| $$

(8.114)

for all x ₁,x ₂∈X satisfying ∥x ₁∥,∥x ₂∥≤2K ₀+4. There is δ ₀∈(0,1) such that

$$ f^0(y,Vy) \le-\delta_0 $$

(8.115)

for each y∈X satisfying ∥y∥≤K ₀+4 and $\varXi (y) \le-\bar{\varepsilon }/4$. Choose δ ₁∈(0,1) such that

$$ 4\delta_1(K_0+L_0) < \delta_0. $$

(8.116)

By A(iii), there is a positive $\bar{\alpha}$ such that the following conditions hold:

$$ 8\bar{\alpha}(L_0+1) (K_0+1)<\min\{1,\bar{\varepsilon }\}; $$

(8.117)

for each x ₁,x ₂∈X satisfying

$$ \begin{gathered} \|x_1\|,\|x_2\| \le\bar{K}+4, \quad\quad\min\bigl\{ \varXi (x_1),\varXi (x_2)\bigr\} \le-\bar{\varepsilon }/4,\\ \|x_1-x_2\|\le\bar{\alpha}(K_0+1), \end{gathered} $$

(8.118)

the following inequality is true:

$$ H\bigl(\partial f(x_1),\partial f(x_2) \bigr)<\delta_1/2. $$

(8.119)

Next, choose a positive number δ ₂ such that

$$ 8\delta_2(L_0+1) <\delta_1 \delta_0. $$

(8.120)

Finally, choose a positive number γ and define a neighborhood $\mathcal{U}$ such that

$$\begin{aligned} \displaystyle \gamma<\delta_0 /4,& \end{aligned}$$

(8.121)

$$\begin{aligned} \displaystyle {\mathcal{U}}=\bigl\{ W \in{ \mathcal{A}}: \|Wx-Vx\| \le\delta_2, x \in X \hbox{ and } \|x\| \le\bar{K}\bigr\} .& \end{aligned}$$

(8.122)

Assume that $W \in{ \mathcal{U}}$, x∈X satisfies (8.111), and that $\beta\in(0,\bar{\alpha}]$. We intend to show that (8.112)) holds. To this end, we first note that (8.111), (8.113), (8.117) and (8.122) yield

$$ \begin{gathered} \|x+\beta Vx\| \le\bar{K}+\beta K_0 \le\bar{K}+\bar{\alpha}K_0 \le \bar{K}+1 , \\ \|x+\beta Wx\| \le\delta_2 \beta+\|x+\beta Vx\| \le\bar{K}+1+\bar{\alpha}\delta_2 \le\bar{K}+2. \end{gathered} $$

(8.123)

By these inequalities, the definition of L ₀ (see (8.114)) and (8.122),

$$ \bigl|f(x+\beta Vx)-f(x+\beta Wx)\bigr| \le L_0\beta\|Wx- Vx\| \le L_0\beta\delta_2. $$

(8.124)

Next we estimate f(x)−f(x+βVx). By [89], there exist θ∈[0,β] and l∈∂f(x+θVx) such that

$$ f(x+\beta Vx)-f(x)=l(Vx)\beta. $$

(8.125)

By (8.111), (8.114) and (8.117),

$$ \begin{gathered} \|x\| \le\bar{K},\quad\quad \|Vx\| \le K_0,\quad\quad \|\theta Vx\| \le \bar{\alpha}K_0, \quad\hbox{and} \\ \|x+\theta Vx\| \le \bar{K}+1. \end{gathered} $$

(8.126)

Note that (8.126) and the definition of L ₀ (see (8.114)) imply that

$$ \|l\|_* \le L_0. $$

(8.127)

It also follows from (8.111), (8.126) and the definition of $\bar{\alpha}$ (see (8.118) and (8.119)) that H(∂f(x),∂f(x+θVx))<δ ₁. Therefore there exists $\bar{l} \in\partial f(x)$ such that $\|\bar{l}-l\|_{*} \le \delta_{1}$. When combined with (8.125) and (8.126), this fact implies that

$$\begin{aligned} f(x+\beta Vx)-f(x) =& \beta l(Vx) \le\beta\bar{l}(Vx) + \beta\|\bar{l}-l\|_*\|Vx\| \\ \le&\beta \bar{l} (Vx)+ \beta\delta_1K_0. \end{aligned}$$

(8.128)

It follows from the definition of δ ₀ (see (8.115)) and (8.111) that $\beta\bar{l}(Vx) \le-\beta\delta_{0}$. Combining this inequality with (8.128) and (8.116), we see that f(x+βVx)−f(x)≤−βδ ₀+βδ ₁ K ₀≤−βδ ₀/2. It now follows from this inequality, (8.120), (8.124) and (8.121) that f(x+βWx)−f(x)≤ f(x+βVx)−f(x)+f(x+βWx)−f(x+βVx)≤−βδ ₀/2+L ₀ βδ ₂≤−βδ ₀/4≤−γβ. Thus (8.112) holds and Lemma 8.12 is proved. □

8.10 Proofs of Theorems 8.9 and 8.10

Proof of Theorem 8.9

To show that assertion (i) holds, suppose that

$$ \begin{gathered}\{x_i\}_{i=0}^{\infty} \subset X,\quad\quad x_{i+1} \in P_Wx_i,\quad i=0,1,\dots , \\ \sup\bigl\{ \|x_i\|: i=0,1,\dots\bigr\} <\infty. \end{gathered} $$

(8.129)

We claim that

$$ \liminf_{i \to\infty} \varXi (x_i) \ge0. $$

(8.130)

Assume the contrary. Then there exist ε>0 and a strictly increasing sequence of natural numbers $\{i_{k}\}_{k=1}^{\infty}$ such that

$$ \varXi (x_{i_k}) \le-\varepsilon ,\quad k=1,2,\dots. $$

(8.131)

Choose a number S>0 such that

$$ \|x_{i}\|\le S,\quad i=1,2,\dots. $$

(8.132)

By Lemma 8.12, there exist numbers α,γ∈(0,1) such that for each x∈X satisfying

$$ \|x\| \le S\quad \hbox{and} \quad \varXi (x) \le-\varepsilon , $$

(8.133)

and each β∈(0,α], we have

$$ f(x)-f(x+\beta Wx) \ge\gamma\beta. $$

(8.134)

It follows from (8.129), (8.82), (8.83), the definitions of α and γ, (8.132) and (8.131) that for each integer k≥1, $f(x_{i_{k}})-f(x_{i_{k}+1}) \ge f(x_{i_{k}})-f(x_{i_{k}}+\alpha Wx_{i_{k}}) \ge \gamma\alpha$. Since this inequality holds for all integers k≥1, we conclude that lim _n→∞(f(x ₀)−f(x _n ))=∞. This contradicts our assumption that f is bounded from below. Therefore (8.130) and assertion (i) are indeed true, as claimed.

We turn now to assertion (ii). Let $\mathbf{a}=\{a_{i}\}_{i=0}^{\infty} \subset(0,1)$ satisfy (8.84) and let a bounded $\{x_{i}\}_{i=0}^{\infty} \subset X$ satisfy (8.85) for all integers i≥0. We will show that

$$ \limsup_{i \to\infty} \varXi (x_i) \ge0. $$

(8.135)

Indeed, assume that (8.135) is not true. Then there exist ε>0 and an integer i ₁≥0 such that

$$ \varXi (x_i) \le-\varepsilon ,\quad i \ge i_1. $$

(8.136)

Since the sequence $\{x_{i}\}_{1=0}^{\infty}$ is bounded, there exists a number S>0 such that

$$ S>\|x_i\|, \quad i=0,1,\dots. $$

(8.137)

By Lemma 8.12, there exist numbers α,γ∈(0,1) such that for each x∈X satisfying (8.133) and each β∈(0,α], inequality (8.134) holds. Since a _i →0 as i→∞, there exists a natural number i ₀≥i ₁ such that

$$ a_i<\alpha\quad\hbox{for all integers } i \ge i_0. $$

(8.138)

Let i≥i ₀ be an integer. Then it follows from (8.137), (8.136), the definitions of α and γ, and (8.138) that f(x _i )−f(x _i +a _i Wx _i )≥γa _i , x _i+1=x _i +a _i Wx _i , and f(x _i )−f(x _i+1)≥γa _i . Since $\sum_{i=0}^{\infty}a_{i}=\infty$, we conclude that lim _n→∞(f(x ₀)−f(x _n ))=∞. The contradiction we have reached shows that (8.135), assertion (ii) and Theorem 8.9 itself are all true. □

Proof of Theorem 8.10

Let

$$\begin{aligned} \displaystyle K_0 >\sup\bigl\{ f(x): x \in X, \|x\| \le K+1\bigr\} ,& \end{aligned}$$

(8.139)

$$\begin{aligned} \displaystyle E_0=\bigl\{ x \in X: f(x) \le K_0+1\bigr\} .& \end{aligned}$$

(8.140)

It is clear that E ₀ is bounded and closed. Choose

$$ K_1 >\sup\bigl\{ \|x\|: x \in E_0\bigr\} +1+K. $$

(8.141)

By Lemma 8.12, there exist a neighborhood $\mathcal{U}$ of V in $\mathcal{A}$ and numbers α,γ∈(0,1) such that for each $W \in{ \mathcal{U}}$, each x∈X satisfying

$$ \|x\| \le K_1\quad \hbox{and}\quad \varXi (x) \le-\varepsilon , $$

(8.142)

and each β∈(0,α],

$$ f(x)-f(x+\beta Wx) \ge\gamma\beta. $$

(8.143)

Now choose a natural number N ₀ which satisfies

$$ N_0>(\alpha\gamma)^{-1} \bigl(K_0+4+\bigl|\inf(f)\bigr|\bigr). $$

(8.144)

Let $W \in{ \mathcal{U}}$, $\{x_{i}\}_{i=0}^{n} \subset X$, where the integer n≥N ₀,

$$\begin{aligned} \displaystyle \|x_0\| \le K,\quad\hbox{and}\quad x_{i+1} \in P_Wx_i,\quad i=0,\dots, n-1,& \end{aligned}$$

(8.145)

$$\begin{aligned} \displaystyle B=\bigl\{ i \in\{0,\dots, n-1\}: \varXi (x_i) \le- \varepsilon \bigr\} \quad\hbox{and}\quad m=\operatorname{Card}(B).& \end{aligned}$$

(8.146)

By (8.145) and (8.139)–(8.141), we have

$$ \|x_i\| \le K_1, \quad i=0,\dots, n. $$

(8.147)

Let i∈B. It follows from (8.147), (8.146) and the definitions of ${\mathcal{U}}$, α and γ (see (8.142) and (8.143)) that f(x _i )−f(x _i+1)≥f(x _i )−f(x _i +αWx _i )≥γα. Summing up from i=0 to n−1, we conclude that

$$f(x_0)-f(x_{n}) \ge\gamma\alpha\operatorname{Card}(B)=m\gamma \alpha. $$

It follows from this inequality, (8.139), (8.145) and (8.144) that

$$m \le\bigl[\bigl|\inf(f)\bigr|+K_0\bigr](\alpha\gamma)^{-1}<N_0. $$

Thus we see that assertion (i) is proved.

To prove assertion (ii), let a sequence $\mathbf{a}=\{a_{i}\}_{i=0}^{\infty} \subset(0,1)$ satisfy

$$ \lim_{i \to\infty} a_i=0 \quad\hbox{and} \quad\sum _{i=0}^{\infty}a_i=\infty. $$

(8.148)

Clearly, there exists a natural number N ₁ such that

$$ a_i \le\alpha\quad\hbox{for all } i \ge N_1. $$

(8.149)

Choose a natural number N>N ₁+4 such that

$$ \gamma\sum_{i=N_1}^{N-1}a_i >K_0+4+\bigl|\inf(f)\bigr|. $$

(8.150)

Now assume that $W \in{ \mathcal{U}}$, $\{x_{i}\}_{i=0}^{N} \subset X$, ∥x ₀∥≤K, and that (8.85) holds for all i=0,…,N−1. We will show that

$$ \max\bigl\{ \varXi (x_i): i=0,\dots,N\bigr\} \ge- \varepsilon . $$

(8.151)

Assume the contrary. Then

$$ \varXi (x_i)\le-\varepsilon ,\quad i=0,\dots, N. $$

(8.152)

Since ∥x ₀∥≤K, we see by (8.85) and (8.139)–(8.141) that

$$ \|x_i\| \le K_1,\quad i=0,\dots, N. $$

(8.153)

Let i∈{N ₁,…,N−1}. It follows from (8.153), (8.152), (8.149) and the definitions of α and γ (see (8.142)) and (8.143)) that

$$f(x_i)-f(x_i+a_iWx_i) \ge \gamma a_i. $$

This implies that

$$f(x_{N_1})-f(x_N) \ge\gamma\sum _{i=N_1}^{N-1}a_i. $$

By this inequality, (8.85), the inequality ∥x ₀∥≤K, (8.139) and (8.150), we obtain that

$$\inf(f) \le f(x_N) \le f(x_{N_1})-\gamma\sum _{i=N_1}^{N-1}a_i \le K_0 - \gamma \sum_{i=N_1}^{N-1}a_i <-4-\bigl| \inf(f)\bigr|. $$

The contradiction we have reached proves (8.151) and assertion (ii). □

8.11 Continuous Descent Methods

Let (X ^∗,∥⋅∥_∗) be the dual space of the Banach space (X,∥⋅∥), and let f:X→R ¹ be a convex continuous function which is bounded from below. Recall that for each pair of sets A,B⊂X ^∗,

$$H(A,B)=\max\Bigl\{ \sup_{x \in A} \inf_{y \in B} \|x-y\|_*, \sup_{y \in B} \inf_{x \in A}\|x-y\|_*\Bigr\} $$

is the Hausdorff distance between A and B.

For each x∈X, let

$$\partial f(x):=\bigl\{ l \in X^*: f(y)-f(x) \ge l(y-x) \hbox{ for all } y \in X \bigr\} $$

be the subdifferential of f at x. It is well known that the set ∂f(x) is nonempty and norm-bounded. Set

$$\inf(f):=\inf\bigl\{ f(x): x \in X\bigr\} . $$

Denote by $\mathcal{A}$ the set of all mappings V:X→X such that V is bounded on every bounded subset of X (that is, for each K ₀>0, there is K ₁>0 such that ∥Vx∥≤K ₁ if ∥x∥≤K ₀), and for each x∈X and each l∈∂f(x), l(Vx)≤0. We denote by ${\mathcal{A}}_{c}$ the set of all continuous $V \in \mathcal{A}$, by ${\mathcal{A}}_{u}$ the set of all $V \in \mathcal{A}$ which are uniformly continuous on each bounded subset of X, and by ${\mathcal{A}}_{au}$ the set of all $V \in{ \mathcal{A}}$ which are uniformly continuous on the subsets

$$\bigl\{ x \in X: \|x\| \le n \hbox{ and } f(x) \ge\inf(f)+1/n\bigr\} $$

for each integer n≥1. Finally, let ${\mathcal{A}}_{auc}={\mathcal{A}}_{au} \cap{ \mathcal{A}}_{c}$.

Our results are valid in any Banach space and for those convex functions which satisfy the following two assumptions.

A(i):

There exists a bounded set X ₀⊂X such that

$$\inf(f)=\inf\bigl\{ f(x): x \in X\bigr\} =\inf\bigl\{ f(x): x \in X_0 \bigr\} ; $$

A(ii):

for each r>0, the function f is Lipschitzian on the ball {x∈X:∥x∥≤r}.

Note that assumption A(i) clearly holds if lim_{∥x∥→∞} f(x)=∞.

We recall that a mapping $V \in{ \mathcal{A}}$ is regular if for any natural number n, there exists a positive number δ(n) such that for each x∈X satisfying

$$\|x\| \le n \quad\hbox{and}\quad f(x) \ge\inf(f)+1/n, $$

and for each l∈∂f(x), we have

$$l(Vx) \le-\delta(n). $$

Denote by ${\mathcal{F}}$ the set of all regular vector fields $V \in{ \mathcal{A}}$.

Let T>0, x ₀∈X and let u:[0,T]→X be a Bochner integrable function. Set

$$x(t)=x_0+\int_0^tu(s)\,ds, \quad t \in[0,T]. $$

Then x:[0,T]→X is differentiable and x′(t)=u(t) for almost every t∈[0,T]. Recall that the function f:X→R ¹ is assumed to be convex and continuous, and therefore it is, in fact, locally Lipschitzian. It follows that its restriction to the set {x(t):t∈[0,T]} is Lipschitzian. Indeed, since the set {x(t):t∈[0,T]} is compact, the closure of its convex hull C is both compact and convex, and so the restriction of f to C is Lipschitzian. Hence the function (f⋅x)(t):=f(x(t)), t∈[0,T], is absolutely continuous. It follows that for almost every t∈[0,T], both the derivatives x′(t) and (f⋅x)′(t) exist:

$$\begin{gathered} x'(t)=\lim_{h \to0}h^{-1} \bigl[x(t+h)-x(t)\bigr], \\ (f\cdot x)'(t)=\lim_{h \to0} h^{-1} \bigl[f\bigl(x(t+h)\bigr)-f\bigl(x(t)\bigr)\bigr].\end{gathered} $$

We continue with the following fact.

Proposition 8.13

Assume that t∈[0,T] and that both the derivatives x′(t) and (f⋅x)′(t) exist. Then

$$ (f\cdot x)'(t)=\lim_{h \to0}h^{-1} \bigl[f\bigl(x(t)+hx'(t)\bigr)-f\bigl(x(t)\bigr)\bigr]. $$

(8.154)

Proof

There exist a neighborhood $\mathcal{U}$ of x(t) in X and a constant L>0 such that

$$ \bigl|f(z_1)-f(z_2)\bigr| \le L \|z_1-z_2\| \quad\hbox{for all } z_1,z_2 \in{ \mathcal{U}}. $$

(8.155)

Let ε>0 be given. There exists δ>0 such that

$$ x(t+h), x(t)+hx'(t) \in{ \mathcal{U}} \quad\hbox {for each } h \in[-\delta, \delta] \cap[-t,T-t], $$

(8.156)

and such that for each h∈[(−δ,δ)∖{0}]∩[−t,T−t],

$$ \bigl\| x(t+h)-x(t)-hx'(t)\bigr\| <\varepsilon |h|. $$

(8.157)

Let

$$ h \in\bigl[(-\delta,\delta) \setminus\{0\}\bigr]\cap[-t,T-t]. $$

(8.158)

It follows from (8.156), (8.155) and (8.157) that

$$ \bigl|f\bigl(x(t+h)\bigr)-f\bigl(x(t)+hx'(t)\bigr)\bigr| \le L \bigl\| x(t+h)-x(t)-hx'(t)\bigr\| <L\varepsilon |h|. $$

(8.159)

Clearly,

$$\begin{aligned} \bigl[f\bigl(x(t+h)\bigr)-f\bigl(x(t)\bigr)\bigr]h^{-1} =&\bigl[f \bigl(x(t+h)\bigr)-f\bigl(x(t)+hx'(t)\bigr)\bigr]h^{-1} \\ &{}+\bigl[f\bigl(x(t)+hx'(t)\bigr)-f\bigl(x(t)\bigr) \bigr]h^{-1}. \end{aligned}$$

(8.160)

Relations (8.159) and (8.160) imply that

$$\begin{aligned} &\bigl|\bigl[f\bigl(x(t+h)\bigr)-f\bigl(x(t)\bigr)\bigr]h^{-1}- \bigl[f \bigl(x(t)+hx'(t)\bigr)-f\bigl(x(t)\bigr)\bigr]h^{-1}\bigr| \\ &\quad\le \bigl|f\bigl(x(t+h)\bigr)-f\bigl(x(t)+hx'(t)\bigr) \bigr|\bigl|h^{-1}\bigr| \le L \varepsilon . \end{aligned}$$

Since ε is an arbitrary positive number, we conclude that (8.154) holds. □

Assume now that $V \in{ \mathcal{A}}$ and that the differentiable function x:[0,T]→X satisfies

$$ x'(t)=V\bigl(x(t)\bigr)\quad \hbox{for a.e. } t \in[0,T]. $$

(8.161)

Then by Proposition 8.13, (f⋅x)′(t)≤0 for a.e. t∈[0,T], and f(x(t)) is decreasing on [0,T].

In the sequel we denote by μ(E) the Lebesgue measure of E⊂R ¹.

In the next two sections, we prove the following two results which were obtained in [148].

Theorem 8.14

Let $V \in{ \mathcal{A}}$ be regular, let x:[0,∞)→X be differentiable and suppose that

$$ x'(t)=V\bigl(x(t)\bigr) \quad\textit{for a.e.}\ t \in[0, \infty). $$

(8.162)

Assume that there exists a positive number r such that

$$ \mu\bigl(\bigl\{ t \in[0,T]: \bigl\| x(t)\bigr\| \le r\bigr\} \bigr) \to\infty \quad\textit{as } T \to\infty. $$

(8.163)

Then lim _t→∞ f(x(t))=inf(f).

Theorem 8.15

Let $V \in{ \mathcal{A}}$ be regular, let f be Lipschitzian on bounded subsets of X, and assume that lim_{∥x∥→∞} f(x)=∞. Let K ₀ and ε>0 be positive. Then there exist N ₀>0 and δ>0 such that for each T≥N ₀ and each differentiable mapping x:[0,T]→X satisfying

$$\bigl\| x(0)\bigr\| \le K_0 \quad\textit{and}\quad \bigl\| x'(t)-V\bigl(x(t) \bigr)\bigr\| \le\delta \quad\textit{for a.e. } t \in[0,T], $$

the following inequality holds for all t∈[N ₀,T]:

$$f\bigl(x(t)\bigr) \le\inf(f)+\varepsilon . $$

8.12 Proof of Theorem 8.14

Assume the contrary. Since f(x(t)) is decreasing on [0,∞), this means that there exists ε>0 such that

$$ \lim_{t \to\infty} f\bigl(x(t)\bigr)>\inf(f)+\varepsilon . $$

(8.164)

Then by Proposition 8.13 and (8.162), we have for each T>0,

$$\begin{aligned} f\bigl(x(T)\bigr)-f\bigl(x(0)\bigr) =&\int_0^T(f \cdot x)'(t)\,dt \\ =&\int_0^Tf^0\bigl(x(t),x'(t) \bigr)\,dt = \int_0^T f^0\bigl(x(t),V \bigl(x(t)\bigr)\bigr)\,dt \\ \le&\int_{\varOmega _T}f^0\bigl(x(t),V \bigl(x(t)\bigr)\bigr)\,dt, \end{aligned}$$

(8.165)

where

$$ \varOmega _T=\bigl\{ t \in[0,T]: \bigl\| x(t)\bigr\| \le r\bigr\} . $$

(8.166)

Since V is regular, there exists δ>0 such that for each x∈X satisfying

$$ \|x\| \le r+1\quad \hbox{and}\quad f(x) \ge\inf(f)+\varepsilon /2, $$

(8.167)

and each l∈∂f(x), we have

$$ l(Vx) \le-\delta. $$

(8.168)

It follows from (8.165), (8.166), (8.164), the definition of δ (see (8.167) and (8.168)) and (8.163) that for each T>0,

$$f\bigl(x(T)\bigr)-f\bigl(x(0)\bigr) \le\int_{\varOmega _T}f^0 \bigl(x(t),V\bigl(x(t)\bigr)\bigr)\,dt \le-\delta\mu(\varOmega _T) \to - \infty $$

as T→∞, a contradiction. The contradiction we have reached proves Theorem 8.14.

8.13 Proof of Theorem 8.15

We may assume without loss of generality that ε<1/2. Choose

$$ K_1 >\sup\bigl\{ f(x): x \in X \hbox{ and } \|x\| \le K_0+1\bigr\} . $$

(8.169)

The set

$$ \bigl\{ x \in X: f(x) \le K_1 +\bigl|\inf(f)\bigr|+4\bigr\} $$

(8.170)

is bounded. Therefore there exists

$$K_2>K_0+K_1 $$

such that

$$ \hbox{if } f(x) \le K_1+\bigl|\inf(f)\bigr|+4, \hbox{ then } \|x\| \le K_2. $$

(8.171)

There exists a number K ₃>K ₂+1 such that

$$\begin{aligned} & \sup\bigl\{ f(x): x \in X \hbox{ and } \|x\| \le K_2+1\bigr\} +2 \\ &\quad <\inf\bigl\{ f(x): x \in X \hbox{ and } \|x\| \ge K_3\bigr\} . \end{aligned}$$

(8.172)

There exists a number L ₀>0 such that

$$ \bigl|f(x_1)-f(x_2)\bigr| \le L_0 \|x_1-x_2\| $$

(8.173)

for each x ₁,x ₂∈X satisfying

$$ \|x_1\|,\|x_2\| \le K_3+1. $$

(8.174)

Fix an integer

$$ n>K_3+8/\varepsilon . $$

(8.175)

There exists a positive number δ(n)<1 such that:

(P1):

for each x∈X satisfying

$$\|x\| \le n \quad\hbox{and}\quad f(x) \ge\inf(f)+1/n, $$

and each l∈∂f(x), we have

$$l(Vx) \le-\delta(n). $$

Choose a natural number N ₀>8 such that

$$ 8^{-1}\delta(n)N_0 >\bigl|\inf(f)\bigr|+\sup\bigl\{ \bigl|f(z)\bigr|: z \in X \hbox{ and } \|z\| \le K_2\bigr\} +4 $$

(8.176)

and a positive number δ which satisfies

$$ 8\delta(N_0+1) (L_0+1)<\varepsilon \quad\text{and}\quad (1+L_0)\delta< \delta(n)/2. $$

(8.177)

Let T≥N ₀ and let x:[0,T]→X be a differentiable function such that

$$ \bigl\| x(0)\bigr\| \le K_2 $$

(8.178)

and

$$ \bigl\| x'(t)-V\bigl(x(t)\bigr)\bigr\| \le\delta\quad\hbox{for a.e. } t \in[0,T]. $$

(8.179)

We claim that

$$ \bigl\| x(t)\bigr\| \le K_3, \quad t \in\bigl[0,\min \{2N_0,T\}\bigr]. $$

(8.180)

Assume the contrary. Then there exists t ₀∈(0,min{2N ₀,T}] such that

$$ \bigl\| x(t)\bigr\| \le K_3,\quad t \in[0,t_0) \quad\hbox{and}\quad \bigl\| x(t_0)\bigr\| =K_3. $$

(8.181)

It follows from Proposition 8.13, the convexity of directional derivatives, the inequality f ⁰(x(t),Vx(t))≤0, which holds for all t∈[0,T], (8.181), the definition of L ₀ (see (8.173), (8.174) and (8.179)) that

$$\begin{aligned} &f\bigl(x(t_0)\bigr)-f\bigl(x(0)\bigr) \\ &\quad= \int_0^{t_0}(f \cdot x)'(t)\,dt =\int_0^{t_0}f^0 \bigl(x(t),x'(t)\bigr)\,dt \\ &\quad\le \int_0^{t_0}f^0\bigl(x(t),V\bigl(x(t) \bigr)\bigr)\,dt+\int_0^{t_0}f^0 \bigl(x(t),x'(t)-V\bigl(x(t)\bigr)\bigr)\,dt \\ &\quad\le \int_0^{t_0}f^0 \bigl(x(t),x'(t)-V\bigl(x(t)\bigr)\bigr)\,dt \le \int _0^{t_0}L_0\bigl\| x'(t)-V \bigl(x(t)\bigr)\bigr\| \,dt \le t_0L_0\delta. \end{aligned}$$

Thus by (8.177),

$$f\bigl(x(t_0)\bigr) \le f\bigl(x(0)\bigr)+2N_0L_0 \delta<f\bigl(x(0)\bigr)+1. $$

Since ∥x(0)∥≤K ₂ (see (8.178)) and ∥x(t ₀)∥=K ₃, the inequality just obtained contradicts (8.172). The contradiction we have reached proves (8.180).

We now claim that there exists a number

$$ t_0 \in[1,N_0] $$

(8.182)

such that

$$ f\bigl(x(t_0)\bigr) \le\inf(f)+\varepsilon /8. $$

(8.183)

Assume the contrary. Then

$$ f\bigl(x(t)\bigr)>\inf(f)+\varepsilon /8 \quad\hbox{and}\quad \bigl\| x(t)\bigr\| \le K_3,\quad t \in[1,N_0]. $$

(8.184)

It follows from (8.184), Property (P1) and (8.175) that

$$ f^0\bigl(x(t),V\bigl(x(t)\bigr)\bigr) \le-\delta(n),\quad t \in[1,N_0]. $$

(8.185)

By (8.185), (8.184), (8.179), (8.177), the convexity of the directional derivatives of f, and the definition of L ₀ (see (8.173) and (8.174)), we have, for almost every t∈[1,N ₀],

$$\begin{aligned} f^0\bigl(x(t),x'(t)\bigr) \le& f^0 \bigl(x(t),V\bigl(x(t)\bigr)\bigr)+f^0\bigl(x(t),x'(t)-V \bigl(x(t)\bigr)\bigr) \\ \le&-\delta(n)+L_0\bigl\| x'(t)-V\bigl(x(t) \bigr)\bigr\| \le-\delta(n)+L_0\delta \\ \le&-\delta(n)/2. \end{aligned}$$

(8.186)

It follows from the convexity of the directional derivatives of f, the inclusion $V \in{ \mathcal{A}}$, (8.179), (8.180) and the definition of L ₀ (see (8.173) and (8.174)), that for almost every t∈[0,1],

$$\begin{aligned} f^0\bigl(x(t),x'(t)\bigr) \le& f^0 \bigl(x(t),V\bigl(x(t)\bigr)\bigr)+f^0\bigl(x(t),x'(t)-V \bigl(x(t)\bigr)\bigr) \\ \le& f^0\bigl(x(t),x'(t)-V\bigl(x(t) \bigr)\bigr) \le L_0\bigl\| x'(t)-V\bigl(x(t)\bigr)\bigr\| \\ \le& L_0\delta. \end{aligned}$$

(8.187)

Inequalities (8.178), (8.186) and (8.187) imply that

$$\begin{aligned} &\inf(f)-\sup\bigl\{ f(z): z \in X, \|z\| \le K_2\bigr\} \\ &\quad \le f \bigl(x(N_0)\bigr)-f\bigl(x(0)\bigr) \\ &\quad=\int_0^{N_0}f^0\bigl(x(t),x'(t) \bigr)\,dt= \int_0^1f^0 \bigl(x(t),x'(t)\bigr)\,dt+\int_1^{N_0}f^0 \bigl(x(t),x'(t)\bigr)\,dt \\ &\quad\le-2^{-1}\delta(n)N_0/2+1. \end{aligned}$$

This contradicts (8.176). The contradiction we have reached yields the existence of a point t ₀ which satisfies both (8.182) and (8.183). Clearly, ∥x(t ₀)|≤K ₂. Having established (8.180) and the existence of such a point t ₀ for an arbitrary mapping x satisfying both (8.178) and (8.179), we now consider the mapping x ₀(t)=x(t+t ₀), t∈[0,T−t ₀]. Evidently, (8.178) and (8.179) hold true with x replaced by x ₀ and T replaced by T−t ₀. Hence, if T−t ₀≥N ₀, then we have

$$\bigl\| x(t)\bigr\| =\bigl\| x_0(t-t_0)\bigr\| \le K_3,\quad t \in \bigl[t_0,t_0+\min\{2N_0,T\}\bigr], $$

and there exists

$$t_1 \in[t_0+1,t_0+N_0] $$

for which

$$f\bigl(x(t_1)\bigr) \le\inf(f)+\varepsilon /8. $$

Repeating this procedure, we obtain by induction a finite sequence of points $\{t_{i}\}_{i=0}^{q}$ such that

$$\begin{gathered} t_0 \in[1,N_0],\quad\quad t_{i+1}-t_i \in[1,N_0],\quad i=0,\dots, q-1,\quad\quad T-t_q<N_0, \\ f\bigl(x(t_i)\bigr)\le\inf(f)+\varepsilon /8,\quad i=0,\dots,q, \\ \bigl\| x(t)\bigr\| \le K_3,\quad t \in[t_0,T].\end{gathered} $$

Let i∈{0,…,q}, t≤T, and 0<t−t _i ≤N ₀. Then by Proposition 8.13, the convexity of the directional derivative of f, the inclusion $V \in{ \mathcal{A}}$, the definition of L ₀ (see (8.173) and (8.174)), (7.177) and (8.179), we have

$$\begin{aligned} f\bigl(x(t)\bigr)-f\bigl(x(t_i)\bigr) =&\int_{t_i}^tf^0 \bigl(x(t),x'(t)\bigr)\,dt \\ \le&\int_{t_i}^tf^0\bigl(x(t),V \bigl(x(t)\bigr)\bigr)\,dt+\int_{t_i}^tf^0 \bigl(x(t),x'(t)-V\bigl(x(t)\bigr)\bigr)\,dt \\ \le&\int_{t_i}^tf^0 \bigl(x(t),x'(t)-V\bigl(x(t)\bigr)\bigr)\,dt \\ \le&\int _{t_i}^tL_0\bigl\| x'(t)-V \bigl(x(t)\bigr)\bigr\| \,dt \\ \le& L_0\delta(t-t_i) \le2N_0L_0 \delta<\varepsilon /4 \end{aligned}$$

and hence

$$f\bigl(x(t)\bigr) \le f\bigl(x(t_i)\bigr)+\varepsilon /4 \le\inf(f)+ \varepsilon /2. $$

This completes the proof of Theorem 8.15.

8.14 Regular Vector-Fields

In the previous sections of this chapter, given a continuous convex function f on a Banach space X, we associate with f a complete metric space ${\mathcal{A}}$ of mappings V:X→X such that f ⁰(x,Vx)≤0 for all x∈X. Here f ⁰(x,u) is the right-hand derivative of f at x in the direction of u∈X. We call such mappings descent vector-fields (with respect to f). We identified a regularity property of such vector-fields and showed that regular vector-fields generate convergent discrete descent methods. This has turned out to be true for continuous descent methods as well. Such results are significant because most of the elements in ${\mathcal{A}}$ are, in fact, regular. Here by “most” we mean an everywhere dense G _δ subset of $\mathcal{A}$. Thus it is important to know when a given descent vector-field V:X→X is regular. In [163] we established necessary and sufficient conditions for regularity: see Theorems 8.18–8.21 below.

More precisely, let (X,∥⋅∥) be a Banach space and let (X ^∗,∥⋅∥_∗) be its dual.

For each h:X→R ¹, set inf(h)={h(z):z∈X}.

Let U be a nonempty, open subset of X and let f:U→R ¹ be a locally Lipschitzian function.

For each x∈U, let

$$ f^0(x,h)=\limsup_{t \to0^+, y\to x} \bigl[f(y+th)-f(y)\bigr]/t,\quad h \in X, $$

(8.188)

be the Clarke derivative of f at the point x, and let

$$ \partial f(x)=\bigl\{ l \in X^*: f^0(x,h) \ge l(h) \hbox{ for all } h \in X\bigr\} $$

(8.189)

be the Clarke subdifferential of f at x.

For each x∈U, set

$$ \varXi _f(x):=\inf\bigl\{ f^0(x,u): u \in X, \|u\| \le1\bigr\} . $$

(8.190)

Clearly, Ξ _f (x)≤0 for all x∈X and Ξ _f (x)=0 if and only if 0∈∂f(x).

For each x∈U, set

$$ \tilde{\varXi }_f(x)=\inf\bigl\{ f^0(x,h): h \in X, \|h\| =1\bigr\} . $$

(8.191)

Let x∈U. Clearly, $\tilde{\varXi }_{f}(x)\ge \varXi _{f}(x)$ and 0∈∂f(x) if and only if $\tilde{\varXi }_{f}(x) \ge0$.

In the next section we prove the following two propositions.

Proposition 8.16

Let x∈U. If $\tilde{\varXi }_{f}(x) \ge0$, then Ξ _f (x)=0. If $\tilde{\varXi }_{f}(x)<0$, then $\varXi _{f}(x)= \tilde{\varXi }_{f}(x)$.

Proposition 8.17

For each x∈U,

$$ \varXi _f(x)=-\inf\bigl\{ \|l\|_*: l \in\partial f(x) \bigr\} . $$

(8.192)

Assume now that f:X→R ¹ is a continuous and convex function which is bounded from below. It is known that f is locally Lipschitzian. It is also known (see Chap. 2, Sect. 2 of [41]) that in this case

$$f^0(x,h)=\lim_{t \to0^+} \bigl[f(x+th)-f(x)\bigr]/t,\quad x,h \in X. $$

Recall that a mapping V:X→X is called regular if V is bounded on every bounded subset of X, f ⁰(x,Vx)≤0 for all x∈X, and if for any natural number n, there exists a positive number δ(n) such that for each x∈X satisfying ∥x∥≤n and f(x)≥inf(f)+1/n, we have

$$f^0(x,Vx) \le-\delta(n). $$

We now present four results which were established in [163]. Their proofs are given in subsequent sections.

Theorem 8.18

Let f:X→R ¹ be a convex and continuous function which is bounded from below, let $\bar{x} \in X$ satisfy

$$ f(\bar{x}) =\inf\bigl\{ f(z): z \in X\bigr\} , $$

(8.193)

and let the following property hold:

(P1):

for every sequence $\{y_{i}\}_{i=1}^{\infty} \subset X$ satisfying $\lim_{i \to\infty}f(y_{i})=f(\bar{x})$, $\lim_{i \to\infty}y_{i} =\bar{x}$ in the norm topology.

For each natural number n, let ϕ _n :[0,∞)→[0,∞) be an increasing function such that ϕ _n (0)=0 and the following property holds:

(P2):

for each ε>0, there exists δ:=δ(ε,n)>0 such that for each t≥0 satisfying ϕ _n (t)≤δ, the inequality t≤ε holds.

If V:X→X is bounded on bounded subsets of X,

$$ f^0(x,Vx) \le0\quad \textit{for all}\ x \in X, $$

(8.194)

and if for each natural number n and each x∈X satisfying ∥x∥≤n, we have

$$ f^0(x,Vx) \le-\phi_n\bigl(- \varXi _f(x)\bigr), $$

(8.195)

then V is regular.

Theorem 8.19

Assume that f:X→R ¹ is a convex and continuous function, $\bar{x} \in X$,

$$f(\bar{x})=\inf(f), $$

property (P1) holds and the following property also holds:

(P3):

if $\{x_{i}\}_{i=1}^{\infty} \subset X$ converges to $\bar{x}$ in the norm topology, then

$$\lim_{i \to\infty} \varXi _f(x_i)=0. $$

Assume that V:X→X is regular and let n≥1 be an integer. Then there exists an increasing function ϕ _n :[0,∞)→[0,∞) such that ϕ _n (0)=0, property (P2) holds, and for each x∈X satisfying ∥x∥≤n, we have

$$f^0(x,Vx) \le-\phi_n\bigl(-\varXi _f(x)\bigr). $$

Assume now that f:X→R ¹ is merely locally Lipschitzian. Recall that in this case a mapping V:X→X is called regular if V is bounded on every bounded subset of X,

$$ f^0(x,Vx) \le0 \quad\hbox{for all } x \in X, $$

(8.196)

and for any natural number n, there exists δ(n)>0 such that for each x∈X satisfying ∥x∥≤n and Ξ _f (x)≤−1/n, we have f ⁰(x,Vx)≤−δ(n).

Theorem 8.20

Let f:X→R ¹ be a locally Lipschitzian function. For each natural number n, let ϕ _n :[0,∞)→[0,∞) be an increasing function such that ϕ _n (0)=0 and property (P2) holds.

Assume that V:X→X is bounded on every bounded subset of X,

$$f^0(x,Vx) \le0\quad\textit{for all}\ x\in X, $$

and for each natural number n and each x∈X satisfying ∥x∥≤n, we have

$$ f^0(x,Vx) \le-\phi_n\bigl(- \varXi _f(x)\bigr). $$

(8.197)

Then V is regular.

Theorem 8.21

Assume that the function f:X→R ¹ is locally Lipschitzian and that V:X→X is regular.

Then for each natural number n, there exists an increasing function ϕ _n :[0,∞)→[0,∞) such that (P2) holds and for each natural number n and each x∈X satisfying ∥x∥≤n, (8.197) holds.

8.15 Proofs of Propositions 8.16 and 8.17

Proof of Proposition 8.16

Assume that $\tilde{\varXi }_{f}(x)\ge0$. Then 0∈∂f(x) and Ξ _f (x)=0. Assume that $\tilde{\varXi }_{f}(x) <0$. Then by definition (see (8.191)),

$$ \inf\bigl\{ f^0(x,h): h \in X, \|h\|=1\bigr\} =\tilde{\varXi }_f(x)<0. $$

(8.198)

By (8.198) and the homogeneity of f ⁰(x,⋅),

$$ f^0(x,h) \ge\tilde{\varXi }_f(x)\|h\| \quad\text{for all } h \in X. $$

(8.199)

By (8.198), (8.191), (8.190) and (8.199),

$$\begin{aligned} 0 >&\tilde{\varXi }_f(x) \ge \varXi _f(x) =\inf\bigl\{ f^ 0(x,h): h \in X, \|h\| \le1\bigr\} \\ \ge&\inf\bigl\{ \tilde{\varXi }_f(x)\|h\|: h \in X, \|h\| \le1\bigr\} = \tilde{\varXi }_f(x). \end{aligned}$$

This implies that

$$\tilde{\varXi }_f(x)=\varXi _f(x), $$

as claimed. Proposition 8.16 is proved. □

We precede the proof of Proposition 8.17 with the following lemma.

Lemma 8.22

Let x∈U and c>0 be given. Then the following statements are equivalent:

• (i) Ξ _f (x)≥−c;

• (ii) $\tilde{\varXi }_{f}(x) \ge-c$;

• (iii) there is l∈∂f(x) such that ∥l∥_∗≤c.

Proof

By Proposition 8.16,

$$\varXi _f(x) \ge-c \quad\hbox{if and only if}\quad \tilde{\varXi }_f(x) \ge-c. $$

It follows from (8.191) that $\tilde{\varXi }_{f}(x)\ge-c$ if and only if

$$f^0(x,h) \ge-c \quad\hbox{for all } h \in X \hbox{ satisfying } \|h\|=1, $$

which is, in its turn, equivalent to the following relation:

$$f^0(x,h) \ge-c\|h\| \quad\hbox{for all } h \in X. $$

Rewriting this last inequality as

$$f^0(x,h)+c\|h\| \ge0\quad \hbox{for all } h \in X, $$

we see that it is equivalent to the inclusion

$$0 \in\partial f(x) +c\bigl\{ l \in X^*: \|l\|_* \le1\bigr\} . $$

Thus we have proved that (ii) is equivalent to (iii). This completes the proof of Lemma 8.22. □

Proof of Proposition 8.17

Clearly, equality (8.192) holds if either one of its sides equals zero. Therefore we only need to prove (8.192) in the case where

$$ \varXi _f(x)<0 \quad\hbox{and}\quad \inf\bigl\{ \|l\|_*: l \in \partial f(x)\bigr\} >0. $$

(8.200)

Assume that (8.200) holds. By Lemma 8.22, there is $\bar{l}$ such that

$$ \bar{l} \in\partial f(x)\quad \hbox{and}\quad \|\bar{l}\|_* \le- \varXi _f(x). $$

(8.201)

Hence

$$ -\inf\bigl\{ \|l\|_*: l \in\partial f(x)\bigr\} \ge-\|\bar{l}\|_* \ge \varXi _f(x). $$

(8.202)

Let ε be any positive number. There is l _ε ∈∂f(x) such that

$$ \|l_{\varepsilon }\|_* \le\inf\bigl\{ \|l\|_*: l \in \partial f(x) \bigr\} +\varepsilon . $$

(8.203)

By (8.203) and Lemma 8.22,

$$\varXi _f(x) \ge-\varepsilon -\inf\bigl\{ \|l\|_*: l \in\partial f(x)\bigr\} . $$

Since ε is any positive number, we conclude that

$$\varXi _f(x)\ge-\inf\bigl\{ \|l\|_*: l \in\partial f(x)\bigr\} . $$

When combined with (8.202), this inequality completes the proof of Proposition 8.17. □

8.16 An Auxiliary Result

Proposition 8.23

Let g:X→R ¹ be a convex and continuous function, $\bar{x} \in X$,

$$ g(\bar{x})=\inf\bigl\{ g(z): z \in X\bigr\} , $$

(8.204)

and let the following property hold:

(P4):

for any sequence $\{y_{i}\}_{i=1}^{\infty} \subset X$ satisfying $\lim_{i \to\infty}g(y_{i})=g(\bar{x})$, we have $\lim_{i \to \infty}\|y_{i}-\bar{x}\|=0$.

Assume that $\{x_{i}\}_{i=1}^{\infty} \subset X$,

$$ \sup\bigl\{ \|x_i\|: i=1,2,\dots\bigr\} <\infty\quad\textit{and}\quad \lim _{i \to\infty} \varXi _g(x_i)=0. $$

(8.205)

Then $\lim_{i \to \infty}\|x_{i}-\bar{x}\|=0$.

Proof

By (8.205) and Proposition 8.17, there exists a sequence $\{l_{i}\}_{i=1}^{\infty} \subset X^{*}$ such that

$$ \lim_{i \to\infty}\|l_i\|_*=0 \quad\hbox{and}\quad l_i\in\partial g(x_i)\quad \hbox{for all integers } i \ge1. $$

(8.206)

Choose a number M>0 such that

$$ \|x_i\| \le M\quad\hbox{for all integers } i \ge1 $$

(8.207)

and let i≥1 be an integer. By (8.206),

$$ g(z)-l_i(z) \ge g(x_i)-l_i(x_i)\quad \hbox{for all } z \in X. $$

(8.208)

It follows from (8.208), (8.207) and (8.206) that

$$\begin{aligned} g(\bar{x})-g(x_i) =&g(\bar{x})-l_i(\bar{x})- \bigl(g(x_i)-l_i(x_i)\bigr)+l_i( \bar{x}-x_i) \\ \ge& l_i(\bar{x}-x_i) \ge-\|l_i\|\|\bar{x}-x_i\| \ge-\|l_i\|\bigl(M+\|\bar{x}\|\bigr) \to0 \quad\hbox{as } i \to\infty \end{aligned}$$

and therefore

$$\liminf_{i \to\infty}\bigl(g(\bar{x})-g(x_i)\bigr) \ge0. $$

Together with (P4) this implies that $\lim_{i \to\infty}\|x_{i}-\bar{x}\|=0$. Proposition 8.23 is proved. □

8.17 Proof of Theorem 8.18

To show that V is regular, let n be a natural number. We have to find a positive number δ=δ(n) such that for each x∈X satisfying ∥x∥≤n and f(x)≥inf(f)+1/n,

$$f^0(x,Vx) \le-\delta. $$

Assume the contrary. Then for each natural number k, there exists x _k ∈X satisfying

$$ \|x_k\| \le n,\quad\quad f(x_k) \ge\inf(f)+1/n, $$

(8.209)

and

$$ f^0(x_k,Vx_k)>-1/k. $$

(8.210)

It follows from (8.210), (8.209) and (8.195) that for each natural number k,

$$-k^{-1}<f^0(x_k,Vx_k) \le- \phi_n\bigl(-\varXi _f(x_k)\bigr) $$

and hence ϕ _n (−Ξ _f (x _k ))<k ⁻¹.

Together with (P2) this inequality implies that lim _k→∞ Ξ _f (x _k )=0. When combined with Proposition 8.23 and (8.209), this implies $\lim_{k \to\infty}\|x_{k}-\bar{x}\|=0$. Since f is continuous,

$$\lim_{k \to\infty}f(x_k)=f(\bar{x})=\inf(f). $$

This, however, contradicts (8.209). The contradiction we have reached proves that V is indeed regular, as asserted.

8.18 Proof of Theorem 8.19

In what follows we make the convention that the infimum over the empty set is infinity. Set ϕ _n (0)=0 and let t>0. Put

$$ \phi_n(t)=\min\bigl\{ \inf\bigl\{ -f^0(x,Vx): x \in X, \|x\|\le n \hbox{ and } \varXi _f(x)\le-t\bigr\} ,1\bigr\} . $$

(8.211)

Clearly, ϕ _n :[0,∞)→[0,1] is well defined and increasing.

We show that for each x∈X satisfying ∥x∥≤n,

$$ f^0(x,Vx) \le-\phi_n\bigl(- \varXi _f(x)\bigr). $$

(8.212)

Let x∈X with ∥x∥≤n. If Ξ _f (x)=0, then it is obvious that (8.212) holds.

Assume now that

$$ \varXi _f(x)<0. $$

(8.213)

Then by (8.211)), (8.213) and the inequality ∥x∥≤n,

$$\begin{aligned} \phi_n\bigl(-\varXi _f(x)\bigr) =&\min\bigl\{ \inf \bigl\{ -f^0(y,Vy): y \in X, \|y\| \le n \hbox{ and } \varXi _f(y) \le \varXi _f(x)\bigr\} ,1\bigr\} \\ \le&\min\bigl\{ 1,-f^0(x,Vx)\bigr\} \le-f^0(x,Vx) \end{aligned}$$

and hence

$$f^0(x,Vx) \le-\phi_n\bigl(-\varXi _f(x)\bigr). $$

Thus (8.212) holds for each x∈X satisfying ∥x∥≤n.

Next we show that (P2) holds. To this end, let ε>0 be given. We claim that there is δ>0 such that for each t≥0 satisfying ϕ _n (t)≤δ, the inequality t≤ε holds.

Assume the contrary. Then for each natural number i, there exists t _i ≥0 such that

$$ \phi_n(t_i) \le(4i)^{-1},\quad t_i>\varepsilon . $$

(8.214)

By (8.214) and (8.211), for each natural number i, there exists a point x _i ∈X such that

$$ \|x_i\|\le n, \quad\quad \varXi _f(x_i) \le-t_i <-\varepsilon , $$

(8.215)

and

$$ f^0(x_i,Vx_i ) \ge-(2i)^{-1}. $$

(8.216)

Now it follows from (8.215), (8.216) and the definition of regularity that

$$\lim_{i \to\infty}f(x_i)=f(\bar{x}). $$

Together with (P1) this implies that $\lim_{i \to\infty}\|x_{i}-\bar{x}\|=0$. When combined with (P3), this inequality implies that lim _i→∞ Ξ _f (x _i )=0. This, however, contradicts (8.215). The contradiction we have reached proves Theorem 8.19.

8.19 Proof of Theorem 8.20

Let n be a given natural number. We need to show that there exists δ>0 such that for each x∈X satisfying

$$ \|x\|\le n \quad\hbox{and}\quad \varXi _f(x)<-1/n, $$

(8.217)

we have

$$f^0(x,Vx) \le-\delta. $$

Assume the contrary. Then for each natural number k, there exists x _k ∈X such that

$$ \|x_k\| \le n,\quad\quad \varXi _f(x_k) \le-1/n, $$

(8.218)

and

$$f^0(x_k,Vx_k)>-1/k. $$

By (8.218) and (8.197),

$$-1/k<f^0(x_k,Vx_k) \le-\phi_n \bigl(-\varXi _f(x_k)\bigr) $$

and

$$ \phi\bigl(-\varXi _f(x_k)\bigr) \le1/k. $$

(8.219)

It now follows from (8.219) and property (P2) that

$$\limsup_{k \to\infty}\bigl(-\varXi _f(x_k) \bigr)=0 $$

and

$$\lim_{k \to\infty} \varXi _f(x_k)=0. $$

The last equality contradicts (8.218) and this contradiction proves Theorem 8.20.

8.20 Proof of Theorem 8.21

Set ϕ _n (0)=0 and let t>0. Define

$$ \phi_n(t)=\min\bigl\{ \inf\bigl\{ -f^0(x,Vx): x \in X, \|x\| \le n, \varXi _f(x) \le-t\bigr\} ,1\bigr\} . $$

(8.220)

Clearly, ϕ:[0,∞)→[0,1] is well defined and increasing.

We show that for each x∈X satisfying ∥x∥≤n,

$$ f^0(x,Vx) \le-\phi_n\bigl(- \varXi _f(x)\bigr). $$

(8.221)

Consider x∈X with

$$ \|x\|\le n. $$

(8.222)

If Ξ _f (x)=0, then (8.221) clearly holds. Assume that

$$ \varXi _f(x)<0. $$

(8.223)

Then by (8.220), (8.221), (8.222) and (8.223),

$$\begin{aligned} \phi_n\bigl(-\varXi _f(x)\bigr) =&\min \bigl\{ \inf\bigl\{ -f^0(y,Vy): y \in X, \|y\|\le n, \varXi _f(y) \le \varXi _f(x)\bigr\} ,1\bigr\} \\ \le&\min\bigl\{ 1,-f^0(x,Vx)\bigr\} \le-f^0(x,Vx) \end{aligned}$$

and hence (8.221) holds for all x∈X satisfying ∥x∥≤n, as claimed.

Now we show that property (P2) also holds. To this end, let ε be positive.

We claim that there is δ>0 such that for each t≥0 satisfying ϕ _n (t)≤δ, the inequality t≤ε holds.

Assume the contrary. Then for each natural number i, there exists t _i ≥0 such that

$$ \phi(t_i) \le(4i)^{-1},\quad\quad t_i> \varepsilon . $$

(8.224)

Let i be a natural number. By (8.224) and (8.220), there exists x _i ∈X such that

$$ \|x_i\|\le n,\quad\quad \varXi _f(x_i) \le-t_i <-\varepsilon , $$

(8.225)

and

$$-f^0(x_i,Vx_i) \le(2i)^{-1}. $$

Clearly,

$$ f^0(x_i,Vx_i) \ge-(2i)^{-1}. $$

(8.226)

Choose a natural number p such that

$$ p>n \quad\hbox{and}\quad 1/p<\varepsilon . $$

(8.227)

Since V is regular, there is δ>0 such that

$$ \hbox{if } x \in X, \|x\|\le p \hbox{ and } \varXi _f(x)<-1/p, \hbox{ then } f^0(x,Vx)<-\delta. $$

(8.228)

Choose a natural number j such that

$$ 1/j<\delta. $$

(8.229)

Then for all integers i≥j, it follows from (8.225) and (8.227) that

$$\varXi _f(x_i)<-\varepsilon <-1/p \quad\hbox{and}\quad \|x_i\| \le p. $$

Together with (8.228) and (8.229), this implies that for all integers i≥j,

$$f^0(x_i,Vx_i)<-\delta<-j^{-1}<-(i)^{-1}. $$

Since this contradicts (8.226), the proof of Theorem 8.21 is complete.

8.21 Most Continuous Descent Methods Converge

Let (X,∥⋅∥) be a Banach space and let f:X→R ¹ be a convex continuous function which satisfies the following conditions:

C(i):

lim_{∥x∥→∞} f(x)=∞;

C(ii):

there is $\bar{x} \in X$ such that $f(\bar{x}) \le f(x)$ for all x∈X;

C(iii):

if $\{x_{n}\}_{n=1}^{\infty} \subset X$ and $\lim_{n \to\infty} f(x_{n})=f(\bar{x})$, then

$$\lim_{n \to\infty}\|x_n-\bar{x} \|=0. $$

By C(iii), the point $\bar{x}$, where the minimum of f is attained, is unique.

For each x∈X, let

$$ f^0(x,u)=\lim_{t \to0^+} \bigl[f(x+tu)-f(x)\bigr]/t,\quad u \in X. $$

(8.230)

Let (X ^∗,∥⋅∥_∗) be the dual space of (X,∥⋅∥).

For each x∈X, let

$$\partial f(x)=\bigl\{ l \in X^*: f(y)-f(x) \ge l(y-x) \hbox{ for all } y \in X \bigr\} $$

be the subdifferential of f at x. It is well known that the set ∂f(x) is nonempty and norm-bounded.

For each x∈X and r>0, set

$$ B(x,r)=\bigl\{ z \in X: \|z-x\|\le r\bigr\} \quad\hbox{and}\quad B(r)=B(0,r). $$

(8.231)

For each mapping A:X→X and each r>0, put

$$ \operatorname{Lip}(A,r):=\sup\bigl\{ \|Ax-Ay\|/\|x-y\|: x,y \in B(r)\hbox{ and } x \not=y\bigr\} . $$

(8.232)

Denote by ${\mathcal{A}}_{l}$ the set of all mappings V:X→X such that $\operatorname{Lip}(V,r)<\infty$ for each positive r (this means that the restriction of V to any bounded subset of X is Lipschitzian) and f ⁰(x,Vx)≤0 for all x∈X.

For the set ${\mathcal{A}}_{l}$ we consider the uniformity determined by the base

$$\begin{aligned} E_s(n,\varepsilon ) =&\bigl\{ (V_1,V_2) \in{ \mathcal{A}}_l\times{ \mathcal{A}}_l: \operatorname{Lip}(V_1-V_2,n) \le \varepsilon \\ & \hbox{and } \|V_1x-V_2 x\| \le \varepsilon \hbox{ for all } x \in B(n)\bigr\} . \end{aligned}$$

(8.233)

Clearly, this uniform space ${\mathcal{A}}_{l}$ is metrizable and complete. The topology induced by this uniformity in ${\mathcal{A}}_{l}$ will be called the strong topology.

We also equip the space ${\mathcal{A}}_{l}$ with the uniformity determined by the base

$$\begin{aligned} E_w(n,\varepsilon ) =&\bigl\{ (V_1,V_2) \in{ \mathcal{A}}_l \times{ \mathcal{A}}_l: \|V_1x-V_2x \| \le \varepsilon \\ &\hbox{for all } x \in B(n)\bigr\} \end{aligned}$$

(8.234)

where n,ε>0. The topology induced by this uniformity will be called the weak topology.

The following existence result is proved in the next section.

Proposition 8.24

Let x ₀∈X and $V \in{ \mathcal{A}}_{l}$. Then there exists a unique continuously differentiable mapping x:[0,∞)→X such that

$$\begin{gathered} x'(t)=Vx(t),\quad t \in[0,\infty), \\ x(0)=x_0.\end{gathered} $$

In the subsequent sections we prove the following result which was obtained in [1].

Theorem 8.25

There exists a set ${\mathcal{F}} \subset {\mathcal{A}}_{l}$ which is a countable intersection of open (in the weak topology) everywhere dense (in the strong topology) subsets of ${\mathcal{A}}_{l}$ such that for each $V \in{ \mathcal{F}}$, the following property holds:

For each ε>0 and each n>0, there exist T _εn >0 and a neighborhood ${\mathcal{U}}$ of V in ${\mathcal{A}}_{l}$ with the weak topology such that for each $W \in{ \mathcal{U}}$ and each differentiable mapping y:[0,∞)→X satisfying

$$\bigl|f\bigl(y(0)\bigr)\bigr| \le n \quad\textit{and}\quad y'(t)=Wy(t)\quad\textit{for all}\ t \ge0, $$

the inequality $\|y(t)-\bar{x}\| \le \varepsilon$ holds for all t≥T _εn .

8.22 Proof of Proposition 8.24

Since V is locally Lipschitzian, there exists a unique differentiable function x:I→X, where I is an interval of the form [0,b), b>0, such that

$$ x(0)=x_0,\quad\quad x'(t)=Vx(t),\quad t \in I. $$

(8.235)

We may and will assume that I is the maximal interval of this form on which the solution exists.

We need to show that b=∞. Suppose, by contradiction, that b<∞.

By Proposition 8.13 and the relation $V \in{ \mathcal{A}}_{l}$, the function f(x(t)) is decreasing on I. By C(i), the set {x(t):t∈[0,b)} is bounded. Thus there is K ₀>0 such that

$$ \bigl\| x(t)\bigr\| \le K_0 \quad\hbox{for all } t \in[0,b). $$

(8.236)

Since V is Lipschitzian on bounded subsets of X, there is K ₁>0 such that

$$ \hbox{if } z \in X, \|z\| \le K_0,\hbox{ then } \|Vz \| \le K_1. $$

(8.237)

Let ε>0 be given. Then it follows from (8.235), (8.236) and (8.237) that for each t ₁,t ₂∈[0,b) such that 0<t ₂−t ₁<ε/K ₁,

$$\begin{aligned} \bigl\| x(t_2)-x(t_1)\bigr\| =&\biggl\| \int_{t_1}^{t_2}x'(t)\,dt \biggr\| =\biggl\| \int_{t_1}^{t_2}Vx(t)\,dt\biggr\| \\ \le&\int_{t_1}^{t_2}\bigl\| Vx(t)\bigr\| \,dt \le\int _{t_1}^{t_2}K_1 \,dt=K_1(t_2-t_1)< \varepsilon . \end{aligned}$$

Hence there exists $z_{0}=\lim_{t \to b^{-}}x(t)$ in the norm topology. It follows that there exists a unique solution of the initial value problem

$$z'(t)=Vz(t),\quad\quad z(b)=z_0, $$

defined on a neighborhood of b, and this implies that our solution x(⋅) can be extended to an open interval larger than I. The contradiction we have reached completes the proof of Proposition 8.24.

8.23 Proof of Theorem 8.25

For each $V \in{ \mathcal{A}}_{l}$ and each γ∈(0,1), set

$$ V_{\gamma}x=Vx+\gamma(\bar{x}-x),\quad x \in X. $$

(8.238)

We first prove several lemmata.

Lemma 8.26

Let $V \in{ \mathcal{A}}_{l}$ and γ∈(0,1). Then $V_{\gamma} \in{ \mathcal{A}}_{l}$.

Proof

Clearly, V _γ is Lipschitzian on any bounded subset of X. Let x∈X. Then by (8.238), the subadditivity and positive homogeneity of the directional derivative of a convex function, the relation $V \in{ \mathcal{A}}_{l}$, and C(ii),

$$\begin{aligned} f^0(x,V_{\gamma}x) =&f^0\bigl(x,Vx+\gamma(\bar{x}-x) \bigr)\le f^0(x,Vx)+\gamma f^0(x,\bar{x}-x) \\ \le&\gamma f^0(x,\bar{x}-x) \le\gamma\bigl(f(\bar{x})-f(x)\bigr) \le0. \end{aligned}$$

This completes the proof of Lemma 8.26. □

It is easy to see that the following lemma also holds.

Lemma 8.27

Let $V \in{ \mathcal{A}}_{l}$. Then $\lim_{\gamma \to0^{+}}V_{\gamma}=V$ in the strong topology.

Lemma 8.28

Let $V \in{ \mathcal{A}}_{l}$, γ∈(0,1), ε>0, and let x∈X satisfy $f(x) \ge f(\bar{x})+\varepsilon$. Then f ⁰(x,V _γ x)≤−γε.

Proof

It follows from (8.238), the properties of the directional derivative of a convex function, and the relation $V \in {\mathcal{A}}_{l}$ that

$$\begin{aligned} f^0(x, V_{\gamma}x) =&f^0\bigl(x,Vx+\gamma(\bar{x}-x)\bigr) \le f^0(x,Vx)+\gamma f^0(x,\bar{x}-x) \\ \le&\gamma f^0(x,\bar{x}-x) \le\gamma\bigl(f(\bar{x})-f(x)\bigr) \le- \varepsilon \gamma. \end{aligned}$$

The lemma is proved. □

Lemma 8.29

Let $V \in{ \mathcal{A}}_{l}$, γ∈(0,1), and let x∈C ¹([0,∞);X) satisfy

$$ x'(t)=V_{\gamma}x(t),\quad t \in[0,\infty). $$

(8.239)

Assume that T ₀,ε>0 are such that

$$ T_0>\bigl(f\bigl(x(0)\bigr)-f(\bar{x})\bigr) (\gamma \varepsilon )^{-1}. $$

(8.240)

Then for each t≥T ₀, $f(x(t)) \le f(\bar{x})+\varepsilon$.

Proof

Since the function f(x(⋅)) is decreasing on [0,∞) (see Proposition 8.13, Lemma 8.26 and (8.239)), it is sufficient to show that

$$ f\bigl(x(T_0)\bigr) \le f(\bar{x})+\varepsilon . $$

(8.241)

Assume the contrary. Then $f(x(T_{0}))>f(\bar{x})+\varepsilon$, and since f(x(⋅)) is decreasing on [0,∞), we have

$$ f\bigl(x(t)\bigr)>f(\bar{x})+\varepsilon \quad\hbox{for all } t \in[0,T_0]. $$

(8.242)

When combined with Lemma 8.28, inequality (8.242) implies that

$$ f^0(x(t),V_{\gamma}\bigl(x(t)\bigr) \le-\gamma \varepsilon \quad\hbox{for all } t \in [0,T_0]. $$

(8.243)

It now follows from Proposition 8.13, (8.239) and (8.243) that

$$\begin{aligned} f\bigl(x(T_0)\bigr)-f\bigl(x(0)\bigr) =&\int_0^{T_0}(f \circ x)'(t)\,dt=\int_0^{T_0}f^0 \bigl(x(t),x'(t)\bigr)\,dt \\ =&\int_0^{T_0}f^0\bigl(x(t),V_{\gamma}x(t) \bigr)\,dt \le T_0(-\gamma \varepsilon ), \end{aligned}$$

whence

$$T_0\gamma \varepsilon \le f\bigl(x(0)\bigr)-f\bigl(x(T_0) \bigr) < f\bigl(x(0)\bigr)-f(\bar{x}). $$

This contradicts (8.240). The contradiction we have reached proves the lemma. □

Lemma 8.30

Let $V \in{ \mathcal{A}}_{l}$, γ∈(0,1), ε>0 and n>0. Then there exist a neighborhood $\mathcal{U}$ of V _γ in ${\mathcal{A}}_{l}$ with the weak topology and τ>0 such that for each $W \in{ \mathcal{U}}$ and each continuously differentiable mapping x:[0,∞)→X satisfying

$$ x'(t)=Wx(t),\quad t \in[0,\infty), $$

(8.244)

and

$$ \bigl|f\bigl(x(0)\bigr)\bigr| \le n, $$

(8.245)

the following inequality holds:

$$ \bigl\| x(t)-\bar{x}\bigr\| \le \varepsilon \quad\textit{for all}\ t \ge\tau. $$

(8.246)

Proof

By C(i), there is n ₁>n such that

$$ \hbox{if } z \in X, f(z)\le n, \hbox{ then } \|z\| \le n_1. $$

(8.247)

By C(iii), there is δ ₁>0 such that

$$ \hbox{if } z \in X \hbox{ and } f(z) \le f(\bar{x})+ \delta_1, \hbox{ then } \|z-\bar{x}\| \le \varepsilon . $$

(8.248)

Since f is continuous, there is ε ₁>0 such that

$$ \bigl|f(\bar{x})-f(z)\bigr| \le\delta_1\quad \hbox{for each } z \in X \hbox{ satisfying } \|z-\bar{x}\| \le \varepsilon _1. $$

(8.249)

In view of C(iii), there exists δ ₀∈(0,1) such that

$$ \hbox{if } z \in X \hbox{ and } f(z) \le f(\bar{x})+ \delta_0, \hbox{ then } \|z-\bar{x}\| \le \varepsilon _1/4. $$

(8.250)

Since $V_{\gamma} \in{ \mathcal{A}}_{l}$, there is L>0 such that

$$ \|V_{\gamma}z_1-V_{\gamma}z_2 \|\le L\|z_1-z_2\| \quad\hbox{for all } z_1,z_2 \in B(n_1). $$

(8.251)

Fix

$$ \tau>\bigl(n-f(\bar{x})+1\bigr) (\gamma\delta_0)^{-1}+1 $$

(8.252)

and choose a positive number Δ such that

$$ \varDelta \tau e^{L\tau} \le \varepsilon _1 /4. $$

(8.253)

Set

$$ {\mathcal{U}}=\bigl\{ W \in{ \mathcal{A}}_l: \|Wz-V_{\gamma}z\| \le \varDelta \hbox{ for all }z \in B(n_1)\bigr\} . $$

(8.254)

Assume that

$$ W \in{ \mathcal{U}} $$

(8.255)

and that x∈C ¹([0,∞);X) satisfies (8.244) and (8.245). We have to prove (8.246). In view of (8.248), it is sufficient to show that

$$f\bigl(x(t)\bigr) \le f(\bar{x})+\delta_1 \quad\hbox{for all } t \ge\tau. $$

Since the function f(x(⋅)) is decreasing on [0,∞), in order to prove the lemma we only need to show that

$$f\bigl(x(\tau)\bigr) \le f(\bar{x})+\delta_1. $$

By (8.249), this inequality will follow from the inequality

$$ \bigl\| x(\tau)-\bar{x}\bigr\| \le \varepsilon _1. $$

(8.256)

We now prove (8.256).

To this end, consider a continuously differentiable mapping y:[0,∞)→X which satisfies

$$ y'(t)=V_{\gamma}y(t),\quad t \in[0,\infty), $$

(8.257)

and

$$ y(0)=x(0). $$

(8.258)

Since the functions f(x(⋅)) and f(y(⋅)) are decreasing on [0,∞), we obtain by (8.258) and (8.245) that for each s≥0,

$$f\bigl(x(s)\bigr),f\bigl(y(s)\bigr) \le f\bigl(x(0)\bigr) \le n. $$

When combined with (8.247), this inequality implies that

$$ \bigl\| x(s)\bigr\| ,\bigl\| y(s)\bigr\| \le n_1 \quad\hbox{for all } s \ge0. $$

(8.259)

It follows from Lemma 8.29 (with x=y, ε=δ ₀), (8.258), (8.257), (8.252) and (8.245) that

$$f\bigl(y(\tau)\bigr) \le f(\bar{x})+\delta_0. $$

This inequality and (8.250) imply that

$$ \bigl\| y(\tau)-\bar{x}\bigr\| \le \varepsilon _1/4. $$

(8.260)

Now we estimate ∥x(τ)−y(τ)∥. It follows from (8.257), (8.244) and (8.258) that for each s∈[0,τ],

$$\begin{aligned} \bigl\| y(s)-x(s)\bigr\| =&\biggl\| y(0)+\int_0^sV_{\gamma}y(t)\,dt- \biggl(x(0)+\int_0^sWx(t)\,dt\biggr)\biggr\| \\ =&\biggl\| \int_0^s\bigl(V_{\gamma}y(t)-Wx(t) \bigr)\,dt\biggr\| \le \int_0^s\bigl\| V_{\gamma}y(t)-Wx(t) \bigr\| \,dt \\ \le&\int_0^s\bigl\| V_{\gamma}y(t)-V_{\gamma}x(t) \bigr\| \,dt+\int_0^s\bigl\| V_{\gamma }x(t)-Wx(t)\bigr\| \,dt. \end{aligned}$$

(8.261)

By (8.259) and (8.254), for each s∈(0,τ], we have

$$ \int_0^s\bigl\| V_{\gamma}x(t)-Wx(t) \bigr\| \,dt \le\int_0^s\varDelta \,dt \le \varDelta s \le \varDelta \tau. $$

(8.262)

By (8.259) and (8.251), for each s∈[0,τ],

$$ \int_0^s\bigl\| V_{\gamma}y(t)-V_{\gamma}x(t) \bigr\| \,dt \le \int_0^sL\bigl\| y(t)-x(t)\bigr\| \,dt. $$

(8.263)

It follows from (8.261), (8.262) and (8.263) that for each s∈[0,τ],

$$ \bigl\| y(s)-x(s)\bigr\| \le \varDelta \tau+\int_0^sL \bigl\| y(t)-x(t)\bigr\| \,dt. $$

(8.264)

Applying Gronwall’s inequality, we obtain that

$$\bigl\| y(\tau)-x(\tau)\bigr\| \le \varDelta \tau e^{\int_0^{\tau}L\,dt} =\varDelta \tau e^{L\tau}. $$

When combined with (8.253), this inequality implies that

$$\bigl\| y(\tau)-x(\tau)\bigr\| \le \varepsilon _1 /4. $$

Together with (8.260), this implies that $\|x(\tau)-\bar{x}\| \le \varepsilon _{1}/2$. Lemma 8.30 is proved. □

Completion of the proof of Theorem 8.25

Let $V \in{ \mathcal{A}}_{\gamma}$, γ∈(0,1), and let i be a natural number. By Lemma 8.30, there exist an open neighborhood ${\mathcal{U}}(V,\gamma,i)$ of V _γ in ${\mathcal{A}}_{l}$ with the weak topology and a positive number τ(V,γ,i) such that the following property holds:

(P):

For each $W \in{ \mathcal{U}}(V,\gamma,i)$ and each continuously differentiable mapping x:[0,∞)→X satisfying

$$\begin{gathered} x'(t)=Wx(t), \quad t \in[0,\infty), \\ \bigl|f\bigl(x(0)\bigr)\bigr| \le i,\end{gathered} $$

the following inequality holds:

$$\bigl\| x(t)-\bar{x}\bigr\| \le i^{-1} \quad\hbox{for all } t \ge\tau(V,\gamma,i). $$

Set

$$ {\mathcal{F}}:=\bigcap_{i=1}^{\infty} \bigcup\bigl\{ {\mathcal{U}}(V,\gamma,i): V \in {\mathcal{A}}_l, \gamma\in(0,1)\bigr\} . $$

(8.265)

By Lemma 8.27, ${\mathcal{F}}$ is a countable intersection of open (in the weak topology) everywhere dense (in the strong topology) subsets of ${\mathcal{A}}_{l}$.

Let $\tilde{V} \in{ \mathcal{F}}$ and let n,ε>0 be given. Choose a natural number i such that

$$ i >n,\quad\quad i >\varepsilon ^{-1}. $$

(8.266)

By (8.265), there are $V \in{ \mathcal{A}}_{l}$ and γ∈(0,1) such that

$$ \tilde{V} \in{ \mathcal{U}}(V,\gamma,i). $$

(8.267)

We claim show that the assertion of Theorem 8.15 holds with ${\mathcal{U}}={\mathcal{U}}(V,\gamma,i)$ and T _εn =τ(V,γ,i).

Assume that $W \in{ \mathcal{U}}(V,\gamma,i)$ and that the continuously differentiable mapping y:[0,∞)→X satisfies

$$ \bigl|f\bigl(y(0)\bigr)\bigr| \le n,\quad\quad y'(t)=Wy(t) \quad\hbox{for all } t \ge0. $$

(8.268)

Then by (8.268), (8.266) and property (P), it follows that

$$\bigl\| y(t)-\bar{x}\bigr\| \le i^{-1} \quad\hbox{for all } t \ge\tau(V,\gamma,i). $$

When combined with (8.266), this inequality implies that $\|y(t)-\bar{x}\| \le \varepsilon$ for all t≥τ(V,γ,i). Theorem 8.25 is established. □