1 Introduction

This note discusses methods for the global solution of semi-infinite programs (SIP). Specifically, the method from [4] is considered, and it is shown with a counterexample that the lower bounds do not always converge. Throughout we use notation as close as possible to that used in [4], embellishing it only as necessary with, for instance, iteration counters.

Consider a SIP in the general form

$$\begin{aligned} f^* = \inf _{x}\;&f(x) \\ \mathrm {s.t.}\;\nonumber&x \in X, \\ \nonumber&g(x,y) \le 0,\; \forall y \in Y, \end{aligned}$$
(SIP)

for subsets X, Y of finite dimensional real vector spaces and \(f : X \rightarrow \mathbb {R}\), \(g : X \times Y \rightarrow \mathbb {R}\). We may view Y as an index set, with potentially uncountably infinite cardinality. Important to validating the feasibility of a point x is the lower-level program:

$$\begin{aligned} \sup _y \left\{ g(x,y) : y \in Y \right\} . \end{aligned}$$
(LLP)

Global solution of (SIP) often involves the construction of convergent upper and lower bounds. The approach in [4] to obtain a lower bound is a modification of the constraint-generation/discretization method of [2]. The claim is that the lower-level program may be solved approximately; the exact nature of the approximation is important to the convergence of the lower bounds and this is the subject of the present note.

2 Sketch of the lower bounding procedure and claim

The setting of the method is the following. The method is iterative and at iteration k, for a given finite subset \(Y^{LBD,k} \subset Y\), a lower bound of \(f^*\) is obtained from the finite program

$$\begin{aligned} f^{LBD,k}&= \inf _{x}\; f(x) \\ \mathrm {s.t.}\;\nonumber&x \in X, \\ \nonumber&g(x,y) \le 0, \;\forall y \in Y^{LBD,k}. \end{aligned}$$
(1)

This is indeed a lower bound since fewer constraints are enforced, and thus (1) is a relaxation of (SIP). Assume that the lower bounding problem (1) is feasible (otherwise we can conclude that (SIP) is infeasible). Let \(\bar{x}^k\) be a (global) minimizer of the lower bounding problem (1). In [4], Lemma 2.2 supposes that we either verify \(\sup _y \left\{ g(\bar{x}^k,y) : y \in Y \right\} \le 0\), or else find \(\bar{y}^k \in Y\) such that \(g(\bar{x}^k,\bar{y}^k) > 0\). If \(\sup _y \left\{ g(\bar{x}^k,y) : y \in Y \right\} \le 0\), then \(\bar{x}^k\) is feasible in (SIP) and thus optimal (since it also solves a relaxation). Otherwise, set \(Y^{LBD,k+1} = Y^{LBD,k} \cup \left\{ \bar{y}^k \right\} \) and we iterate.

In the case that we can find \(\bar{y}^k \in Y\) such that \(g(\bar{x}^k,\bar{y}^k) > 0\), the point \(\bar{x}^k\) is infeasible in (SIP), and \(\bar{y}^k\) yields a constraint violation that makes \(\bar{x}^k\) infeasible in the following iteration of (1). While it is proven in [4] that the sequence of \(g(\bar{x}^k,\bar{y}^k)\) converges to zero, it is not shown that \(\sup _y \left\{ g(\bar{x}^k, y) : y \in Y \right\} \) converges to zero. This is critical as \(\sup _y \left\{ g(x,y) : y \in Y \right\} \le 0\) is the definition of a feasible point x. The counterexample exploits this by constructing a sequence of solutions for which the sequence of constraint violations \(g(\bar{x}^k,\bar{y}^k)\) converges to zero while \(\sup _y \left\{ g(\bar{x}^k, y) : y \in Y \right\} \) converges to a strictly positive value.

The precise statement of the claim is repeated here (again, with only minor embellishments to the notation to help keep track of iterations).

Lemma 2.1

(Lemma 2.2 in [4]) Take any \(Y^{LBD,0} \subset Y\). Assume that X and Y are compact and that g is continuous on \(X \times Y\). Suppose that at each iteration of the lower bounding procedure the lower-level program is solved approximately for the solution of the lower bounding problem \(\bar{x}^k\) either establishing \(\max _{y \in Y} g(\bar{x}^k,y) \le 0\), or furnishing a point \(\bar{y}^k\) such that \(g(\bar{x}^k,\bar{y}^k) > 0\). Then, the lower bounding procedure converges to the optimal objective value, i.e. \(f^{LBD,k} \rightarrow f^*\).

3 Correction

3.1 Counterexample

We now present a counterexample to the claim in Lemma 2.1. Consider

$$\begin{aligned} \inf _{x}\;&-x \\ \mathrm {s.t.}\;\nonumber&x \in [-1,1], \\ \nonumber&2x - y \le 0,\; \forall y \in [-1,1], \end{aligned}$$
(CEx)

thus we define \(X = Y = [-1,1]\), \(f : x \mapsto -x\), \(g : (x,y) \mapsto 2x - y\). The behavior to note is this: We are trying to maximize x; The feasible set is

$$\begin{aligned} \left\{ x \in [-1,1] : x \le \left( \frac{1}{2}\right) y, \forall y \in [-1,1] \right\} = \left[ -1,-\frac{1}{2}\right] ; \end{aligned}$$

The infimum, consequently, is \(\frac{1}{2}\). See Fig. 1.

Fig. 1
figure 1

Visualization of counterexample (CEx). The box represents \([-1,1] \times [-1,1]\). The shaded grey area is the subset of (x, y) such that \(2x - y > 0\). The dashed line represents the approximate minimizers used in the counterexample

Full size image

Beginning with \(Y^{LBD,1} = \emptyset \), the minimizer of the lower bounding problem is \(\bar{x}^1 = 1\). Now, assume that solving the resulting (LLP) approximately, we get \(\bar{y}^1 = 1\) which we note satisfies

$$\begin{aligned} 2\bar{x}^1 - \bar{y}^1 = 1 > 0 \end{aligned}$$

as required by Lemma 2.1.

The next iteration, with \(Y^{LBD,2} = \left\{ 1 \right\} \), adds the constraint \(2x - 1 \le 0\) to the lower bounding problem; the feasible set is \([-1,\frac{1}{2}]\) so the minimizer is \(\bar{x}^2 = \frac{1}{2}\). Again, assume that solving the lower-level program approximately yields \(\bar{y}^2 = \frac{1}{2}\); again we get

$$\begin{aligned} 2\bar{x}^2 - \bar{y}^2 = \frac{1}{2} > 0 \end{aligned}$$

as required by Lemma 2.1.

The third iteration, with \(Y^{LBD,3} = \left\{ 1, \frac{1}{2} \right\} \), adds the constraint \(2x - \frac{1}{2} \le 0\) to the lower bounding problem; the feasible set is \([-1,\frac{1}{4}]\) so the minimizer is \(\bar{x}^3 = \frac{1}{4}\). Again, assume that solving the lower-level program approximately yields \(\bar{y}^3 = \frac{1}{4}\); again we get

$$\begin{aligned} 2\bar{x}^3 - \bar{y}^3 = \frac{1}{4} > 0 \end{aligned}$$

as required by Lemma 2.1.

Proceeding in this way, we construct \(\bar{x}^k\) and \(\bar{y}^k\) so that \(g(\bar{x}^k,\bar{y}^k) > 0\) and the lower bounds satisfy \(f^{LBD,k} = -\bar{x}^k = -\frac{1}{2^{k-1}}\), for all k. Consequently, they converge to 0, which we note is strictly less than the infimum of \(\frac{1}{2}\).

3.2 Modified claim

We now present a modification of the claim in order to demonstrate what kind of approximate solution of the lower-level program suffices to establish convergence of the lower bounds. To state the result, let the optimal objective value of (LLP) as a function of x be

$$\begin{aligned} g^*(x) = \sup _y\left\{ g(x,y) : y \in Y \right\} . \end{aligned}$$

The proof of the following result has a similar structure to the original proof of [4, Lemma 2.2].

Lemma 3.1

Choose any finite \(Y^{LBD,0} \subset Y\), and \(\alpha \in (0,1)\). Assume that X and Y are compact and that f and g are continuous. Suppose that at each iteration k of the lower bounding procedure (LLP) is solved approximately for the solution \(\bar{x}^k\) of the lower bounding problem (1), either establishing that \(g^*(\bar{x}^k) \le 0\) or furnishing a point \(\bar{y}^k\) such that

$$\begin{aligned} g(\bar{x}^k,\bar{y}^k) \ge \alpha g^*(\bar{x}^k) > 0. \end{aligned}$$

Then, the lower bounding procedure converges to the optimal objective value, i.e. \(f^{LBD,k} \rightarrow f^*\).

Proof

First, if the lower bounding problem (1) is ever infeasible for some iteration k, then (SIP) is infeasible and we can set \(f^{LBD,k} = +\infty = f^*\). Otherwise, since X is compact, \(Y^{LBD,k}\) is finite, and f and g are continuous, for every iteration the lower bounding problem has a solution by Weierstrass’ (extreme value) theorem. If at some iteration k the lower bounding problem furnishes a point \(\bar{x}^k\) for which \(g^*(\bar{x}^k) \le 0\), then \(\bar{x}^k\) is feasible for (SIP), and thus optimal. The corresponding lower bound \(f^{LBD,k}\), and all subsequent lower bounds, equal \(f^*\).

Otherwise, we have an infinite sequence of solutions to the lower bounding problems. Since X is compact we can move to a subsequence \(\left( \bar{x}^{k} \right) _{k \in \mathbb {N}} \subset X\) which converges to \(x^* \in X\). By construction of the lower bounding problem we have

$$\begin{aligned} g(\bar{x}^{\ell },\bar{y}^k) \le 0, \quad \forall \ell ,k : \ell > k. \end{aligned}$$

By continuity and compactness of \(X \times Y\) we have uniform continuity of g, and so for any \(\epsilon > 0\), there exists a \(\delta > 0\) such that

$$\begin{aligned} g(x,\bar{y}^k)< \epsilon , \quad \forall x : \left\| x - \bar{x}^{\ell } \right\| < \delta , \quad \forall \ell ,k : \ell > k. \end{aligned}$$
(2)

Since the (sub)sequence \(\left( \bar{x}^k \right) _{k \in \mathbb {N}}\) converges, there is an index K sufficiently large that

$$\begin{aligned} \left\| \bar{x}^{\ell } - \bar{x}^k \right\| < \delta , \quad \forall \ell ,k : \ell > k \ge K. \end{aligned}$$
(3)

Using (3), we can substitute \(x = \bar{x}^k\) in (2) to get that for any \(\epsilon > 0\), there exists K such that

$$\begin{aligned} g(\bar{x}^k,\bar{y}^k) < \epsilon , \quad \forall k \ge K. \end{aligned}$$

By assumption \(g(\bar{x}^k,\bar{y}^k) > 0\) for all k, and so combined with the above we have that \(g(\bar{x}^k,\bar{y}^k) \rightarrow 0\).

Combining \(g(\bar{x}^k,\bar{y}^k) \rightarrow 0\) with \( g(\bar{x}^k,\bar{y}^k) \ge \alpha g^*(\bar{x}^k) > 0, \) for all k, we see \( g^*(\bar{x}^k) \rightarrow 0. \) Meanwhile \(g^* : X \rightarrow \mathbb {R}\) is a continuous function, by classic parametric optimization results like [1, Theorem 1.4.16] (using continuity of g and compactness of Y)Footnote 1. Thus

$$\begin{aligned} g^*(x^*) = \lim _{k \rightarrow \infty } g^*(\bar{x}^k) = 0. \end{aligned}$$

Thus \(x^*\) is feasible in (SIP) and so \(f^* \le f(x^*)\). But since the lower bounding problem is a relaxation, \(f^{LBD,k} = f(\bar{x}^k) \le f^*\) for all k, and so by continuity of f, \(f(x^*) \le f^*\). Combining these inequalities we see \(f^{LBD,k} \rightarrow f(x^*) = f^*\). Since the entire sequence of lower bounds is a nondecreasing sequence, we see that the entire sequence converges to \(f^*\) (without moving to a subsequence). \(\square \)

4 Remarks

The main contribution of [4] is a novel upper bounding procedure, which still stands, and combined with the modified lower bounding procedure from Lemma 3.1 or the original procedure from [2], the overall global solution method for (SIP) is still effective. Furthermore, the corrections made here likely do not influence how methods for (SIP) are followed in practice; a relative optimality tolerance akin to the condition required in Lemma 3.1 is typically used anyway.

The counterexample that has been presented may seem contrived. However, as the lower bounding method for SIP from [4] is adapted to give a lower bounding method for generalized semi-infinite programs (GSIP) in [5], a modification of the counterexample reveals that similar behavior may occur (and in a more natural way) when constructing the lower bounds for a GSIP. Consequently, the lower bounds fail to converge to the infimum. See [3].