1 Introduction

As discussed in [3–6], the two most central open questions in monotone operator theory are probably in a general real Banach space whether (1) the sum of two maximally monotone operators \(A+B\) is maximally monotone under Rockafellar’s constraint qualification \(\mathrm{dom}A \cap \mathrm{int}\mathrm{dom}B\ne \emptyset \) [12]; and whether (2) \( \overline{\mathrm{dom}A}\) is always convex. Rockafellar showed that \(\overline{\mathrm{dom}A}\) is convex if \(\mathrm{int}\mathrm{dom}A\ne \varnothing \) [11].

Since a positive answer to various restricted versions of (1) implies a positive answer to (2) [6, 16], we have determined to make what progress we may with the later question.

1.1 Preliminary matters

Throughout this note, we assume that \(X\) is a general real Banach space with norm \(\Vert \cdot \Vert \), that \(X^*\) is its continuous dual space, and that \(\langle {{\cdot },{\cdot }}\rangle \) denotes the usual pairing between these spaces. Let \(A:X\rightrightarrows X^*\) be a set-valued operator (also known as multifunction) from \(X\) to \(X^*\), i.e., for every \(x\in X, Ax\subseteq X^*\), and let \(\mathrm{gra}A = \big \{{(x,x^*)\in X\times X^*} \mid {x^*\in Ax}\big \}\) be the graph of \(A\). Then \(A\) is said to be monotone if

$$\begin{aligned} \big (\forall (x,x^*)\in \mathrm{gra}A\big )\big (\forall (y,y^*)\in \mathrm{gra}A\big ) \quad \langle {{x-y},{x^*-y^*}}\rangle \ge 0, \end{aligned}$$
(1)

and maximally monotone if no proper enlargement (in the sense of graph inclusion) of \(A\) is monotone. We say \((x,x^*)\) is monotonically related to \(\mathrm{gra}A\) if

$$\begin{aligned} \langle x-y,x^*-y^*\rangle \ge 0,\quad \forall (y,y^*)\in \mathrm{gra}A. \end{aligned}$$

We say \(A\) is a linear relation if \(\mathrm{gra}A\) is a linear subspace. Let \(A:X\rightrightarrows X^*\) be such that \(\mathrm{gra}A\ne \varnothing \). The Fitzpatrick function associated with \(A\) [8] is defined by

$$\begin{aligned} F_A:X\times X^*\rightarrow \,\left]-\infty ,+\infty \right]:(x,x^*)\mapsto \sup _{(a,a^*)\in \mathrm{gra}A} \big (\langle {{x},{a^*}}\rangle +\langle {{a},{x^*}}\rangle -\langle {{a},{a^*}}\rangle \big ). \end{aligned}$$

Monotone operators have proven to be a key class of objects in modern Optimization and Analysis; see, e.g., the books [2, 6, 7, 10, 13, 14, 16, 26–28] and the references therein.

Throughout, we adopt standard convex analysis notation. Given a subset \(C\) of \(X, \mathrm{int}C\) is the interior of \(C, \overline{C}\) is the norm closure of \(C\), and \(\mathrm{conv}{C}\) is the convex hull of \(C\). The indicator function of \(C\), written as \(\iota _C\), is defined at \(x\in X\) by

$$\begin{aligned} \iota _C (x)={\left\{ \begin{array}{ll} 0,\,&\text{ if}\,\,x\in C;\\ +\infty ,\,&\text{ otherwise}. \end{array}\right.} \end{aligned}$$
(2)

The distance function to the set \(C\), written as \(\text{ dist}(\cdot , C)\), is defined by \(x\mapsto \inf _{c\in C}\Vert x-c\Vert \). For every \(x\in X\), the normal cone operator of \(C\) at \(x\) is defined by \(N_C(x)= \big \{{x^*\in X^*} \mid {\sup _{c\in C}\langle {{c-x},{x^*}}\rangle \le 0}\big \}\), if \(x\in C\); and \(N_C(x):=\varnothing \), if \(x\notin C\).

We also say a set \(C\) is nearly convex if \(\overline{C}\) is convex.

Let \(f:X\rightarrow \,\left]-\infty ,+\infty \right]\). Then \(\mathrm{dom}f= f^{-1}(\mathbb R )\) is the domain of \(f\), and \(f^*:X^*\rightarrow \,\left[-\infty ,+\infty \right]:x^*\mapsto \sup _{x\in X}(\langle {{x},{x^*}}\rangle -f(x))\) is the Fenchel conjugate of \(f\). We say \(f\) is proper if \(\mathrm{dom}f\ne \varnothing \). Let \(f\) be proper. The subdifferential of \(f\) is defined by

$$\begin{aligned} \partial f:X\rightrightarrows X^*:x\mapsto \{x^*\in X^*\mid (\forall y\in X)\; \langle {{y-x},{x^*}}\rangle + f(x)\le f(y)\}. \end{aligned}$$

Let \(Y\) be another real Banach space. We denote \(P_X: X\times Y\rightarrow X:(x,y)\mapsto x\) and \(\mathbb N :=\{1,2,3,\ldots \}\).

The paper is organized as follows. In Sect. 2, we collect auxiliary results for future reference and for the reader’s convenience.

In Sect. 3, we first present an efficient sufficient condition for the closure of the domain of a maximally monotone operator \(A\) to be convex in Theorem 3.3. To the best of our knowledge, this recaptures and extends all known general conditions for \(\mathrm{dom}A\) to be nearly convex. We then turn to our main result (Theorem 3.6) which provides an affirmative answer to a problem posed by Simons in [16, Problem 28.3, page 112].

Let \(A:X\rightrightarrows X^*\) be maximally monotone. Is it necessarily true that

$$\begin{aligned} \overline{P_X\left[\mathrm{dom}F_A\right]}=\overline{\mathrm{conv}\left[\mathrm{dom}A\right]} ? \end{aligned}$$

Simons [15], showed the answer is yes under the constraint that \(\mathrm{int}\mathrm{dom}A\ne \varnothing \). (This is also an immediate consequence of Corollary 3.4 below.)

2 Auxiliary results

The basic building blocks we appeal to are as follows. We first record Fitzpatrick’s result connecting \(A\) to the corresponding Fitzpatrick function.

Fact 2.1

(Fitzpatrick) (See [8, Proposition 3.2, Theorem 3.4 and Corollary 3.9].) Let \(A:X\rightrightarrows X^*\) be monotone with \(\mathrm{dom} A\ne \varnothing \). Then \(F_A\) is proper lower semicontinuous, convex and \(F_A=\langle \cdot ,\cdot \rangle \) on \(\mathrm{gra} A\). Moreover, if \(A\) is maximally monotone, for every \((x,x^*)\in X\times X^*\), the inequality

$$\begin{aligned} \langle {{x},{x^*}}\rangle \le F_A(x,x^*) \end{aligned}$$

is true, and equality holds if and only if \((x,x^*)\in \mathrm{gra} A\).

We turn to two technical results of Simons that we shall exploit.

Fact 2.2

(Simons) (See [16, Lemma 19.18].) Let \(A:X\rightrightarrows X^*\) be monotone with \(\mathrm{gra} A\ne \varnothing \) and \((z,z^*)\in \mathrm{dom} F_A\). Let \(F:\mathrm{gra} A\rightarrow \mathbb R \) be such that \(\inf F>0\). Then there exists \(r\in \mathbb R \) such that

$$\begin{aligned} \frac{\langle z-a,z^*-a^*\rangle }{F(a,a^*)}\ge r,\quad \forall (a,a^*)\in \mathrm{gra} A. \end{aligned}$$

Fact 2.3

(Simons) (See [16], Thereom 27.5].) Let \(A:X\rightrightarrows X^*\) be monotone. Assume the implication that

$$\begin{aligned} z\notin \overline{\mathrm{dom}A}\quad \Rightarrow \quad \sup _{(a,a^*)\in \mathrm{gra}A} \frac{\langle z-a, a^*\rangle }{\Vert z-a\Vert }=+\infty \end{aligned}$$

holds. Then

$$\begin{aligned} \overline{\mathrm{dom}A}=\overline{\mathrm{conv}\left[\mathrm{dom}A\right]}=\overline{P_X\left[\mathrm{dom}F_A\right]}, \end{aligned}$$

and hence \(\mathrm{dom}A\) is nearly convex: \(\overline{\mathrm{dom}A}\) is convex, since \(P_X\left[\mathrm{dom}F_A\right]\) is the projection of a convex set.

Let us recall two definitions:

Definition 2.4

((FPV) and (BR)) Let \(A:X\rightrightarrows X^*\) be maximally monotone. We say \(A\) is type Fitzpatrick-Phelps-Veronas (FPV) if for every open convex set \(U\subseteq X\) such that \(U\cap \mathrm{dom}A\ne \varnothing \), the implication

$$\begin{aligned} x\in U\,\text{ and}\,(x,x^*)\,\text{ is} \text{ monotonically} \text{ related} \text{ to}\,\, \mathrm{gra}A\cap (U\times X^*) \Rightarrow (x,x^*)\in \mathrm{gra}A \end{aligned}$$

holds.

We say \(A\) is of of “Brønsted-Rockafellar” (BR) type [17] if whenever \((x,x^*)\in X\times X^*, \alpha ,\beta >0\) while

$$\begin{aligned} \inf _{(a,a^*)\in \mathrm{gra}A} \langle x-a,x^*-a^*\rangle >-\alpha \beta \end{aligned}$$

then there exists \((b,b^*)\in \mathrm{gra}A\) such that \(\Vert x-b\Vert <\alpha ,\Vert x^*-b^*\Vert <\beta \).

Note that if \(\le \) is used throughout we get a nominally slightly stronger property [19]. We also remark that unlike most properties studied for monotone operators (BR) is an isometric property not a priori preserved under Banach space isomorphism [1].

Likewise in [18, 19] a notion of Verona regularity of a maximally monotone operator \(A\) is introduced and studied. In [21, Theorems 1&2] it is shown that \(A\) is Verona regular, if and only if, \(A+M\) is maximally monotone whenever \(M\) is maximally monotone and has bounded range, if and only if, \(A+ \lambda \partial \Vert \cdot -x\Vert \) is maximally monotone for all \(\lambda >0, x \in X\).

We finish this section with a result on linear mappings.

Fact 2.5

(See [25, Corollary 3.3].) Let \(A:X\rightrightarrows X^*\) be a maximally monotone linear relation, and \(f:X\rightarrow \,\left]-\infty ,+\infty \right]\) be a proper lower semicontinuous convex function with \(\mathrm{dom}A\cap \mathrm{int}\mathrm{dom}\partial f\ne \varnothing \). Then \(A+\partial f\) is of type \((FPV)\). In particular \(A\) is of type \((FPV)\).

 

3 Domain near convexity

A useful general criterion for (4) to hold is the following result abstracted from [19]. Let us denote \(J_p := \frac{1}{p} \partial \Vert \cdot \Vert ^p.\) Thus, \(J_2=J\) is the classical duality map, while \(J_1\) is a maximally monotone operator with bounded range.

Proposition 3.1

Let \( 1 \le p < \infty \) be given. Suppose that \(A\) is monotone and that for all \(z\notin \overline{\mathrm{dom}A}\) and all sufficiently large \(\lambda >0\)

$$\begin{aligned} (z,0)\,~ \text{ is} \text{ not} \text{ monotonically} \text{ related} \text{ to} \mathrm{gra}\big (A+ \lambda J_p(\cdot -z)\big ). \end{aligned}$$
(3)

Then

$$\begin{aligned} \overline{\mathrm{dom}A}=\overline{\mathrm{conv}\left[\mathrm{dom}A\right]}=\overline{P_X\left[\mathrm{dom}F_A\right]}, \end{aligned}$$
(4)

and in particular \(\mathrm{dom}A\) is nearly convex.

 

Proof

Let \(z\in X\) be such that

$$\begin{aligned} z\notin \overline{\mathrm{dom}A}. \end{aligned}$$
(5)

We have \(\alpha := \text{ dist}(z, \overline{\mathrm{dom}A})>0\). Since \((z,0)\) is not monotonically related to \(\mathrm{gra}\big (A +\lambda J_p(-z+\cdot )\big )\), there exist \((a_{\lambda }, a_{\lambda }^*)\in \mathrm{gra}A\) and \(b^*_{\lambda }\in J_p(-z+a_{\lambda })\) such that

$$\begin{aligned}&\langle z-a_{\lambda }, {a^*_{\lambda }}+\lambda b^*_{\lambda }\rangle >0\\&\quad \Rightarrow \langle z-a_{\lambda }, {a^*_{\lambda }}\rangle >\lambda \langle -z+a_{\lambda }, b^*_{\lambda }\rangle \\&\quad \Rightarrow \langle z-a_{\lambda }, {a^*_{\lambda }}\rangle >\lambda \Vert -z+a_{\lambda }\Vert ^p \quad (\text{ by}\,\,b^*_{\lambda }\in J_p(-z+a_{\lambda }))\\&\quad \Rightarrow \frac{ \langle z-a_{\lambda }, a^*_{\lambda }\rangle }{\Vert -z+a_{\lambda }\Vert }> \lambda \Vert -z+a_{\lambda }\Vert ^{p-1}\quad (\text{ since}\,\,\Vert -z+a_{\lambda }\Vert \ge \alpha >0)\\&\quad \Rightarrow \frac{\langle z-a_{\lambda }, a^*_{\lambda }\rangle }{\Vert -z+a_{\lambda }\Vert } > \lambda \alpha ^{p-1}\quad \left(\text{ by} \Vert -z+a_{\lambda }\Vert \ge \alpha \right)\\&\quad \Rightarrow \sup _{\lambda } \frac{\langle z-a_{\lambda }, a^*_{\lambda }\rangle }{\Vert z-a_{\lambda }\Vert }=+\infty . \end{aligned}$$

By Fact 2.3, \(\overline{\mathrm{dom}A}=\overline{\mathrm{conv}\left[\mathrm{dom}A\right]}=\overline{P_X\left[\mathrm{dom}F_A\right]}\). Hence \(\overline{\mathrm{dom}A}\) is convex. \(\square \)

We also provide a (potentially) weaker criterion for equality of \(\overline{\mathrm{conv}\left[\mathrm{dom}A\right]}=\overline{P_X\left[\mathrm{dom}F_A\right]}\) for a monotone operator.

Proposition 3.2

Let \(A:X\rightrightarrows X^*\) be monotone. Assume the implication that

$$\begin{aligned} z\notin \overline{\mathrm{conv}\left[\mathrm{dom}A\right]}\quad \Rightarrow \quad \sup _{(a,a^*)\in \mathrm{gra}A} \frac{\langle z-a, a^*\rangle }{\Vert z-a\Vert }=+\infty \end{aligned}$$
(6)

holds. Then

$$\begin{aligned} \overline{\mathrm{conv}\left[\mathrm{dom}A\right]}=\overline{P_X\left[\mathrm{dom}F_A\right]}. \end{aligned}$$

 

Proof

By Fact 2.1, \(\mathrm{conv}\left[\mathrm{dom}A\right]\subseteq P_X\left[\mathrm{dom}F_A\right]\) and thus \(\overline{\mathrm{conv}\left[\mathrm{dom}A\right]}\subseteq \overline{P_X\left[\mathrm{dom}F_A\right]}\). It suffices to show that

$$\begin{aligned} P_X\left[\mathrm{dom}F_A\right]\subseteq \overline{\mathrm{conv}\left[\mathrm{dom}A\right]}. \end{aligned}$$
(7)

Suppose to the contrary that there exists \((z,z^*)\in \mathrm{dom}F_A\) such that \(z\notin \overline{\mathrm{conv}\left[\mathrm{dom}A\right]}\). Define \(F:\mathrm{gra}A: (a,a^*)\mapsto \Vert z-a\Vert \). We have \(\inf F=\text{ dist}(z, \mathrm{dom}A)>0\). Then by Fact 2.2, there exists \(r\in \mathbb R \) such that for every \((a,a^*)\in \mathrm{gra}A\),

$$\begin{aligned} \Vert z^*\Vert -\frac{\langle z-a,a^*\rangle }{\Vert z-a\Vert }\ge \frac{\langle z-a,z^*\rangle }{\Vert z-a\Vert }-\frac{\langle z-a,a^*\rangle }{\Vert z-a\Vert }=\frac{\langle z-a,z^*-a^*\rangle }{\Vert z-a\Vert }\ge r. \end{aligned}$$

Then we have

$$\begin{aligned} \sup _{(a,a^*)\in \mathrm{gra}A}\frac{\langle z-a,a^*\rangle }{\Vert z-a\Vert }\le \Vert z^*\Vert - r<+\infty , ~\text{ which} \text{ contradicts}\ (6). \end{aligned}$$

Hence (7) holds. \(\square \)

The proof of Proposition 3.2 above follows closely that of [16], Thereom 27.5].

We now turn to our synthesis of known results.

Theorem 3.3

(Domain near convexity) Let \(A: X\rightrightarrows X^*\) be monotone. Fix \(p\) with \(1 \le p < \infty \).

  1. (i)

    First suppose that for all \(z\notin \overline{\mathrm{dom}A}\) and all sufficiently large \(\lambda >0\)

    $$\begin{aligned} A+ \lambda J_p(\cdot -z)\,~ \text{ is} \text{ maximally} \text{ monotone}. \end{aligned}$$

    Then \(\mathrm{dom}A\) is nearly convex since

    $$\begin{aligned} \overline{\mathrm{dom}A}=\overline{\mathrm{conv}\left[\mathrm{dom}A\right]}=\overline{P_X\left[\mathrm{dom}F_A\right]}. \end{aligned}$$
  2. (ii)

    If we only suppose that for all \(z\notin \overline{\mathrm{conv}\left[\mathrm{dom}A\right]}\) and all sufficiently large \(\lambda >0\)

    $$\begin{aligned} (z,0)\,~ \text{ is} \text{ not} \text{ monotonically} \text{ related} \text{ to} \mathrm{gra}\big (A+ \lambda J_p(\cdot -z)\big ). \end{aligned}$$

    Then

    $$\begin{aligned} \overline{\mathrm{conv}\left[\mathrm{dom}A\right]}=\overline{P_X\left[\mathrm{dom}F_A\right]}. \end{aligned}$$

 

Proof

 

  • Part (i). Suppose \(z\notin \overline{\mathrm{dom}A}\). Then \((z,0)\notin \mathrm{gra}\big (A+ \lambda J_p(\cdot -z)\big )\). Since \(A+ \lambda J_p(\cdot -z)\) is maximally monotone, \((z,0)\) cannot be monotonically related to \(\mathrm{gra}\big (A+ \lambda J_p(\cdot -z)\big )\). Then we may directly apply Proposition 3.1.

  • Part (ii). Suppose now that \(z\notin \overline{\mathrm{conv}\left[\mathrm{dom}A\right]}\). We follow the lines of the proof of Proposition 3.1 to obtain \(\sup _{(a,a^*)\in \mathrm{gra}A} \frac{\langle z-a, a^*\rangle }{\Vert z-a\Vert }=+\infty \). We then apply Proposition 3.2 and obtain

    $$\begin{aligned} \overline{\mathrm{conv}\left[\mathrm{dom}A\right]}=\overline{P_X\left[\mathrm{dom}F_A\right]}, \end{aligned}$$

    as asserted.

When \(A\) is maximally monotone, Part (i) of Theorem 3.3 recaptures a great many attractive cases.

Corollary 3.4

(Conditions for near convexity) Let \(A:X\rightrightarrows X^*\) be maximally monotone. Then \(\overline{\mathrm{dom}A}=\overline{\mathrm{conv}\left[\mathrm{dom}A\right]}=\overline{P_X\left[\mathrm{dom}F_A\right]}\) (and thus \(\mathrm{dom}A\) is nearly convex) as soon as any of the following hold:

  1. (i)

    \(X\) is reflexive;

  2. (ii)

    \(A\) is Verona regular;

  3. (iii)

    \(A\) is of type (FPV); as holds if

  4. (iv)

    \(\mathrm{int}\mathrm{dom}A \ne \emptyset \); or if

  5. (v)

    \(\mathrm{gra}A\) is affine.

 

Proof

 

  • Part (i). First, in reflexive spaces, we have a Rockafellar’s sum theorem implying that \(A+\lambda J_p(\cdot -z)\) is always maximally monotone for every \(z\in X\) since \(\mathrm{dom}J_p(\cdot -z)=X\), see [6, 12, 16]).

  • Part (ii). Next observe that setting \(p=1\) shows that Verona regular mappings imply \(\overline{\mathrm{dom}A}=\overline{\mathrm{conv}\left[\mathrm{dom}A\right]}=\overline{P_X\left[\mathrm{dom}F_A\right]}\), since as noted above the Veronas show the maximal monotonicity of \(A+\lambda J_1(\cdot -z)\) for all \(\lambda >0,z \in X\) is equivalent to their definition.

  • Part (iii). Likewise, [23, Corollary 3.6] gives another proof that type (FPV) mappings have this property. Or see [16, Theorem 44.2], [24, Proposition 5.2.1, page 107].

  • Part (iv). This holds for various reasons. For instance it is shown in [4] that \(A\) is type (FPV). It also follows from the sum theorem in [4, Theorem 9.6.1] which applies to two maximally monotone operators \(A,B\) such that \(\mathrm{int}\mathrm{dom}A \cap \mathrm{int}\mathrm{dom}B \ne \emptyset ,\) and so shows that the maximal monotonicity of \(A+\lambda J_p(\cdot -z)\) for all \(p \ge 1, \lambda >0,z \in X\).

  • Part (v). By Fact 2.5, we have that \(A\) is type (FPV) and that \(A+\lambda J_p(\cdot -z)\) is maximally monotone for all \(\lambda >0,z \in X\). Then we directly apply Theorem 3.3. Or see [16, Theorem 46.1].

In both cases (ii) and (iii), it is still entirely possible that all maximally monotone operators have the desired property. The only other general conditions we know for near convexity are a relativization of Part (iv), namely, \(^{ic}(\mathrm{conv}\mathrm{dom}A)\ne \varnothing \) [22, Corollary 5] and the result below which follows directly from the definition of operators of type (BR):

Proposition 3.5

Let \(A:X\rightrightarrows X^*\) be maximally monotone and \((x,x^*)\in X\times X^*\). Assume that \(A\) is of type (BR) and that \(\inf _{(a,a^*)\in \mathrm{gra}A}\langle x-a,x^*-a^*\rangle >-\infty \). Then \(x\in \overline{\mathrm{dom}A}\) and \(x^*\in \overline{\mathrm{ran}A}\). In particular,

$$\begin{aligned} \overline{\mathrm{dom}A}=\overline{P_X\left[\mathrm{dom}F_A\right]}\quad \text{ and} \quad \overline{\mathrm{ran}A}=\overline{P_{X^*}\left[\mathrm{dom}F_A\right]}. \end{aligned}$$

Consequently, both \(\mathrm{dom}A\) and \(\mathrm{ran}A\) are nearly convex.

 

Proof

By an easy argument, see [20, Lemma 1(a) and (b)] or [1], Fact 2.10], we have \(x\in \overline{\mathrm{dom}A}\) and \(x^*\in \overline{\mathrm{ran}A}\).

We now show that \(\overline{\mathrm{dom}A}=\overline{P_X\left[\mathrm{dom}F_A\right]}\). By Fact 2.1, it suffices to show that

$$\begin{aligned} P_X\left[\mathrm{dom}F_A\right]\subseteq \overline{\mathrm{dom}A}. \end{aligned}$$
(8)

Let \((z,z^*)\in \mathrm{dom}F_A\). Then we have \(\inf _{(a,a^*)\in \mathrm{gra}A}\langle x-a,x^*-a^*\rangle >-\infty \). Hence \(z\in \overline{\mathrm{dom}A}\) by the above result. Thus (8) holds and then \(\overline{\mathrm{dom}A}=\overline{P_X\left[\mathrm{dom}F_A\right]}\). Similarly, we have \(\overline{\mathrm{ran}A}=\overline{P_{X^*}\left[\mathrm{dom}F_A\right]}\). Hence \(\mathrm{dom}A\) and \(\mathrm{ran}A\) both are nearly convex. \(\square \)

Note the different flavour of Proposition 3.5 with its implications for near convexity of the range and domain. We recall also that all dense type (D) operators are of type (BR) (see [9, Theorem 1.4(4)]). A detailed analysis of (BR) and type (D) operators is made in [1].

We now come to our main new result.

Theorem 3.6

Let \(A:X\rightrightarrows X^*\) be maximally monotone. Then

$$\begin{aligned} \overline{\mathrm{conv}\left[\mathrm{dom}A\right]}=\overline{P_X\left[\mathrm{dom}F_A\right]}. \end{aligned}$$

 

Proof

By Fact 2.1, it suffices to show that

$$\begin{aligned} P_X\left[\mathrm{dom}F_A\right]\subseteq \overline{\mathrm{conv}\left[\mathrm{dom}A\right]}. \end{aligned}$$
(9)

Let \((z,z^*)\in \mathrm{dom}F_A\). We shall show that

$$\begin{aligned} z\in \overline{\mathrm{conv}\left[\mathrm{dom}A\right]}. \end{aligned}$$
(10)

Suppose to the contrary that

$$\begin{aligned} z\notin \overline{\mathrm{conv}\left[\mathrm{dom}A\right]}. \end{aligned}$$
(11)

Define \(B:X\rightrightarrows X^*\) by

$$\begin{aligned} \mathrm{gra}B:=\mathrm{gra}A-\{(0,z^*)\}\!. \end{aligned}$$
(12)

Then we have

$$\begin{aligned} F_B(z,0)&= \sup _{(x,x^*)\in \mathrm{gra}B}\big \{\langle z,x^*\rangle -\langle x,x^*\rangle \big \}\nonumber \\&= \sup _{(y,y^*)\in \mathrm{gra}A}\big \{\langle z,y^*-z^*\rangle -\langle y,y^*-z^*\rangle \big \}\nonumber \\&= \sup _{(y,y^*)\in \mathrm{gra}A}\big \{\langle z,-z^*\rangle +\langle z,y^*\rangle +\langle y,z^*\rangle -\langle y,y^*\rangle \big \}\nonumber \\&= \langle z,-z^*\rangle +F_A(z,z^*). \end{aligned}$$
(13)

Since \((z,z^*)\in \mathrm{dom}F_A\), by (13), there exists \(r\in \mathbb R \) such that

$$\begin{aligned} F_B(z,0)\le r. \end{aligned}$$
(14)

By construction, \(B\) is maximally monotone and \(\mathrm{dom}B=\mathrm{dom}A\). Then \(z\notin \overline{\mathrm{conv}\left[\mathrm{dom}B\right]}\) by (11). By the Separation theorem, there exist \(\delta >0\) and \(y_0^*\in X^*\) with \(\Vert y^*_0\Vert =1\) such that

$$\begin{aligned} \langle y_0^*, z-b\rangle >\delta , \quad \forall b\in \mathrm{conv}\left[\mathrm{dom}B\right]\!. \end{aligned}$$
(15)

Let \(n\in \mathbb N \). Since \(z\notin \overline{\mathrm{conv}\left[\mathrm{dom}B\right]}, (z, ny_0^*)\notin \mathrm{gra}B\). By the maximal monotonicity of \(B\), there exists \((b_n, b_n^*)\in \mathrm{gra}B\) such that

$$\begin{aligned}&\langle z-b_n, b^*_n\rangle +\langle ny_0^*, b_n\rangle >\langle n y_0^*, z\rangle \quad \Rightarrow \quad \langle z-b_n, b^*_n\rangle >\langle ny_0^*, z-b_n\rangle \nonumber \\&\quad \Rightarrow \langle z-b_n, b^*_n\rangle >n\delta \quad (\text{ by}\ (15)). \end{aligned}$$
(16)

Then we have

$$\begin{aligned} F_B(z,0)&\ge \sup _{n\in \mathbb N }\big \{\langle z-b_n, b^*_n\rangle \big \}\\&\ge \sup _{n\in \mathbb N }\big \{n\delta \big \}\\&= +\infty , \end{aligned}$$

which contradicts (14). Hence \(z\in \overline{\mathrm{conv}\left[\mathrm{dom}A\right]}\) and in consequence (9) holds. \(\square \)

 

Remark 3.7

As already noted Theorem 3.6 provides an affirmative answer to a question posed by Simons in [16, Problem 28.3, page 112].

Question:

The following question is still unresolved:

 

Let \(A:X\rightrightarrows X^*\) be maximally monotone. Is \(\overline{\mathrm{dom}A}\) necessarily convex? In light of Theorem  3.6 this is equivalent to asking if

$$\begin{aligned} \overline{\mathrm{dom}A}=\overline{P_X\left[\mathrm{dom}F_A\right]} \end{aligned}$$

always holds?