Second-order characterizations of quasiconvexity and pseudoconvexity for differentiable functions with Lipschitzian derivatives

Abstract

For a $\mathcal {C}^2$-smooth function on a finite-dimensional space, a necessary condition for its quasiconvexity is the positive semidefiniteness of its Hessian matrix on the subspace orthogonal to its gradient, whereas a sufficient condition for its strict pseudoconvexity is the positive definiteness of its Hessian matrix on the subspace orthogonal to its gradient. Our aim in this paper is to extend those conditions for $\mathcal {C}^{1,1}$-smooth functions by using the Fréchet and Mordukhovich second-order subdifferentials.

1 Introduction

Since the notion of convexity does not satisfy a variety of mathematical models used in sciences, economics, and engineering, various generalizations of convex functions have been introduced in literature [7, 11, 18] such as (strictly) quasiconvex and (strictly) pseudoconvex functions. Those functions share many nice properties of convex functions and cover some models which are effective and adaptable to real-world situations. To be more specific, the quasiconvexity of a function ensures the convexity of its sub-level sets, and the pseudoconvexity implies that its critical points are minimizers.

First-order characterizations for quasiconvexity and pseudoconvexity can be found in [4, 7, 10, 11] for smooth functions and [1, 2, 5, 14,15,16, 22, 23, 25] for nonsmooth ones. The well-known second-order necessary condition for the quasiconvexity of $\mathcal {C}^2$-smooth functions (see for instance [3, 7, 9, 11]) states that the Hessian matrix of a quasiconvex function is positive-semidefinite on the subspace orthogonal to its gradient. Furthermore, if the Hessian matrix of a $\mathcal {C}^2$-smooth function is positive definite on the subspace orthogonal to its gradient then the given function is strictly pseudoconvex [11]. Using some kinds of generalized second-order derivatives, many authors established some second-order criteria for the quasiconvexity and pseudoconvexity of functions without the $\mathcal {C}^2$-smooth property. By employing Taylor’s formula and an estimation formula of generalized Hessian, Luc [17] set up the necessary and sufficient conditions for the quasiconvexity of $\mathcal {C}^{1,1}$-smooth functions. In [12] Ginchev and Ivanov introduced the concept of second-order upper Dini-directional derivatives and utilized it to characterize the pseudoconvexity of radially upper semicontinuous functions. By using the theory of viscosity solutions of partial differential equations, Barron, Goebel, and Jensen [6] obtained some necessary conditions and sufficient ones for the quasiconvexity of upper semicontinuous functions.

It can be seen that the Fréchet and the Mordukhovich second-order subdifferentials play a crucial role in variational analysis [19, 20, 24]. Recently, Nadi and Zafarani [21] established some characterizations of the quasimonotone and pseudomonotone of set-valued mappings in terms of their Fréchet coderivatives. Using the relationship between generalized monotone mappings and generalized convex functions, they presented some second-order characterizations of quasiconvex [21, Corollary 3.16] and pseudoconvex functions [21, Corollary 3.20] via their Fréchet second-order subdifferentials. In this paper, utilizing Fréchet and Mordukhovich second-order subdifferentials, we establish directly some necessary and sufficient conditions for quasiconvexity and pseudoconvexity of $\mathcal {C}^{1,1}$-smooth functions without using characterizations of generalized monotone mappings. For the necessity, we prove that the Fréchet second-order subdifferential of a pseudoconvex function is positive semidefinite on the subspace orthogonal to its gradient while the Mordukhovich second-order subdifferential of a quasiconvex function is only positive semidefinite along its some selection. It is noted that although the latter can be implied from [21, Corollary 3.16], we give another simpler proof of this result via the mean value inequality and some facts of quasiconvex functions. For the sufficiency, we propose two conditions guaranteeing the strict pseudoconvexity. The first one is the positive definiteness of the Mordukhovich second-order subdifferential of a given function on the subspace orthogonal to its gradient. The second one claims that the Fréchet second-order subdifferential of a given function has some selection which is positive on the subspace orthogonal to its gradient. Moreover, a second-order sufficient condition for the strict quasiconvexity is also established by using Fréchet second-order subdifferentials. Throughout the paper, we proposed a variety of examples to illustrate and analyze the obtained results.

The paper is organized as follows. Some background material from variational analysis and generalized convexity are recalled in Sect. 2. Section 3 presents some second-order conditions for quasiconvexity and pseudoconvexity of $\mathcal {C}^{1,1}$-smooth functions. Sufficient conditions are given in Sect. 4. Conclusions and further investigations are discussed in the last section.

2 Preliminaries

To begin with, some necessary notions from [20] will be recalled. Let F be a set-valued mapping between Euclidean spaces $\mathbb {R}^n$ and $\mathbb {R}^m$. As usual, the effective domain and the graph of F are given, respectively, by

$$\begin{aligned} \text {dom}F:=\{x\in \mathbb {R}^n|F(x)\ne \emptyset \}\quad \text {and}\quad \text {gph}F := \{(x,y)\in \mathbb {R}^n\times \mathbb {R}^m|y\in F(x)\}. \end{aligned}$$

The sequential Painlevé–Kuratowski outer limit of F as $x\rightarrow \bar{x}$ is defined as

$$\begin{aligned} \begin{array}{rl} \underset{x\rightarrow \bar{x}}{\mathop {\mathrm{Lim}\,\mathrm{sup}}}\;F(x): =\{y\in \mathbb {R}^n\, |\, &{}\exists \text{ sequences } x_k\rightarrow \bar{x},\ y_k\rightarrow y,\\ &{}\text{ with } y_k\in F(x_k)\ \text{ for } \text{ all }\ k=1,2,\ldots \}. \end{array} \end{aligned}$$

(1)

Let us consider an extended-real-valued function $\varphi :\mathbb {R}^n\rightarrow \overline{\mathbb {R}}:=(-\infty ,\infty ]$. We always assume that $\varphi$ is proper and lower semicontinuous. The Fréchet subdifferential of $\varphi$ at $\bar{x}\in \text {dom}\varphi :=\{x\in \mathbb {R}^n:\varphi (x)<\infty \}$ (known as the presubdifferential and as the regular subdifferential) is

$$\begin{aligned} \widehat{\partial }\varphi \left( {\bar{x} } \right) : = \left\{ {x^* \in {\mathbb {R}^n}\, |\,\mathop {\lim \inf }\limits _{x \rightarrow \overline{x} } \frac{{\varphi (x) - \varphi \left( {\bar{x} } \right) - \left\langle {{x^*},x - \bar{x}} \right\rangle }}{{\left\| {x - \bar{x} } \right\| }} \ge 0} \right\} . \end{aligned}$$

(2)

Then the limiting subdifferential of $\varphi$ at $\bar{x}$ (known also the general or basic subdifferential) is defined via the outer limit (1)

$$\begin{aligned} \partial \varphi (\bar{x}):= \underset{x \overset{\varphi }{\rightarrow } \bar{x}}{\mathop {\mathrm{Lim}\,\mathrm{sup}}} \; \widehat{\partial }\varphi (x), \end{aligned}$$

(3)

where $x \overset{\varphi }{\rightarrow } \bar{x}$ signifies that $x\rightarrow \bar{x}$ with $\varphi (x)\rightarrow \varphi (\bar{x})$. Observe that both Fréchet and limiting subdifferentials reduce to the classical Fréchet derivative for continuously differentiable functions.

Given a set $\Omega \subset \mathbb {R}^n$ with its indicator function $\delta _\Omega (x)$ equal to 0 for $x\in \Omega$ and to $\infty$ otherwise, the Fréchet and the Mordukhovich normal cones to $\Omega$ at $\bar{x}\in \Omega$ are defined, respectively, via the corresponding subdifferentials (2) and (3) by

$$\begin{aligned} \widehat{N}(\bar{x};\Omega ):=\widehat{\partial }\delta _\Omega (\bar{x}) \quad \text {and}\quad N(\bar{x};\Omega ):=\partial \delta _\Omega (\bar{x}). \end{aligned}$$

(4)

The Fréchet and Mordukhovich coderivatives of F at $(\bar{x},\bar{y})\in \text {gph}F$ are defined, respectively, via corresponding normal cones (4) by

$$\begin{aligned} D^*F(\overline{x} ,\overline{y} )(y^*):= & {} \{ x^* \in {\mathbb {R}^n}:({x^*}, - {y^*}) \in {N}((\bar{x},\bar{y}), \text {gph}F) \},\\ \widehat{D}^*F(\overline{x} ,\overline{y} )(y^*):= & {} \{ x^* \in {\mathbb {R}^n}:({x^*}, - {y^*}) \in \widehat{N}((\bar{x},\bar{y}), \text {gph}F) \}. \end{aligned}$$

We omit $\bar{y} = f(\bar{x})$ in the above coderivative notions if $F:= f:\mathbb {R}^n \rightarrow \mathbb {R}^m$ is single-valued.

Definition 2.1

Let $\varphi : \mathbb {R}^n \rightarrow \overline{\mathbb {R}}$ be a function with a finite value at $\bar{x}$.

• (i) For any $\bar{y} \in \partial \varphi (\bar{x})$, the map $\partial ^2 \varphi (\bar{x},\bar{y}): \mathbb {R}^n \rightrightarrows \mathbb {R}^n$ with the values

  $$\begin{aligned} \partial ^2 \varphi (\bar{x},\bar{y})(u)= (D^*\partial \varphi )(\bar{x},\bar{y})(u) \quad (u \in \mathbb {R}^n) \end{aligned}$$

  is said to be the Mordukhovich second-order subdifferential of $\varphi$ at $\bar{x}$ relative to $\bar{y}$.

• (ii) For any $\bar{y} \in \widehat{\partial } \varphi (\bar{x})$, the map $\widehat{\partial }^2 \varphi (\bar{x},\bar{y}): \mathbb {R}^n \rightrightarrows \mathbb {R}^n$ with the values

  $$\begin{aligned} \widehat{\partial }^2 \varphi (\bar{x},\bar{y})(u)= (\widehat{D}^*\widehat{\partial }\varphi )(\bar{x},\bar{y})(u) \quad (u \in \mathbb {R}^n) \end{aligned}$$

  is said to be the Fréchet second-order subdifferential of $\varphi$ at $\bar{x}$ relative to $\bar{y}$. We omit $\bar{y} = \nabla \varphi (\bar{x})$ in the above second-order subdifferentials if $\varphi \in \mathcal {C}^1$ around $\bar{x}$, i.e., continuously Fréchet differentiable in a neighborhood of $\bar{x}$.

In general, the Fréchet second-order subdifferential and the Mordukhovich one are incomparable. However, if $\varphi \in \mathcal {C}^1$ around $\bar{x}$, then

$$\begin{aligned} \widehat{\partial }^2 \varphi (\bar{x})(u)\subset \partial ^2 \varphi (\bar{x})(u), \quad \forall u \in \mathbb {R}^n. \end{aligned}$$

(5)

If $\varphi \in \mathcal {C}^{1,1}$ around $\bar{x}$, i.e., Fréchet differentiable around $\bar{x}$ with the gradient $\nabla \varphi$ being locally Lipschitzian around $\bar{x}$ then the calculation of second-order subdifferentials can be essentially simplified due to the following scalarization formulas (see [19, Proposition 3.5] and [20, Proposition 1.120])

$$\begin{aligned} \widehat{\partial }^2 \varphi (\bar{x})(u)=\widehat{\partial }\langle u,\nabla \varphi \rangle (\bar{x}), \quad {\partial }^2 \varphi (\bar{x})(u)=\partial \langle u,\nabla \varphi \rangle (\bar{x}). \end{aligned}$$

(6)

In this case, Mordukhovich second-order subdifferentials are nonempty [20, Corollary 2.25] while Fréchet ones may be empty. If $\varphi \in \mathcal {C}^2$ around $\bar{x}$, i.e., $\varphi$ is twice continuously Fréchet differentiable in a neighborhood of $\bar{x}$, then

$$\begin{aligned} \partial ^2 \varphi (\bar{x})(u)=\widehat{\partial }^2 \varphi (\bar{x})(u)= \{\nabla ^2 \varphi (\bar{x})u \}, \quad \forall u \in \mathbb {R}^n. \end{aligned}$$

(7)

Let us recall some well-known notions of generalized convexity.

Definition 2.2

• (a) A function $\varphi : \mathbb {R}^n \rightarrow \mathbb {R}$ is said to be quasiconvex if

  $$\begin{aligned} \varphi ((1-\lambda )x + \lambda y) \le \max \{\varphi (x),\varphi (y)\} \end{aligned}$$

  for every $x, y \in \mathbb {R}^n$ and for every $\lambda \in [0,1]$.

• (b) A function $\varphi : \mathbb {R}^n \rightarrow \mathbb {R}$ is said to be strictly quasiconvex if

  $$\begin{aligned} \varphi ((1-\lambda )x + \lambda y) < \max \{\varphi (x),\varphi (y)\} \end{aligned}$$

  for every $x, y \in \mathbb {R}^n, x\ne y$ and for every $\lambda \in (0,1)$.

• (c) A differentiable function $\varphi : \mathbb {R}^n \rightarrow \mathbb {R}$ is called pseudoconvex if

  $$\begin{aligned} x,y \in \mathbb {R}^n, \varphi (x)>\varphi (y) \Longrightarrow \langle \nabla \varphi (x), y - x \rangle <0. \end{aligned}$$

• (d) A differentiable function $\varphi : \mathbb {R}^n \rightarrow \mathbb {R}$ is called strictly pseudoconvex if

  $$\begin{aligned} x,y \in \mathbb {R}^n,\; x\ne y,\; \varphi (x)\ge \varphi (y) \Longrightarrow \langle \nabla \varphi (x), y - x \rangle <0. \end{aligned}$$

It follows, immediately, from the given definitions, that a strictly quasiconvex (pseudoconvex) function is quasiconvex (pseudoconvex). For differentiable functions, (strict) pseudoconvexity implies (strict) quasiconvexity. The next theorem shall point out that within the class of (strictly) quasiconvex functions, (strict) pseudoconvexity may be specified by means of its behaviour at critical points.

Theorem 2.1

[7, Theorem 3.2.9] Let $\varphi :\mathbb {R}^n\rightarrow \mathbb {R}$ be a continuously differentiable function. Then, $\varphi$ is (strictly) pseudoconvex if and only if the following conditions hold:

• (i) $\varphi$ is quasiconvex;

• (ii) If $\nabla \varphi (x)=0$ then x is a (strict) local minimizer for $\varphi$.

Finally, we consider a lemma which will be used in the sequel.

Lemma 2.1

Let $\varphi :\mathbb {R}^n\rightarrow \mathbb {R}$ be a differentiable function. If $\varphi$ is not strictly quasiconvex, then there exist $x_1, x_2\in \mathbb {R}^n, x_1\ne x_2$ and $t_0\in (0,1)$ such that $\langle \nabla \varphi (x_1+t_0(x_2-x_1)),x_2-x_1\rangle =0$ and

$$\begin{aligned} \varphi (x_1+t(x_2-x_1))\le \varphi (x_1+t_0(x_2-x_1)), \quad \forall t\in [0,1]. \end{aligned}$$

(8)

Proof

Since $\varphi$ is not strictly quasiconvex, there exist $x_1,x_2\in \mathbb {R}^n, \alpha \in (0,1)$ such that $x_1\ne x_2$ and

$$\begin{aligned} \varphi ((1-\alpha )x_1+\alpha x_2)\ge \max \{\varphi (x_1),\varphi (x_2)\}. \end{aligned}$$

(9)

Consider the function $f:\mathbb {R}\rightarrow \mathbb {R}$ given by

$$\begin{aligned} f(t)=\varphi (x_1+t(x_2-x_1)),\quad \forall t\in \mathbb {R}. \end{aligned}$$

Then, thanks to the Weierstrass theorem and (9), we can find a number $t_0\in (0,1)$ for which the function f admits a maximum on the interval [0, 1] at $t_{0}$. Hence, (8) is satisfied and by the Fermat rule we have

$$\begin{aligned} 0=\nabla f(t_0)=\langle \nabla \varphi (x_1+t_0(x_2-x_1)),x_2-x_1\rangle . \end{aligned}$$

$\square$

3 Necessary conditions

Let us recall the well-known second-order necessary condition for quasiconvexity of $\mathcal {C}^2$-smooth functions.

Theorem 3.1

(see [3, Lemma 6.2] or [7, Theorem 3.4.2]) Let $\varphi :\mathbb {R}^n \rightarrow \mathbb {R}$ be a $\mathcal {C}^2$-smooth function. If $\varphi$ is quasiconvex, then

$$\begin{aligned} x, u \in \mathbb {R}^n, \langle \nabla \varphi (x),u \rangle = 0 \Longrightarrow \langle \nabla ^2\varphi (x)u,u \rangle \ge 0. \end{aligned}$$

(10)

By using the mean value inequality in terms of limiting subdifferential for Lipschitzian functions [20, Corollary 3.51 ] we extend the above result to $\mathcal {C}^{1,1}$-smooth functions.

Proposition 3.1

Let $\varphi :\mathbb {R}^n\rightarrow \mathbb {R}$ be a Lipschitz continuous function on an open set containing [a, b]. Then one has

$$\begin{aligned} \langle x^*,b-a\rangle \ge \varphi (b)-\varphi (a)\quad \text {for some}\quad x^*\in \partial \varphi (c),\quad c\in [a,b). \end{aligned}$$

Theorem 3.2

Let $\varphi :\mathbb {R}^n\rightarrow \mathbb {R}$ be a $\mathcal {C}^{1,1}$-smooth function. If $\varphi$ is quasiconvex then

$$\begin{aligned} x, u \in \mathbb {R}^n, \langle \nabla \varphi (x), u \rangle = 0 \Longrightarrow \langle z, u \rangle \ge 0 \; \text {for some} \; z \in {\partial }^2 \varphi (x)(u). \end{aligned}$$

(11)

Proof

Let $x,u\in \mathbb {R}^n$ be such that $\langle \nabla \varphi (x),u\rangle =0$. If $u=0$ then $\langle z,u\rangle = 0$ for all $z\in {\partial }^2 \varphi (x)(u)$. Otherwise, consider the function $f:\mathbb {R}^n\rightarrow \mathbb {R}$ given by

$$\begin{aligned} f(y):=\langle \nabla \varphi (y),u\rangle \quad \forall y\in \mathbb {R}^n. \end{aligned}$$

Then, $f(x)=0$ and f is locally Lipschitz continuous on $\mathbb {R}^n$ by the $\mathcal {C}^{1,1}$-smoothness of $\varphi$. Moreover, $\partial f$ is locally bounded (see [20, Corollary 1.81] or [24, Theorem 9.13]), robust [24, Proposition 8.7] on $\mathbb {R}^n$ and for every $y\in \mathbb {R}^n$

$$\begin{aligned} \partial f(y)=\partial \langle u,\nabla \varphi \rangle (y)=\partial ^2\varphi (y)(u). \end{aligned}$$

For the sequences $x_k:=x+(1/k)u,\; x^\prime _k:=x-(1/k)u\; (k\in \mathbb {N})$, one has $x_k\rightarrow x, x^\prime _k\rightarrow x$ and, in view of Proposition 3.1, there exist $\theta _k \in [0,1/k)$, $\theta ^\prime _k\in (0,1/k]$ and $z_k\in \partial f(x+\theta _ku), z^\prime _k\in \partial f(x-\theta _k^\prime u)$ such that

$$\begin{aligned} \langle z_k,(1/k)u\rangle= & {} \langle z_k,x_k-x\rangle \ge f(x_k)-f(x)=f(x_k),\\ \langle z^\prime _k,(1/k)u\rangle= & {} \langle z^\prime _k,x-x^\prime _k\rangle \ge f(x)-f(x^\prime _k)=-f(x^\prime _k). \end{aligned}$$

By the quasiconvexity of $\varphi$, it follows from [11, Proposition 1] that

$$\begin{aligned} 0\ge & {} \min \{\langle \nabla \varphi (x_k),x^\prime _k-x_k\rangle , \langle \nabla \varphi (x^\prime _k),x_k-x^\prime _k\rangle \}\\= & {} \min \{(-2/k)f(x_k), (2/k)f(x^\prime _k)\}\\\ge & {} \min \{(-2/k^2)\langle z_k,u\rangle , (-2/k^2)\langle z^\prime _k,u\rangle \}. \end{aligned}$$

Therefore, $\max \{\langle z_k,u\rangle ,\langle z^\prime _k,u\rangle \}\ge 0$ for all $k\in \mathbb {N}$. Since $\partial f$ is locally bounded around x, the sequences $(z_k), (z^\prime _k)$ are bounded. Without loss of generality, we can assume that $z_k\rightarrow z$ and $z^\prime _k\rightarrow z^\prime$. It follows that $\max \{\langle z,u\rangle ,\langle z^\prime ,u\rangle \}\ge 0$ and by the robustness of $\partial f$ we have $z,z^\prime \in \partial f(x)=\partial ^2\varphi (x)(u)$. The proof is complete.$\square$

Remark 3.1

Theorem 3.2 can be deduced directly from [21, Corollary 3.16] when $\varphi$ is a $\mathcal {C}^{1,1}$-smooth function. The mean value inequality allows us to give a simpler proof for this result.

The following example shows that the inequality in (11) may not be true for all points belonging to the second-order Mordukhovich subdifferentials even for pseudoconvex functions.

Example 3.1

[13, Remark 3.1] Let $\varphi : \mathbb {R} \rightarrow \mathbb {R}$ be defined by

$$\begin{aligned} \varphi (x):= \int _0^{|x|} {\phi (t)dt}, \end{aligned}$$

where

$$\begin{aligned} \phi (t) = {\left\{ \begin{array}{ll} 2t^2 + t^2 \sin \left( \frac{1}{t} \right) &{} \text {if}\quad t > 0,\\ 0 &{} \text {if}\quad t=0.\\ \end{array}\right. } \end{aligned}$$

Observe that $\varphi$ is a pseudoconvex $\mathcal {C}^{1,1}$-smooth function. Indeed, for every $x\in \mathbb {R}$, we have

$$\begin{aligned} \nabla \varphi (x) = {\left\{ \begin{array}{ll} \phi (x)&{} \text {if}\quad x \ge 0,\\ -\phi (-x) &{} \text {if} \quad x< 0. \end{array}\right. } \end{aligned}$$

Hence, $\nabla \varphi$ is locally Lipschitz and so it is $\mathcal {C}^{1,1}$-smooth. Moreover, $\nabla \varphi (x) = 0$ if and only if $x = 0$ and 0 is a local minimum of $\varphi$. It follows from [7, Theorem 3.2.7] that $\varphi$ is a pseudoconvex function. Clearly, one has $\partial ^2 \varphi (0)(u) = [-|u|,|u|]$ for each $u \in \mathbb {R}$. Thus, for all $u\ne 0$, there exists $z^* \in \partial ^2 \varphi (0)(u)$ such that $\langle z^*, u \rangle < 0$.

Although the pseudoconvexity does not imply the positive semidefiniteness of the second-order Mordukhovich subdifferential, it guarantees the positive semidefiniteness of the second-order Fréchet subdifferential.

Theorem 3.3

Let $\varphi :\mathbb {R}^n\rightarrow \mathbb {R}$ be a $\mathcal {C}^{1,1}$-smooth function. If $\varphi$ is pseudoconvex then

$$\begin{aligned} x, u \in \mathbb {R}^n, \langle \nabla \varphi (x), u \rangle = 0 \Longrightarrow \langle z, u \rangle \ge 0 \; \text {for all} \; z \in \widehat{\partial }^2 \varphi (x)(u). \end{aligned}$$

(12)

Proof

Suppose to the contrary that there exist $x,u\in \mathbb {R}^n$ and $z\in \widehat{\partial }^2 \varphi (x)(u)$ such that $\langle \nabla \varphi (x), u \rangle = 0$ and $\langle z,u\rangle <0$. By (6), we have $z\in \widehat{\partial }\langle u,\nabla \varphi \rangle (x)$ and so

$$\begin{aligned} 0\le & {} \liminf _{y\rightarrow x}\frac{\langle u,\nabla \varphi (y)\rangle -\langle u,\nabla \varphi (x)\rangle -\langle z,y-x\rangle }{\Vert y-x\Vert }\nonumber \\= & {} \liminf _{y\rightarrow x}\frac{\langle u,\nabla \varphi (y)\rangle -\langle z,y-x\rangle }{\Vert y-x\Vert }. \end{aligned}$$

(13)

For the sequence $x_k:=x-(1/k)u\; (k\in \mathbb {N})$, one has $x_k\rightarrow x$ and

$$\begin{aligned} \langle \nabla \varphi (x), x_k - x \rangle = \langle \nabla \varphi (x), -(1/k)u \rangle =0. \end{aligned}$$

The pseudoconvexity of $\varphi$ implies that $\varphi (x_k)\ge \varphi (x)$ and by the classical mean value theorem there exists $\theta _k\in (0,1/k)$ such that

$$\begin{aligned} 0\le \varphi (x_k)-\varphi (x)=\langle \nabla \varphi (x-\theta _ku),x_k-x\rangle =\langle \nabla \varphi (x-\theta _ku),(-1/k)u\rangle . \end{aligned}$$

For the sequence $y_k:=x-\theta _ku\; (k\in \mathbb {N})$, one has $y_k\rightarrow x$ and $\langle \nabla \varphi (y_k),u\rangle \le 0$. Therefore, by (13) we have

$$\begin{aligned} 0\le & {} \liminf _{k\rightarrow \infty }\frac{\langle u,\nabla \varphi (y_k)\rangle -\langle z,y_k-x\rangle }{\Vert y_k-x\Vert }\\\le & {} \liminf _{k\rightarrow \infty }\frac{\langle z,\theta _ku\rangle }{\Vert \theta _ku\Vert }\\= & {} \frac{\langle z,u\rangle }{\Vert u\Vert } \end{aligned}$$

which is a contradiction to $\langle z,u\rangle <0$.$\square$

The next example shows that (12) is violated if the pseudoconvexity is relaxed to quasiconvexity.

Example 3.2

Let $\varphi : \mathbb {R} \rightarrow \mathbb {R}$ be given by

$$\begin{aligned} \varphi (x) := \frac{1}{2} x|x|, \quad \forall x \in \mathbb {R}. \end{aligned}$$

Observe that $\varphi$ is a quasiconvex $\mathcal {C}^{1,1}$-smooth function and $\nabla \varphi (x) = |x|$ for every $x \in \mathbb {R}$. It is clear that

$$\begin{aligned} N((0,0);\text {gph}\nabla \varphi ) = \{(x,y)\in \mathbb {R}^2|\; y = |x|, y \le -|x| \}. \end{aligned}$$

Note that the Fréchet coderivative of $\nabla \varphi$ in this case is given by

$$\begin{aligned} \widehat{\partial }^2 \varphi (0)(u) =\widehat{D}^* \nabla \varphi (0)(u) = {\left\{ \begin{array}{ll} [-u,u]&{} \text {if}\quad u \ge 0,\\ \emptyset &{} \text {if}\quad u< 0. \end{array}\right. } \end{aligned}$$

Observe that for $z=-1, u=1$, we have $z \in \widehat{\partial }^2 \varphi (0)(u)$ and $\langle z,u \rangle < 0$.

4 Sufficient conditions

A second-order sufficient condition for the strict pseudoconvexity in the $\mathcal {C}^2$-smooth case is recalled in the following theorem.

Theorem 4.1

[11, Proposition 4] Let $\varphi : \mathbb {R}^n \rightarrow \mathbb {R}$ be a $\mathcal {C}^2$-smooth function satisfying

$$\begin{aligned} x\in \mathbb {R}^n, u \in \mathbb {R}^n \setminus \{0\}, \langle \nabla \varphi (x),u \rangle =0 \Longrightarrow \langle \nabla ^2\varphi (x)u,u \rangle >0. \end{aligned}$$

(14)

Then, $\varphi$ is a strictly pseudoconvex function.

Our aim in this section is to establish some similar versions of Theorem 4.1 in the $\mathcal {C}^{1,1}$-smooth case by using the Fréchet and Mordukhovich second-order subdifferentials. The first version is the replacement of the Hessian matrices in (14) by the Mordukhovich second-order subdifferentials. Our proof is based on Theorem 2.1 and the following sufficient optimality condition for $\mathcal {C}^{1,1}$-smooth functions.

Proposition 4.1

[8, Corollary 4.8] Suppose that $\varphi :\mathbb {R}^n\rightarrow \mathbb {R}$ is a $\mathcal {C}^{1,1}$-smooth function and $x\in \mathbb {R}^n$. If $\nabla \varphi (x)=0$ and

$$\begin{aligned} \langle z,u\rangle >0\; \text {for all}\; z\in \partial ^2\varphi (x)(u), u\in \mathbb {R}^n \end{aligned}$$

then x is a strict local minimizer of $\varphi$.

Theorem 4.2

Let $\varphi :\mathbb {R}^n\rightarrow \mathbb {R}$ be a $\mathcal {C}^{1,1}$-smooth function satisfying

$$\begin{aligned} x \in \mathbb {R}^n, u \in \mathbb {R}^n \setminus \{0\}, \langle \nabla \varphi (x), u \rangle =0 \Longrightarrow \langle z, u \rangle >0 \; \text {for all} \; z \in \partial ^2 \varphi (x)(u). \end{aligned}$$

(15)

Then $\varphi$ is a strictly pseudoconvex function.

Proof

Observe that if $\nabla \varphi (x)=0$, then (15) implies the positive semidefiniteness of $\partial ^2\varphi (x)$ and so, by Proposition 4.1, x is a strict local minimizer of $\varphi$. Hence, it follows from Theorem 2.1 that $\varphi$ is strictly pseudoconvex if and only if $\varphi$ is quasiconvex.

Assume that $\varphi$ is not quasiconvex. Then, by Lemma 2.1, there exist $x_1,x_2\in \mathbb {R}^n, x_1\ne x_2$ and $t_0\in (0,1)$ such that $\langle \nabla \varphi (x_1+t_0(x_2-x_1)),x_2-x_1\rangle =0$ and (8) is satisfied. Let $\bar{x}:=x_1+t_0(x_2-x_1)$ and $u:=x_2-x_1$. It follows that $u\ne 0$ and $\langle \nabla \varphi (\bar{x}),u\rangle =0$ and so, by (15),

$$\begin{aligned} \langle z,u\rangle >0, \quad \forall z\in \partial ^2\varphi (\bar{x})(u). \end{aligned}$$

(16)

For the sequence $x_k:=\bar{x}+(1/k)u$ ($k\in \mathbb {N}$) we have $x_k\rightarrow \bar{x}$. For sufficiently large k, we have $t_0+1/k\in (0,1)$ and so $\varphi (x_k)\le \varphi (\bar{x})$ by (8). Applying the classical mean value theorem, for sufficiently large k, there exists $\theta _k\in (0, 1/k)$ such that

$$\begin{aligned} \langle \nabla \varphi (\bar{x}+\theta _ku),(1/k)u\rangle =\varphi (x_k)-\varphi (\bar{x})\le 0. \end{aligned}$$

(17)

Consider the function $\phi :\mathbb {R}^n\rightarrow \mathbb {R}$ given by

$$\begin{aligned} \phi (x)=\langle u,\nabla \varphi \rangle (x), \quad \forall x\in \mathbb {R}^n. \end{aligned}$$

Applying Proposition 3.1, for every k, there exist $\gamma _k\in (0,\theta _k]$ and $z_k\in \partial \phi (\bar{x}+\gamma _ku)$ such that

$$\begin{aligned} \langle z_k,-\theta _ku\rangle\ge & {} \phi (\bar{x})-\phi (\bar{x}+\theta _ku)\\= & {} \langle u,\nabla \varphi (\bar{x})\rangle -\langle u,\nabla \varphi (\bar{x}+\theta _ku)\rangle \\= & {} -\langle u,\nabla \varphi (\bar{x}+\theta _ku)\rangle . \end{aligned}$$

Combining the above inequality with (17) we have $\langle z_k,u\rangle \le 0$ for sufficiently large k. Since $\partial \phi$ is locally bounded at $\bar{x}$, the sequence $(z_k)$ is bounded. Without loss of generality, we can assume that $z_k\rightarrow z$. It follows that $\langle z,u\rangle \le 0$ and by the robustness of $\partial \phi$ we have $z\in \partial \phi (\bar{x})=\partial ^2\varphi (\bar{x})(u)$. This is a contradiction to (16). The proof is complete.$\square$

We consider two examples to analyze (15). The first one shows that (15) cannot be relaxed to the following condition

$$\begin{aligned} x \in \mathbb {R}^n, u \in \mathbb {R}^n \setminus \{0\}, \langle \nabla \varphi (x), u \rangle =0 \Longrightarrow \langle z, u \rangle >0 \; \text {for some} \; z \in \partial ^2 \varphi (x)(u) \end{aligned}$$

(18)

Moreover, (18) is not sufficient for the quasiconvexity of $\varphi$.

Example 4.1

Let $\varphi : \mathbb {R} \rightarrow \mathbb {R}$ be the function given by

$$\begin{aligned} \varphi (x):= \int _0^{x} {\phi (t)dt}, \end{aligned}$$

where

$$\begin{aligned} \phi (t) = {\left\{ \begin{array}{ll} -2t^2 + t^2 \sin (\frac{1}{t})&{} \text {if}\quad t > 0,\\ 0 &{} \text {if}\quad t= 0,\\ 2t^2 + t^2 \sin (\frac{1}{t}) &{} \text {if}\quad t < 0.\\ \end{array}\right. } \end{aligned}$$

Observe that $\varphi$ is a $\mathcal {C}^{1,1}$-smooth function and $\nabla \varphi (x) = \phi (x)$ for every $x \in \mathbb {R}$. Moreover, we have $\partial ^2 \varphi (0)(u) = \left[ -|u|,|u|\right]$ for all $u \in \mathbb {R}$. Let $x \in \mathbb {R}$, $u \in \mathbb {R} \setminus \{0\}$ be such that $\langle \nabla \varphi (x), u \rangle = 0$. It follows that $\nabla \varphi (x) = 0$, or equivalently $x = 0$. For $z^* = u \in \partial ^2 \varphi (0)(u)$, we have $\langle z^*, u \rangle = |u|^2 > 0$. The condition (18) holds for $\varphi$. However, $\varphi$ is not quasiconvex. Indeed, for $x =\displaystyle \frac{1}{\pi }, y =-\frac{1}{\pi }$, we have

$$\begin{aligned} \langle \nabla \varphi (x), y -x \rangle = \frac{4}{\pi ^3} >0,\quad \langle \nabla \varphi (y), y -x \rangle = -\frac{4}{\pi ^3}<0. \end{aligned}$$

By [11, Proposition 1], $\varphi$ is not quasiconvex.

The second example points out that we cannot replace the Mordukhovich second-order subdifferential in (15) by the Fréchet second-order one since it may be empty.

Example 4.2

Let $\varphi : \mathbb {R} \rightarrow \mathbb {R}$ be the function given by

$$\begin{aligned} \varphi (x):= \int _0^{x} {\phi (t)dt} \quad \forall x \in \mathbb {R}, \end{aligned}$$

where

$$\begin{aligned} \phi (t) := {\left\{ \begin{array}{ll} -2t - t\sin (\log (|t|))&{} \text {if}\quad t \ne 0,\\ 0 &{} \text {if}\quad t = 0 \end{array}\right. } \end{aligned}$$

is a locally Lipschitz function. Hence, $\varphi$ is $\mathcal {C}^{1,1}$-smooth and $\nabla \varphi (x)=\phi (x)$ for every $x\in \mathbb {R}$. Let $x, u \in \mathbb {R}$, $u \ne 0$ such that $\langle \nabla \varphi (x),u\rangle =0$. Then, $\nabla \varphi (x) = 0$ and so $x = 0$. We have $\widehat{\partial }^2 \varphi (0)(u) = \emptyset$. Thus, the below condition holds

$$\begin{aligned} x \in \mathbb {R}, u \in \mathbb {R}{\setminus } \{0\}, \langle \nabla \varphi (x),u\rangle = 0 \Longrightarrow \langle z,u\rangle >0\; \text {for all} \; z \in \widehat{\partial }^2\varphi (x)(u). \end{aligned}$$

However, $\varphi$ is not a pseudoconvex function. Indeed, for $x = 0, y = 1$, we have

$$\begin{aligned} \langle \nabla \varphi (x), y-x\rangle =0, \; \langle \nabla \varphi (y), y-x\rangle =-2 <0. \end{aligned}$$

By [11, Proposition 2], $\varphi$ is not pseudoconvex.

When the Fréchet second-order subdifferential is nonempty, we can use it to characterize the strict quasiconvexity and strict pseudoconvexity of $\mathcal {C}^{1,1}$-smooth functions.

Theorem 4.3

Let $\varphi :\mathbb {R}^n\rightarrow \mathbb {R}$ be a $\mathcal {C}^{1,1}$-smooth function satisfying

$$\begin{aligned} x \in \mathbb {R}^n, u \in \mathbb {R}^n {\setminus } \{0\}, \langle \nabla \varphi (x), u \rangle =0 \Longrightarrow \langle z, u \rangle >0\ \text {for some}\ z \in \widehat{\partial }^2\varphi (x)(u) \cup -\widehat{\partial }^2\varphi (x)(-u)\nonumber \\ \end{aligned}$$

(19)

Then $\varphi$ is a strictly quasiconvex function.

Proof

Assume that $\varphi$ is not strictly quasiconvex. Then, by Lemma 2.1, there exist $x_1,x_2\in \mathbb {R}^n$ with $x_1\ne x_2$ and $t_0\in (0,1)$ such that $\langle \nabla \varphi (x_1+t_0(x_2-x_1)),x_2-x_1\rangle =0$ and (8) is satisfied. Let $x:=x_1+t_0(x_2-x_1)$ and $u:=x_2-x_1$. It follows that $u\ne 0$ and $\langle \nabla \varphi (x),u\rangle =0$ and so, by (19), there exists $z\in \widehat{\partial }^2\varphi (x)(u) \cup -\widehat{\partial }^2\varphi (x)(-u)$ such that $\langle z,u\rangle >0$. Since

$$\begin{aligned} \widehat{\partial }^2\varphi (x)(u) \cup -\widehat{\partial }^2\varphi (x)(-u) = \widehat{\partial }\langle u,\nabla \varphi \rangle (x) \cup -\widehat{\partial }\langle -u,\nabla \varphi \rangle (x) \end{aligned}$$

it must happen one of the following cases.

Case 1: $z\in \widehat{\partial }\langle u,\nabla \varphi \rangle (x)$. Since $\langle \nabla \varphi (x),u\rangle =0$, we have

$$\begin{aligned} 0\le & {} \liminf _{y\rightarrow x}\frac{\langle u,\nabla \varphi (y)\rangle -\langle u,\nabla \varphi (x)\rangle -\langle z,y-x\rangle }{\Vert y-x\Vert }\nonumber \\= & {} \liminf _{y\rightarrow x}\frac{\langle u,\nabla \varphi (y)\rangle -\langle z,y-x\rangle }{\Vert y-x\Vert }. \end{aligned}$$

(20)

For the sequence $x_k:=x+(1/k)u$ ($k\in \mathbb {N}$) we have $x_k\rightarrow x$. For sufficiently large k, we have $t_0+1/k\in (0,1)$ and so $\varphi (x_k)\le \varphi (x)$ by (8). Applying the classical mean value theorem, for sufficiently large k, there exists $\theta _k\in (0, 1/k)$ such that

$$\begin{aligned} \langle \nabla \varphi (x+\theta _ku),(1/k)u\rangle =\varphi (x_k)-\varphi (x)\le 0. \end{aligned}$$

(21)

For the sequence $y_k:=x+\theta _ku\; (k\in \mathbb {N})$ we have $y_k\rightarrow x$ and $\langle \nabla \varphi (y_k),u\rangle \le 0$ by (21) for every $k\in \mathbb {N}$. It follows from (20) that

$$\begin{aligned} 0\le & {} \liminf _{k\rightarrow \infty }\frac{\langle u,\nabla \varphi (y_k)\rangle -\langle z,y_k-x\rangle }{\Vert y_k-x\Vert }\\\le & {} \liminf _{k\rightarrow \infty }\frac{-\langle z,\theta _ku\rangle }{\Vert \theta _ku\Vert }\\= & {} \frac{-\langle z,u\rangle }{\Vert u\Vert } \end{aligned}$$

which is a contradiction to $\langle z,u\rangle > 0$.

Case 2. $z\in -\widehat{\partial }\langle -u,\nabla \varphi \rangle (x)$. Repeating the proof of Case 1. with u, z being replaced by $-u,-z$ we also get a contradiction.$\square$

Remark 4.1

Observe that the strict quasiconvexity in Theorem 4.3 cannot be improved to strict pseudoconvexity. Indeed, let $\varphi$ be the function given in Example 3.2. We have

$$\begin{aligned} \widehat{\partial }^2\varphi (0)(u)\cup -\widehat{\partial }^2\varphi (0)(-u) =\left[ -|u|,|u|\right] , \end{aligned}$$

for all $u \in \mathbb {R}$. Observe that if $x \in \mathbb {R}$, $u\in \mathbb {R}\setminus \{0\}$ are such that $\langle \nabla \varphi (x), u \rangle = 0$ then $x = 0$. Hence, with $z:= u \in \widehat{\partial }^2\varphi (0)(u)\cup -\widehat{\partial }^2\varphi (0)(-u)$ we have $\langle z, u \rangle = |u|^2 > 0$ and so (19) holds while $\varphi$ is not strictly pseudoconvex.

We now improve (19) to get another characterization for the strict pseudoconvexity.

Theorem 4.4

Let $\varphi : \mathbb {R}^n \rightarrow \mathbb {R}$ be a $\mathcal {C}^{1,1}$-smooth function satisfying

$$\begin{aligned} x \in \mathbb {R}^n, u \in \mathbb {R}^n{\setminus } \{0\}, \langle \nabla \varphi (x), u\rangle =0 \Longrightarrow \langle z,u \rangle >0 \; \text {for some} \; z \in \widehat{\partial }^2\varphi (x)(u). \end{aligned}$$

(22)

Then $\varphi$ is a strictly pseudoconvex function.

Proof

By Theorem 4.3, $\varphi$ is strictly quasiconvex. We will use Theorem 2.1 to prove the strict pseudoconvexity of $\varphi$. Let $x \in \mathbb {R}^n$ such that $\nabla \varphi (x)=0$. It follows from (6) and (22) that

$$\begin{aligned} \widehat{\partial }\langle u,\nabla \varphi \rangle (x)=\widehat{\partial }^2 \varphi (x)(u)\ne \emptyset \quad \text {and}\quad \widehat{\partial }\langle -u,\nabla \varphi \rangle (x)=\widehat{\partial }^2 \varphi (x)(-u)\ne \emptyset \end{aligned}$$

for every $u\in \mathbb {R}^n\setminus \{0\}$. By [20, Proposition 1.87], the scalar function $\langle u, \nabla \varphi \rangle$ is differentiable at x for every $u\in \mathbb {R}^n\setminus \{0\}$. Hence, $\varphi$ is twice differentiable at x and

$$\begin{aligned} \widehat{\partial }^2\varphi (x)(u)=\{\nabla \langle u, \nabla \varphi \rangle (x)\}=\{\nabla ^2\varphi (x)u\} \end{aligned}$$

for every $u\in \mathbb {R}^n\setminus \{0\}$. Again, by (22), its Hessian $\nabla ^2\varphi (x)$ is positive definite. Moreover, by [24, Theorem 13.2], the Hessian matrix $\nabla ^2\varphi (x)$ also furnishes a quadratic expansion for $\varphi$ at x. Therefore, since $\nabla ^2\varphi (x)$ is positive definite and $\nabla \varphi (x)=0$, it yields that x is a strict local minimizer of $\varphi$. By Theorem 2.1, $\varphi$ is strictly pseudoconvex. $\square$

Remark 4.2

According to the proof of Theorem 4.4, the condition (22) also implies that $\varphi$ is twice differentiable at its critical points.

In the two next examples, we will show that (22) and (15) are incomparable.

Example 4.3

Let $\varphi : \mathbb {R} \rightarrow \mathbb {\mathbb {R}}$ be the function defined by

$$\begin{aligned} \varphi (x) = {\left\{ \begin{array}{ll} \frac{1}{2}x^2 &{} \text {if}\quad x \le 0,\\ 3x^2 &{} \text {if}\quad x> 0. \end{array}\right. } \end{aligned}$$

Then, $\varphi$ is $\mathcal {C}^{1,1}$-smooth and

$$\begin{aligned} \nabla \varphi (x) = {\left\{ \begin{array}{ll} x &{} \text {if}\quad x \le 0,\\ 6x &{} \text {if}\quad x> 0. \end{array}\right. } \end{aligned}$$

Let $x, u \in \mathbb {R}$, $u \ne 0$ such that $\langle \nabla \varphi (x),u \rangle =0$. Then, $\nabla \varphi (x) =0$ and so $x = 0$. Clearly,

$$\begin{aligned} \widehat{\partial }^2 \varphi (0)(u) = {\left\{ \begin{array}{ll} [u,6u] &{} \text {if}\quad u \ge 0,\\ \emptyset &{} \text {if}\quad u< 0, \end{array}\right. } \quad \text {and} \quad {\partial }^2 \varphi (0)(u) = {\left\{ \begin{array}{ll} [u,6u] &{} \text {if}\quad u \ge 0,\\ \{u,6u\} &{} \text {if}\quad u< 0. \end{array}\right. } \end{aligned}$$

Hence, (15) holds while (22) is not satisfied.

Example 4.4

Let $\varphi : \mathbb {R} \rightarrow \mathbb {R}$ be the function defined by

$$\begin{aligned} \varphi (x):= \int _0^{x} {\phi (t)dt}, \end{aligned}$$

where $\phi : \mathbb {R} \rightarrow \mathbb {R}$ is given by

$$\begin{aligned} \phi (t) = {\left\{ \begin{array}{ll} \displaystyle \frac{1}{2\pi } &{} \text {if} \quad \displaystyle t\ge \frac{1}{\pi },\\ \displaystyle \frac{t}{2} + t^2 \sin \left( \frac{1}{t} \right) &{} \text {if}\quad \displaystyle 0<|t|<\frac{1}{\pi },\\ 0 &{} \text {if}\quad t=0.\\ \displaystyle -\frac{1}{2\pi } &{} \text {if} \quad \displaystyle t\le -\frac{1}{\pi }. \end{array}\right. } \end{aligned}$$

Since $\phi$ is locally Lipschitz, $\varphi$ is $\mathcal {C}^{1,1}$-smooth and $\nabla \varphi (x)=\phi (x)$ for every $x\in \mathbb {R}$. Moreover, $\varphi$ is twice differentiable everywhere except the points $\displaystyle \frac{1}{\pi }$ and $\displaystyle -\frac{1}{\pi }$. Let $x, u \in \mathbb {R}^n$, $u \ne 0$ such that $\langle \nabla \varphi (x),u \rangle =0$. Then $\nabla \varphi (x)=0$. We have

$$\begin{aligned} \left| {x\sin \frac{1}{x}} \right| = \left| x \right| \left| {\sin \frac{1}{x}} \right| \le \left| x \right| < \frac{1}{\pi }. \end{aligned}$$

when $-\frac{1}{\pi }<x < \frac{1}{\pi }$. Hence,

$$\begin{aligned} x \sin \frac{1}{x} + \frac{1}{2}> \frac{1}{2} - \frac{1}{\pi }>0, \quad \text {for all} \; \; x \in \left( { - \frac{1}{\pi },\frac{1}{\pi }} \right) . \end{aligned}$$

Therefore, $\nabla \varphi (x) = 0$ if and only if $x = 0$. Clearly,

$$\begin{aligned} \widehat{\partial }^2 \varphi (0)(u) = \left\{ {\frac{1}{2}u} \right\} \quad \text {and} \quad {\partial }^2 \varphi (0)(u) = {\left\{ \begin{array}{ll} \left[ -\frac{1}{2}u,\frac{3}{2}u\right] &{}\text {if}\quad u\ge 0,\\ \left[ \frac{3}{2}u,-\frac{1}{2}u\right] &{}\text {if}\quad u<0. \end{array}\right. } \end{aligned}$$

Hence, (22) holds while (15) is not satisfied.

5 Conclusions and further investigations

Several second-order necessary and sufficient conditions for the (strict) quasiconvexity and the (strict) pseudoconvexity of $\mathcal {C}^{1,1}$-smooth functions have been established on finite-dimensional Euclidean spaces. We also propose many examples to analyze and illustrate our results. Further investigations are needed to solve the following questions:

• 1. How to extend our results to wider classes of smooth and non-smooth functions on infinite-dimensional Hilbert or even Banach spaces?

• 2. How to apply our results to construct second-order necessary and sufficient conditions for nonlinear programming problems with non-convex and $\mathcal {C}^{1,1}$-smooth data?