Higher-order optimality conditions for strict and weak efficient solutions in set-valued optimization

Abstract

In this paper, we introduce a notion of higher-order Studniarski epiderivative of a set-valued map and study its properties. Then, we discuss their applications to optimality conditions in set-valued optimization. Higher-order optimality conditions for strict and weak efficient solutions of a constrained set-valued optimization problem are established. Some remarks on the existing results in the literature are given from our results.

1 Introduction

Set-valued optimization problems have been recently received much attention from mathematicians since many practical models involve set-valued maps. However, the idea of studying optimality conditions in terms of Gâteaux and Fréchet derivatives for smooth single-valued optimization problems still plays a crucial role for modern researches. Thus, several generalized derivatives have been proposed to replace classical derivatives in set-valued optimization. Most of them are based on the graph and epigraph of kinds of tangency (local linear approximations of a set). Let A be a subset of a normed space X. The earliest tangency notion is the contingent cone $T_A(x_0)$ of A at $x_0 \in \mathrm{cl}A$ (the closure of A), defined respectively (shortly resp) as, see [3],

$$\begin{aligned} T_A(x_0) := \{u \in X| \exists t_{n} \rightarrow 0^{+}, \exists u_{n} \rightarrow u, x_0 + t_{n}u_{n} \in A\}. \end{aligned}$$

For higher-order contingent set, with $u_1,\ldots ,u_{m - 1}\in X$, we get

$$\begin{aligned}&T_A^m(x_0,u_1,\ldots ,u_{m-1})\\&\quad := \{u \in X| \exists t_{n} \rightarrow 0^{+}, \exists u_{n} \rightarrow u, x_0 + t_nu_1 + \cdots + t_n^{m-1}u_{m-1} + t_{n}^mu_{n} \in A\}. \end{aligned}$$

Let a set-valued map $F:X\rightarrow 2^Y$ (Y is also a normed space), the corresponding higher-order contingent derivative of F at $(x_0,y_0)\in \mathrm{gr}F$ with respect to $(u_1,v_1),\ldots ,(u_{m-1},v_{m-1})$ $\in X\times Y$ is the set-valued map $D^m F(x_0,y_0,u_1,v_1,\ldots ,u_{m-1},v_{m-1}):X\rightarrow 2^Y$, defined by its graph:

$$\begin{aligned} \mathrm{gr}D^mF(x_0,y_0,u_1,v_1,\ldots ,u_{m-1},v_{m-1}):=T^m_{\mathrm{gr}F}(x_0,y_0,u_1,v_1,\ldots ,u_{m-1},v_{m-1}). \end{aligned}$$

Inspired by [14], another higher-order contingent derivative, called the Studniarski derivative, was proposed by

$$\begin{aligned}&D^mF(x_0,y_0)(u)\\&\quad :=\{v\in Y| \exists t_n\rightarrow 0^+, \exists (u_n,v_n)\rightarrow (u,v), y_0 + t_n^mv_n\in F(x_0+t_nu_n)\}. \end{aligned}$$

A direction of the higher-order Studniarski derivative does not depend on lower-order directions. Some applications of this derivative in nonsmooth optimization were mentioned in [1, 2, 9, 13, 15].

On the other hand, it is well-known that, considering minimization problems, epigraphs play a vital role. They are even more important than the more classic and basic geometrical notion of graph. Therefore, epiderivatives based on epigraphs, in a similar manner as the contingent derivative is based on graphs, have certain advantages over other kinds of derivatives, see [3] for epiderivatives of extended-real valued functions, and [4, 8] for that of set-valued maps. For higher-order epiderivatives and applications to set-valued optimization, the reader is referred to [6, 12, 16, 17]. All these higher-order epiderivatives are defined upon informations of lower-order approximating directions.

Since higher-order considerations are of great importance in mathematics (especially in optimality conditions), motivated by [14], we propose a notion of higher-order Studniarski epiderivative in the paper. Then, we apply this object to optimality conditions in set-valued optimization.

The lay-out of this paper is as follows. In Sect. 2, we recall some notions and preliminaries needed for our results. In Sect. 3, the Studniarski epiderivative of a set-valued map is introduced and its properties are discussed. In Sect. 4, we establish higher-order optimality conditions for weak efficient solutions and strict efficient solutions of a generalized set-valued optimization problem in terms of this epiderivative. From our results, some remarks on the existing ones in the literature are given.

2 Preliminaries

Throughout the paper, let X, Y be normed spaces and $C \subseteq Y$ be a pointed closed convex cone. $B_X(x,\delta )$ stands for the closed ball with radius $\delta > 0$ and centered at $x\in X$ and $\mathcal {U}(x)$ for the set of all neighborhoods of x. For $A \subseteq Y$, $\mathrm{int }A$ and $\mathrm{cl} A$ denote the interior and closure of A, resp. $Y^*$ is used for the dual space of Y and $\langle {.,.}\rangle$ for the canonical pairing. With the cone C and the subset A above, we use the following cones

$$\begin{aligned} \begin{array}{c} \displaystyle \mathrm{cone} A := \{\lambda a | \lambda \ge 0,\, a\in A\},\quad A(u):= \mathrm{cone}(A+u)\; \mathrm{for}\;u\in Y,\\ \displaystyle C^*:=\{y^*\in Y^*|\langle {y^*,c}\rangle \ge 0,\; \forall c\in C\}. \end{array} \end{aligned}$$

A nonempty convex subset B of the cone C is said to be a base of C if $C=\mathrm{cone}B$ and $0\not \in \mathrm{cl}B$. For $A\subseteq Y$, $y_0\in A$ is an efficient point of A ($y_0\in \mathrm{Min}_C A$) if $(A-y_0)\cap (-C{\setminus }\{0\})=\emptyset$. If $\mathrm{int}C\ne \emptyset$, then $y_0\in A$ is a weak efficient point of A ($y_0 \in \mathrm{WMin}_C A$) if $(A-y_0)\cap (-\mathrm{int}C)=\emptyset$.

Let $S\subseteq X$, the domain, graph, and epigraph of a set-valued map $F:S\rightarrow 2^Y$ are defined by, resp,

$$\begin{aligned}&domF := \{x \in X| F(x) \ne \emptyset \}, grF := \{(x, y) \in X\times Y| y \in F(x)\},\\&epiF := \{ (x, y) \in X\times Y| y \in F(x) + C\}. \end{aligned}$$

Definition 2.1

Let $F:S\rightarrow 2^Y$, $(x_0,y_0)\in \mathrm{gr}F$, and an integer $m\ge 1$.

• (i) The point $(x_0,y_0)$ is said to be a local efficient solution of F on S if there exists $U\in \mathcal {U}(x_0)$ such that $y_0\in \mathrm{Min}_C F(S\cap U)$.

• (ii) Suppose that $\mathrm{int}C\ne \emptyset$, the point $(x_0,y_0)$ is said to be a local weak efficient solution of F on S if there exists $U\in \mathcal {U}(x_0)$ such that $y_0\in \mathrm{WMin}_C F(S\cap U)$.

• (iii) [7] The point $(x_0,y_0)$ is said to be a local strict efficient solution of order m of F on S if $y_0\in \mathrm{Min}_C F(x_0)$ and there exist $\alpha > 0$, $U\in \mathcal {U}(x_0)$ such that for all $x\in (S\cap U){\setminus } \{x_0\}$,

  $$\begin{aligned} (F(x)+C)\cap B_{Y}(y_0,\alpha ||x-x_0||^m)=\emptyset . \end{aligned}$$

  The set of local strict efficient solutions of order m of F on S is denoted by $m{\text{- }}\mathrm{Str}_CF(S\cap U)$. If $U=X$, then we get corresponding definitions for global solutions. It is obvious to see that $m{\text{- }}\mathrm{Str}_CF(S)\subseteq \mathrm{Min}_CF(S)\subseteq \mathrm{WMin}_CF(S)$. The following example shows a case where the above inclusions may be strict.

Example 2.1

Let $X=\mathbb {R}$, $Y=\mathbb {R}^2$, $C=\mathbb {R}^2_+$, $S=\{0,1/n\}_{n\in \mathbb {N}}$, and $F: S\rightarrow 2^Y$ be defined by

$$\begin{aligned} F(x):=\left\{ \begin{array}{l@{\quad }l} \{(y_1,y_2)\in Y| y_1^2 + y_2^2 \le 2\} + C,&{}\mathrm{if}\; x=0,\\ \{(y_1,y_2)\in Y| y_1^2 + y_2^2 \le 2, y_1 + y_2 = -2 + 2x^2\}+C,&{}\mathrm{if}\; x\in \{1/n\}_{n\in \mathbb {N}}. \end{array}\right. \end{aligned}$$

By calculating, we get

$$\begin{aligned}&2{\text{- }}\mathrm{Str}_CF(S)=\{(-1,-1)\},\\&\quad \mathrm{Min}_CF(S)=\{(y_1,y_2)\in Y|y^2_1 + y^2_2 =2, y_1\le 0, y_2\le 0\}, \end{aligned}$$

$$\begin{aligned} \mathrm{WMin}_CF(S)= & {} \{(y_1,y_2)\in Y|y^2_1 + y^2_2 =2, y_1\le 0, y_2\le 0\}\\&\cup \,\{(y_1,y_2)\in Y| y_1\ge 0, y_2 = -\sqrt{2}\}\\&\cup \, \{(y_1,y_2)\in Y| y_1=-\sqrt{2}, y_2\ge 0\}. \end{aligned}$$

Thus,

$$\begin{aligned} 2{\text{- }}\mathrm{Str}_CF(S)\varsubsetneq \mathrm{Min}_CF(S)\varsubsetneq \mathrm{WMin}_CF(S). \end{aligned}$$

Definition 2.2

Let $S\subseteq X$ and $F:S\rightarrow 2^Y$.

• (i) The set S is said to be convex if for all $x_1,x_2\in S$, $\lambda \in [0,1]$,

  $$\begin{aligned} \lambda x_1 + (1-\lambda )x_2\in S. \end{aligned}$$

• (ii) [18, 19] The set S is said to be invex if there exists $\eta : X\times X \rightarrow X$ such that for all $x_1,x_2\in S$, $\lambda \in [0,1]$,

  $$\begin{aligned} x_2 + \lambda \eta (x_1,x_2)\in S. \end{aligned}$$

• (iii) The map F is said to be C-convex on a convex subset S if for all $x_1,x_2\in S$, $\lambda \in [0,1]$,

  $$\begin{aligned} \lambda F(x_1) + (1-\lambda )F(x_2)\subseteq F(\lambda x_1 + (1-\lambda )x_2) + C. \end{aligned}$$

• (iv) [5] The map F is said to be C-preinvex with respect to $\eta$ on S if for all $x_1,x_2\in S$, $\lambda \in [0,1]$,

  $$\begin{aligned} \lambda F(x_1) + (1-\lambda )F(x_2)\subseteq F(x_2 + \lambda \eta (x_1,x_2)) + C. \end{aligned}$$

• (v) [20] The map F is said to be nearly C-subconvexlike if $\mathrm{cl}\mathrm{cone}(F(S)+C)$ is convex.

Remark 2.1

• (i) The convexity of S (C-convexity of F) is a special case of the invexity of S (C-preinvexity of F, resp) with $\eta (x,y)=x-y$.

• (ii) If F is C-convex, then F is nearly C-subconvexlike.

• (iii) In general, the preinvexness is incomparable with the near subconvexlikeness, see Remark 2.1(iii) in [11].

3 Higher-order Studniarski epiderivative

Definition 3.1

• (i) The mth-order Studniarski set of $A\subseteq X\times Y$ at $(x_0,y_0)\in \mathrm{cl}A$ is defined by

  $$\begin{aligned} S^{m}_A(x_0,y_0):= & {} \{(u,v)\in X\times Y| \exists t_n\rightarrow 0^+, \exists (u_n,v_n)\rightarrow (u,v),\\&(x_0+t_nu_n,y_0+t_n^mv_n)\in A\}. \end{aligned}$$

• (ii) For $F:X\rightarrow 2^Y$ and $(x_0,y_0)\in \mathrm{gr}F$, a single-valued map $ED^{m}F(x_0,y_0): X\rightarrow Y$ whose epigraph equals the mth-order Studniarski set of the epigraph of F at $(x_0,y_0)$, i.e.,

  $$\begin{aligned} \mathrm{epi}ED^{m}F(x_0,y_0)=S^{m}_{\mathrm{epi}F}(x_0,y_0), \end{aligned}$$

  is called the mth-order Studniarski epiderivative of F at $(x_0,y_0)$.

Remark 3.1

• (i) If the mth-order Studniarski epiderivative exists, then it is unique. The 1st-order Studniarski set and the 1st-order Studniarski epiderivative coincide with the contingent cone and the contingent epiderivative, resp.

• (ii) For comparison results (e.g., Proposition 3.1 below), recall that the mth-order Studniarski derivative of F at $(x_0,y_0)$, see [1], is the set-valued map $D^{m}F(x_0,y_0): X\rightarrow 2^Y$ such that $\mathrm{gr}D^{m}F(x_0,y_0)=S^{m}_{\mathrm{gr}F}(x_0,y_0)$. Equivalently, for all $u\in X$,

  $$\begin{aligned}&{D}^{m}F(x_0,y_0)(u)\\&\quad :=\{v\in Y|\; \exists t_n\rightarrow 0^+, \exists (u_n,v_n)\rightarrow (u,v), y_0+t_n^mv_n\in F(x_0+t_nu_n)\}. \end{aligned}$$

  The following definition will be necessary in the sequel.

Definition 3.2

For $u\in X$, $F:X \rightarrow 2^Y$ is called mth-order u-directionally compact at $(x_0,y_0)\in \mathrm{gr}F$ if, for every $t_n\rightarrow 0^+$ and $u_n \rightarrow u$, any sequence $v_n$, with $y_0 + t_n^m v_n \in F(x_0 + t_nu_n)$, contains a convergent subsequence. If this is satisfied for every $u\in X$, then “u-directionally” is replaced by “directionally”.

To get a basic relation between $ED^{m}F(x_0,y_0)$ and ${D}^{m}F(x_0,y_0)$, we need the following property of the latter.

Lemma 3.1

Let $F:X \rightarrow 2^Y$ be mth-order u-directionally compact at $(x_0,y_0)\in \mathrm{gr}F$. Then,

$$\begin{aligned} {D}^{m}(F+C)(x_0,y_0)(u)={D}^{m}F(x_0,y_0)(u)+C, \end{aligned}$$

where $(F+C)(.):=F(.)+C$.

Proof

“$\supseteq$”: Let $v=y+c$ for some $y\in {D}^{m}F(x_0,y_0)(u)$ and $c\in C$. Then, there exist $t_n \rightarrow 0^+$, $y_n\rightarrow y$, and $u_n\rightarrow u$ such that, for all n,

$$\begin{aligned} y_0+t_n^m(y_n + c) \in F(x_0 + t_nu_n) + C =(F+C)(x_0 + t_nu_n). \end{aligned}$$

Because $y_n + c \rightarrow y + c$, $v\in {D}^{m}(F+C)(x_0,y_0)(u)$.

“$\subseteq$”: Let $v\in {D}^{m}(F+C)(x_0,y_0)(u)$, i.e., there exist $t_n \rightarrow 0^+$, $(u_n,v_n)\rightarrow (u,v)$, and $c_n\in C$ such that, for all n,

$$\begin{aligned} y_0+t_n^m\left( v_n - \dfrac{c_n}{t_{n}^m}\right) \in F(x_0 + t_nu_n). \end{aligned}$$

By the assumed compactness, $v_n -c_n/t_{n}^m$ (or a subsequence) converges to some y. Hence, $y \in {D}^{m}F(x_0,y_0)(u)$ and $c_n/t_{n}^m \rightarrow v-y\in C$. Thus, $v\in {D}^{m}F(x_0,y_0)(u) + C$. $\square$

Proposition 3.1

Let $F : X\rightarrow 2^Y$ be mth-order directionally compact at $(x_0,y_0)\in \mathrm{gr}F$ and $ED^{m}F(x_0,y_0)$ exist. Then, $\mathrm{dom}(ED^{m}F(x_0,y_0))=\mathrm{dom}({D}^{m}F(x_0,y_0))$ and for every $u\in \mathrm{dom}(ED^{m}F(x_0,y_0))$,

$$\begin{aligned} ED^{m}F(x_0,y_0)(u)= \mathrm{Min}{D}^{m}F(x_0,y_0)(u). \end{aligned}$$

Proof

It follows from Definition 3.1(ii) and Remark 3.1(ii) that

$$\begin{aligned} \mathrm{epi}(ED^{m}F(x_0,y_0)) = S^m_{\mathrm{epi}F}(x_0, y_0) = \mathrm{gr}(D^m(F + C)(x_0, y_0)). \end{aligned}$$

This means that, for $u \in \mathrm{dom}(D^m(F + C)(x_0, y_0))$,

$$\begin{aligned} ED^{m}F(x_0,y_0)(u) + C = D^m(F + C)(x_0, y_0)(u). \end{aligned}$$

Hence,

$$\begin{aligned} ED^{m}F(x_0,y_0)(u)= & {} \mathrm{Min}(ED^{m}F(x_0,y_0)(u) + C) \\= & {} \mathrm{Min}D^m(F + C)(x_0, y_0)(u)\\= & {} \mathrm{Min} (D^mF(x_0, y_0)(u) + C)\;\mathrm{(Lemma\; 3.1)}\\= & {} \mathrm{Min}D^mF(x_0, y_0)(u). \end{aligned}$$

$\square$

Example 3.1

Let $X=Y=\mathbb {R}$, $C=\mathbb {R}_+$, $F(x) := \{y\in Y| y\ge x^2\}$, and $(x_0,y_0)=(0,0)$. It is easy to check that F is 2nd-order directionally compact at $(x_0, y_0)$. Direct calculations yield, for all $u\in X$,

$$\begin{aligned} ED^{2}F(x_0,y_0)(u)= \mathrm{Min}{D}^{2}F(x_0,y_0)(u)=\{u^2\}. \end{aligned}$$

The compactness assumption in Proposition 3.1 is a sufficient condition, but not necessary, as shown by the following example.

Example 3.2

Let $X = \mathbb {R}$, $Y = \mathbb {R}^2$, $C=\mathbb {R}_+^2$, $F(x) \equiv \mathbb {R}_+^2$, and $(x_0,y_0) = (0,(0,0))$. For all $m\ge 1$, F is not mth-order directionally compact at $(x_0, y_0)$ because, for any $u\in X$ and sequence $u_n\rightarrow u$, by choosing $t_n =1/n$, $y_n = (n,n)$, we have $y_0 +t_n^my_n \in F(x_0 + t_nu_n)$ for all n. But, $y_n$ does not contain any convergent subsequence. By calculating, one has, for all $u\in X$, ${D}^{m}F(x_0,y_0)(u) = \mathbb {R}_+^2$ and

$$\begin{aligned} ED^{m}F(x_0,y_0)(u)= \mathrm{Min}{D}^{m}F(x_0,y_0)(u)=\{(0,0)\}. \end{aligned}$$

The next proposition gives us an existence condition (inspired by [10]) for higher-order Studniarski epiderivative in the case of $Y=\mathbb {R}$.

Proposition 3.2

For $F :X \rightarrow 2^{\mathbb {R}}$ and $(x_0,y_0)\in \mathrm{gr}F$, assume that there are functions $f, g:X \rightarrow \mathbb {R}$ with $\mathrm{epi}g\subseteq S^{m}_{\mathrm{epi}F}(x_0,y_0)\subseteq \mathrm{epi}f$. Then, $ED^{m}F(x_0, y_0)$ is explicitly expressed as, for $x\in X$,

$$\begin{aligned} ED^{m}F(x_0, y_0)(x) = \mathrm{min}\{y \in \mathbb {R}| (x, y) \in S^{m}_{\mathrm{epi}F} (x_0, y_0)\}. \end{aligned}$$

(1)

Proof

Let $h :X \rightarrow \mathbb {R}\cup \{-\infty \}$ be defined by, for $x\in X$,

$$\begin{aligned} h(x) = \mathrm{inf}\{y \in \mathbb {R}| (x, y) \in S^{m}_{\mathrm{epi}F}(x_0, y_0)\}. \end{aligned}$$

The function h is well-defined on X since for every $x \in X$, there exists $y \in \mathbb {R}$ with $(x, y)\in S^{m}_{\mathrm{epi}F}(x_0,y_0)$ (by $\mathrm{epi}g\subseteq S^{m}_{\mathrm{epi}F}(x_0,y_0)$). We now claim that

$$\begin{aligned} h(x) = \mathrm{min}\{y \in \mathbb {R}| (x, y) \in S^{m}_{\mathrm{epi}F}(x_0, y_0)\}. \end{aligned}$$

(2)

In fact, for $x \in X$, there is an infimal sequence $y_n$ such that $y_n \rightarrow h(x)$ and $(x, y_n)\in S^{m}_{\mathrm{epi}F}(x_0,y_0)$. Thus, $(x,h(x))\in S^{m}_{\mathrm{epi}F}(x_0,y_0)$ since the mth-order Studniarski set is closed. By assumption, $-\infty < f(x)\le h(x)$, and hence (2) holds. Next, we prove that $\mathrm{epi}h= S^{m}_{\mathrm{epi}F}(x_0, y_0)$. Let $(x,\alpha )\in S^{m}_{\mathrm{epi}F}(x_0, y_0)$, it follows from (2) that $(x,\alpha )\in \mathrm{epi}h$. For the reverse inclusion, take $(x,\alpha )\in \mathrm{epi}h$. Because $(x,h(x))\in S^{m}_{\mathrm{epi}F}(x_0,y_0)$, there exist $t_n\rightarrow 0^+$ and $(x_n,y_n)\rightarrow (x,h(x))$ such that $y_0 + t_n^my_n\in F(x_0+t_nx_n)+\mathbb {R}_+$. Therefore,

$$\begin{aligned}&y_0+t_n^m(y_n + \alpha - h(x))\in F(x_0+t_nx_n) + t_n^m(\alpha - h(x))\\&\quad +\mathbb {R}_+\subseteq F(x_0+t_nx_n) +\mathbb {R}_+. \end{aligned}$$

By setting $(\overline{x}_n,\overline{y}_n)=(x_n, y_n + \alpha - h(x))$, which tends to $(x, \alpha )$, we get

$$\begin{aligned} (x_0+t_n\overline{x}_n,y_0+t_n^m\overline{y}_n)\in \mathrm{epi}F, \end{aligned}$$

i.e., $(x,\alpha )\in S^{m}_{\mathrm{epi}F}(x_0, y_0)$. Hence, h is the mth-order Studniarski epiderivative of F at $(x_0,y_0)$ and it follows from the uniqueness of $ED^{m}F(x_0, y_0)$ that (1) is satisfied. $\square$

The following proposition collects some properties of $ED_R^{m}F(x_0,y_0)$.

Proposition 3.3

Let $F:X\rightarrow 2^Y$ and $(x_0,y_0)\in \mathrm{gr} F$. Then,

• (i) If $ED^{m}F(x_0,y_0)$ exists, then $ED^mF(x_0,y_0)(0)=0$.

• (ii) Suppose that $S^m_{\mathrm{epi}F}(x_0,y_0)$ is a cone. If $ED^{m}F(x_0,y_0)$ exists, then it is positively homogeneous. If, additionally, $S^m_{\mathrm{epi}F}(x_0,y_0)$ is convex, then $ED^{m}F(x_0,y_0)$ is subadditive.

• (iii) If $Y=\mathbb {R}$ and $ED^{m}F(x_0, y_0)$ is expressed by (1), then this derivative is mth-order positively homogeneous. $\square$

Proof

• (i) Since $(0,ED^mF(x_0,y_0)(0))\in \mathrm{epi}ED^mF(x_0,y_0)$, then for all $t>0$,

  $$\begin{aligned} (t\cdot 0,t^m ED^mF(x_0,y_0)(0))\in \mathrm{epi}ED^mF(x_0,y_0), \end{aligned}$$

  i.e.,

  $$\begin{aligned} t^mED^mF(x_0,y_0)(0)\in ED^mF(x_0,y_0)(0)+C, \end{aligned}$$

  which implies that $(t^m-1)ED^mF(x_0,y_0)(0)\in C$ for all $t>0$. Take $t=2$ and $t=1/2$, then $ED^mF(x_0,y_0)(0)\in C\cap (-C)$. Since C is pointed ($C\cap (-C)=\{0\}$), we get that $ED^mF(x_0,y_0)(0)=0.$

• (ii) Let $t>0$ and $x\in X$. Since $(x,ED^mF(x_0,y_0)(x))\in \mathrm{epi}ED^mF(x_0,y_0)$ and $S^m_{\mathrm{epi}F}(x_0,y_0)$ is a cone, we have

  $$\begin{aligned} (tx,tED^mF(x_0,y_0)(x))\in \mathrm{epi}ED^mF(x_0,y_0) = S^m_{\mathrm{epi}F}(x_0,y_0), \end{aligned}$$

  i.e.,

  $$\begin{aligned} tED^mF(x_0,y_0)(x)\in ED^mF(x_0,y_0)(tx) + C, \end{aligned}$$

  which implies

  $$\begin{aligned} tED^mF(x_0,y_0)(x) - ED^mF(x_0,y_0)(tx) \in C. \end{aligned}$$

  (3)

  Moreover, since $(tx,ED^mF(x_0,y_0)(tx))\in \mathrm{epi}ED^mF(x_0,y_0)$, we get

  $$\begin{aligned} \left( x,\dfrac{1}{t}ED^mF(x_0,y_0)(tx)\right) \in \mathrm{epi}ED^mF(x_0,y_0), \end{aligned}$$

  which means

  $$\begin{aligned} \dfrac{1}{t}ED^mF(x_0,y_0)(tx)\in ED^mF(x_0,y_0)(x) + C, \end{aligned}$$

  equivalently,

  $$\begin{aligned} tED^mF(x_0,y_0)(x) - ED^mF(x_0,y_0)(tx) \in -C. \end{aligned}$$

  (4)

  Since C is pointed, it follows from (3) and (4) that for all $t>0$, $x\in X$,

  $$\begin{aligned} tED^mF(x_0,y_0)(x) = ED^mF(x_0,y_0)(tx). \end{aligned}$$

  On the other hand, from (i), it follows that $ED^mF(x_0,y_0)(0)=0$. Hence, $ED^mF(x_0,y_0)$ is positively homogeneous.

  Next, we prove the subadditivity of $ED^mF(x_0,y_0)$. Let $x_1, x_2\in X$, since $S^m_{\mathrm{epi}F}(x_0,y_0)$ is convex and $(x_i,ED^mF(x_0,y_0)(x_i)){\,\in \,}\mathrm{epi}ED^mF(x_0,y_0)$, $i=1,2$, we get

  $$\begin{aligned}&\left( {\dfrac{1}{2}x_1+\dfrac{1}{2}x_2, \dfrac{1}{2}ED^mF(x_0,y_0)(x_1)+\dfrac{1}{2}ED^mF(x_0,y_0)(x_2)}\right) \\&\quad \in \mathrm{epi}ED^mF(x_0,y_0), \end{aligned}$$

  i.e.,

  $$\begin{aligned}&\dfrac{1}{2}(ED^mF(x_0,y_0)(x_1)+ ED^mF(x_0,y_0)(x_2))\\&\quad \in ED^mF(x_0,y_0)\left( \dfrac{1}{2}(x_1 + x_2)\right) + C\\&\quad \in \dfrac{1}{2}ED^mF(x_0,y_0)(x_1+x_2) +C. \end{aligned}$$

  Thus,

  $$\begin{aligned} ED^mF(x_0,y_0)(x_1)+ ED^mF(x_0,y_0)(x_2)\in ED^mF(x_0,y_0)(x_1+x_2) +C, \end{aligned}$$

  which means that $ED^mF(x_0,y_0)$ is subadditive.

• (iii) For a general $F : X\rightarrow 2^Y$, observe that, for $t>0$,

  $$\begin{aligned} (tx,y)\in S^{m}_{\mathrm{epi}F}(x_0,y_0) \Longleftrightarrow \left( x, \dfrac{y}{t^m}\right) \in S^{m}_{\mathrm{epi}F}(x_0,y_0). \end{aligned}$$

  Therefore, by (1),

  $$\begin{aligned} ED^{m}F(x_0, y_0)(tx) = \mathrm{min}\left\{ y\in \mathbb {R}\big |\left( x,\dfrac{y}{t^m}\right) \in S^m_{\mathrm{epi}F}(x_0,y_0)\right\} . \end{aligned}$$

  Set $z:=y/t^m$, we obtain

  $$\begin{aligned} ED^{m}F(x_0, y_0)(tx)= & {} t^m\mathrm{min}\{z\in \mathbb {R}|(x,z)\in S^m_{\mathrm{epi}F}(x_0,y_0)\}\\= & {} t^mED^{m}F(x_0, y_0)(x), \end{aligned}$$

  i.e., $ED^mF(x_0,y_0)$ is mth-order positively homogeneous.

Definition 3.3

Let $F:X\rightarrow 2^Y$, $(x_0,y_0)\in \mathrm{gr}F$, and $ED^{m}F(x_0,y_0)$ exists. The map F is said to have a mth-order radial-Studniarski epiderivative at $(x_0,y_0)$ if

$$\begin{aligned} \mathrm{epi}ED^mF(x_0,y_0)= & {} \{(u,v)\in X\times Y| \exists t_n >0, \exists (u_n,v_n)\rightarrow (u,v),\\&(x_0+t_nu_n,y_0+t_n^mv_n)\in \mathrm{epi}F\} \end{aligned}$$

Proposition 3.4

Let $F:S\rightarrow 2^Y$, $(x_0,y_0)\in \mathrm{gr}F$, and $ED^{m}F(x_0,y_0)$ exists. Suppose that F is C-preinvex with respect to $\eta$ on S. Then, for all $x\in X$,

• (i) $F(x)-y_0\subseteq ED^1F(x_0,y_0)(\eta (x,x_0)) + C$.

• (ii) For $m\ge 2$, if F has a mth-order radial-Studniarski epiderivative at $(x_0,y_0)$, then

  $$\begin{aligned} F(x)-y_0\subseteq ED^mF(x_0,y_0)(\eta (x,x_0)) + C. \end{aligned}$$

Proof

• (i) Let $(x,y)\in \mathrm{gr}F$. Since F is C-preinvex with respect to $\eta$ on S, for all $x\in S$, $\lambda \in [0,1]$,

  $$\begin{aligned} (1-\lambda )F(x_0) + \lambda F(x)\subseteq F(x_0 + \lambda \eta (x,x_0))+C, \end{aligned}$$

  which implies that

  $$\begin{aligned} y-y_0\in \dfrac{F(x_0+\lambda \eta (x,x_0))+C-y_0}{\lambda }. \end{aligned}$$

  (5)

  With an arbitrary sequence $t_n\rightarrow 0^+$, ones have $t_n\in [0,1]$ for n large enough. Since (5) is fulfilled for all $\lambda \in [0,1]$, then for n large enough,

  $$\begin{aligned} y-y_0\in \dfrac{F(x_0+ t_n \eta (x,x_0))+C-y_0}{t_n}. \end{aligned}$$

  By setting $u_n:=\eta (x,x_0)$ ($u_n\rightarrow \eta (x,x_0)$), $v_n:=y-y_0$ ($v_n\rightarrow y-y_0$) for n large enough, it follows that $y_0 + t_nv_n\in F(x_0 + t_nu_n) + C$. Hence, $y-y_0\in ED^1F(x_0,y_0)(\eta (x,x_0))+C$.

• (ii) Let any $(x,y)\in \mathrm{gr}F$, it follows from (5) with $\lambda = 1$ that

  $$\begin{aligned} y-y_0\in F(x_0+ \eta (x,x_0))+C-y_0. \end{aligned}$$

  Then, there exist $t_n:=1$, $u_n:=\eta (x,x_0)$ ($u_n\rightarrow \eta (x,x_0)$), and $v_n:=y-y_0$ ($v_n\rightarrow y-y_0$) for all n such that $y_0+ t_n^mv_n \in F(x_0+t_nu_n) + C$. Since F has a mth-order radial-Studniarski derivative at $(x_0,y_0)$, we get that $(\eta (x,x_0),y-y_0)\in \mathrm{epi}ED^mF(x_0,y_0)$, i.e., $y-y_0\in ED^mF(x_0,y_0)(\eta (x,x_0))$. $\square$

4 Applications of higher-order Studniarski epiderivatives

Let X, Y, Z be normed spaces, and $C\subseteq Y$, $D\subseteq Z$ be closed pointed convex cones with $\mathrm{int}(C\times D)\ne \emptyset$. We consider the following constrained set-valued optimization problem

$$\begin{aligned} \mathrm{(SOP)}\left\{ \begin{array}{l} \mathrm{Min}\;F(x),\\ \mathrm{s.t.}\; x\in S,\\ G(x)\cap (-D)\ne \emptyset , \end{array}\right. \end{aligned}$$

where $S\subseteq X$, $F:X\rightarrow 2^Y$, and $G:X\rightarrow 2^Z$ with $\mathrm{dom}F\cup \mathrm{dom}G\subseteq S$. Then, $A:=\{x\in S| G(x)\cap (-D)\ne \emptyset \}$ denotes the feasible solution set of (SOP).

A point $(x_0,y_0)\in \mathrm{gr}F$ is said to be a local weak efficient solution (efficient solution, strict efficient solution of order m) of (SOP) if $x_0\in A$ and there exists $U\in \mathcal {U}(x_0)$: $y_0\in \mathrm{WMin}_CF(A\cap U)$ ($y_0\in \mathrm{Min}_CF(A\cap U)$, $y_0\in m{\text{- }}\mathrm{Str}_CF(A\cap U)$, resp).

We need the following definitions for our results:

• A subset $S\subseteq X$ is invex near $x_0$ if there exists $U\in \mathcal {U}(x_0)$ such that $S\cap \overline{U}$ is invex for all $\overline{U}\in \mathcal {U}(x_0): \overline{U}\subseteq U$.

• A map $F:S\rightarrow 2^Y$ is nearly C-subconvexlike on S near $x_0\in S$ (C-preinvex with respect to $\eta$ on S near $x_0$) if there exists $U\in \mathcal {U}(x_0)$ such that F is nearly C-subconvexlike on $S\cap \overline{U}$ (C-preinvex with respect to $\eta$ on $S\cap \overline{U}$, resp) for all $\overline{U}\in \mathcal {U}(x_0)$: $\overline{U}\subseteq U$.

Let $(x_0,y_0)\in \mathrm{gr}F$, $z_0\in G(x_0)\cap (-D)$. We assume that $ED^m(F,G)(x_0,(y_0,z_0))$ exists and set $\Omega :=\mathrm{dom}ED^m(F,G)(x_0,(y_0,z_0))$. Firstly, necessary conditions in Fritz–John and Kuhn–Tucker types for weak efficient solutions of (SOP) are given as follows.

Theorem 4.1

Suppose that $(x_0,y_0)$ is a local weak efficient solution of (SOP) and either of the following conditions is satisfied

• (i) $(F-y_0,G)$ is nearly $(C\times D)$-subconvexlike on S near $x_0$, where $(F-y_0)(.):=F(.)-y_0$;

• (ii) S is invex near $x_0$ and (F, G) is $(C\times D)$-preinvex with respect to $\eta$ on S near $x_0$.

Then, there exists $(c^*,d^*)\in (C^*\times D^*){\setminus }\{(0,0)\}$ such that for all $(y,z)\in ED^m(F,G)(x_0,$ $(y_0,z_0))(\Omega )$,

$$\begin{aligned} \langle {c^*,y}\rangle + \langle {d^*,z}\rangle \ge 0, \end{aligned}$$

(6)

and

$$\begin{aligned} \langle {d^*,z_0}\rangle =0. \end{aligned}$$

(7)

If, additionally, for all $U\in \mathcal {U}(x_0)$, there exists $(\overline{x},\overline{z})\in \mathrm{gr}G$: $\overline{x}\in S\,\cap \, U$ and $\langle {d^*,\overline{z}}\rangle < 0$, then $c^*\ne 0$.

Proof

Since $(x_0,y_0)$ is a local weak efficient solution of (SOP), then there exists $U\in \mathcal {U}(x_0)$ such that

$$\begin{aligned} ((F,G)(S\cap U)-(y_0,0))\cap (-\mathrm{int}(C\times D))=\emptyset . \end{aligned}$$

(8)

Indeed, suppose to the contrary, i.e., for every $U\in \mathcal {U}(x_0)$, there are $x\in S\cap U$ and $(y,z)\in (F,G)(x)$ with $(y-y_0,z)\in -\mathrm{int}(C\times D)$, which implies that $G(S\cap U)\cap -D\ne \emptyset$, i.e., $x\in A\cap U$. Thus, $y_0\not \in \mathrm{WMin_CF(A\cap U)}$ (since $y-y_0\in -\mathrm{int C}$), which contradicts the local weak efficiency of $(x_0,y_0)$. Consequently, it follows from (8) that

$$\begin{aligned} ((F,G)(S\cap U)+C\times D-(y_0,0))\cap (-\mathrm{int}(C\times D))=\emptyset . \end{aligned}$$

(9)

$\bullet$ If the condition (i) holds, there is $\overline{U}\in \mathcal {U}(x_0)$ such that $\mathrm{cl}\mathrm{cone}((F,G)(S\cap \hat{U})+C\times D -(y_0,0))$ is convex for all $\hat{U}\in \mathcal {U}(x_0)$: $\hat{U}\subseteq \overline{U}$. Set $\hat{U}:=\overline{U}\cap U \in \mathcal {U}(x_0)$, it follows from (9) that

$$\begin{aligned} ((F,G)(S\cap \hat{U})+C\times D-(y_0,0))\cap (-\mathrm{int}(C\times D))=\emptyset , \end{aligned}$$

thus

$$\begin{aligned} \mathrm{cl}\mathrm{cone}((F,G)(S\cap \hat{U})+C\times D -(y_0,0))\cap (-\mathrm{int}(C\times D))=\emptyset , \end{aligned}$$

i.e., we can separate $\mathrm{cl}\mathrm{cone}((F,G)(S\cap \hat{U})+C\times D -(y_0,0))$ and $-\mathrm{int}(C\times D)$.

$\bullet$ If the condition (ii) is satisfied, we have $\widetilde{U},\overline{U}\in \mathcal {U}(x_0)$ such that $S\cap \hat{U}$ is invex and (F, G) is $(C\times D)$-preinvex with respect to $\eta$ on $S\cap \hat{U}$ for any neighborhood $\hat{U}$ ($\subseteq \overline{U}\cap \widetilde{U}$) of $x_0$. Let $\hat{U}:=U\cap \overline{U}\cap \widetilde{U}$, we claim that $H:=(F,G)(S\cap \hat{U})+C\times D-(y_0,0)$ is convex. Take $h_i\in H$, $i=1,2$, then there exist $x_i\in S\cap \hat{U}$, $(y_i,z_i)\in (F,G)(x_i)$, and $(c_i,d_i)\in C\times D$ such that $h_i=(y_i,z_i)+(c_i,d_i)-(y_0,0)$, $i=1,2$. By the assumption, we get, for all $x_1,x_2\in S\cap \hat{U}$, $\lambda \in [0,1]$,

$$\begin{aligned} \lambda (F,G)(x_1)+ (1-\lambda ) (F,G)(x_2)\subseteq (F,G)(x_2 + \lambda \eta (x_1,x_2)) + C\times D, \end{aligned}$$

so

$$\begin{aligned}&\lambda ((y_1,z_1)+(c_1,d_1)-(y_0,0)) + (1-\lambda )((y_2,z_2)+(c_2,d_2)-(y_0,0))\\&\quad \in (F,G)(x_2+\lambda \eta (x_1,x_2))+(C\times D)-(y_0,0), \end{aligned}$$

where $x_2+\lambda \eta (x_1,x_2)\in S\cap \hat{U}$ (since S is invex near $x_0$). Therefore, $\lambda h_1 + (1-\lambda ) h_2\in H$, i.e., H is convex. Thus, by (9), H and $-\mathrm{int}(C\times D)$ can be separated.

From two above cases, there exist $\hat{U}\in \mathcal {U}(x_0)$ and $(c^*,d^*)\in (X^*\times Y^*){\setminus }\{(0,0)\}$ such that for all $(y,z)\in (F,G)(S\cap \hat{U})$, $(c,d)\in C\times D$,

$$\begin{aligned} \langle {c^*,y+c-y_0}\rangle + \langle {d^*,z+d}\rangle \ge 0. \end{aligned}$$

(10)

Take $y=y_0$, $c=0$, $z=z_0$ in (10), we get $\langle {d^*,z_0 + d}\rangle \ge 0$, which implies that $\langle {d^*,z_0}\rangle \ge 0$ ($d=0$) and $\langle {d^*,d}\rangle \ge 0$ for all $d\in D$ (since D is a cone), i.e., $d^*\in D^*$. On the other hand, $\langle {d^*,z_0}\rangle \le 0$ ($z_0\in -D$). Consequently, (7) hold.

With $y=y_0$, $z=z_0$, and $d=-z_0$, it follows from (10) that $\langle {c^*,c}\rangle \ge 0$ for all $c\in C$, i.e., $c^*\in C^*$.

Let $(y,z)\in ED^m(F,G)(x_0,(y_0,z_0))(\Omega )$, then there exists $x\in \Omega$ such that $(x,(y,z))\in \mathrm{epi}ED^m(F,G)(x_0,(y_0,z_0))$. By the definition, there are $t_n\rightarrow 0^+$, $\{x_n\}_{n\in \mathbb {N}}\subseteq S$, $(y_n,z_n)\in (F,G)(x_n)$, and $(c_n,d_n)\in C\times D$ such that

$$\begin{aligned} \dfrac{x_n-x_0}{t_n}\rightarrow x,\quad \dfrac{y_n+c_n-y_0}{t_n^m}\rightarrow y,\quad \dfrac{z_n+d_n-z_0}{t_n^m}\rightarrow z. \end{aligned}$$

Set $u_n:=\dfrac{x_n-x_0}{t_n}$ ($u_n\rightarrow x$), then $x_n = x_0 +t_nu_n \rightarrow x_0$. Thus, $x_n\in S\cap \hat{U}$ with large enough n, and from (7), (10), we get

$$\begin{aligned} \left\langle {c^*,\dfrac{y_n+c_n-y_0}{t_n^m}} \right\rangle + \left\langle {d^*,\dfrac{z_n+d_n-z_0}{t_n^m}} \right\rangle \ge 0. \end{aligned}$$

Hence, we obtain (6) as $n\rightarrow +\infty$.

Finally, we prove that $c^*\ne 0$. Suppose to the contrary, i.e., $c^*=0$, it follows from (10) that $\left\langle {d^*,z+d} \right\rangle \ge 0$ for all $z\in G(S\cap \hat{U})$ and $d\in D$. By the assumption, with $z=\overline{z}$ and $d=0$, we get $\left\langle {d^*,\overline{z}} \right\rangle \ge 0$, which is a contradiction. Therefore, $c^*\ne \emptyset$. $\square$

For global solutions, terminology “near $x_0$” is omitted in conditions (i), (ii), while the condition ensuring $c^*\ne 0$ is reduced to $\exists (\overline{x},\overline{z})\in \mathrm{gr}G: \langle {d^*,\overline{z}\rangle } <0$. Theorem 4.1 is also a necessary condition for efficient solutions and strict efficient solutions of (SOP) since these solutions are included in the set of weak efficient solutions. For this reason, it is enough to establish sufficient conditions only for strict efficient solutions.

Inspired by [1], we get a sufficient condition for strict efficient solutions based on the stableness of objective and constraint maps. Recall that a map $F:X\rightarrow 2^Y$ is called to be stable of order m at $(x_0,y_0)$ if there exists $\lambda >0$ and $U\in \mathcal {U}(x_0)$ such that for all $x\in U{\setminus } \{0\}$,

$$\begin{aligned} F(x)\subseteq \{y_0\} + \lambda ||x-x_0||^m B_Y(0,1). \end{aligned}$$

Theorem 4.2

Let X, Y, Z are finite-dimensional and $y_0\in \mathrm{Min}_CF(x_0)$. Suppose that

• (i) (F, G) is stable of order m at $(x_0,(y_0,z_0))$;

• (ii) $ED^m(F,G)(x_0,$ $(y_0,z_0))(u)\ne \{0\}$ for all $u\in \Omega {\setminus } \{0\}$;

• (iii) there exist $c^*\in C^*{\setminus }\{0\}$, $d^*\in D^*$ such that (7) holds and for all $(y,z)\in ED^m(F,G)(x_0,(y_0,z_0))(\Omega {\setminus }\{0\})$,

  $$\begin{aligned} \langle {c^*,y}\rangle + \langle {d^*,z}\rangle > 0. \end{aligned}$$

  (11)

  Then, $(x_0,y_0)$ is a local strict efficient solution of order m of (SOP).

Proof

Suppose that $(x_0,y_0)$ is not a local strict efficient solution of order m of (SOP), then there exists $x_n\in A\cap B_X(0,n^{-1}){\setminus }\{x_0\}$ such that for all n,

$$\begin{aligned} (F(x_n)+C)\cap B_Y(y_0,n^{-1}||x_n-x_0||^m)\ne \emptyset . \end{aligned}$$

Thus, we get $(y_n,z_n)\in (F,G)(x_n)$ and $(c_n,d_n)\in C\times D$ with $z_n= -d_n$ and

$$\begin{aligned} y_n + c_n \in y_0 + ||x_n-x_0||^m B_Y(0,n^{-1}), \end{aligned}$$

which implies

$$\begin{aligned} (y_n-y_0,z_n-z_0)\in & {} -(c_n,d_n)-(0,z_0) + ||x_n-x_0||^m (B_{Y}(0,n^{-1}) \times \{0\})\nonumber \\\subseteq & {} -(C\times D(z_0)) + ||x_n-x_0||^m B_{Y\times Z}(0,n^{-1}). \end{aligned}$$

(12)

Moreover, from (i), there exists $\lambda > 0$ such that

$$\begin{aligned} (y_n,z_n)\in (F,G)(x_n)\subseteq \{(y_0,z_0)\} + \lambda ||x_n-x_0||^m B_{Y\times Z} (0,1). \end{aligned}$$

Therefore,

$$\begin{aligned} \left( \dfrac{y_n-y_0}{||x_n-x_0||^m},\dfrac{z_n-z_0}{||x_n-x_0||^m}\right) \in \lambda B_{Y\times Z}(0,1). \end{aligned}$$

(13)

Since $Y\times Z$ is finite-dimensional, the sequence $||x_n-x_0||^{-m}(y_n-y_0,z_n-z_0)$ (or its subsequence) converges to some $(y,z)\in Y\times Z$. It follows from (12) that $(y,z)\in -(C\times D(z_0))$. On the other hand, we have

$$\begin{aligned} (y_n,z_n)\in (F,G)(x_n)=(F,G)(x_0 + t_nu_n), \end{aligned}$$

where $t_n:=||x_n-x_0||\rightarrow 0^+$ and $u_n:=||x_n-x_0||^{-1}(x_n-x_0)\rightarrow u$ with $||u||=1$ (since X is finite-dimensional). It means that $(u,y,z)\in S^m_{\mathrm{epi}(F,G)}(x_0,(y_0,z_0))=\mathrm{epi}ED^m(F,G)(x_0,(y_0,z_0)),$ or $(y,z)\in ED^m(F,G)(x_0,(y_0,z_0))(u)+C\times D$. So,

$$\begin{aligned} ED^m(F,G)(x_0,(y_0,z_0))(u)\in (y,z)-(C\times D)\subseteq -(C\times D(z_0)). \end{aligned}$$

Therefore, there exist $(c,d)\in C\times D$ and $t>0$ such that $ED^m(F,G)(x_0,(y_0,z_0))(u)=-(c,td+tz_0)$ (${\ne }(0,0)$ (from (ii))). Hence, with $c^*\in C^*{\setminus }\{0\}$ and $d^*\in D^*$ in (iii), we obtain

$$\begin{aligned} \langle {(c^*,d^*), ED^m(F,G)(x_0,(y_0,z_0))(u)}\rangle = \langle {c^*,-c}\rangle + t\langle {d^*,-d}\rangle + t\langle {d^*,-z_0}\rangle \le 0, \end{aligned}$$

which contradicts assumption (iii). $\square$

From Theorem 4.2, we have some remarks on the earlier results in the literature as follows.

Remark 4.1

To get a sufficient condition for a local isolated minimizer of order m (in terms of the Studniarski derivative ${D}^m(F,G)_+(x_0,(y_0,z_0))(x)$, see Definition 3.1 in [1]), we proposed the following assumption (see Theorem 3.8 in [1]): for every $x\in \Omega {\setminus }\{0\}$,

$$\begin{aligned} {D}^m(F,G)_+(x_0,(y_0,z_0))(x)\cap -(\mathrm{cl}C\times \mathrm{cl}D(z_0))=\emptyset , \end{aligned}$$

However, (0, 0) possibly belongs to ${D}^m(F,G)(x_0,(y_0,z_0))(x)$ even $x\ne 0$ (see some examples in [1]). For this case, the above assumption cannot be employed. To overcome the situation, a condition that $(0,0)\not \in {D}^m(F,G)_+(x_0,(y_0,z_0))(x)$ for all $x\ne 0$ ($0\not \in {D}^m(F)_+(x_0,y_0)(x)$ for all $x\ne 0$) should be supplemented in Theorems 3.8, 3.10 (Theorem 3.9, resp) in [1] (without changing their proofs).

Theorem 4.2 is also a sufficient condition for efficient solutions and weak efficient solutions of (SOP). However, for these solutions, we have other conditions as follows.

Theorem 4.3

Suppose that (F, G) is $(C\times D)$-preinvex with respect to $\eta$ on S and (F, G) has a mth-order radial Studniarski epiderivative at $(x_0,(y_0,z_0))$ for all $m\ge 2$. If there exist $m\ge 1$, $c^*\in C^*{\setminus }\{0\}$ and $d^*\in D^*$ such that (7) and (11) hold for all $(y,z)\in ED^m(F,G)(x_0,(y_0,z_0))(\Omega {\setminus } \{0\})$, then $(x_0,y_0)$ is a local efficient solution of (SOP).

Proof

Suppose that $(x_0,y_0)$ is not a local efficient solution of (SOP), i.e., for all $U\in \mathcal {U}(x_0)$, there exists $x\in A\cap U$ such that $(F(x)-y_0)\cap (-C{\setminus } \{0\})\ne \emptyset$. Then, we get $z\in G(x)$ with $z\in -D$ and

$$\begin{aligned} (y-y_0,z-z_0)\in -(C{\setminus }\{0\}\times D(z_0)) \end{aligned}$$

It follows from Proposition 3.4 that

$$\begin{aligned} (y-y_0,z-z_0)\in ED^m(F,G)(x_0,(y_0,z_0))(\eta (x,x_0)) + C\times D. \end{aligned}$$

If $ED^m(F,G)(x_0,(y_0,z_0))(\eta (x,x_0))=0$, then $y-y_0\in C$, which contradicts the fact that $y-y_0\in -C{\setminus }\{0\}$ and C is pointed. Thus, $ED^m(F,G)(x_0,(y_0,z_0))(\eta (x,x_0))\ne 0$ and

$$\begin{aligned}&ED^m(F,G)(x_0,(y_0,z_0))(\eta (x,x_0))\in (y-y_0,z-z_0) - (C\times D)\\&\quad \subseteq -(C{\setminus }\{0\}\times D(z_0)). \end{aligned}$$

Hence, with $(c^*, d^*)$ in the assumption, we obtain

$$\begin{aligned} \langle {(c^*,d^*),ED^m(F,G)(x_0,(y_0,z_0))(\eta (x,x_0))}\rangle \le 0, \end{aligned}$$

which is a contradiction. $\square$

Theorem 4.4

Suppose that (F, G) is $(C\times D)$-preinvex with respect to $\eta$ on S and (F, G) has a mth-order radial Studniarski epiderivative at $(x_0,(y_0,z_0))$ for all $m\ge 2$. If there exist $m\ge 1$, $c^*\in C^*{\setminus }\{0\}$ and $d^*\in D^*$ such that (6) and (7) hold for all $(y,z)\in ED^m(F,G)(x_0,(y_0,z_0))(\Omega {\setminus } \{0\})$, then $(x_0,y_0)$ is a local weak efficient solution of (SOP).

Proof

The proof is similar to that of Theorem 4.3. $\square$