Optimality conditions for nonsmooth multiobjective bilevel optimization problems

Abstract

This article is devoted to the study of a nonsmooth multiobjective bilevel optimization problem, which involves the vector-valued objective functions in both levels of the considered program. We first formulate a relaxation multiobjective formulation for the multiobjective bilevel problem and examine the relationships of solutions between them. We then establish Fritz John (FJ) and Karush–Kuhn–Tucker (KKT) necessary conditions for the nonsmooth multiobjective bilevel optimization problem via its relaxation. This is done by studying a related multiobjective optimization problem with operator constraints.

1 Introduction

A multiobjective optimization problem (also known as a multi-criteria optimization problem or a vector optimization problem) (Ehrgott 2005; Jahn 2004; Luc 1989) is a mathematical problem of making the best possible choices (i.e. optimal solutions) that satisfy two or more conflicting objectives from a set of feasible choices, described by the constraints of the problem. Most of the real-life optimization problems are multi-objective in their nature. For instance, multi-objective optimization is a natural setting for investment portfolio management as such problems have to deal with the conflicting notions of revenue and risk. In this case, the multi-objective optimization problem corresponds to choosing a portfolio allocation that optimizes the tradeoff of risk and revenue, among all possible allocations that satisfy certain constraints.

A bilevel optimization problem is a combination of two programming problems where the constraint region of the upper-level optimization problem is determined by the solution set to the lower-level one. When the objective functions of both levels are vector-valued, one has the notion of multiobjective bilevel optimization problem.

Single-objective/or scalar bilevel optimization problems have been investigated intensively in the literature; see e.g., Colson et al. (2007), Bard (1998), Dempe (2002), Dempe and Dutta (2012), Dempe et al. (2012, 2014), Dempe and Zemkoho (2012, 2014), Ye et al. (1997), Jeyakumar et al. (2016), Chuong and Jeyakumar (2017) and Jeyakumar and Li (2015), Calvete et al. (2012). However, there are only few works dealing with multiobjective/or vector bilevel optimization problems Bonnel (2006), Bonnel and Morgan (2006), Bonnel and Collonge (2015), Dempe et al. (2013), Dempe and Franke (2012), Eichfelder (2010), Gadhi and Dempe (2012) and Ye (2011). Here, we describe briefly some of them. The authors in Dempe and Franke (2012) formulated the lower-level problem as a nonlinear programming one and then solved the problem with some scalarization methods. By using a Pascoletti and Serafini scalarization approach, paper Eichfelder (2010) solved a multiobjective bilevel optimization problem via an iterative process without any convex assumption. For the bilevel optimization problems where the upper-level is a scalar optimization problem and the lower-level is a vector optimization problem, we refer the reader to Bonnel and Morgan (2006) for an approach by using a penalty function. For the bilevel optimization problems with vector-valued objectives only in the upper-level one, we refer the reader to Gadhi and Dempe (2012) for an approach by means of the Hiriart–Urruty scalarization function, and to Ye (2011) for optimality conditions in a smooth setting.

We are now interested in a nonsmooth multiobjective bilevel optimization problem of the form:

where S(x) denotes the set of (weakly) Pareto solutions of the lower-level multiobjective optimization problem

and “$V-\mathrm{min}$” stands for vector minimization. Here, the functions $F_i, f_j, g_t:\mathbb {R}^n\times \mathbb {R}^m\rightarrow \mathbb {R}, i=1,\ldots ,p, j=1,\ldots ,q, t=1,\ldots ,r$, and $G_k:\mathbb {R}^n\rightarrow \mathbb {R}, k=1,\ldots ,l$ are assumed to be locally Lipschitz, and moreover, the lower-level multiobjective problem $\mathrm{P}_x$ is assumed to be convex for each $x\in \mathbb {R}^n$. The model of multiobjective bilevel optimization problem like $\mathrm{P}$ was introduced by Eichfelder (2010) under the term optimistic bilevel programming.

Based on the definitions in multiobjective optimization and multiobjective bilevel programming [cf. Dempe and Franke 2012; Ehrgott 2005; Eichfelder 2010; Gadhi and Dempe 2012; Ye 2011], we present the following concepts of optimal/Pareto solutions. For the sake of convenience, we denote the feasible sets of $\mathrm{P}$ and $\mathrm{P}_x$ respectively by

$$\begin{aligned} C:=&\left\{ (x,y)\in \mathbb {R}^n\times \mathbb {R}^m\mid y\in S(x),\, G_k(x) \le 0, \ k=1,\ldots ,l\right\} , \end{aligned}$$

(1.1)

$$\begin{aligned} C_x:=&\left\{ y\in \mathbb {R}^m\mid \; g_t(x,y)\le 0,\, t=1,\ldots ,r\right\} . \end{aligned}$$

(1.2)

Definition 1.1

• (i) Let $x\in \mathbb {R}^n$. We say that $\bar{y}\in C_x$ is a weak Pareto solution of problem $\mathrm{P}_x$, and write $\bar{y}\in S(x)$, if there does not exist $\hat{y}\in C_x$ such that

  $$\begin{aligned} f_j(x,\hat{y})<f_j(x,\bar{y}), \quad j=1,\ldots ,q. \end{aligned}$$

• (ii) We say that $(\bar{x},\bar{y})\in C$ is called a local weak Pareto solution of problem $\mathrm{P}$, and write $(\bar{x},\bar{y})\in locS(P)$, if there is a neighborhood U of $(\bar{x},\bar{y})$ such that there does not exist $(\hat{x},\hat{y})\in C\cap U$ satisfying

  $$\begin{aligned} F_i(\hat{x},\hat{y})< F_i(\bar{x},\bar{y}),\quad i=1,\ldots ,p. \end{aligned}$$

In this work, we provide necessary optimality conditions for local weak Pareto solutions of the multiobjective bilevel optimization problem $\mathrm{P}$. More precisely, we first derive a relaxation problem for the problem $\mathrm{P}$ and examine the relationships of local weak Pareto solutions between them. We then establish Fritz John (FJ) and Karush–Kuhn–Tucker (KKT) necessary conditions for the problem $\mathrm{P}$ via its relaxation.

It is known that the task of identifying and locating the optimal/Pareto solutions of the upper-level multiobjective problem becomes extremely difficult due to lack of mathematically tractable representations of the optimal/Pareto solutions of many lower-level optimization problems (Dempe 2002; Eichfelder 2010; Ye 2011). The representation of optimal solutions of a simple one-dimensional lower-level optimization problem results in hard nonlinear constraints, often nonsmooth/nonlinear constraints. These non-standard constraints produce complex nonsmoothness in the underlying multiobjective optimization problem even if all the constraint functions are just simple one-dimensional quadratic functions. As an illustration, the constraint

$$\begin{aligned} y\in S(x)=\mathrm{arg}\min \limits _{y\in \mathbb {R}}{\{g_0(x,y):=y\mid g_1(x,y):=y^2-x \le 0\}} \end{aligned}$$

is given by $y=h(x):=-\sqrt{x}$ with $x \ge 0$, which leads to a nonsmooth constraint.

Our approach in this paper is to convert the multiobjective bilevel optimization problem $\mathrm{P}$ into a multiobjective optimization problem with operator constraints (i.e., constraints described by inverse maps of sets) and obtain optimality conditions by applying some advanced tools of variational analysis and generalized differentiation (Mordukhovich 2006a; Rockafellar and Wets 1998) to the transformed problem. This approach is inspired by a recent progress in studying scalar bilevel optimization problems of Dempe and Zemkoho (2014), where one can see almost important extensions of concepts like M-, C-, and S-stationarity as well as corresponding results from the smooth case to a nonsmooth one. In what follows, we will exploit some techniques and ideas from Dempe and Zemkoho (2014) to examine the M-type stationarity condition. These techniques can be found in Dempe et al. (2012) for an optimistic bilevel program and in Dempe et al. (2014) for the more challenging pessimistic version of bilevel programming problems.

We would also like to point out that employing advanced tools of variational analysis and generalized differentiation to study a vector nonsmooth optimization problem with a general form of equilibrium constraints was given by Bao et al. (2007), where the reader can also find applications for deriving necessary optimality conditions in bilevel programming via coderivatives of set-valued mappings. In fact, there are some similarities between the class of multiobjective bilevel programs and the class of multiobjective optimization problems with equilibrium constraints; see, e.g., the paper by Mordukhovich (2009). It is worth mentioning here that the recent paper by Zemkoho (2016) proposed a novel approach to dealing with optimistic smooth bilevel programs by way of set-valued optimization techniques. It would be of interest to explore how the results in Zemkoho (2016) can be extended to the nonsmooth setting of problem $\mathrm{P}$. Moreover, since the set of local Pareto solutions of problem $\mathrm{P}$ is contained in the set of local weak Pareto solutions of problem $\mathrm{P}$, it would be possible to formulate the corresponding results for the local Pareto solutions. These would also form interesting topics for further research.

The rest of the paper is organized as follows. Section 2 contains some basic definitions from variational analysis and several auxiliary results. Section 3 investigates some stability properties of the map of nonsmooth Lagrange multipliers and evaluates the coderivative of the subdifferential multifunction for the lower-level optimization problem $\mathrm{P}_x$. In Sect. 4, we first address a relaxation reformulation for the problem $\mathrm{P}$ and examine the relationships of solutions between them. We then derive FJ and KKT necessary conditions for the relaxation reformulation of problem $\mathrm{P}$.

2 Preliminaries and auxiliary results

Throughout the paper we use the standard notation of variational analysis; see e.g., Mordukhovich (2006a) and Rockafellar and Wets (1998). All spaces under consideration are finite dimensional spaces whose norms are always denoted by $\Vert \cdot \Vert .$ The symbol $\langle \cdot \,,\cdot \rangle$ stands for the inner product in the referred space. For $f:\mathbb {R}^n\rightarrow \mathbb {R}^m$ and $v\in \mathbb {R}^m$, the scalarization function $\langle v,f\rangle :\mathbb {R}^n\rightarrow \mathbb {R}$ is defined by $\langle v,f\rangle (x):=\langle v,f(x)\rangle$ for all $x\in \mathbb {R}^n.$ The closed unit ball (resp., the unit sphere) in $\mathbb {R}^n$ is denoted by $I\!\!B_{n}$ (resp., $\mathbb {S}_n$) or sometimes by $I\!\!B$ (resp., $\mathbb {S}$) for simplicity if there is no confusion, while $I\!\!B_r(\bar{x}):=\bar{x}+rI\!\!B_n$ for $\bar{x}\in \mathbb {R}^n$ and $r>0.$ The topological closure and the topological interior of a set $\Omega \subset \mathbb {R}^n$ are denoted by $\mathrm{cl\;}\Omega$ and $\mathrm{int\;}\Omega$, respectively. As usual, the origin of any space is denoted by 0 but we may use $0_n$ for the origin of $\mathbb {R}^n$ in situations where some confusion might be possible. For two vectors $x, y\in \mathbb {R}^n$, we may write (x, y) instead of $(x,y)^\top$, and $x\ge y$ means that $x-y\in \mathbb {R}^n_+$, where $\mathbb {R}^n_+$ denotes the nonnegative orthant of $\mathbb {R}^n.$ Also, $\mathbb {R}^n_-=-\mathbb {R}^n_+$ is the nonpositive orthant of $\mathbb {R}^n.$

A set $\Omega \subset \mathbb {R}^n$ is called closed around$\bar{x}\in \Omega$ if there is a neighborhood U of $\bar{x}$ such that $\Omega \cap \mathrm{cl\,}U$ is closed. A multifunction $F:\mathbb {R}^n\rightrightarrows R^m$ is closed at $\bar{x}\in \mathbb {R}^n$ if for every sequence $x_k \rightarrow \bar{x}$ and $y_k\rightarrow \bar{y}$ with $y_k \in F(x_k)$, one has $\bar{y}\in F(\bar{x}).$ If F is closed at very point $\bar{x}\in \mathbb {R}^n$, then we say that F is closed (or graph-closed). We say that (cf. Mordukhovich 2006a, Definition 1.63) $F:\mathbb {R}^n\rightrightarrows R^m$ is inner semicompact at $\bar{x}\in \mathbb {R}^n$ if for every sequence $x_k \rightarrow \bar{x}$ there is a sequence $y_k \in F(x_k)$ that contains a convergent subsequence as $k\rightarrow \infty$. It is clear that when F has nonempty values around $\bar{x}$, the inner semicompact property is guaranteed by the local boundedness of F at this point; that is, there exists a neighborhood U of $\bar{x}$ such that F(U) is bounded.

Given a set-valued mapping $F:\mathbb {R}^n\rightrightarrows \mathbb {R}^n$, we denote by

$$\begin{aligned} \begin{array}{ll} \displaystyle \mathop {\mathrm{Lim}\,\mathrm{sup}}_{x\rightarrow \bar{x}}F(x):=&{}\Bigg \{v\in \mathbb {R}^n\big |\exists \; \text{ sequences } \;x_n\rightarrow \bar{x}\; \text{ and } \; v_n\rightarrow v\\ &{}\quad \text{ with } \;v_n\in F(x_n)\; \text{ for } \text{ all } \; n\in \mathbb {N}\Bigg \} \end{array} \end{aligned}$$

the sequential Painlevé–Kuratowski upper/outer limit of F as $x\rightarrow \bar{x},$ where $\mathbb {N}:=\{1,2,\ldots \}$.

Let $\Omega \subset X$ be closed around $\bar{x}\in \Omega$.

The regular/Fréchet normal cone to $\Omega$ at $\bar{x}\in \Omega$ is defined by

$$\begin{aligned} \widehat{N}(\bar{x};\Omega ):= \left\{ v\in \mathbb {R}^n\Big |\limsup _ {x\xrightarrow {\Omega }\bar{x}}\frac{\langle v, x-\bar{x}\rangle }{\Vert x-\bar{x}\Vert }\le 0\right\} , \end{aligned}$$

(2.1)

where $x\xrightarrow {\Omega }\bar{x}$ means that $x\rightarrow \bar{x}$ with $x\in \Omega$. If $\bar{x}\notin \Omega ,$ we put $\widehat{N}(\bar{x};\Omega ):=\emptyset .$

The limiting/Mordukhovich normal cone$N(\bar{x};\Omega )$ to $\Omega$ at $\bar{x}\in \Omega$ is obtained from Fréchet normal cones by taking the sequential Painlevé–Kuratowski upper limits as

$$\begin{aligned} N(\bar{x};\Omega ):=\mathop {\mathrm{Lim}\,\mathrm{sup}}\limits _{x\xrightarrow {\Omega }\bar{x}} \widehat{N}(x;\Omega ). \end{aligned}$$

(2.2)

If $\bar{x}\notin \Omega ,$ we put $N(\bar{x};\Omega ):=\emptyset .$

It is clear by (2.1) and (2.2) that

$$\begin{aligned} \widehat{N}(\bar{x};\Omega )\subset N(\bar{x};\Omega ). \end{aligned}$$

(2.3)

If the inclusion (2.3) holds as equality, then we say that $\Omega$ is normally regular at $\bar{x}$. The class of normally regular sets is sufficiently large including, besides convex ones, many other sets important in variational analysis and optimization; see Mordukhovich (2006a) for more details.

Let $\Phi :\mathbb {R}^n\rightrightarrows \mathbb {R}^m$ be a set-valued mapping with the graph

$$\begin{aligned} \text{ gph }\,\Phi := \left\{ (x,y)\in \mathbb {R}^n\times \mathbb {R}^m\big |\;y\in \Phi (x)\right\} . \end{aligned}$$

The normal/Mordukhovich coderivative of $\Phi$ at $(\bar{x},\bar{y})\in \mathrm{gph\,} \Phi$ is defined by

$$\begin{aligned} D^* \Phi (\bar{x},\bar{y})(y^*):=\left\{ x^*\in \mathbb {R}^n\big |\;(x^*,-y^*)\in N((\bar{x},\bar{y});\text{ gph }\,\Phi )\right\} ,\quad y^*\in \mathbb {R}^m. \end{aligned}$$

(2.4)

When $\Phi$ is a single-valued mapping, to simplify the notation, one writes $D^*\Phi (\bar{x})(y^*)$ instead of $D^*\Phi (\bar{x}, \Phi (\bar{x}))(y^*)$.

If the single-valued mapping $\Phi :\mathbb {R}^n\rightarrow \mathbb {R}^m$ is strictly differentiable at $\bar{x}\in \mathbb {R}^n$ with the derivative $\nabla \Phi (\bar{x})$, then we have

$$\begin{aligned} D^*\Phi (\bar{x})(y^*)=\{\nabla \Phi (\bar{x})^\top y^*\},\quad y^*\in \mathbb {R}^m. \end{aligned}$$

A calculation for the coderivative of the inverse image of a set via a strict differentiable function is taken from Mordukhovich (2006a, Theorem 1.17).

Lemma 2.1

Let $f: \mathbb {R}^n\rightarrow \mathbb {R}^m$ be strictly differentiable at $\bar{x}\in \mathbb {R}^n$ with the surjective derivative $\nabla f(\bar{x})$, and let $\Theta \subset \mathbb {R}^m$ be a closed set such that $f(\bar{x})\in \Theta .$ Then we have

$$\begin{aligned} N\left( \bar{x};f^{-1}(\Theta )\right) = \nabla f(\bar{x})^\top N\left( f(\bar{x});\Theta \right) . \end{aligned}$$

(2.5)

In what follows, we also use an evaluation for the coderivative of a product set-valued mapping (cf. Dempe and Zemkoho 2014, Proposition 3.2).

Lemma 2.2

Let $\Phi _i: \mathbb {R}^n\rightrightarrows \mathbb {R}^m, i=1,\ldots , q,$ be closed set-valued maps, and let $\Phi :\mathbb {R}^n\rightrightarrows \mathbb {R}^{m\times q}$ be defined by $\Phi (x):=\Phi _1(x)\times \cdots \times \Phi _q(x).$ Assume that the following qualification condition

$$\begin{aligned} \left[ \sum _{i=1}^q v^i=0, v^i\in D^*\Phi _i(\bar{x},\bar{y}^i)(0), i=1,\ldots ,q\right] \Rightarrow v^1=\cdots =v^q=0 \end{aligned}$$

(2.6)

holds at $(\bar{x},\bar{y})$ with $\bar{x}\in \mathbb {R}^m$ and $\bar{y}:=(\bar{y}^1,\ldots ,\bar{y}^q)\in \Phi (\bar{x}).$ Then, we have

$$\begin{aligned} D^*\Phi (\bar{x},\bar{y})(v)\subseteq \sum _{i=1}^qD^*\Phi _i(\bar{x},\bar{y}^i)(v^i), \end{aligned}$$

(2.7)

where $v:=(v^1,\ldots ,v^q)\in \mathbb {R}^{m\times q}.$ If in addition $\mathrm{gph\,}\Phi _i$ is normally regular at $(\bar{x},\bar{y}^i)$ for $i=1,\ldots ,q,$ then the inclusion (2.7) holds as equality.

For an extended real-valued function $\varphi :\mathbb {R}^n\rightarrow \overline{\mathbb {R}}:=[-\infty ,\infty ]$, we set

$$\begin{aligned} \text{ dom }\,\varphi :=\{x\in \mathbb {R}^n\mid \varphi (x)<\infty \},\quad \text{ epi }\,\varphi :=\{(x,\mu )\in \mathbb {R}^n\times \mathbb {R}\; |\; \mu \ge \varphi (x)\}. \end{aligned}$$

The limiting/Mordukhovich subdifferential of $\varphi$ at $\bar{x}\in \mathbb {R}^n$ with $|\varphi (\bar{x})|<\infty$ is defined by

$$\begin{aligned} \partial \varphi (\bar{x}):=\{v\in \mathbb {R}^n\; |\; (v,-1)\in N((\bar{x},\varphi (\bar{x}));\text{ epi }\,\varphi )\}. \end{aligned}$$

(2.8)

If $|\varphi (\bar{x})|=\infty ,$ then one puts $\partial \varphi (\bar{x}):=\emptyset .$ It is known (cf. Mordukhovich 2006a, Theorem 1.93) that when $\varphi$ is a convex function, the limiting subdifferential coincides with the subdifferential in the sense of convex analysis (cf. Rockafellar 1970); i.e.,

$$\begin{aligned}\partial \varphi (\bar{x}):=\{v\in \mathbb {R}^n\; |\; \langle v,x-\bar{x}\rangle \le \varphi (x)-\varphi (\bar{x})\quad \text{ for } \text{ all }\quad x\in \mathbb {R}^n\}. \end{aligned}$$

For a function $\varphi$ locally Lipschitz continuous at $\bar{x}\in \mathbb {R}^n$ with modulus $\ell >0$, it holds [see Mordukhovich (2006a, Corollaries 1.81 and 2.25)] that

$$\begin{aligned} \partial \varphi (\bar{x})\ne \emptyset \; \text{ and } \; ||v||\le \ell \quad \forall v\in \partial \varphi (\bar{x}). \end{aligned}$$

(2.9)

Given a nonempty set $\Omega \subset \mathbb {R}^n$, the distance function $d(\cdot ;\Omega ):\mathbb {R}^n\rightarrow \mathbb {R}$ defined by $d(x;\Omega ):=\inf _{u\in \Omega }{||x-u||}$ is globally Lipschitz continuous on $\mathbb {R}^n$ with modulus $\ell =1$, and furthermore, if $\Omega$ is closed around $\bar{x}\in \Omega$, then one has (cf. Rockafellar and Wets 1998, Example 8.53)

$$\begin{aligned} \partial d(\bar{x};\Omega )=N(\bar{x};\Omega )\cap I\!\!B_n. \end{aligned}$$

(2.10)

Let $\Omega \subset \mathbb {R}^n.$ Considering the indicator function $\delta (\cdot ;\Omega )$ defined by $\delta (x;\Omega ):=0$ for $x\in \Omega$ and by $\delta (x;\Omega ):=\infty$ otherwise, we have [see Mordukhovich (2006a, Proposition 1.79)]:

$$\begin{aligned} N(\bar{x};\Omega )=\partial \delta (\bar{x};\Omega )\quad \text{ for } \text{ any }\quad \bar{x}\in \Omega . \end{aligned}$$

(2.11)

The following limiting subdifferential sum rule is needed for our study.

Lemma 2.3

(See Mordukhovich 2006a, Theorem 3.36) Let $\varphi _i: \mathbb {R}^n\rightarrow \overline{\mathbb {R}}, i=1,2,\ldots ,n, n\ge 2,$ be lower semicontinuous around $\bar{x}\in \mathbb {R}^n,$ and let all but one of these functions be Lipschitz continuous around $\bar{x}.$ Then one has

$$\begin{aligned} \partial (\varphi _1+\varphi _2+\cdots +\varphi _n)(\bar{x})\subset \partial \varphi _1(\bar{x})+\partial \varphi _2(\bar{x})+\cdots +\partial \varphi _n(\bar{x}). \end{aligned}$$

(2.12)

The following result is known as the normal cone to a complementarity set, which can be found in Outrata (1999, Lemma 2.2) or Ye (2000, Proposition 3.7).

Lemma 2.4

Let $\Theta :=\{(a,b)\in \mathbb {R}^r\times \mathbb {R}^r\mid a\in \mathbb {R}^r_+, b\in \mathbb {R}^r_+, \langle a,b\rangle =0\},$ and let $(x,y)\in \Theta .$ One has

$$\begin{aligned} N\left( (x,y);\Theta \right) =\left\{ (u,v)\in \mathbb {R}^r\times \mathbb {R}^r\Bigg | \begin{array}{ll} &{}u_i=0, i\in I_y,\\ &{} v_i=0, i\in I_x, \\ &{} (u_i<0 \wedge v_i <0) \vee u_iv_i=0, i\in I_{xy}\end{array}\right\} , \end{aligned}$$

(2.13)

where $x:=(x_1,\ldots ,x_r), y:=(y_1,\ldots ,y_r), u=(u_1,\ldots ,u_r), v:=(v_1,\ldots ,v_r)$ and $I_x:=\{i=1,\ldots ,r\mid x_i=0, y_i>0\}, I_y:=\{i=1,\ldots ,r\mid x_i>0, y_i=0\}, I_{xy}:=\{i=1,\ldots ,r\mid x_i=0, y_i=0\}.$

Let us provide the computation for the limiting normal cone to an intersection of the nonnegative orthant with the unit sphere in $\mathbb {R}^n$, which will be used in the sequel.

Lemma 2.5

Let $\bar{x}:=(\bar{x}_1,\ldots ,\bar{x}_n)\in \Theta :=\mathbb {R}^n_+\cap \mathbb {S}_n.$ We have

$$\begin{aligned} N(\bar{x};\Theta ):=\left\{ v:=(v_1,\ldots ,v_n)\in \mathbb {R}^n\Big | \begin{array}{ll} v_i\le 0, &{}i\in I(\bar{x}),\\ v_i:=\lambda \bar{x}_i, \lambda \in \mathbb {R}, &{}i\notin I(\bar{x})\end{array}\right\} , \end{aligned}$$

where $I(\bar{x}):=\{i=1,\ldots ,n\mid \bar{x}_i=0\}.$

Proof

Let $f:\mathbb {R}^n\rightarrow \mathbb {R}$ be defined by $f(x):=||x||-1$. We see that $\mathbb {S}_n=f^{-1}(0):=\{x\in \mathbb {R}^n\mid f(x)=0\}$, and that f is strictly differentiable at every point $x\in \mathbb {S}_n$ with the surjective derivative $\nabla f(x)=\frac{x}{||x||}=x.$ Applying Lemma 2.1 and Mordukhovich (2006a, Corollary 1.15), we have for each $x\in \mathbb {S}_n$,

$$\begin{aligned} \widehat{N}\left( x;f^{-1}(0)\right) = N\left( x;f^{-1}(0)\right) =\{\lambda x\mid \lambda \in \mathbb {R}\}. \end{aligned}$$

(2.14)

Since $\mathbb {R}^n_+$ is a convex cone, it holds (see, e.g., Henrion and Outrata 2008, Proposition 3.7) that

$$\begin{aligned} \widehat{N}\left( x;\mathbb {R}^n_+\right) = N\left( x;\mathbb {R}^n_+\right) =\{(\mu _1,\ldots ,\mu _n)\in \mathbb {R}^n\mid \mu _i\le 0, \mu _ix_i=0, i=1,\ldots ,n\}\nonumber \\ \end{aligned}$$

(2.15)

for each $x:=(x_1,\ldots ,x_n)\in \mathbb {R}^n_+.$ Now, consider $\bar{x}:=(\bar{x}_1,\ldots ,\bar{x}_n)\in \Theta :=\mathbb {R}^n_+\cap \mathbb {S}_n.$ Since it can be checked that $N\left( \bar{x};\mathbb {R}^n_+\right) \cap \left( - N\left( \bar{x};\mathbb {S}_n\right) \right) =\{0\},$ we conclude by Mordukhovich (2006a, Theorem 3.4 and Corollary 3.5) that

$$\begin{aligned} N(\bar{x};\Theta )= N\left( \bar{x};\mathbb {R}^n_+\right) + N\left( \bar{x};\mathbb {S}_n\right) . \end{aligned}$$

This together with (2.14) and (2.15) finishes the proof. $\Box$

The partial first-order subdifferential mapping $\partial _y \varphi :\mathbb {R}^n\times \mathbb {R}^m \rightrightarrows \mathbb {R}^m$ of $\varphi :\mathbb {R}^n\times \mathbb {R}^m \rightarrow \overline{\mathbb {R}}$ with respect to y is defined by

$$\begin{aligned} \partial _y\varphi (x,y):=\{\text{ set } \text{ of } \text{ subgradients } u \text{ of } \varphi _x:=\varphi (x,\cdot ) \text{ at } y\}=\partial \varphi _x(y). \end{aligned}$$

(2.16)

The definition of $\partial _x\varphi (x,y)$ is defined similarly. We have a relation between the partial first-order subdifferential of a function at a given point and its subdifferential at the corresponding point as follows.

Lemma 2.6

(See Mordukhovich 2006a, Corollary 3.44) Let $\varphi :\mathbb {R}^n\times \mathbb {R}^m \rightarrow \overline{\mathbb {R}}$ be locally Lipschitz continuous at $(\bar{x},\bar{y})\in \mathbb {R}^n\times \mathbb {R}^m.$ One has

$$\begin{aligned} \partial _x\varphi (\bar{x},\bar{y})\subseteq \{v\in \mathbb {R}^n\mid \exists u\in \mathbb {R}^m \text{ with } (v,u)\in \partial \varphi (\bar{x},\bar{y})\}. \end{aligned}$$

Let $\varphi$ be finite at $(\bar{x},\bar{y})\in \mathbb {R}^n\times \mathbb {R}^m,$ and let $\bar{u}\in \partial _y \varphi (\bar{x},\bar{y}).$ The partial second-order subdifferential $\partial ^2_y \varphi (\bar{x},\bar{y},\bar{u}):\mathbb {R}^m \rightrightarrows \mathbb {R}^n\times \mathbb {R}^m$ of $\varphi$ with respect to y at $(\bar{x},\bar{y})$ relate to $\bar{u}$ is defined by (cf. Mordukhovich 1992; Levy et al. 2000; Mordukhovich and Rockafellar 2012)

$$\begin{aligned} \partial ^2_y\varphi (\bar{x},\bar{y},\bar{u})(w):=D^*(\partial _y\varphi )(\bar{x},\bar{y},\bar{u})(w),\quad w\in \mathbb {R}^m. \end{aligned}$$

(2.17)

Note that the scheme of deriving second-order subdifferentials as the coderivative of the first-order ones was suggested by Mordukhovich (1992), see Mordukhovich (2006a, b) for more discussions about this. The reader is referred to Mordukhovich et al. (2014) for various calculus rules for partial second-order subdifferentials. Let us quote representations of the partial second-order subdifferential for smooth functions with Lipschitz continuity or strictly differentiability of partial derivatives.

Lemma 2.7

(See Mordukhovich et al. 2014, Propositions 2.2 and 2.4) Let $\varphi :\mathbb {R}^n\times \mathbb {R}^m \rightarrow \mathbb {R}$ be continuously differentiable around $(\bar{x},\bar{y})\in \mathbb {R}^n\times \mathbb {R}^m,$ and let its partial derivative $\nabla _y\varphi :\mathbb {R}^n\times \mathbb {R}^m\rightarrow \mathbb {R}^m$ be Lipschitz continuous around $(\bar{x},\bar{y}).$ Then we have

$$\begin{aligned} \partial ^2_y\varphi (\bar{x},\bar{y},\bar{u})(w)=\partial \langle w,\nabla _y\varphi \rangle (\bar{x},\bar{y})\ne \emptyset \text{ for } \text{ all } w\in \mathbb {R}^m, \end{aligned}$$

where $\bar{u}:=\nabla _y\varphi (\bar{x},\bar{y}).$ If in addition $\nabla _y\varphi$ is strictly differentiable at $(\bar{x},\bar{y})$ with the partial derivatives denoted by $\nabla ^2_{yx}\varphi (\bar{x},\bar{y})$ and $\nabla ^2_{yy}\varphi (\bar{x},\bar{y})$, then

$$\begin{aligned} \partial ^2_y\varphi (\bar{x},\bar{y},\bar{u})(w)=\left\{ (\nabla ^2_{yx}\varphi (\bar{x},\bar{y})^\top w,\nabla ^2_{yy}\varphi (\bar{x},\bar{y})^\top w)\right\} \text{ for } \text{ all } w\in \mathbb {R}^m. \end{aligned}$$

Let $F_i:\mathbb {R}^n\rightarrow \mathbb {R}, i=1,\ldots ,p$, and $\psi :\mathbb {R}^n\rightarrow \mathbb {R}^m$ be locally Lipschitz continuous. The forthcoming lemma provides Fritz John (FJ) and Karush–Kuhn–Tucker (KKT) conditions for the multiobjective optimization problem:

$$\begin{aligned} V-\min {\{(F_1(x),\ldots ,F_p(x))\mid x\in \Delta \cap \psi ^{-1}(\Theta )\}}, \end{aligned}$$

(2.18)

where $\Delta \subset \mathbb {R}^n$ and $\Theta \subset \mathbb {R}^m$ are nonempty closed sets.

Lemma 2.8

Let $\bar{x}$ be a local weak Pareto solution of problem (2.18). Then there exist $\nu :=(\nu _1,\ldots ,\nu _p)\in \mathbb {R}^p_+$ and $v\in N\left( \psi (\bar{x});\Theta \right)$, not all zero, such that

$$\begin{aligned} 0\in \sum _{i=1}^p\nu _i \partial F_i(\bar{x})+\partial \langle v,\psi \rangle (\bar{x})+N(\bar{x};\Delta ).\end{aligned}$$

(2.19)

In addition, $\nu \ne 0$ provided that the following qualification condition holds:

$$\begin{aligned} \big [ 0\in \partial \langle v,\psi \rangle (\bar{x})+N(\bar{x};\Delta ),\; v\in N\big (\psi (\bar{x});\Theta \big )\big ]\Rightarrow v=0. \end{aligned}$$

(2.20)

Proof

The proof of this lemma can be derived from several corresponding results of multiobjective optimization problems with a general preference such as Ye and Zhu (2003, Theorem 1.2) or Bellaassali and Jourani (2008, Theorem 4.1). $\Box$

3 Lower-level multiobjective optimization problem

In this section, we examine some stability properties of the map of nonsmooth Lagrange multipliers and evaluate the coderivative of the subdifferential multifunction for the lower-level multiobjective optimization problem defined by $\mathrm{P}_x$. For this purpose, we consider $x\in \mathbb {R}^n$ as a parameter and recall the problem as follows:

$$\begin{aligned} V-\min _{y}\{(f_1(x,y),\ldots , f_q(x,y))\mid g_t(x,y) \le 0, t=1,\ldots ,r\}, \end{aligned}$$

(3.1)

where the functions $f_j(x,\cdot ), g_t(x,\cdot ), j=1,\ldots ,q, t=1,\ldots ,r,$ are real-valued convex for all $x\in \mathbb {R}^n.$

Consider a subdifferential multifunction $L:\mathbb {R}^n\times \mathbb {R}^m\times \mathbb {R}^q\times \mathbb {R}^r \rightrightarrows \mathbb {R}^m$ of functions involved in the problem (3.1) defined by

$$\begin{aligned} L(x,y,\alpha ,\lambda ):= \sum _{j=1}^q\alpha _j\partial _y f_j(x,y)+\sum _{t=1}^r\lambda _t\partial _y g_t(x,y), \end{aligned}$$

(3.2)

where $\alpha :=(\alpha _1,\ldots ,\alpha _q)$ and $\lambda :=(\lambda _1,\ldots ,\lambda _r).$ Then, the multifunction $\Lambda :\mathbb {R}^n\times \mathbb {R}^m\rightrightarrows \mathbb {R}^q\times \mathbb {R}^r$ of nonsmooth Lagrange multipliers of problem (3.1) can be expressed via the multifunction L as

$$\begin{aligned} \Lambda (x,y):&=\Big \{(\alpha ,\lambda )\in (\mathbb {R}^q_+\cap \mathbb {S}_q)\times \mathbb {R}^r_+\mid 0\in L(x,y,\alpha ,\lambda ), g_t(x,y)\nonumber \\&\le 0, \lambda _tg_t(x,y)=0, t=1,\ldots , r \Big \},\nonumber \\&=\Big \{(\alpha ,\lambda )\in (\mathbb {R}^q_+\cap \mathbb {S}_q)\times \mathbb {R}^r_+\mid 0\in \sum _{j=1}^q\alpha _j\partial _y f_j(x,y)+\sum _{t=1}^r\lambda _t\partial _y g_t(x,y),\nonumber \\&\qquad \qquad g_t(x,y)\le 0, \lambda _tg_t(x,y)=0, t=1,\ldots , r \Big \}, \end{aligned}$$

(3.3)

where $\alpha :=(\alpha _1,\ldots ,\alpha _q)$ and $\lambda :=(\lambda _1,\ldots ,\lambda _r).$

Definition 3.1

We say that the Slater constraint qualification (SCQ) for the problem (3.1) is satisfied at $\bar{x}\in \mathbb {R}^n$ if there exists $\hat{y}\in \mathbb {R}^m$ such that

$$\begin{aligned} g_t(\bar{x},\hat{y})< 0,\quad t=1,\ldots , r. \end{aligned}$$

(3.4)

Remark 3.2

If the (SCQ) in (3.4) is satisfied at $x\in \mathbb {R}^n,$ then we have the following assertion (see, e.g., Ehrgott 2005, Theorem 4.1, p. 97): $y\in S(x)$if and only if there is$(\alpha ,\lambda )\in \Lambda (x,y)$; i.e., there exist$\alpha :=(\alpha _1,\ldots ,\alpha _q)\in \mathbb {R}^q_+\cap \mathbb {S}_q$and$\lambda :=(\lambda _1,\ldots ,\lambda _r)\in \mathbb {R}^r_+$such that

$$\begin{aligned} {\left\{ \begin{array}{ll} 0\in \sum _{j=1}^q\alpha _j\partial _y f_j(x,y)+\sum _{t=1}^r\lambda _t\partial _y g_t(x,y), \\ g_t(x,y)\le 0, \lambda _tg_t(x,y)=0, t=1,\ldots , r.\end{array}\right. } \end{aligned}$$

(3.5)

In our framework, the closedness, nonemptyness and/or the local boundedness of partial subdifferential of functions involved in the problem (3.1) as well as the subdifferential multifunction L defined in (3.2) hold without any condition.

Proposition 3.3

Consider the functions involved in the problem (3.1) and the subdifferential multifunction L in (3.2). We have the following assertions:

• (i) The partial subdifferentials $\partial _y f_j, j=1,\ldots , q, \partial _y g_t, t=1,\ldots , r$ are nonempty, closed and locally bounded at any $(\bar{x},\bar{y})\in R^n\times \mathbb {R}^m$.

• (ii) The multifunction L in (3.2) is closed.

Proof

We first justify (i). We only need to justify the nonemptiness, closedness and local boundedness of $f_j$ at $(\bar{x},\bar{y})$ for $j\in \{1,\ldots ,q\}$ due to the similarity of functions.

Since $f_j$ is locally Lipschitz continuous at $(\bar{x},\bar{y}),$ there exists $r>0$ such that $f_j$ is Lipschitz continuous on $I\!\!B_r(\bar{x},\bar{y})$ with modulus $\ell >0.$ Taking $r_1\in (0,r)$, we assert that $\partial _yf_j\big (I\!\!B_{r_1}(\bar{x},\bar{y})\big )$ is a bounded set in $\mathbb {R}^m.$ Indeed, picking any $(x,y)\in I\!\!B_{r_1}(\bar{x},\bar{y})$, it holds that $I\!\!B_{r}(\bar{x},\bar{y})$ is a neighborhood of (x, y), and thus $f_j$ is locally Lipschitz continuous at (x, y) with the same modulus $\ell .$ By virtue of (2.9),

$$\begin{aligned} \partial f_j(x,y)\ne \emptyset \, \text{ and } \, ||w^*||\le \ell \quad \forall w^*\in \partial f_j(x,y). \end{aligned}$$

(3.6)

Now, let $v^*\in \partial _y f_j(x,y)$. Thanks to Lemma 2.6, we find $u^*\in \mathbb {R}^n$ such that $(u^*,v^*)\in \partial f_j(x,y).$ Then, it follows by (3.6) that

$$\begin{aligned} ||v^*||\le ||(u^*,v^*)||\le \ell . \end{aligned}$$

Consequently, $\partial _y f_j$ is nonempty and locally bounded at $(\bar{x},\bar{y}).$

To prove the closedness of $\partial _y f_j$, let $(x^k,y^k)\rightarrow (\bar{x},\bar{y})$ and $v^{*k}\rightarrow v^*$ with $v^{*k}\in \partial _y f_j(x^k,y^k)$. For each $k\in \mathbb {N}$, due to the convexity of $f_j(x^k,\cdot )$, the relation $v^{*k}\in \partial _y f_j(x^k,y^k)$ means that

$$\begin{aligned} \langle v^{*k},y-y^k\rangle \le f_j(x^k,y)-f_j(x^k,y^k) \quad \forall y\in \mathbb {R}^m. \end{aligned}$$

(3.7)

Letting $k\rightarrow \infty$ in (3.7), we arrive at

$$\begin{aligned} \langle v^{*},y-\bar{y}\rangle \le f_j(\bar{x},y)-f_j(\bar{x},\bar{y}) \quad \forall y\in \mathbb {R}^m, \end{aligned}$$

i.e., $v^{*}\in \partial _y f_j(\bar{x},\bar{y}).$ So, $\partial _y f_j$ is closed at $(\bar{x},\bar{y}).$

To justify (ii), letting any $(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda })\in \mathbb {R}^n\times \mathbb {R}^m\times \mathbb {R}^q\times \mathbb {R}^r,$ we will prove that L is closed at this point. Assume that $(x^k,y^k,\alpha ^k,\lambda ^k)\rightarrow (\bar{x},\bar{y},\bar{\alpha },\bar{\lambda })$ and $w^{k}\rightarrow \bar{w}$ with $w^{k}\in L(x^k,y^k,\alpha ^k,\lambda ^k)$, where $\alpha ^k:=(\alpha ^k_1,\ldots ,\alpha ^k_q), \bar{\alpha }:=(\bar{\alpha }_1,\ldots ,\bar{\alpha }_q)\in \mathbb {R}^q$ and $\lambda ^k:=(\lambda _1^k,\ldots ,\lambda _r^k), \bar{\lambda }:= (\bar{\lambda }_1,\ldots ,\bar{\lambda }_r) \in \mathbb {R}^r$ for all $k\in \mathbb {N}.$ For each $k\in \mathbb {N}$, the relation $w^{k}\in L(x^k,y^k,\alpha ^k,\lambda ^k)$ means that there exist $v^{*k}_j\in \partial _y f_j(x^k,y^k), j=1,\ldots ,q, u^{*k}_t\in \partial _y g_t(x^k,y^k),t=1,\ldots ,r,$ such that

$$\begin{aligned} w^k= \sum _{j=1}^q\alpha ^k_jv^{*k}_j+\sum _{t=1}^r\lambda ^k_t u^{*k}_t. \end{aligned}$$

(3.8)

Since $(x^k,y^k)\rightarrow (\bar{x},\bar{y})$ and by (i), $\partial _y f_j, j=1,\ldots , q, \partial _y g_t, t=1,\ldots , r$ are locally bounded at $(\bar{x},\bar{y})$, it follows that the sequences $\{v^{*k}_j\}, j=1,\ldots , q, \{u^{*k}_t\}, t=1,\ldots , r,$ are bounded. So, by taking subsequences if necessary, we may assume that $v^{*k}_j\rightarrow \bar{v}^{*}_j\in \mathbb {R}^m, j=1,\ldots , q, u^{*k}_t\rightarrow \bar{u}^{*}_t\in \mathbb {R}^m, t=1,\ldots , r$ as $k\rightarrow \infty .$ Moreover, we confirm that $\bar{v}^{*}_j\in \partial _y f_j(\bar{x},\bar{y}), j=1,\ldots , q$ and $\bar{u}^{*}_t\in \partial _y g_t(\bar{x},\bar{y}), t=1,\ldots , r$ due to the closedness of $\partial _y f_j, j=1,\ldots , q, \partial _y g_t, t=1,\ldots , r,$ at $(\bar{x},\bar{y})$ as in (i).

Now, letting $k\rightarrow \infty$ in (3.8), we arrive at

$$\begin{aligned} \bar{w}=\sum _{j=1}^q\bar{\alpha }_j\bar{v}^{*}_j+\sum _{t=1}^r\bar{\lambda }_t \bar{u}^{*}_t\in \sum _{j=1}^q\bar{\alpha }_j\partial _y f_j(\bar{x},\bar{y})+\sum _{t=1}^r\bar{\lambda }_t \partial _y g_t(\bar{x},\bar{y}), \end{aligned}$$

showing that $\bar{w}\in L(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda })$. Hence, L is closed at $(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda })$, which completes the proof. $\Box$

Using the above obtained results, we are able to derive the closedness of the nonsmooth Lagrange multipliers multifunction $\Lambda$ in (3.3) without any condition, and to justify the local boundedness of this mapping under the fulfilment of (SCQ).

Proposition 3.4

Consider the multifunction $\Lambda$ in (3.3). We have the following assertions.

• (i) The multifunction $\Lambda$ is closed at any $(\bar{x},\bar{y})\in \mathbb {R}^n\times \mathbb {R}^m.$

• (ii) If in addition the (SCQ) is satisfied at $\bar{x}\in \mathbb {R}^n,$ then $\Lambda$ is locally bounded at $(\bar{x},\bar{y})$.

Proof

We first justify (i). Let $(x^k,y^k)\rightarrow (\bar{x},\bar{y})$, and let $(\alpha ^k,\lambda ^k)\in \Lambda (x^k,y^k)$ be such that $(\alpha ^k,\lambda ^k)\rightarrow (\bar{\alpha },\bar{\lambda })$, where $\alpha ^k:=(\alpha ^k_1,\ldots ,\alpha ^k_q), \bar{\alpha }:=(\bar{\alpha }_1,\ldots ,\bar{\alpha }_q)\in \mathbb {R}^q_+$ and $\lambda ^k:=(\lambda _1^k,\ldots ,\lambda _r^k), \bar{\lambda }:= (\bar{\lambda }_1,\ldots ,\bar{\lambda }_r) \in \mathbb {R}^r_+$ for all $k\in \mathbb {N}.$ Then,

$$\begin{aligned}&0\in L(x^k,y^k,\alpha ^k,\lambda ^k), \alpha ^k\in \mathbb {S}_q, \end{aligned}$$

(3.9)

$$\begin{aligned}&g_t(x^k,y^k)\le 0, \lambda ^k_tg_t(x^k,y^k)=0, t=1,\ldots , r, k\in \mathbb {N}. \end{aligned}$$

(3.10)

Taking into account the closedness of the mapping L as shown in Proposition 3.3(ii), by letting $k\rightarrow \infty$ in (3.9) and (3.10), we obtain that

$$\begin{aligned}&0\in L(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda }), \bar{\alpha }\in \mathbb {S}_q,\\&g_t(\bar{x},\bar{y})\le 0, \bar{\lambda }_tg_t(\bar{x},\bar{y})=0, t=1,\ldots , r, \end{aligned}$$

which shows that $(\bar{\alpha },\bar{\lambda })\in \Lambda (\bar{x},\bar{y})$, and consequently, $\Lambda$ is closed at $(\bar{x},\bar{y}).$

To prove (ii), we now suppose for contradiction that there exist sequences $(x^k,y^k)\rightarrow (\bar{x},\bar{y})$ and $(\alpha ^k,\lambda ^k)\in \Lambda (x^k,y^k)$ with $||(\alpha ^k,\lambda ^k)||\rightarrow \infty$ as $k\rightarrow \infty .$ The relation $(\alpha ^k,\lambda ^k)\in \Lambda (x^k,y^k)$ means that $\alpha ^k:=(\alpha ^k_1,\ldots ,\alpha ^k_q)\in \mathbb {R}^q_+\cap \mathbb {S}_q, \lambda ^k:=(\lambda _1^k,\ldots ,\lambda _r^k) \in \mathbb {R}^r_+$ and

$$\begin{aligned}&0\in \sum _{j=1}^q\alpha ^k_j\partial _y f_j(x^k,y^k)+\sum _{t=1}^r\lambda ^k_t\partial _y g_t(x^k,y^k), \end{aligned}$$

(3.11)

$$\begin{aligned}&g_t(x^k,y^k)\le 0, \lambda ^k_tg_t(x^k,y^k)=0, t=1,\ldots , r, k\in \mathbb {N}. \end{aligned}$$

(3.12)

On the one side, by $||\alpha ^k||=1$, it ensures that $||\lambda ^k||\rightarrow \infty$ as $k\rightarrow \infty .$ Letting $\tilde{\lambda }_t^k:=\frac{\lambda _t^k}{||\lambda ^k||}, t=1,\ldots ,r,$ we obtain that $||\tilde{\lambda }^k||=1$, where $\tilde{\lambda }^k:=(\tilde{\lambda }_1^k,\ldots ,\tilde{\lambda }_r^k)\in \mathbb {R}^r_+$ for all $k\in \mathbb {N}$. By passing to a subsequence if necessary, one may assume that $\tilde{\lambda }^k\rightarrow \tilde{\lambda }$ as $k\rightarrow \infty ,$ where $\tilde{\lambda }:=(\tilde{\lambda }_1,\ldots ,\tilde{\lambda }_r)\in \mathbb {R}^r_+$ with $||\tilde{\lambda }||=1.$ On the other side, for each $k\in \mathbb {N}$, dividing (3.11) and (3.12) by $||\lambda ^k||$, we get

$$\begin{aligned}&0\in \sum _{j=1}^q\frac{\alpha ^k_j}{||\lambda ^k||}\partial _y f_j(x^k,y^k)+\sum _{t=1}^r\tilde{\lambda }^k_t \partial _y g_t(x^k,y^k), \end{aligned}$$

(3.13)

$$\begin{aligned}&g_t(x^k,y^k)\le 0, \tilde{\lambda }^k_tg_t(x^k,y^k)=0, t=1,\ldots , r. \end{aligned}$$

(3.14)

Observe that $\frac{\alpha ^k_j}{||\lambda ^k||}\rightarrow 0$ as $k\rightarrow \infty$ for $j=1,\ldots ,q$. Due to the closedness and local boundedness of the subdifferentials $\partial _yf_j, j=1,\ldots ,q, \partial _yg_t, t=1,\ldots ,r,$ at $(\bar{x},\bar{y})$ as shown in Proposition 3.3(i), by passing to the limit as $k\rightarrow \infty$ in (3.13) and (3.14) we come to the following relations

$$\begin{aligned}&0\in \sum _{t=1}^r\tilde{\lambda }_t\partial _y g_t(\bar{x},\bar{y}), \end{aligned}$$

(3.15)

$$\begin{aligned}&g_t(\bar{x},\bar{y})\le 0, \tilde{\lambda }_tg_t(\bar{x},\bar{y})=0, t=1,\ldots , r. \end{aligned}$$

(3.16)

Let the (SCQ) be satisfied at $\bar{x},$ i.e., (3.4) holds true. By (3.15), there are $\tilde{u}^{*}_t\in \partial _y g_t(\bar{x},\bar{y}), t=1,\ldots ,r,$ such that $\sum _{t=1}^r\tilde{\lambda }_t\tilde{u}^{*}_t=0.$ The relations $\tilde{u}^{*}_t\in \partial _y g_t(\bar{x},\bar{y}), t=1,\ldots ,r,$ mean that

$$\begin{aligned} \langle \tilde{u}^{*}_t,y-\bar{y}\rangle \le g_t(\bar{x},y)-g_t(\bar{x},\bar{y}) \quad \forall y\in \mathbb {R}^m, t=1,\ldots ,r. \end{aligned}$$

(3.17)

Hence, by summing up these inequalities after substituting $y:=\hat{y}$ into (3.17) and multiplying them by $\tilde{\lambda }_t$ for $t=1,\ldots , r$, respectively, we arrive at

$$\begin{aligned} 0=\left\langle \sum _{t=1}^r\tilde{\lambda }_t\tilde{u}^{*}_t,\hat{y}-\bar{y}\right\rangle \le \sum _{t=1}^r\tilde{\lambda }_t g_t(\bar{x},\hat{y})-\sum _{t=1}^r\tilde{\lambda }_t g_t(\bar{x},\bar{y})< 0, \end{aligned}$$

where the last inequality holds due to (3.16), (3.4) and the fact that $||\tilde{\lambda }||=1.$ So, we obtain a contradiction, which finishes the proof. $\Box$

Remark 3.5

We have used Proposition 3.4(i) to give a proof for Proposition 3.4(ii). In fact, the proof of Proposition 3.4(ii) can be derived from a more general result given in Li and Zhang (2010). In the case of the lower-level optimization problem is scalar one (i.e., $q=1$), Proposition 3.4 agrees with Dempe and Zemkoho (2014, Theorem 3.1).

We next provide calculus rules for computing or estimating the coderivative of L given in (3.2), which will be employed in the sequel. The statement (i) of Theorem 3.6 below reduces to Dempe and Zemkoho (2014, Theorem 3.3) for the case, where the lower-level optimization problem is scalar one (i.e., $q=1$), and an additional assumption on the local boundedness and closedness of the partial subdifferentials of $f_j, j=1,\ldots , q, g_t, t=1,\ldots , r$ has been imposed.

Theorem 3.6

Let $(\bar{x},\bar{y})\in \mathbb {R}^n\times R^m, \bar{\alpha }:=(\bar{\alpha }_1,\ldots ,\bar{\alpha }_q)\in \mathbb {R}^q$ and $\bar{\lambda }:=(\bar{\lambda }_1,\ldots ,\bar{\lambda }_r)\in \mathbb {R}^r.$

• (i) Let $\bar{w}\in L(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda })$. Assume that for all $z:=(v^1,\ldots ,v^q,u^1,\ldots ,u^r)$ with $v^j:=(v^j_1,\ldots ,v^j_m)\in \partial _yf_j(\bar{x},\bar{y}), j=1,\ldots , q, u^t:=(u^t_1,\ldots ,u^t_m)\in \partial _y g_t(\bar{x},\bar{y}), t=1,\ldots , r$ and $\sum _{j=1}^q\bar{\alpha }_jv^j+\sum _{t=1}^r\bar{\lambda }_tu^t=\bar{w}$, the following qualification condition holds:

  $$\begin{aligned}&\left[ v^j_*\in \partial ^2_yf_j(\bar{x},\bar{y},v^j)(0), j=1,\ldots , q, u^t_*\in \partial ^2_yg_t(\bar{x},\bar{y},u^t)(0), t\right. \nonumber \\&\left. =1,\ldots , r\mid \sum _{j=1}^qv^j_*+\sum _{t=1}^ru^t_*=0\right] \nonumber \\&\Rightarrow v^j_*=u^t_*=0 \text{ for } \text{ all } j=1,\ldots , q, t=1,\ldots ,r. \end{aligned}$$

  (3.18)

  Then for each $w:=(w_1,\ldots ,w_m)\in \mathbb {R}^m,$ we have

  $$\begin{aligned}&D^*L(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda },\bar{w})(w)\subseteq \mathop {\mathop {\bigcup }\limits _{z:\sum _{j=1}^q\bar{\alpha }_jv^j+\sum _{t=1}^r\bar{\lambda }_tu^t=\bar{w},}}\limits _{v^j\in \partial _yf_j(\bar{x},\bar{y}),u^t\in \partial _y g_t(\bar{x},\bar{y})} \Big [\Big (\sum _{j=1}^q\partial ^2_yf_j(\bar{x},\bar{y},v^j)(\bar{\alpha }_jw)\nonumber \\&+\sum _{t=1}^r\partial ^2_yg_t(\bar{x},\bar{y},u^t)(\bar{\lambda }_tw)\Big )\times \Big \{\big (\sum _{i=1}^mv^1_iw_i,\ldots ,\sum _{i=1}^mv^q_iw_i,\sum _{i=1}^mu^1_iw_i,\ldots ,\sum _{i=1}^mu^r_iw_i\big )\Big \}\Big ]. \end{aligned}$$

  (3.19)

• (ii) Let $f_j, j=1,\ldots , q, g_t, t=1,\ldots , r$ be continuously differentiable around $(\bar{x},\bar{y})$, and let the corresponding partial derivatives $\nabla _y f_j, j=1,\ldots , q, \nabla _y g_t, t=1,\ldots , r$ be Lipschitz continuous around this point. Let $\nabla _y f_j(\bar{x},\bar{y}):=(v^j_1,\ldots ,v^j_m), j=1,\ldots ,q, \nabla _y g_t (\bar{x},\bar{y}):= (u^t_1,\ldots ,u^t_m), t=1,\ldots , r.$ Then the qualification condition (3.18) is automatically satisfied, and for each $w:=(w_1,\ldots ,w_m)\in \mathbb {R}^m,$ it holds that

  $$\begin{aligned}&D^*L\big (\bar{x},\bar{y},\bar{\alpha },\bar{\lambda },L(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda })\big )(w)\subseteq \Big (\sum _{j=1}^q\partial \langle \bar{\alpha }_jw,\nabla _y f_j\rangle (\bar{x},\bar{y})\nonumber \\&+\sum _{t=1}^r\partial \langle \bar{\lambda }_tw, \nabla _y g_t\rangle (\bar{x},\bar{y})\Big )\times \Big (\sum _{i=1}^mv^1_iw_i,\ldots ,\sum _{i=1}^mv^q_iw_i, \sum _{i=1}^mu^1_iw_i,\ldots ,\sum _{i=1}^mu^r_iw_i\Big ). \end{aligned}$$

  (3.20)

  If in addition $\nabla _y f_j, j =1,\ldots , q, \nabla _y g_t, t=1,\ldots , r$ are strictly differentiable at $(\bar{x},\bar{y})$ with the partial derivatives $\nabla ^2_{yx}f_j(\bar{x},\bar{y}), \nabla ^2_{yy}f_j(\bar{x},\bar{y}), j=1,\ldots ,q, \nabla ^2_{yx}g_t(\bar{x},\bar{y}), \nabla ^2_{yy}g_t(\bar{x},\bar{y}), t=1,\ldots ,r,$ then the inclusion (3.20) holds as equality, i.e.,

  $$\begin{aligned}&D^*L\big (\bar{x},\bar{y},\bar{\alpha },\bar{\lambda },L(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda })\big )(w)= \Big \{\Big (\sum _{j=1}^q\bar{\alpha }_j(\nabla ^2_{yx}f_j(\bar{x},\bar{y})^\top w,\nabla ^2_{yy}f_j(\bar{x},\bar{y})^\top w)\\&\quad +\sum _{t=1}^r\bar{\lambda }_t(\nabla ^2_{yx}g_t(\bar{x},\bar{y})^\top w, \nabla ^2_{yy}g_t(\bar{x},\bar{y})^\top w)\Big )\\&\quad \times \Big (\sum _{i=1}^mv^1_iw_i,\ldots ,\sum _{i=1}^mv^q_iw_i, \sum _{i=1}^mu^1_iw_i,\ldots ,\sum _{i=1}^mu^r_iw_i\Big )\Big \}. \end{aligned}$$

Proof

We first prove (i). Consider a function $\Phi :\mathbb {R}^{m(q+r)}\times \mathbb {R}^{q}\times \mathbb {R}^r\rightarrow \mathbb {R}^m$ and a set-valued map $\Gamma :\mathbb {R}^n\times \mathbb {R}^m\times \mathbb {R}^q\times \mathbb {R}^r\rightrightarrows \mathbb {R}^{m(q+r)}\times \mathbb {R}^{q}\times \mathbb {R}^r$ defined, respectively, by

$$\begin{aligned}&\Phi (z,\alpha ,\lambda ):= \sum _{j=1}^q\alpha _jv^j+\sum _{t=1}^r\lambda _tu^t,\quad \alpha :=(\alpha _1,\ldots ,\alpha _q)\in \mathbb {R}^q, \lambda :=(\lambda _1,\ldots ,\lambda _r)\in \mathbb {R}^r,\\&z:=(v^1,\ldots ,v^q,u^1,\ldots ,u^r)\in \mathbb {R}^{m(q+r)} \text{ with } v^j:=(v^j_1,\ldots ,v^j_m)\in \mathbb {R}^m, j=1,\ldots , q, \\&u^t:=(u^t_1,\ldots ,u^t_m)\in \mathbb {R}^m, t=1,\ldots , r, \end{aligned}$$

and

$$\begin{aligned} \Gamma (x,y,\alpha ,\lambda ):= \{(z,\alpha ,\lambda )\mid z\in \Gamma _0(x,y)\},\quad x\in \mathbb {R}^n, y\in \mathbb {R}^m, \alpha \in \mathbb {R}^q, \lambda \in \mathbb {R}^r, \end{aligned}$$

with $\Gamma _0(x,y):=\partial _yf_1(x,y)\times \cdots \times \partial _yf_q(x,y)\times \partial _yg_1(x,y)\times \cdots \times \partial _yg_r(x,y).$ Then, it holds that $L=\Phi \circ \Gamma .$ Moreover, we see that the function $\Phi$ is continuously differentiable at every point $(z,\alpha ,\lambda )\in \mathbb {R}^{m(q+r)}\times \mathbb {R}^{q}\times \mathbb {R}^r$, and the set-valued map $\Gamma$ is closed inasmuch as $\partial _yf_j, j=1,\ldots , q, \partial _y g_t, t=1,\ldots , r,$ are closed by virtue of Proposition 3.3(i). Let us consider a set-valued map $M: \mathbb {R}^n\times \mathbb {R}^m\times \mathbb {R}^q\times \mathbb {R}^r\times \mathbb {R}^m\rightrightarrows \mathbb {R}^{m(q+r)}\times \mathbb {R}^{q}\times \mathbb {R}^r$ defined by

$$\begin{aligned} M(x,y,\alpha ,\lambda , w):=\Gamma (x,y,\alpha ,\lambda )\cap \Phi ^{-1}(w)= \{(z,\alpha ,\lambda )\mid z\in \Gamma _0(x,y), \Phi (z,\alpha ,\lambda )=w\} \end{aligned}$$

for $x\in \mathbb {R}^n, y\in \mathbb {R}^m, \alpha \in \mathbb {R}^q, \lambda \in \mathbb {R}^r, w\in \mathbb {R}^m.$ Invoking Proposition 3.3(i) again, it holds that $\partial _yf_j, j=1,\ldots , q, \partial _y g_t, t=1,\ldots , r,$ are locally bounded at $(\bar{x},\bar{y})$, and thus, M is locally bounded at $(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda },\bar{w})$. Moreover, it is easy to see that M has nonempty values around $(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda },\bar{w})$ due to the nonemptyness of $\partial _yf_j, j=1,\ldots , q, \partial _y g_t, t=1,\ldots , r,$ and thus, M is inner semicompact at this point. Applying Mordukhovich (2006a, Theorem 1.65(ii)), we have the following estimate

$$\begin{aligned}&D^*L(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda },\bar{w})(w)\nonumber \\&\quad \subseteq \bigcup _{(z,\alpha ,\lambda )\in M(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda },\bar{w})}\Big [D^*\Gamma ((\bar{x},\bar{y},\bar{\alpha },\bar{\lambda }),(z,\alpha ,\lambda ))(\nabla \Phi (z,\alpha ,\lambda )^\top w)\Big ], \end{aligned}$$

(3.21)

where $\nabla \Phi (z,\alpha ,\lambda )$ stands for the derivative of $\Phi$ at $(z,\alpha ,\lambda ).$ For each $w:=(w_1,\ldots ,w_m)\in \mathbb {R}^m,$ a direct computation shows that

$$\begin{aligned} \nabla \Phi (z,\alpha ,\lambda )^\top w&=\Big [\alpha _1 w,\ldots ,\alpha _q w, \lambda _1 w,\ldots ,\lambda _r w,\sum _{i=1}^mv^1_iw_i,\ldots ,\sum _{i=1}^mv^q_iw_i,\sum _{i=1}^mu^1_iw_i,\ldots ,\nonumber \\&\quad \sum _{i=1}^mu^r_iw_i\Big ]. \end{aligned}$$

(3.22)

To estimate the coderivative of $\Gamma$, we construct a function $\Psi :\mathbb {R}^n\times \mathbb {R}^m\times \mathbb {R}^q\times \mathbb {R}^r\times \mathbb {R}^{m(q+r)}\times \mathbb {R}^{q}\times \mathbb {R}^r\rightarrow \mathbb {R}^n\times \mathbb {R}^m\times \mathbb {R}^{m(q+r)}\times \mathbb {R}^{q}\times \mathbb {R}^r$ defined by

$$\begin{aligned} \Psi (x,y,\alpha ,\lambda ,z, \tilde{\alpha },\tilde{\lambda }):=(x,y,z, \alpha -\tilde{\alpha },\lambda -\tilde{\lambda }). \end{aligned}$$

Then, it holds that $\mathrm{gph\,}\Gamma =\Psi ^{-1}(\Theta )$ with $\Theta :=\mathrm{gph\,}\Gamma _0\times \{0_q\}\times \{0_r\}$, and that $\Psi$ is strictly differentiable at any $(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda },z,\tilde{\alpha },\tilde{\lambda })$ with the derivative $\nabla \Psi (\bar{x},\bar{y},\bar{\alpha },\bar{\lambda },z, \tilde{\alpha },\tilde{\lambda })$ and thus,

$$\begin{aligned} D^*\Psi (\bar{x},\bar{y},\bar{\alpha },\bar{\lambda },z, \tilde{\alpha },\tilde{\lambda })(a,b,c,d,e)= & {} \nabla \Psi (\bar{x},\bar{y},\bar{\alpha },\bar{\lambda },z, \tilde{\alpha },\tilde{\lambda })^\top (a,b,c,d,e)\\= & {} (a,b,d, e,c,-d,-e) \end{aligned}$$

for each $(a,b,c,d,e)\in N\big (\Psi (\bar{x},\bar{y},\bar{\alpha },\bar{\lambda },z,\tilde{\alpha },\tilde{\lambda });\Theta \big )=N\big ((\bar{x},\bar{y},z);\mathrm{gph\,}\Gamma _0\big )\times \mathbb {R}^q\times \mathbb {R}^r.$ It can be checked that $\nabla \Psi (\bar{x},\bar{y},\bar{\alpha },\bar{\lambda },z, \tilde{\alpha },\tilde{\lambda })$ is surjective, and then applying Lemma 2.1 allows us to assert that

$$\begin{aligned}&N\big ((\bar{x},\bar{y},\bar{\alpha },\bar{\lambda },z,\alpha ,\lambda ); \mathrm{gph\,}\Gamma \big )\\&\quad = \big \{(a,b,d,e,c,-d,-e)\mid (a,b,c,d,e)\in N\big ((\bar{x},\bar{y},z);\mathrm{gph\,}\Gamma _0\big )\times \mathbb {R}^q\times \mathbb {R}^r \big \}. \end{aligned}$$

Thus, it follows by the definition of coderivative (2.4) that

$$\begin{aligned} D^*\Gamma ((\bar{x},\bar{y},\bar{\alpha },\bar{\lambda }),(z, \alpha ,\lambda ))(s,\gamma ,\theta )= D^*\Gamma _0(\bar{x},\bar{y},z)(s)\times \{\gamma \}\times \{\theta \} \end{aligned}$$

for each $(s,\gamma ,\theta )\in \mathbb {R}^{m(q+r)}\times \mathbb {R}^{q}\times \mathbb {R}^r.$ Combining this with (3.21) and (3.22) yields

$$\begin{aligned} D^*L(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda },\bar{w})(w)&\subseteq \mathop {\mathop {\bigcup }\limits _{z: z\in \Gamma _0(\bar{x},\bar{y}),}}\limits _{\Phi (z,\bar{\alpha },\bar{\lambda })=\bar{w}}\Big [D^*\Gamma ((\bar{x},\bar{y},\bar{\alpha },\bar{\lambda }),(z,\bar{\alpha },\bar{\lambda }))(\nabla \Phi (z,\bar{\alpha },\bar{\lambda })^\top w)\Big ]\nonumber \\&\subseteq \mathop {\mathop {\bigcup }\limits _{z: z\in \Gamma _0(\bar{x},\bar{y}),}}\limits _{\Phi (z,\bar{\alpha },\bar{\lambda })=\bar{w}} \Big [D^*\Gamma _0(\bar{x},\bar{y},z)(\bar{\alpha }_1 w,\ldots ,\bar{\alpha }_q w, \bar{\lambda }_1 w,\ldots ,\bar{\lambda }_r w)\nonumber \\&\quad \times \big \{\big (\sum _{i=1}^mv^1_iw_i,\ldots ,\sum _{i=1}^mv^q_iw_i,\sum _{i=1}^mu^1_iw_i, \ldots ,\sum _{i=1}^mu^r_iw_i\big )\big \}\Big ]. \end{aligned}$$

(3.23)

Taking into account the definition of partial second-order subdifferential in (2.17), we assert that condition (2.6) is satisfied under the fulfillment of (3.18). Moreover, since $\partial _yf_j, j=1,\ldots , q, \partial _y g_t, t=1,\ldots , r,$ are closed, we invoke Lemma 2.2 to obtain that

$$\begin{aligned}&D^*\Gamma _0(\bar{x},\bar{y},z)(\bar{\alpha }_1 w,\ldots ,\bar{\alpha }_q w, \bar{\lambda }_1 w,\ldots ,\bar{\lambda }_r w)\subseteq \sum _{j=1}^q\partial ^2_yf_j(\bar{x},\bar{y},v^j)(\bar{\alpha }_jw)\nonumber \\&\quad +\sum _{t=1}^r\partial ^2_yg_t(\bar{x},\bar{y},u^t)(\bar{\lambda }_tw). \end{aligned}$$

(3.24)

Now, the estimate (3.19) follows by (3.23) and (3.24), which ends the proof (i).

Let us now prove (ii). Under additional assumptions, we assert by Lemma 2.7 that for each $w\in \mathbb {R}^m$,

$$\begin{aligned}&\partial _y^2f_j(\bar{x},\bar{y}, \nabla _{y}f_j(\bar{x},\bar{y}))(w)=\partial \langle w,\nabla _y f_j\rangle (\bar{x},\bar{y}), j=1,\ldots , q,\\&\partial _y^2g_t(\bar{x},\bar{y}, \nabla _{y}g_t(\bar{x},\bar{y}))(w)=\partial \langle w,\nabla _y g_t\rangle (\bar{x},\bar{y}), t=1,\ldots , r. \end{aligned}$$

Then,

$$\begin{aligned} \partial _y^2f_j(\bar{x},\bar{y}, \nabla _{y}f_j(\bar{x},\bar{y}))(0)= & {} \{0\}, j=1,\ldots , q, \partial _y^2g_t(\bar{x},\bar{y}, \nabla _{y}g_t(\bar{x},\bar{y}))(0)=\{0\}, t\\= & {} 1,\ldots , r, \end{aligned}$$

and thus, the qualification condition (3.18) is satisfied. So, the inclusion (3.20) holds by virtue of (3.19) and Lemma 2.7. If in addition $\nabla _y f_j, j=1,\ldots , q, \nabla _y g_t, t=1,\ldots , r$ are strictly differentiable at $(\bar{x},\bar{y})$, then one can employ Mordukhovich (2006a, Theorem 1.65(iii)) instead of Mordukhovich (2006a, Theorem 1.65(ii)) to obtain an equality in (3.21), and thus, (3.23) becomes as equality as well. In this circumstance, the inclusion (3.24) becomes as equality due to Lemma 2.2 under normal regularities of $\mathrm{gph\,}\nabla _yf_j, j=1,\ldots , q,$ and $\mathrm{gph\,}\nabla _yg_t, t=1,\ldots ,r.$$\Box$

4 Optimality via KKT relaxation schemes for multiobjective bilevel optimization

An often-used approach to dealing with a bilevel optimization problem is to replace the lower-level problem by its KKT relaxation schemes. Inspired by the scalar one in Dempe and Zemkoho (2014), we first propose a KKT relaxation multiobjective problem for the multiobjective bilevel optimization problem $\mathrm{P}$ and explore relationships of solutions between them. We then establish FJ and KKT necessary conditions in terms of the first- and second-orders limiting subdifferentials for a KKT relaxation multiobjective formulation of the multiobjective bilevel optimization problem $\mathrm{P}$. In this vein, we are able to obtain necessary conditions in the forms of the M-,C-, and S-type stationarity. For the sake of concise presentation, we only provide FJ and KKT necessary conditions that are of the M-type stationarity.

A KKT relaxation multiobjective formulation of the problem $\mathrm{P}$ reads as follows.

where $L(x,y,\alpha ,\lambda )$ is defined as in (3.2). The constraint set of the problem $\mathrm{RP}$ is denoted by

$$\begin{aligned} C_R&:=\big \{(x,y,\alpha ,\lambda )\in \mathbb {R}^{n+m+q+r}\mid 0\in L(x,y,\alpha ,\lambda ),\, G_k(x) \le 0, \ k=1,\ldots ,l,\nonumber \\&\quad g_t(x,y)\le 0, \lambda _tg_t(x,y)=0, t=1,\ldots , r,\nonumber \\&\quad \alpha :=(\alpha _1,\ldots ,\alpha _q)\in \mathbb {R}^q_+\cap \mathbb {S}_q, \lambda :=(\lambda _1,\ldots ,\lambda _r)\in \mathbb {R}^r_+\big \}. \end{aligned}$$

(4.1)

It should be noted here that a local weak Pareto solution of the problem $\mathrm{RP}$ is similarly defined as in Definition 1.1(ii), and we denote by locS(RP) the set of local weak Pareto solution of this problem.

The first theorem describes a relationship between local weak Pareto solutions of problem $\mathrm{P}$ and problem $\mathrm{RP}$. The statement of this theorem is inspired by the scalar one in Dempe and Zemkoho (2014, Theorem 4.1).

Theorem 4.1

Consider the map $\Lambda$ given by (3.3), and let the (SCQ) be satisfied at $\bar{x}\in \mathbb {R}^n.$

• (i) If $(\bar{x},\bar{y})\in locS(P),$ then for each $(\bar{\alpha },\bar{\lambda })\in \Lambda (\bar{x},\bar{y})$, one has $(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda })\in locS(RP).$

• (ii) If $(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda })\in locS(RP)$ for all $(\bar{\alpha },\bar{\lambda })\in \Lambda (\bar{x},\bar{y}),$ then it holds that $(\bar{x},\bar{y})\in locS(P).$

Proof

To justify (i), we let $(\bar{x},\bar{y})\in locS(P).$ Suppose for contradiction that there is $(\tilde{\alpha },\tilde{\lambda })\in \Lambda (\bar{x},\bar{y})$ such that $(\bar{x},\bar{y},\tilde{\alpha },\tilde{\lambda })\notin locS(RP).$ Then, we can find a sequence $\{(x^n,y^n,\alpha ^n,\lambda ^n)\}\subset C_R$ such that $(x^n,y^n,\alpha ^n,\lambda ^n)\rightarrow (\bar{x},\bar{y},\tilde{\alpha },\tilde{\lambda })$ and

$$\begin{aligned} F_i(x^n,y^n)< F_i(\bar{x},\bar{y}),\quad i=1,\ldots ,p,\quad n\in \mathbb {N}, \end{aligned}$$

where $\alpha ^n:=(\alpha ^n_1,\ldots ,\alpha ^n_q)$ and $\lambda ^n:=(\lambda ^n_1,\ldots ,\lambda ^n_r).$ As $\{(x^n,y^n,\alpha ^n,\lambda ^n)\}\subset C_R$, it follows that

$$\begin{aligned}&0\in \sum _{j=1}^q\alpha ^n_j\partial _y f_j(x^n,y^n)+\sum _{t=1}^r\lambda ^n_t\partial _y g_t(x^n,y^n), \end{aligned}$$

(4.2)

$$\begin{aligned}&||\alpha ^n||=1,\; g_t(x^n,y^n)\le 0, \;\lambda ^n_tg_t(x^n,y^n)=0, \; t=1,\ldots , r,\nonumber \\&G_k(x^n) \le 0, \ k=1,\ldots ,l,\quad n\in \mathbb {N}. \end{aligned}$$

(4.3)

Since the (SCQ) is satisfied at $\bar{x}$, i.e., there exists $\hat{y}\in \mathbb {R}^m$ such that

$$\begin{aligned} g_t(\bar{x},\hat{y})< 0,\quad t=1,\ldots , r, \end{aligned}$$

it entails that there is $n^0\in \mathbb {N}$ such that

$$\begin{aligned} g_t(x^n,\hat{y})< 0,\quad t=1,\ldots , r, \end{aligned}$$

for all $n\ge n^0.$ It means that the (SCQ) is satisfied at $x^n$ for all $n\ge n^0,$ without loss of generality we assume that the (SCQ) is satisfied at $x^n$ for all $n\in \mathbb {N}.$ This together with (4.2) and (4.3) confirms that $y^n\in S(x^n)$ by virtue of (3.5) for all $n\in \mathbb {N}$. Hence, we arrive at the following assertions

$$\begin{aligned}&F_i(x^n,y^n)< F_i(\bar{x},\bar{y}),\quad i=1,\ldots ,p,\\&y^n\in S(x^n),\; G_k(x^n) \le 0, \ k=1,\ldots ,l,\; n\in \mathbb {N},\\&(x^n,y^n)\rightarrow (\bar{x},\bar{y}) \text{ as } n\rightarrow \infty , \end{aligned}$$

or equivalently,

$$\begin{aligned}&F_i(x^n,y^n)< F_i(\bar{x},\bar{y}),\quad i=1,\ldots ,p,\\&(x^n,y^n)\in C, \; n\in \mathbb {N},\\&(x^n,y^n)\rightarrow (\bar{x},\bar{y}) \text{ as } n\rightarrow \infty , \end{aligned}$$

where C is given in (1.1), which contradicts the fact that $(\bar{x},\bar{y})\in locS(P).$

To justify (ii), we let $(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda })\in locS(RP)$ for all $(\bar{\alpha },\bar{\lambda })\in \Lambda (\bar{x},\bar{y}).$ Then, for each $(\bar{\alpha },\bar{\lambda })\in \Lambda (\bar{x},\bar{y})$, it holds that

$$\begin{aligned}&0\in \sum _{j=1}^q\bar{\alpha }_j\partial _y f_j(\bar{x},\bar{y})+\sum _{t=1}^r\bar{\lambda }_t\partial _y g_t(\bar{x},\bar{y}), \end{aligned}$$

(4.4)

$$\begin{aligned}&||\bar{\alpha }||=1,\; g_t(\bar{x},\bar{y})\le 0, \;\bar{\lambda }_tg_t(\bar{x},\bar{y})=0, \; t=1,\ldots ,r, \end{aligned}$$

(4.5)

$$\begin{aligned}&G_k(\bar{x}) \le 0, \ k=1,\ldots ,l. \end{aligned}$$

(4.6)

where $\bar{\alpha }:=(\bar{\alpha }_1,\ldots ,\bar{\alpha }_q)$ and $\bar{\lambda }:=(\bar{\lambda }_1,\ldots ,\bar{\lambda }_r).$ With the fulfilment of the (SCQ) at $\bar{x}$, invoking (3.5) again, we get by (4.4) and (4.5) that $\bar{y}\in S(\bar{x}).$ This together with (4.6) yields $(\bar{x},\bar{y})\in C$ defined in (1.1). Arguing by contradiction that $(\bar{x},\bar{y})\notin locS(P).$ This means that there exists $\{(x^n,y^n)\}\subset C$ such that $(x^n,y^n)\rightarrow (\bar{x},\bar{y})$ as $n\rightarrow \infty$ and

$$\begin{aligned} F_i(x^n,y^n)< F_i(\bar{x},\bar{y}),\quad i=1,\ldots ,p,\quad n\in \mathbb {N}. \end{aligned}$$

Similarly as above, we may assume that the (SCQ) is satisfied at $x^n$ for all $n\in \mathbb {N}.$ For each $n\in \mathbb {N},$ the relation $(x^n,y^n)\in C$ entails that $G_k(x^n) \le 0, \ k=1,\ldots ,l$ and $y^n\in S(x^n)$, and then, by (3.5), we find $\alpha ^n:=(\alpha _1^n,\ldots ,\alpha _q^n)\in \mathbb {R}^q_+\cap \mathbb {S}_q$ and $\lambda ^n:=(\lambda _1^n,\ldots ,\lambda _r^n)\in \mathbb {R}^r_+$ such that

$$\begin{aligned}&0\in \sum _{j=1}^q\alpha ^n_j\partial _y f_j(x^n,y^n)+\sum _{t=1}^r\lambda ^n_t\partial _y g_t(x^n,y^n), \\&g_t(x^n,y^n)\le 0, \;\lambda ^n_tg_t(x^n,y^n)=0, \; t=1,\ldots , r, \end{aligned}$$

which means that $(\alpha ^n,\lambda ^n)\in \Lambda (x^n,y^n).$ Since the (SCQ) is satisfied at $\bar{x},$ we apply Proposition 3.4(ii) to conclude that $\Lambda$ is locally bounded at $(\bar{x},\bar{y}).$ Hence, by taking a subsequence if necessary, we may assume that $(\alpha ^n,\lambda ^n)$ converges to some $(\tilde{\alpha },\tilde{\lambda }).$ Furthermore, we asset that $(\tilde{\alpha },\tilde{\lambda })\in \Lambda (\bar{x},\bar{y})$ as $\Lambda$ is closed at $(\bar{x},\bar{y})$ thanks to Proposition 3.4(i). Consequently, we find a sequence $\{(x^n,y^n,\alpha ^n,\lambda ^n)\}\subset C_R$ such that $(x^n,y^n,\alpha ^n,\lambda ^n)\rightarrow (\bar{x},\bar{y},\tilde{\alpha },\tilde{\lambda })$ and

$$\begin{aligned} F_i(x^n,y^n)< F_i(\bar{x},\bar{y}),\quad i=1,\ldots ,p,\quad n\in \mathbb {N}. \end{aligned}$$

This contradicts the fact that $(\bar{x},\bar{y},\tilde{\alpha },\tilde{\lambda })\in locS(RP)$ because of $(\tilde{\alpha },\tilde{\lambda })\in \Lambda (\bar{x},\bar{y}).$ The proof of the theorem is complete. $\Box$

Remark 4.2

As shown by Dempe and Dutta (2012, Example 3.1) with the smooth setting that, in Theorem 4.1(ii), if the condition $(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda })\in locS(RP)$ holds not for all $(\bar{\alpha },\bar{\lambda })\in \Lambda (\bar{x},\bar{y}),$ then the corresponding result may go awry. The reader is referred to Dempe and Dutta (2012) for further study on relationships of global solutions between a smooth scalar bi-level problem and its KKT formulation one.

Let us establish FJ necessary condition for the KKT relaxation multiobjective formulation of the multiobjective bilevel optimization problem $\mathrm{P}$.

Theorem 4.3

Let $(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda })\in locS(RP)$ with $\bar{\alpha }:=(\bar{\alpha }_1,\ldots ,\bar{\alpha }_q)$ and $\bar{\lambda }:=(\bar{\lambda }_1,\ldots ,\bar{\lambda }_r)$. Assume that for all $z:=(v^1,\ldots ,v^q,u^1,\ldots ,u^r)$ with $v^j:=(v^j_1,\ldots ,v^j_m)\in \partial _yf_j(\bar{x},\bar{y}), j=1,\ldots , q, u^t:=(u^t_1,\ldots ,u^t_m)\in \partial _y g_t(\bar{x},\bar{y}), t=1,\ldots , r$ satisfying

$$\begin{aligned} \sum _{j=1}^q\bar{\alpha }_jv^j+\sum _{t=1}^r\bar{\lambda }_tu^t=0, \end{aligned}$$

(4.7)

the qualification condition (3.18) holds. Then there exist $\nu :=(\nu _1,\ldots ,\nu _p)\in \mathbb {R}^p_+, \eta :=(\eta _1,\ldots ,\eta _l)\in \mathbb {R}^l_+, \beta :=(\beta _1,\ldots ,\beta _r)\in \mathbb {R}^r_+, \gamma \in \mathbb {R}_+,$ not all zero, and $z:=(v^1,\ldots ,v^q,u^1,\ldots ,u^r)$ satisfying (4.7) as well as $\omega :=(\omega _1,\ldots ,\omega _m)\in \mathbb {R}^m$ with $||\omega ||\le 1$, such that

$$\begin{aligned}&0\in \sum _{i=1}^p\nu _i\partial F_i(\bar{x},\bar{y})+ \sum _{k=1}^l\eta _k\big (\partial G_k(\bar{x}),0_m\big )+ \sum _{t=1}^r\beta _t\big (\partial g_t(\bar{x},\bar{y})\cup \partial (-g_t)(\bar{x},\bar{y})\big )\nonumber \\&\quad +\sum _{j=1}^q\gamma \partial ^2_yf_j(\bar{x},\bar{y},v^j)(\bar{\alpha }_j\omega )+\sum _{t=1}^r\gamma \partial ^2_yg_t(\bar{x},\bar{y},u^t)(\bar{\lambda }_t\omega ), \end{aligned}$$

(4.8)

$$\begin{aligned}&\quad \eta _kG_k(\bar{x})=0,\; k=1,\ldots , l, \end{aligned}$$

(4.9)

$$\begin{aligned}&\quad \gamma \left( \sum _{i=1}^mv^j_i\omega _i\right) \ge 0,\; j\in J(\bar{\alpha }), \end{aligned}$$

(4.10)

$$\begin{aligned}&\quad \gamma \left( \sum _{i=1}^mu^t_i\omega _i\right) =0,\; t\in I_3(\bar{x},\bar{y},\bar{\lambda }), \end{aligned}$$

(4.11)

$$\begin{aligned}&\quad \beta _t=0,\; t\in I_1(\bar{x},\bar{y},\bar{\lambda }), \end{aligned}$$

(4.12)

$$\begin{aligned}&\quad \beta _t\gamma \left( \sum _{i=1}^mu^t_i\omega _i\right) =0 \vee \left( \beta _t>0\wedge \gamma \left( \sum _{i=1}^mu^t_i\omega _i\right) >0\right) ,\; t\in I_2(\bar{x},\bar{y},\bar{\lambda }), \end{aligned}$$

(4.13)

where

$$\begin{aligned} J(\bar{\alpha }):&=\{j=1,\ldots ,q\mid \bar{\alpha }_j=0\}, \end{aligned}$$

(4.14)

$$\begin{aligned} I_1(\bar{x},\bar{y},\bar{\lambda }):&=\{t=1,\ldots ,r\mid \bar{\lambda }_t=0, g_t(\bar{x},\bar{y})<0\}, \end{aligned}$$

(4.15)

$$\begin{aligned} I_2(\bar{x},\bar{y},\bar{\lambda }):&=\{t=1,\ldots ,r\mid \bar{\lambda }_t=0, g_t(\bar{x},\bar{y})=0\}, \end{aligned}$$

(4.16)

$$\begin{aligned} I_3(\bar{x},\bar{y},\bar{\lambda }):&=\{t=1,\ldots ,r\mid \bar{\lambda }_t>0, g_t(\bar{x},\bar{y})=0\}. \end{aligned}$$

(4.17)

Proof

Note that the set-valued map L given in (3.2) is closed by virtue of Proposition 3.3(ii). Then, the relationship $0\in L(x,y,\alpha ,\lambda )$ is nothing else but $d\big ((x,y,\alpha ,\lambda ,0);\mathrm{gph\,}L\big )=0,$ or equivalently, $d\big ((x,y,\alpha ,\lambda ,0);\mathrm{gph\,}L\big )\le 0$ due to that fact that $d\big ((x,y,\alpha ,\lambda ,0);\mathrm{gph\,}L\big )\ge 0.$ Consider the functions $h:\mathbb {R}^n\times \mathbb {R}^m\times \mathbb {R}^q\times \mathbb {R}^r\rightarrow \mathbb {R}$ and $\psi :\mathbb {R}^n\times \mathbb {R}^m\times \mathbb {R}^q\times \mathbb {R}^r\rightarrow \mathbb {R}^l\times \mathbb {R}\times \mathbb {R}^q\times \mathbb {R}^r\times \mathbb {R}^r$ defined respectively by

$$\begin{aligned} h(x,y,\alpha ,\lambda ):&=d\big ((x,y,\alpha ,\lambda ,0);\mathrm{gph\,}L\big ),\nonumber \\ \psi (x,y,\alpha ,\lambda ):&=[G(x),h(x,y,\alpha ,\lambda ),\alpha ,(\lambda ,-g(x,y))], \end{aligned}$$

(4.18)

where $G(x):=(G_1(x),\ldots ,G_l(x))$ and $g(x,y):=(g_1(x,y),\ldots ,g_r(x,y)).$

Now, the problem $\mathrm{RP}$ can be reformulated in the following form:

$$\begin{aligned} V-\min _{x,y,\alpha ,\lambda }{\big \{ (F_1(x,y),\ldots ,F_p(x,y)) \mid (x,y,\alpha ,\lambda )\in \psi ^{-1}(\Theta )\big \}}, \end{aligned}$$

(4.19)

where $\psi$ was defined by (4.18), $\Theta :=\mathbb {R}^l_{-}\times \mathbb {R}_{-}\times \Theta _1\times \Theta _2$ with $\Theta _1:=\{\alpha :=(\alpha _1,\ldots ,\alpha _q)\in \mathbb {R}^q\mid \alpha \in \mathbb {R}^q_+\cap \mathbb {S}_q\}$ and $\Theta _2:=\{(a,b)\in \mathbb {R}^r\times \mathbb {R}^r\mid a\in \mathbb {R}^r_+, b\in \mathbb {R}^r_+, \langle a,b\rangle =0\}.$ Since $(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda })$ is a local weak Pareto solution of the problem (4.19), applying Lemma 2.8 to this problem, we find $\nu :=(\nu _1,\ldots ,\nu _p)\in \mathbb {R}^p_+$ and $v^*\in N\big (\psi (\bar{x},\bar{y},\bar{\alpha },\bar{\lambda });\Theta \big )$, not all zero, such that

$$\begin{aligned} \nonumber 0&\in \sum _{i=1}^p\nu _i \partial _{x,y,\alpha ,\lambda } F_i(\bar{x},\bar{y})+\partial \langle v^*,\psi \rangle (\bar{x},\bar{y},\bar{\alpha },\bar{\lambda })\\&=\sum _{i=1}^p\nu _i \partial F_i(\bar{x},\bar{y})\times \{(0_q,0_r)\}+\partial \langle v^*,\psi \rangle (\bar{x},\bar{y},\bar{\alpha },\bar{\lambda }). \end{aligned}$$

(4.20)

Using the product rule for limiting normal cones (cf. Mordukhovich 2006a, Proposition 1.2), it holds that

$$\begin{aligned} N\big (\psi (\bar{x},\bar{y},\bar{\alpha },\bar{\lambda });\Theta \big )&= N\big (G(\bar{x});\mathbb {R}^l_{-}\big )\times N\big (h(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda });\mathbb {R}_{-}\big )\nonumber \\&\quad \times N\big (\bar{\alpha };\Theta _1\big )\times N\big ((\bar{\lambda },-g(\bar{x},\bar{y}));\Theta _2\big ). \end{aligned}$$

(4.21)

To calculate the normal cones in the right-hand side of (4.21), we apply Lemmas 2.4 and 2.5 to obtain that

$$\begin{aligned}&N(G(\bar{x});\mathbb {R}^l_{-}):=\{\eta :=(\eta _1,\ldots ,\eta _l)\in \mathbb {R}^l\mid \eta _k\ge 0, \; \eta _kG_k(\bar{x})=0,\; k=1,\ldots , l\}, \end{aligned}$$

(4.22)

$$\begin{aligned}&N(h(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda });\mathbb {R}_{-}):=\{\gamma \in \mathbb {R}\mid \gamma \ge 0\}, \end{aligned}$$

(4.23)

$$\begin{aligned}&N(\bar{\alpha };\Theta _1):=\{\mu :=(\mu _1,\ldots ,\mu _q)\in \mathbb {R}^q\mid \mu _j\le 0,\; j\in J(\bar{\alpha }),\nonumber \\&\qquad \mu _j:=\tilde{\mu }\bar{\alpha }_j, \tilde{\mu }\in \mathbb {R},\; j\notin J(\bar{\alpha })\}, \end{aligned}$$

(4.24)

and

$$\begin{aligned}&N((\bar{\lambda },-g(\bar{x},\bar{y}));\Theta _2):=\{(\zeta ,\tilde{\beta }):=(\zeta _1,\ldots ,\zeta _r,\tilde{\beta }_1,\ldots ,\tilde{\beta }_r)\in \mathbb {R}^r\times \mathbb {R}^r\mid \zeta _t\nonumber \\&\quad =0,\; t\in I_3(\bar{x},\bar{y},\bar{\lambda }),\nonumber \\&\quad \tilde{\beta }_t=0, \; t\in I_1(\bar{x},\bar{y},\bar{\lambda }),\nonumber \\&\quad (\zeta _t<0 \wedge \tilde{\beta }_t<0)\vee \zeta _t\tilde{\beta }_t=0,\; t\in I_2(\bar{x},\bar{y},\bar{\lambda })\}, \end{aligned}$$

(4.25)

where $J(\bar{\alpha }), I_1(\bar{x},\bar{y},\bar{\lambda }), I_2(\bar{x},\bar{y},\bar{\lambda })$ and $I_3(\bar{x},\bar{y},\bar{\lambda })$ were given in (4.14)–(4.17), respectively.

So, it holds that

$$\begin{aligned}&N\big (\psi (\bar{x},\bar{y},\bar{\alpha },\bar{\lambda });\Theta \big )\\&\quad =\big \{(\eta ,\gamma ,\mu ,\zeta ,\tilde{\beta })\mid \eta ,\gamma ,\mu ,\zeta ,\tilde{\beta } \text{ satisfying } (4.22){-}(4.25), \text{ respectively }\big \}. \end{aligned}$$

Denoting by $v^*:=(\eta ,\gamma ,\mu ,\zeta ,\tilde{\beta })\in N\big (\psi (\bar{x},\bar{y},\bar{\alpha },\bar{\lambda });\Theta \big )$, we have

$$\begin{aligned} \langle v^*,\psi \rangle (x,y,\alpha ,\lambda )=\sum _{k=1}^l\eta _k G_k(x)+\gamma h(x,y,\alpha ,\lambda )+ \sum _{j=1}^q\mu _j\alpha _j+\sum _{t=1}^r \zeta _t\lambda _t-\sum _{t=1}^r\tilde{\beta }_tg_t(x,y). \end{aligned}$$

(4.26)

Since the functions given in (4.26) are locally Lipschitz continuous at $(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda })$, applying the limiting subdifferential sum rule in Lemma 2.3 yields the following inclusions

$$\begin{aligned} \partial \langle v^*,\psi \rangle (\bar{x},\bar{y},\bar{\alpha },\bar{\lambda })&\subseteq \sum _{k=1}^l\eta _k \partial G_k(\bar{x})\times \{(0_m,0_q,0_r)\}+\gamma \partial h(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda })\nonumber \\&\quad + \{(0_n,0_m)\}\times \{\mu \}\times \{0_r\}\nonumber \\&\quad +\{(0_n,0_m,0_q)\}\times \{\zeta \}+\sum _{t=1}^r\partial \big (-\tilde{\beta }_tg_t\big )(\bar{x},\bar{y})\times \{(0_q,0_r)\}\nonumber \\&\subseteq \sum _{k=1}^l\eta _k \partial G_k(\bar{x})\times \{(0_m,0_q,0_r)\}+ \gamma \partial h(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda })\nonumber \\&\quad + \sum _{t=1}^r\beta _t\big (\partial g_t(\bar{x},\bar{y})\cup \partial (-g_t)(\bar{x},\bar{y})\big )\times \{(\mu ,\zeta )\}, \end{aligned}$$

(4.27)

where $\beta _t:=|\tilde{\beta }_t|$ for each $t=1,\ldots , r$, and the second inclusion holds inasmuch as $\partial \big (-\tilde{\beta }_tg_t\big )(\bar{x},\bar{y})\subset |\tilde{\beta }_t|\big (\partial g_t(\bar{x},\bar{y})\cup \partial (-g_t)(\bar{x},\bar{y})\big )$ for all $t=1,\ldots , r.$ Due to the Lipschitz continuity of the distance function $d(\cdot ;\mathrm{gph\,}L),$ using Lemma 2.6 allows us to obtain the following ones

$$\begin{aligned}&\partial h(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda })=\partial _{x,y,\alpha ,\lambda } d\big ((\bar{x},\bar{y},\bar{\alpha },\bar{\lambda },0);\mathrm{gph\,}L\big )\nonumber \\&\subseteq \big \{(a,b,c,d)\in \mathbb {R}^{n+m+q+r}\mid \exists e\in \mathbb {R}^m \text{ with } (a,b,c,d,e)\nonumber \\&\quad \in \partial d\big ((\bar{x},\bar{y},\bar{\alpha },\bar{\lambda },0);\mathrm{gph\,}L\big )\big \}\nonumber \\&=\big \{(a,b,c,d)\in \mathbb {R}^{n+m+q+r}\mid \exists e\in \mathbb {R}^m \text{ with } (a,b,c,d,e)\nonumber \\&\quad \in N\big ((\bar{x},\bar{y},\bar{\alpha },\bar{\lambda },0);\mathrm{gph\,}L\big )\cap I\!\!B\big \} \end{aligned}$$

(4.28)

$$\begin{aligned}&\subseteq \bigcup _{\omega \in I\!\!B_m}D^*L\big (\bar{x},\bar{y},\bar{\alpha },\bar{\lambda },0)(\omega ), \end{aligned}$$

(4.29)

where $I\!\!B\subset \mathbb {R}^{n+m+q+r+m},$ and noting further that the equality (4.28) holds by virtue of (2.10) and the inclusion (4.29) is valid because of (2.4). It together with Theorem 3.6, under the fulfillment of condition (3.18), entails that

$$\begin{aligned}&\gamma \partial h(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda })\subseteq \bigcup _{\omega : ||\omega ||\le 1}\mathop {\mathop {\bigcup }\limits _{z: \sum _{j=1}^q\bar{\alpha }_jv^j+\sum _{t=1}^r\bar{\lambda }_tu^t=0,}}\limits _{v^j\in \partial _yf_j(\bar{x},\bar{y}),u^t\in \partial _y g_t(\bar{x},\bar{y})} \gamma \Big [\Big (\sum _{j=1}^q\partial ^2_yf_j(\bar{x},\bar{y},v^j)(\bar{\alpha }_j\omega )\nonumber \\&+\sum _{t=1}^r\partial ^2_yg_t(\bar{x},\bar{y},u^t)(\bar{\lambda }_t\omega )\Big )\times \Big \{\big (\sum _{i=1}^mv^1_i\omega _i,\ldots ,\sum _{i=1}^mv^q_i\omega _i,\sum _{i=1}^mu^1_i\omega _i,\ldots ,\sum _{i=1}^mu^r_i\omega _i\big )\Big \}\Big ], \end{aligned}$$

(4.30)

where $\omega :=(\omega _1,\ldots ,\omega _m)\in \mathbb {R}^m.$ We deduce from (4.27) and (4.30) the following estimate

$$\begin{aligned}&\partial \langle v^*,\psi \rangle (\bar{x},\bar{y},\bar{\alpha },\bar{\lambda })\subseteq \left[ \sum _{k=1}^l\eta _k \big (\partial G_k(\bar{x}),0_m\big )\right. \nonumber \\&\quad \left. +\sum _{t=1}^r\beta _t\big (\partial g_t(\bar{x},\bar{y})\cup \partial (-g_t)(\bar{x},\bar{y})\big )\right] \times \{(\mu ,\zeta )\}\nonumber \\&\quad +\bigcup _{\omega : ||\omega ||\le 1}\mathop {\mathop {\bigcup }\limits _{z: \sum _{j=1}^q\bar{\alpha }_jv^j+\sum _{t=1}^r\bar{\lambda }_tu^t=0,}}\limits _{v^j\in \partial _yf_j(\bar{x},\bar{y}),u^t\in \partial _y g_t(\bar{x},\bar{y})} \gamma \Big [\Big (\sum _{j=1}^q\partial ^2_yf_j(\bar{x},\bar{y},v^j)(\bar{\alpha }_j\omega )\nonumber \\&\quad +\sum _{t=1}^r\partial ^2_yg_t(\bar{x},\bar{y},u^t)(\bar{\lambda }_t\omega )\Big )\nonumber \\&\quad \times \Big \{\big (\sum _{i=1}^mv^1_i\omega _i,\ldots ,\sum _{i=1}^mv^q_i\omega _i,\sum _{i=1}^mu^1_i\omega _i,\ldots ,\sum _{i=1}^mu^r_i\omega _i\big )\Big \}\Big ]. \end{aligned}$$

(4.31)

We assert by the relation (4.20) that there are $u^*_i\in \partial F_i(\bar{x},\bar{y}), i=1,\ldots ,p$ such that

$$\begin{aligned} -\sum _{i=1}^p\nu _iu_i^*\times \{(0_q,0_r)\}\in \partial \langle v^*,\psi \rangle (\bar{x},\bar{y},\bar{\alpha },\bar{\lambda }). \end{aligned}$$

Thus, by (4.31), there exist $\omega :=(\omega _1,\ldots ,\omega _m)\in \mathbb {R}^m$ with $||\omega ||\le 1$ and $z:=(v^1,\ldots ,v^q,u^1,\ldots ,u^r)$ with $v^j:=(v^j_1,\ldots ,v^j_m)\in \partial _yf_j(\bar{x},\bar{y}), j=1,\ldots , q, u^t:=(u^t_1,\ldots ,u^t_m)\in \partial _y g_t(\bar{x},\bar{y}), t=1,\ldots , r$, $\sum _{j=1}^q\bar{\alpha }_jv^j+\sum _{t=1}^r\bar{\lambda }_tu^t=0$ such that

$$\begin{aligned} -\sum _{i=1}^p\nu _iu_i^*\in&\Big [ \sum _{k=1}^l\eta _k \big (\partial G_k(\bar{x}),0_m\big )+\sum _{t=1}^r\beta _t\big (\partial g_t(\bar{x},\bar{y})\cup \partial (-g_t)(\bar{x},\bar{y})\big )\\&+\sum _{j=1}^q\gamma \partial ^2_yf_j(\bar{x},\bar{y},v^j)(\bar{\alpha }_j\omega )+\sum _{t=1}^r\gamma \partial ^2_yg_t(\bar{x},\bar{y},u^t)(\bar{\lambda }_t\omega )\Big ],\\ (0_q,0_r)=&(\mu ,\zeta )+\big (\sum _{i=1}^m\gamma v^1_i\omega _i,\ldots ,\sum _{i=1}^m\gamma v^q_i\omega _i,\sum _{i=1}^m\gamma u^1_i\omega _i,\ldots ,\sum _{i=1}^m\gamma u^r_i\omega _i\big ), \end{aligned}$$

where one can easily check that the conditions (4.8)–(4.13) are satisfied. Here, it should be noted further that if $\nu ,\eta ,\beta ,\gamma$ are simultaneously equal to zero, then it confirms that $\nu =0$ and $v^*=(\eta ,\gamma ,\mu ,\zeta ,\tilde{\beta })=0$, which contradicts the assertion in (4.20). Consequently, $\nu ,\eta ,\beta ,\gamma$ are not all equal to zero. The proof of the theorem is complete. $\Box$

We now provide an example which illustrates the obtained results.

Example 4.4

Consider the problem $\mathrm{P}$ with $F_1, F_2, f_1, f_2, g_1, g_2: \mathbb {R}^2\rightarrow \mathbb {R}$ and $G_1, G_2:\mathbb {R}\rightarrow \mathbb {R}$ defined, respectively, by

$$\begin{aligned}&F_1(x,y):=|x|-|y|,\quad F_2(x,y):=-|x|-y, \quad f_1(x,y):=x+y,\quad f_2(x,y):=|x-y|,\\&g_1(x,y):=y,\quad g_2(x,y):=x+|y|,\quad G_1(x):=x^3,\quad G_2(x):=|x|-1. \end{aligned}$$

Observe that the functions $f_j(x,\cdot ), j=1,2, g_t(x,\cdot ), t=1,2$ are convex for all $x\in \mathbb {R}$, and $\partial _yf_1(x,y)= 1, \partial _yg_1(x,y)=1$ for all $(x,y)\in \mathbb {R}^2,$ as well as

$$\begin{aligned} \partial _yf_2(x,y)={\left\{ \begin{array}{ll} -1 &{} \text{ if } x>y,\\ 1 &{} \text{ if } x<y,\\ {[}-1,1] &{} \text{ if } x=y, \end{array}\right. } \quad \partial _yg_2(x,y)={\left\{ \begin{array}{ll} -1 &{} \text{ if } y<0,\\ 1 &{} \text{ if } y>0,\\ {[}-1,1] &{} \text{ if } y=0. \end{array}\right. } \end{aligned}$$

(4.32)

Then, we can compute the feasible set of the problem $\mathrm{RP}$ defined by (4.1) as

$$\begin{aligned}&C_R:=\big \{(x,y,\alpha ,\lambda )\in \mathbb {R}\times \mathbb {R}\times \mathbb {R}^{2}\times \mathbb {R}^2 \mid 0\\&\quad \in \sum _{j=1}^2\alpha _j\partial _yf_j(x,y)+\sum _{t=1}^2\lambda _t\partial _yg_t(x,y),\\&\quad -1\le x\le 0, y\le 0, x+|y|\le 0,\\&\quad \lambda _1y=0, \lambda _2(x+|y|)=0,\\&\quad \alpha :=(\alpha _1,\alpha _2)\in \mathbb {R}^2_+\cap \mathbb {S}_2, \lambda :=(\lambda _1,\lambda _2)\in \mathbb {R}^2_+\big \}\\&\quad =\big \{(0,0,\alpha ,\lambda )\mid \alpha _1\ge 0, \alpha _2\ge 0, \alpha _1^2+\alpha _2^2=1, \lambda _1\ge 0, \lambda _2\ge 0, -\alpha _1\\&\qquad -\lambda _1\in [-\alpha _2,\alpha _2]+[-\lambda _2,\lambda _2]\big \} \\&\quad \cup \big \{(\upsilon ,\upsilon ,\alpha ,\lambda )\mid -1\le \upsilon <0, \alpha _1\ge 0, \alpha _2\ge 0, \alpha _1^2+\alpha _2^2=1, \lambda _1\\&\qquad = 0, \lambda _2\ge 0,-\alpha _1+\lambda _2\in [-\alpha _2,\alpha _2] \big \}, \end{aligned}$$

and see that $locS(RP)=C_R.$

Let us consider point $(\bar{x},\bar{y}):=(-1,-1)$, and then

$$\begin{aligned} \Lambda (\bar{x},\bar{y})= & {} \{(\alpha ,\lambda )\in \mathbb {R}^2\times \mathbb {R}^2\mid \alpha _1\ge 0, \alpha _2\ge 0, \alpha _1^2+\alpha _2^2=1, \lambda _1\\&= 0, \lambda _2\ge 0,-\alpha _1+\lambda _2\in [-\alpha _2,\alpha _2] \}. \end{aligned}$$

It is easy to see that the (SCQ) is satisfied at $\bar{x}:=-1$ and $(\bar{x},\bar{y},\tilde{\alpha },\tilde{\lambda })\in locS(RP)$ for all $(\tilde{\alpha },\tilde{\lambda })\in \Lambda (\bar{x},\bar{y}).$ Applying now Theorem 4.1(ii) to conclude that $(\bar{x},\bar{y})\in S(P).$

Next, by choosing $\bar{\alpha }:=(0,1),\bar{\lambda }:=(0,0)$, it holds that $(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda })\in locS(RP).$ For all $z:=(v^1,v^2,u^1,u^2)$ with $v^1\in \partial _yf_1(\bar{x},\bar{y}), v^2\in \partial _yf_2(\bar{x},\bar{y}), u^1\in \partial _yg_1(\bar{x},\bar{y}), u^2\in \partial _yg_2(\bar{x},\bar{y})$ satisfying (4.7), we have $v^1=1, v^2=0, u^1=1$ and $u^2=-1$, i.e., we obtain the unique element $z=(1,0,1,-1).$ For each $\omega \in \mathbb {R}$, it is easy to see by Lemma 2.7 that

$$\begin{aligned} \partial ^2_yf_1(\bar{x},\bar{y},v^1)(\omega )=\partial ^2_yg_1(\bar{x},\bar{y},u^1)(\omega )=\partial ^2_yg_2(\bar{x},\bar{y},u^2)(\omega )=\{(0,0)\}. \end{aligned}$$

To deal with the partial second-order subdifferential of $f_2$ at $(\bar{x},\bar{y},v^2)$, we present the graph of $f_2$ as

$$\begin{aligned} \mathrm{gph\,}(\partial _yf_2)&=\{(x,y,-1)\mid x>y\}\cup \{(x,y,1)\mid x<y\}\cup \{(x,y,z)\mid x=y, z\in [-1,1]\}\\&=\Omega _1\cup \Omega _2\cup \Omega _3, \end{aligned}$$

where $\Omega _1:=\{(x,y,-1)\mid x\ge y\}, \Omega _2:=\{(x,y,z)\mid x=y, z\in [-1,1]\}$ and $\Omega _3:=\{(x,y,1)\mid x\le y\}$ are closed sets. Then, we will compute the Fréchet normal cones to $\mathrm{gph\,}(\partial _yf_2)$ at (x, y, z) near $(\bar{x},\bar{y}, v^2)=(-1,-1,0).$ It suffices to consider points $(x,y,z)\in \mathrm{gph\,}(\partial _yf_2)$ such that $-1<z<1$, i.e., $(x,y,z)\in \Omega _2,$ and thus,

$$\begin{aligned} \widehat{N}\big ((x,y,z);\mathrm{gph\,}(\partial _yf_2)\big )=\{(a,-a)\mid a\in \mathbb {R}\}\times \{0\}. \end{aligned}$$

It implies by the definition of limiting normal cone (2.2) that

$$\begin{aligned} N\big ((\bar{x},\bar{y},v^2);\mathrm{gph\,}(\partial _yf_2)\big )=\{(a,-a)\mid a\in \mathbb {R}\}\times \{0\}, \end{aligned}$$

and then, due to (2.4) and (2.17),

$$\begin{aligned} \partial ^2_yf_2(\bar{x},\bar{y},v^2)(\omega )={\left\{ \begin{array}{ll} \{(a,-a)\mid a\in \mathbb {R}\} &{} \text{ if } \omega =0,\\ \emptyset &{} \text{ if } \omega \ne 0.\end{array}\right. } \end{aligned}$$

It is clear that the qualification condition (3.18) holds. Finally, applying Theorem 4.3, we obtain the assertions in (4.8)–(4.13).

In the smooth framework, the result obtained in Theorem 4.3 reduces to the following form, which can be found partially in Ye (2011).

Corollary 4.5

Let $(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda })\in locS(RP)$ with $\bar{\alpha }:=(\bar{\alpha }_1,\ldots ,\bar{\alpha }_q)$ and $\bar{\lambda }:=(\bar{\lambda }_1,\ldots ,\bar{\lambda }_r)$. Let the functions involving in the problem $\mathrm{P}$ be continuously differentiable at the referenced points. Let $\nabla _y f_j(\bar{x},\bar{y}):=(v^j_1,\ldots ,v^j_m), j=1,\ldots ,q, \nabla _y g_t (\bar{x},\bar{y}):= (u^t_1,\ldots ,u^t_m), t=1,\ldots , r.$ Assume further that $\nabla _y f_j, j=1,\ldots , q, \nabla _y g_t, t=1,\ldots , r$ are strictly differentiable at $(\bar{x},\bar{y})$ with the partial derivatives denoted by $\nabla ^2_{yx}f_j(\bar{x},\bar{y}), \nabla ^2_{yy}f_j(\bar{x},\bar{y}), j=1,\ldots ,q, \nabla ^2_{yx}g_t(\bar{x},\bar{y}), \nabla ^2_{yy}g_t(\bar{x},\bar{y}), t=1,\ldots ,r.$ Then there exist $\nu :=(\nu _1,\ldots ,\nu _p)\in \mathbb {R}^p_+, \eta :=(\eta _1,\ldots ,\eta _l)\in \mathbb {R}^l_+, \beta :=(\beta _1,\ldots ,\beta _r)\in \mathbb {R}^r, \gamma \in \mathbb {R}_+$, not all zero, as well as $\omega :=(\omega _1,\ldots ,\omega _m)\in \mathbb {R}^m$ with $||\omega ||\le 1$ such that

$$\begin{aligned} 0&= \sum _{i=1}^p\nu _i\nabla _x F_i(\bar{x},\bar{y})+ \sum _{k=1}^l\eta _k\nabla G_k(\bar{x})+ \sum _{t=1}^r\beta _t\nabla _x g_t(\bar{x},\bar{y})\\&\quad +\sum _{j=1}^q\gamma \bar{\alpha }_j\nabla ^2_{yx}f_j(\bar{x},\bar{y})^\top w+\sum _{t=1}^r\gamma \bar{\lambda }_t\nabla ^2_{yx}g_t(\bar{x},\bar{y})^\top w,\\ 0&= \sum _{i=1}^p\nu _i\nabla _y F_i(\bar{x},\bar{y})+ \sum _{t=1}^r\beta _t\nabla _y g_t(\bar{x},\bar{y}) + \sum _{j=1}^q\gamma \bar{\alpha }_j\nabla ^2_{yy}f_j(\bar{x},\bar{y})^\top w\\&\quad +\sum _{t=1}^r\gamma \bar{\lambda }_t\nabla ^2_{yy}g_t(\bar{x},\bar{y})^\top w,\\&\text{ with } (4.9){-}(4.13). \end{aligned}$$

Proof

The proof follows from Theorems 4.3 and 3.6(ii). $\Box$

In order to obtain a KKT necessary condition for the KKT relaxation multiobjective formulation of the multiobjective bilevel optimization problem $\mathrm{P}$, we need the following qualification constraint condition, which involves the constraint functions of both the upper-level and lower-level multiobjective problems.

Definition 4.6

Let $(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda })$ be a feasible point of the problem $\mathrm{RP}$, where $\bar{\alpha }:=(\bar{\alpha }_1,\ldots ,\bar{\alpha }_q)$ and $\bar{\lambda }:=(\bar{\lambda }_1,\ldots ,\bar{\lambda }_r)$, and let $z:=(v^1,\ldots ,v^q,u^1,\ldots ,u^r)$ with $v^j:=(v^j_1,\ldots ,v^j_m)\in \partial _yf_j(\bar{x},\bar{y}), j=1,\ldots , q, u^t:=(u^t_1,\ldots ,u^t_m)\in \partial _y g_t(\bar{x},\bar{y}), t=1,\ldots , r$ satisfying (4.7) and $\omega :=(\omega _1,\ldots ,\omega _m)\in \mathbb {R}^m$ with $||\omega ||\le 1$. We say that the bi-level qualification condition (BCQ) holds at $(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda })$ if for any $\eta :=(\eta _1,\ldots ,\eta _l)\in \mathbb {R}^l_+, \beta :=(\beta _1,\ldots ,\beta _r)\in \mathbb {R}^r_+, \gamma \in \mathbb {R}_+$ satisfying

$$\begin{aligned} \left. \begin{array}{ll} 0&{}\in \sum _{k=1}^l\eta _k\big (\partial G_k(\bar{x}),0_m\big )+ \sum _{t=1}^r\beta _t\big (\partial g_t(\bar{x},\bar{y})\cup \partial (-g_t)(\bar{x},\bar{y})\big )\\ &{}+ \sum _{j=1}^q\gamma \partial ^2_yf_j(\bar{x},\bar{y},v^j)(\bar{\alpha }_j\omega )+\sum _{t=1}^r\gamma \partial ^2_yg_t(\bar{x},\bar{y},u^t)(\bar{\lambda }_t\omega )\\ &{} \text{ with } (4.9){-}(4.13) \end{array}\right\} \Rightarrow {\left\{ \begin{array}{ll} \eta =0\\ \beta =0\\ \gamma =0.\end{array}\right. } \end{aligned}$$

(4.33)

It should be emphasized here that common regularity conditions for nonlinear programming such as the Mangasarian–Fromovitz constraint qualification are violated at every feasible point for KKT reformulations; see e.g., Ye et al. (1997) and Scheel and Scholtes (2000), and thus this assertion is also valid for our problem $\mathrm{RP}$. So, a qualification condition as above, which glues the constraint functions of both the upper-level and lower-level multiobjective problems together, is essential for investigation of KKT necessary conditions. In the scalar case, i.e., $q=1$, the above-defined (BCQ) reduces to the (CQ) introduced in Dempe and Zemkoho (2014, (5.5)). In this case, as mentioned in Dempe and Zemkoho (2014), the (BCQ) is closely related to the so-called No Nonzero Abnormal Multiplier Constraint Qualification (NNAMCQ) used in Ye (2005) for a smooth mathematical programs with equilibrium constraints. Note further that a slightly different consideration is that we use in Definition 4.6 the term $\sum _{t=1}^r\beta _t\big (\partial g_t(\bar{x},\bar{y})\cup \partial (-g_t)(\bar{x},\bar{y})\big ), \beta _t\ge 0,$ instead of the one $\partial \big (\sum _{t=1}^r\tilde{\beta }_tg_t(\bar{x},\bar{y})\big ), \tilde{\beta }_t\in \mathbb {R},$ as in Dempe and Zemkoho (2014, (5.5)) because it allows us to obtain the corresponding multipliers which are nonnegative that is commonly attained when dealing with optimization problems with inequality constraints.

We are now in a position to establish a KKT necessary condition for the problem $\mathrm{RP}$ under the fulfilment of the (BCQ).

Corollary 4.7

Let $(\bar{x},\bar{y},\bar{\alpha },\bar{\lambda })\in locS(RP)$. Assume that for all $z:=(v^1,\ldots ,v^q,u^1,\ldots ,u^r)$ with $v^j:=(v^j_1,\ldots ,v^j_m)\in \partial _yf_j(\bar{x},\bar{y}), j=1,\ldots , q, u^t:=(u^t_1,\ldots ,u^t_m)\in \partial _y g_t(\bar{x},\bar{y}), t=1,\ldots , r$ satisfying (4.7), the qualification condition (3.18) holds together with the fulfillment of the (BCQ) given in (4.33). Then there exist $\nu :=(\nu _1,\ldots ,\nu _p)\in \mathbb {R}^p_+\setminus \{0\}, \eta :=(\eta _1,\ldots ,\eta _l)\in \mathbb {R}^l_+, \beta :=(\beta _1,\ldots ,\beta _r)\in \mathbb {R}^r_+, \gamma \in \mathbb {R}_+$ and $z:=(v^1,\ldots ,v^q,u^1,\ldots ,u^r)$ satisfying (4.7) as well as $\omega :=(\omega _1,\ldots ,\omega _m)\in \mathbb {R}^m$ with $||\omega ||\le 1$ such that (4.8)–(4.13) hold.

Proof

Thanks to Theorem 4.3, we find $\nu :=(\nu _1,\ldots ,\nu _p)\in \mathbb {R}^p_+, \eta :=(\eta _1,\ldots ,\eta _l)\in \mathbb {R}^l_+, \beta :=(\beta _1,\ldots ,\beta _r)\in \mathbb {R}^r_+, \gamma \in \mathbb {R}_+,$ not all zero, and $z:=(v^1,\ldots ,v^q,u^1,\ldots ,u^r)$ satisfying (4.7) as well as $\omega :=(\omega _1,\ldots ,\omega _m)\in \mathbb {R}^m$ with $||\omega ||\le 1$ such that (4.8)–(4.13) hold. Suppose for contradiction that $\nu =0.$ Then, we have

$$\begin{aligned} \begin{array}{ll} 0&{}\in \sum _{k=1}^l\eta _k\big (\partial G_k(\bar{x}),0_m\big )+ \sum _{t=1}^r\beta _t\big (\partial g_t(\bar{x},\bar{y})\cup \partial (-g_t)(\bar{x},\bar{y})\big )\\ &{}+ \sum _{j=1}^q\gamma \partial ^2_yf_j(\bar{x},\bar{y},v^j)(\bar{\alpha }_j\omega )+\sum _{t=1}^r\gamma \partial ^2_yg_t(\bar{x},\bar{y},u^t)(\bar{\lambda }_t\omega )\\ &{} \text{ with } (4.9){-}(4.13). \end{array} \end{aligned}$$

Invoking the (BCQ) in (4.33), it ensures that $\eta =0, \beta =0$ and $\gamma =0.$ Hence, $\nu ,\eta ,\beta ,\gamma$ are all equal to zero that is absurd. The proof is finished. $\Box$

Remark 4.8

It is worth mentioning here that in the case of the lower-level optimization problem is a scalar one (i.e., $q=1$ and thus, $\bar{\alpha }=1$), the condition (4.10) disappears due to the fact that $J(\bar{\alpha })$ given in (4.14) is an empty set. If we consider both the upper-level and lower-level problems are scalar ones (i.e., $p=1$ and $q=1$), then the result obtained in Corollary 4.7 reduces to the similar one established in Dempe and Zemkoho (2014, Theorem 5.2), where a direct proof has been constructed by verifying a condition of type (2.20) under the fulfilment of the (BCQ) given in (4.33).