Extremal Systems for Sets and Multifunctions in Multiobjective Optimization with Variable Ordering Structures

Abstract

In this paper, we study extremal systems for sets and multifunctions in multiobjective optimization with variable/nonconstant ordering structures, which reduce to vector optimization when an ordering structure is constant, i.e., it is defined by a fixed ordering cone/set. It is important to mention that we do not impose either convexity or nonempty interiority property on ordering structures. Based on these systems, we derive verifiable necessary conditions for nondominated solutions to multiobjective problems with geometric constraints. Examples are provided to illustrate the usage of the obtained results.

1 Introduction

Boris S. Mordukhovich is one of the founders of modern variational analysis and generalized differentiation. Among his best known achievements are the introduction of the most powerful constructions of generalized differentiation (bearing now his name), their developments and applications to broad classes of problems in variational analysis, optimization, equilibrium, control, economics, engineering, and other fields. His theory and various applications have been systematically summarized in the 2-volume monograph [16, 17] in which the driving force in establishing the full calculus for limiting differential objects and applications to various problems of optimization, equilibrium, stability, control, systems with lumped, and distributed parameters, mechanics, economics, and so on is the extremal principle. It plays a fundamental role in variational analysis similar to that of separation theorem of convex analysis. In this paper, we develop new applications of the extremal principle to multiobjective (including vector-valued and set-valued) optimization with variable/nonconstant ordering structures.

Consider a constrained multiobjective optimization problem with respect to a variable ordering structure described by

$$ \textnormal{nondominate} \quad F(x)\quad \mathrm{subject\; to}\quad x\in {\Omega}\quad\mathrm{with\; respect\; to}\quad \mathbf{D}, $$

(P)

where $F: X\rightrightarrows Z$ is a set-valued cost between Banach spaces, Ω is a nonempty set in the domain space X, and $\mathbf {D} : Z \rightrightarrows Z$ is a variable ordering structure of the codomain/image space Z. The purpose of problem (P) is to find a feasible solution that must not be dominated by other feasible solutions of problem (P) with respect to a variable ordering structure D; see Definition 1 below.

Given two distinct vectors z ₁ and z ₂ in a decision vector space Z, we can write z ₂ = z ₁+d for some nonzero vector d∈Z. If z ₁ is preferred by the decision maker to z ₂, then d can be viewed as a domination factor. The set of all the domination factors for z together with the zero vector 0∈Z is denoted by D(z), and the set-valued mapping $\mathbf {D}:Z\rightrightarrows Z$ is called a variable ordering structure. Define an ordering relation induced from D denoted by ≤ _N via

$$\begin{array}{@{}rcl@{}} z_{1}\leq_{N}z_{2}\quad\mathrm{ if\; and\; only\; if }\quad z_{2}\in z_{1}+\mathbf{D}(z_{1}). \end{array} $$

Excluding the trivial case in which a point exists in feasible region that simultaneously maximizes or minimizes all objectives, multiobjective optimization problems have conflicting objectives. In general cases, it is naturally acceptable to say that a good solution must not be dominated by other feasible alternatives.

Definition 1

(Nondominated points to sets and nondominated solutions of problems (P)).

• (i) A point $\bar {z}\in {\Xi }$ is said to be a nondominated point to a nonempty set Ξ⊂Z with respect to the ordering structure D if there is no $z\in {\Xi }\setminus \{\bar {z}\}$ such that $z\leq _{N}\bar {z}$, i.e., $\bar {z}\in z+\mathbf {D}(z)$. The collection of all nondominated points of Ξ with respect to D is denoted by ND (Ξ; D); thus we write $\bar {z}\in \text {ND}\,({\Xi };\mathbf {D})$.

• (ii) A pair $(\bar {x},\bar {z})\in \mathrm {gph\,} F$ is said to be a nondominated solution of problem (P) if $\bar {x}\in {\Omega }$ and $\bar {z}\in \text {ND}\,(F({\Omega });\mathbf {D})$, where F(Ω):=∪{F(x) | x∈Ω} is the image set of F over Ω, i.e., there is no pair (x, z) with x∈Ω, z∈F(x) and $z\neq \bar {z}$ such that

  $$ z \leq_{N} \bar{z} \quad \Longleftrightarrow \quad \bar{z} \in z + \mathbf{D}(z). $$

  (1)

For simplicity, we do not mention $\bar {z}$ when F = f:X → Z is singleton and we say that $(\bar {x},\bar {z})$ is a nondominated solution of F with respect to D when Ω=X.

Observe that if $\bar {z}$ is a nondominated point of Ξ with respect to D, then $(\bar {z},\bar {z})$ is a nondominated solution of the constant set-valued mapping F(z)≡Ξ ∀ z∈Z. Observe also that nondomination is a far-going extension of Pareto efficiency. In fact, when the ordering structure is constant, i.e., D(z) ≡ Θ ∀ z∈Z for some closed, convex and pointed cone Θ⊂Z, the concept of nondomination reduces to the Pareto efficiency of vector optimization ordered by the ordering cone Θ, see, e.g., [13, 14]. In this case, we use the notation ≤ _Θ and the relation

$$z_{1}\leq_{\Theta} z_{2}\quad\mathrm{if\; and\; only\; if}\quad z_{2}-z_{1}\in{\Theta}. $$

Thus, a point $\bar {z}\in {\Xi }$ a Pareto efficient point of Ξ with respect to Θ if and only if there is no other point $z\in {\Xi }\setminus \{\bar {z}\}$ such that $z \leq _{\Theta }\bar {z}$, if and only if

$${\Xi}\cap(\bar{z}-{\Theta})=\{\bar{z}\}\quad\Longleftrightarrow\quad({\Xi}-\bar{z})\cap(-{\Theta})=\{\mathbf{0}\}. $$

The reader is referred to [6, Proposition 3.1] for relations between nondominated and efficient points. The following example shows that the existence of nondominated points/solutions does not guarantee that of efficient ones. Let $\mathbf {D}:\mathbb {R}\rightrightarrows \mathbb {R}$ be a cone-valued ordering structure defined by

$$\begin{array}{@{}rcl@{}} \mathbf{D}(z):=\left\{\begin{array}{ll} \mathbb{R}_{+}&\text{if}\;\;z>0,\\ \mathbb{R}&\text{if}\;\;z=0,\\ \mathbb{R}_{-}&\text{if}\;\;z<0. \end{array}\right. \end{array} $$

(2)

It is easy to check that the function φ(x) := x ³ has no Pareto efficient (either maximum or minimum) solution in the usual sense, but the origin $\bar {x} = 0$ is a nondominated solution of φ with respect to the variable ordering structure D in (2).

The concept of nondomination was introduced and further developed by Yu in [8, 22, 23]. Yu imposed, in his early works, on variable ordering structures, two conditions: (i) if d is a domination factor for z, then any positive multiple of d is also a domination factor for z, and (ii) if d ₁ and d ₂ are two domination factors for z, then d ₁+d ₂ is also a domination factor for z. The validity of these two conditions is equivalent to the convexity and cone-valuedness properties of domination sets. The cone-valuedness condition should be dropped since it leads to the trivial cone, i.e., {0}, in order to apply domination structures to some classes of decision making problems as justified by Bergtresser et al. [9].

To the best of our knowledge, the topic on necessary conditions for nondominated points to sets and solutions to constrained multiobjective optimization problems with respect to variable ordering structures seems to be ‘new’ and underdeveloped even for the class of convex- and cone-valued ordering structures. The only results in this direction which we are familiar with are in [6, 10, 11].

• In [11], Engau formulated necessary conditions for nondominated points to sets with respect to variable ordering structures satisfying that each domination factor set is an ideal-symmetric convex cone. Her technique heavily relies on the geometric angles in $\mathbb {R}^{2}$ and $\mathbb {R}^{3}$. It seems to be not able to extend them to higher dimensions.

• In [10], Eichfelder and Ha obtained generalized Fermat and Lagrange multiplier rules for multiobjective problems with respect to variable Bishop–Phelps-cone-valued ordering structures by using scalarizing techniques to convert the problem under consideration into a scalar one.

• In [6], Bao and Mordukhovich established for the first time the necessary conditions for nondominated points of sets and for the nondominated solutions to vector optimization problems with general geometric constraints for the class of general variable ordering structures satisfying the following three conditions:

  • (A)  (additivity and multiplicity) D(z) is a nonempty convex cone for all z∈dom D;

  • (B)  (nonsubspace property) $\mathbf {D}(\bar {z})$ is not a subspace of Z, i.e., $\mathbf {D}(\bar {z})\setminus (-\mathbf {D}(\bar {z}))\ne \emptyset$;

  • (C)  (nontrivial intersection cone) The common cone ${\Theta }_{\mathbf {D}}:=\bigcap _{z\in \text {dom}\;\mathbf {D}}\mathbf {D}(z)$ is not a trivial cone, i.e., there is some nonzero vector e∈Θ _D .

  Their techniques base on the variational dual-space approach in [1, 4, 6, 16, 17] by using the extremal principle of variational analysis and generalized differentiation.

In contrast to multiobjective optimization with variable ordering structures, we have, in multiobjective optimization with ordering cones, many powerful tools in establishing necessary conditions for Pareto efficient points of constrained and unconstrained multiobjective optimization problems in terms of ordering cones and the given data in both variational dual-space approaches [3–5, 17] and scalarization primal-space approaches [12]. One of the weakest assumptions imposed on ordering cones is the so-called local asymptotic closedness (LAC) property; see Definition 5 below. Unfortunately, combining techniques in [6] for variable convex- and cone-valued ordering structures and arguments in [3–5] for LAC ordering cones does not lead to meaningful necessary nondomination conditions provided that three LAC conditions:

• (A’) D(z) is LAC at the origin for all z ∈ dom D;

• (B’) $\mathbf {D}(\bar {z})\setminus (-\mathbf {D}(\bar {z}))\ne \emptyset$;

• (C’) ${\Theta }_{\mathbf {D}}:=\bigcap _{z\in \mathrm {dom\,} \mathbf {D}}\;\mathbf {D}(z)$ contains a nonzero element and enjoys the LAC property at the origin;

are satisfied. In this paper, we suggest a new way to study multiobjective optimization problems with general variable ordering structures.

Besides the given ordering structure D, we need a multifunction $\mathbf {P} : Z\rightrightarrows Z$ which is called an upper-level-set mapping and defined by

$$\mathbf{P}(z) := I(z) + \mathbf{D}(z) = z + \mathbf{D}(z). $$

We have

$$z_{1}\leq_{N} z_{2} \quad \Longleftrightarrow\quad z_{2} \in \mathbf{P}(z_{1}) \quad \Longleftrightarrow \quad (z_{1}, z_{2}) \in \mathrm{gph\,} \mathbf{P}^{-1}, $$

where P ⁻¹ stands for the inverse of the mapping P. That is that the domination order ≤ _N is nothing but a binary relation deduced from gph P ⁻¹. Precisely, let Q⊂Z × Z be an arbitrary subset of a product space Z × Z, and let $\mathcal {R}$ be a binary relation on Q describing by $z_{1} \mathcal {R}z_{2}$ if and only if (z ₁, z ₂) ∈ Q for all z ₁, z ₂ ∈ Z. The strict preference ≺ on Q is defined by $[z_{1}\prec z_{2} \mathrm { if\; and\; only\; if }\ z_{1} \mathcal {R} z_{2}\ \mathrm { and }\ \neg z_{2} \mathcal {R} z_{1}]$. The indifference relation ∼ on Q is defined by $[z_{1} \sim z_{2} \ \mathrm { if\; and\; only\; if }\ z_{1} \mathcal {R} z_{2}\ \mathrm { and }\ z_{2} \mathcal {R} z_{1}]$. The disjoint union ≼:=≺∪∼ is called a preference on Q.

As we learn from the literature of vector optimization, the ideal way to deal with multiobjective optimization with preferences is to assume that the preference ≼ in question has a utility representation, i.e., there is a utility function $U : Z \to \mathbb {R}$ such that z ₁≼z ₂⇒U(z ₁) ≤ U(z ₂). The existence of such a function allows us to convert a multiobjective optimization problem to a scalar one. However, the class of preferences with utility descriptions is not very broad. In fact, a preference ≺ on a finite-dimensional space Z has a continuous utility function if and only if both lower-level and upper-level sets

$$\{z_{1}\in Z\ | \ z_{1}\preceq z_{2}\} \quad \text{and}\quad \{z_{2} \in Z\ | \ z_{1} \preceq z_{2}\} $$

are closed in Z for all z∈Z. An efficient way is to require that preferences enjoy certain additional properties such as reflexivity, nonreflexitivity, anti-symmetritivity, symmetritivity, and transitivity to have specific structures for Q. It is well recognized that in vector optimization the partial order induced by a closed, convex, and pointed ordering cone is a preference satisfying reflexivity, anti-symmetritivity, and transitivity properties. However, the difficulty is that with many problems it is not possible to either obtain a mathematical representation for a utility function of the preference under consideration or justify the fulfillment of certain additional properties for preferences. It is pointed out in [20, page 22] that “much of the mathematical theory is based on maximality with respect to partial orders or partially preorders, at best. More general concepts of optimality based on preferences satisfying conditions other than merely those of reflexivity and transitivity” should be considered.

In this direction, many authors relaxed the transitivity property of preferences. One way is to impose less restrictive conditions on ordering cones of vector optimization. For example, Bao and Mordukhovich [2] did not require the pointedness and solidness (i.e., nonempty interior) assumptions, Mordukhovich [17], Bao and Mordukhovich [3–5], and Bao [1] did not need the convexity and cone properties, Tammer and Weidner [21] and Rubinov and Gasimov [19] needed a light property for the ordering set Θ⊂Z: there is e ∈ Θ such that Θ + λe ⊂ Θ for all λ > 0, i.e., the recession cone of Θ is nontrivial, and Bao and Mordukhovich [1, 4] further weaken to the LAC property. Another way is to modify the transitivity condition by Zhu [24], Mordukhovich et al. [15], Mordukhovich [17], and Bellaassali and Jourani [7]. They paid their main attention to the class of closed preferences. Recall that a preference is said to be closed if it meets the following requirements:

• Nonreflexivity: z ∉ P(z) ∀ z∈Z;

• Local satiation: z ∈ cl P(z) ∀ z∈U, where U is a neighborhood of $\bar {z}$;

• Almost transitivity: [z ₁ ∈ cl P(z ₂) & z ₂∈P(z ₃)] ⇒ z ₁∈P(z ₃) ∀ z ₁, z ₂, z ₃∈Z.

Although all the techniques used in these papers fundamentally based on the Ekeland variational principle, Mordukhovich et al. [15] finally achieved an advanced tool that is called the (extended) extremal principle for set-valued mappings and extensively used it to establish necessary conditions for Pareto efficient solutions in multiobjective optimization in the sense that a pair $(\bar {x},\bar {z})\in \mathrm {gph\,} F$ is a Pareto efficient solution of problem (P) if there is no pair (x, z) with x∈S, z∈F(x) and $z\neq \bar {z}$ such that

$$ z \in \bar{z} -\mathbf{D}(\bar{z}). $$

(3)

Since then, there have been much further developments and applications of the extended extremal principle for multifunctions.

In this paper, we develop new applications of the extremal principle, exploited before in the case of Pareto efficient solutions, to multiobjective optimization with variable ordering structures. The remaining of the paper is organized as follows. Section 2 presents some of the basic concepts and tools from variational analysis and generalized differentiation broadly used in the sequel; the main tools are versions of the extremal principle for systems of sets or those of multifunctions. In Section 3, we establish relationships between nondomination and extremality in multiobjective optimization with respect to variable ordering structures and derive from them the new verifiable necessary optimality conditions for nondominated solutions of multiobjectve optimization problems with geometric constraints.

Throughout the paper, we employ the standard notation of variational analysis; cf. [16, 18]. For a Banach space X, we denote its norm by ∥⋅∥, consider the dual space X ^∗ equipped with the weak ^∗ topology w ^∗, and denote the canonical pairing between X and X ^∗ by 〈⋅,⋅〉. Given a set-valued mapping $F\colon X\rightrightarrows X^{*}$, the notation

$$\begin{array}{@{}rcl@{}} \begin{array}{ll} \limsup\limits_{x\rightarrow\bar{x}} F(x):=\left\{x^{*}\in X^{*}\left|\right.\exists\; \mathrm{ sequences }\; x_{k}\rightarrow \bar{x} \, \mathrm{ and } \ x^{*}_{k} \mathop{\longrightarrow}\limits^{w^{*}}x^{*}\right.\\ \qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\left.\mathrm{ with } \ x^{*}_{k} \in F(x_{k}) \ \mathrm{ for\; all }\ k\in \mathbb{N} \right\} \end{array} \end{array} $$

(4)

signifies the sequential Painlevé–Kuratowski upper/outer limit with respect to the norm topology of X and the weak ^∗ topology of X ^∗, where $\mathbb {N}:=\{1,2,\ldots \}$.

2 Basic Tools of Variational Analysis

The key tool in this paper is the extremal principle in variational analysis. Since it unconditionally holds in Asplund spaces, all Banach spaces in question are assumed to be Asplund unless otherwise stated. Recall that a Banach space is Asplund if every convex continuous function $\varphi :U \to \mathbb {R}$ defined on an open convex subset U of X is Fréchet differentiable on a dense subset of U. The class of Asplund spaces is quite broad including every reflexive Banach space and every Banach space with a separable dual; in particular, c ₀, ℓ ^p, and L ^p[0,1] for 1<p<∞ are Asplund. It has been comprehensively investigated in geometric theory of Banach spaces and largely employed in variational analysis; see, e.g., [16, 17]. In the sequel, we present several definitions and properties of the basic generalized differential constructions held in the Asplund space setting and enjoying a full calculus.

Let X be a Banach space and Ω⊂X be a nonempty subset of X. The Fréchet/regular normal cone to Ω at x∈Ω is defined by

$$\begin{array}{@{}rcl@{}} \Hat{N}(x;{\Omega}):=\left\{x^{*} \in X^{*} ~ \left|~ \mathop{\limsup}\limits_{u\mathop{\rightarrow}\limits^{\Omega}x}\frac{\langle x^{*},u-x\rangle}{\|u-x\|}\le 0\right\}\right., \end{array} $$

(5)

where $u\mathop {\rightarrow }\limits ^{\Omega }x$ means u → x with u∈Ω. Assume now that X is Asplund and Ω is locally closed around $\bar {x}\in {\Omega }$, i.e., there is a neighborhood U of $\bar {x}$ such that Ω∩cl U is a closed set. The (basic, limiting, Mordukhovich) normal cone to Ω at $\bar {x}$ is defined by the sequential Painlevé–Kuratowski outer limit of Fréchet normal cones to Ω at x as x tends to $\bar {x}$. By (4), we have

$$\begin{array}{@{}rcl@{}} N(\bar{x};{\Omega})& := & \limsup\limits_{x\to\bar{x}}\Hat{N}(x;{\Omega})\\ & = & \left\{ x^{*}\in X^{*} ~\Big |~ \exists ~ x_{k}\rightarrow \bar{x},~ x^{*}_{k}\xrightarrow{w^{*}}x^{*}\ \text{with} \ x^{*}_{k} \in \Hat N(x_{k};{\Omega})\right\}. \end{array} $$

(6)

In contrast to the Fréchet normal cone (5), the limiting normal cone (6) is often nonconvex enjoying nevertheless full calculus. Both cones (6) and (5) reduce to the normal cone of convex analysis when Ω is convex.

Given a set-valued mapping $F\colon X\rightrightarrows Z$ between Banach spaces X and Z. The domain and graph of F are defined by

$$\begin{array}{@{}rcl@{}} \mathrm{dom\,} F:=\{x\in X~|~F(x)\ne\emptyset\}\quad\mathrm{ and }\quad\mathrm{gph\,} F:=\{(x,z)\in X\times Z~|~z\in F(x)\}, \end{array} $$

respectively. The normal coderivative $D^{*}_{N} F(\bar {x},\bar {z})\colon Z^{*}\rightrightarrows X^{*}$ is defined, via the normal cone to the graph of F by

$$\begin{array}{@{}rcl@{}} D^{*}_{N} F(\bar{x},\bar{z})(z^{*}):=\{x^{*}\in X^{*}~|~(x^{*},-z^{*})\in N((\bar{x},\bar{z});\mathrm{gph\,} F)\}. \end{array} $$

(7)

We also need a modification of (7) by using the norm convergence on Z ^∗ and weak ^∗ sequential convergence on X ^∗. It is known as the mixed coderivative $D^{*}_{M} F(\bar {x},\bar {z})\colon Z^{*}\rightrightarrows X^{*}$ with

$$\begin{array}{@{}rcl@{}} D^{*}_{M}F(\bar{x},\bar{z})(z^{*}):=\left\{\left.x^{*}\in X^{*}\right| \exists\; (x_{k},z_{k})\mathop{\longrightarrow}\limits^{\text{gph} F}(\bar{x},\bar{z}),\;x^{*}_{k}\overset{w^{*}}{\to}x^{*},\;z^{*}_{k}\to z^{*}\right.\\ \left.\text{with}\;(x^{*}_{k},-z^{*}_{k})\in\Hat N((x_{k},z_{k});\mathrm{gph\,} F)\right\}. \end{array} $$

(8)

For a single-valued function F = f:X → Z, we omit $\bar {z}=f(\bar {x})$ in the coderivative notation. It follows from (7) and (8) that

$$\begin{array}{@{}rcl@{}} D^{*}_{M} F(\bar{x},\bar{z})(z^{*})\subset D^{*}_{N} F(\bar{x},\bar{z})(z^{*})\;\mathrm{ for\; all }\;z^{*}\in Z^{*}, \end{array} $$

which is strict in many common situations, but holds as equality when, in particular, dimZ<∞. Furthermore, we have

$$\begin{array}{@{}rcl@{}} D^{*}_{N} f(\bar{x})(z^{*}) = D^{*}_{M} f(\bar{x})(z^{*}) =\{\nabla f(\bar{x})^{*}z^{*}\}\;\mathrm{ for\; all }\;z^{*}\in Z^{*} \end{array} $$

provided that f is strictly differentiable at $\bar {x}$; in particular, $f\in {\mathcal {C}}^{1}$ around this point.

An important ingredient of variational analysis and generalized differentiation in infinite-dimensional spaces relates to appropriate “sequential normal compactness” properties of sets and mappings that are automatic in finite dimensions. Given a set Ω⊂X × Z, we say that it is sequentially normally compact (SNC) at $(\bar {x},\bar {z})\in {\Omega }$ if for any sequence of $(x_{k},z_{k},x^{*}_{k},z^{*}_{k})\in X\times Z\times X^{*}\times Z^{*}$ satisfying

$$\begin{array}{@{}rcl@{}} (x_{k},z_{k})\mathop{\to}\limits^{\Omega}(\bar{x},\bar{z})\quad\mathrm{ and }\quad (x^{*}_{k},z^{*}_{k})\in\Hat N((x_{k},z_{k});{\Omega}),\quad k\in\mathbb{N}, \end{array} $$

(9)

the following implication holds:

$$(x^{*}_{k},z^{*}_{k})\mathop{\to}\limits^{w^{*}}(\mathbf{0},\mathbf{0})\Longrightarrow(x^{*}_{k},z^{*}_{k})\mathop{\longrightarrow}\limits^{\|\cdot\|}(\mathbf{0},\mathbf{0})\;\mathrm{ as }\; k\to\infty. $$

The more subtle partial SNC (PSNC) property of Ω at $(\bar {x},\bar {z})$ means that for any sequence $(x_{k},z_{k},x^{*}_{k},z^{*}_{k})$ satisfying (9) we have the implication

$$\begin{array}{@{}rcl@{}} x^{*}_{k}\mathop{\to}\limits^{w^{*}}\mathbf{0}\quad\mathrm{ and }\quad z^{*}_{k}\mathop{\longrightarrow}\limits^{\|\cdot\|}\mathbf{0}\Longrightarrow x^{*}_{k}\mathop{\longrightarrow}\limits^{\|\cdot\|}\mathbf{0}\;\mathrm{ as }\;k\to\infty. \end{array} $$

Applying these properties to graphs of set-valued mappings induces the corresponding properties for mappings. Namely, $F\colon X\rightrightarrows Z$ is S N C at $(\bar {x},\bar {z})\in \mathrm {gph\,} F$ if its graph is SNC at $(\bar {x},\bar {z})$, and that F is PSNC at $(\bar {x},\bar {z})$ if the graph of F is PSNC at $(\bar {x},\bar {z})$. It turns out that the above SNC/PSNC properties are ensured by certain Lipschitzian behavior of sets and mappings; see [16, 17]. In particular, F is PSNC at $(\bar {x},\bar {z})$ if it is Lipschitz-like around this point, i.e., there are neighborhoods U of $\bar {x}$ and V of $\bar {z}$ and a constant ℓ≥0 such that

$$\begin{array}{@{}rcl@{}} F(x)\cap V\subset F(u)+\ell\|x-u\|\mathbb{B}\quad\mathrm{ for\; all }\;x,u\in U. \end{array} $$

This clearly reduces to the classical local Lipschitz continuity for single-valued functions. Furthermore, for Lipschitz-like mappings between general Banach spaces, we have

$$\begin{array}{@{}rcl@{}} D^{*}_{M} F(\bar{x},\bar{z})(\mathbf{0})=\{\mathbf{0}\} \end{array} $$

(10)

by [16, Theorem 1.44], while the coderivative condition (10) together with the PSNC property of F at $(\bar {x},\bar {z})$ provides a complete characterization of the Lipschitz-like property of F around this point for closed-graph mappings between Asplund spaces; see [16, Theorem 4.10].

Finally, we recall the extremal principle for two multifuntions; see [15, Definition 3.3] for a version with n mappings.

Definition 2

(Extremal systems for multifunctions) Let $S_{i} : M_{i} \rightrightarrows X$ for i = 1, 2 be set-valued mappings from metric spaces (M _i , d _i ) into a Banach space X. We say that $\bar {x}$ is a local extremal point of the system {S ₁, S ₂} at $(\bar {s}_{1},\bar {s}_{2})$ provided that $\bar {x} \in S_{1}(\bar {s}_{1})\cap S_{2}(\bar {s}_{2})$ and there exists a neighborhood U of $\bar {x}$ such that for every ε > 0 there are s _i ∈dom S _i satisfying the conditions

$$\begin{array}{@{}rcl@{}} &&d_{i}(s_{i},\bar{s}_{i}) \leq \varepsilon, \quad \mathrm{dist\,}\left(\bar{x};S_{i}(s_{i})\right) \leq \varepsilon\quad\mathrm{ for }\; i=1,2, \quad \text{and}\\ &&\qquad\qquad\qquad\quad S_{1}(s_{1}) \cap S_{2}(s_{2}) \cap U \; =\; \emptyset. \end{array} $$

(11)

In this case, $\{S_{1}, S_{2}, \bar {x}\}$ is called an extremal system for multifunctions at $(\bar {s}_{1},\bar {s}_{2})$.

When S ₁ and S ₂ enjoy the linear transformation property of the form

$$ M_{1} := X, \quad M_{2} := \{\mathbf{0}\},\quad S_{1}(s_{1}) := {\Omega}_{1} + s_{1},\quad\mathrm{ and }~ S_{2}(\mathbf{0}) := {\Omega}_{2}, $$

(12)

the extremal system for two multifunctions S ₁ and S ₂ reduces to that for two sets Ω₁ and Ω₂.

Definition 3

(Extremal systems for sets) Let Ω₁ and Ω₂ be nonempty sets of a space X and $\bar {x}\in {\Omega }_{1}\cap {\Omega }_{2}$. We say that $\bar {x}$ is a local extremal point of the set system {Ω₁, Ω₂} in X if there are a neighborhood U of $\bar {x}$ and a sequence {a _k }⊂Z with ∥a _k ∥→0 such that

$$\begin{array}{@{}rcl@{}} {\Omega}_{1}\cap({\Omega}_{2} + a_{k})\cap U= \emptyset \quad \mathrm{for\; all }~ k\in \mathbb{N}. \end{array} $$

(13)

To formulate the exact extremal principle for multifunctions in the case of infinite-dimensional codomain/image spaces, we need the following normal compactness property for set-valued mappings.

Definition 4

(ISNC property of moving sets) A set-valued mapping $S : M\rightrightarrows X$ between a metric space (M, d) to a Banach space X is said to be imagely SNC (or ISNC, in short) at $(\bar {s},\bar {x}) \in \mathrm {gph\,} S$ if for any sequences $(s_{k}, x_{k}, x^{*}_{k})$ satisfying

$$\begin{array}{@{}rcl@{}} x^{*}_{k} \in \Hat N(s_{k}; S(s_{k})), \quad (s_{k}, x_{k}) \mathop{\longrightarrow}\limits^{\mathrm{gph\,} S} (\bar{s},\bar{x}), \quad \mathrm{ and }~ x^{*}_{k} \mathop{\longrightarrow}\limits^{w^{*}} \mathbf{0}\\ \end{array} $$

one has $\|x^{*}_{k}\|\to 0$ as k → ∞.

Obviously, the ISNC condition for moving sets unconditionally holds when X has finite dimensions. Moreover, if S can be described in the form S(s):=g(s)+Ω for all s around $\bar {s}$, where Ω is a subset of X having the SNC property at $\bar {x}$, and g:M → X is a function being single-valued around $\bar {s}$ and continuous at $\bar {s}$, then S is ISNC at $(\bar {s}, \bar {z})$.

We are now ready to recall a simple version of the exact/limiting extremal principle for systems of multifunctions which were formulated by Mordukhovich, Treman, and Zhu for finitely many sets in [15, Theorem 4.7]. Let $\bar {x}\in S_{1}(\bar {s}_{1})\cap S_{2}(\bar {s}_{2})$ be a local extremal point of the system {S ₁, S ₂} at $(\bar {s}_{1},\bar {s}_{2})$. Assume that the codomain space X is Asplund. Assume also that each mapping S _i is closed-valued around $\bar {s}_{i}$ for i=1,2, and that either S ₁ is ISNC at $(\bar {s}_{1},\bar {x})$ or S ₂ is ISNC at $(\bar {s}_{2},\bar {x})$. Then, the Exact Extremal Principle for Multifunctions S ₁ and S ₂ holds: there is a nonzero dual element x ^∗ ∈ X ^∗ such that

$$x^{*} \in N_{+}(\bar{x};S_{1}(\bar{s}_{1})) \cap (- N_{+}(\bar{x};S_{2}(\bar{s}_{2}))), $$

where $N_{+}(\bar {x};S_{i}(\bar {s}_{i}))$ for i = 1, 2 stand for the imagely normal cones to $S_{i}(\bar {s}_{i})$ at $\bar {x}$ defined by

$$\begin{array}{@{}rcl@{}} N_{+}(\bar{x};S_{i}(\bar{s}_{i})) :=\mathop{\text{Lim}\,\text{sup}}\limits_{(s,x) ~\mathop{\longrightarrow}\limits^{\mathrm{gph\,} S_{i}}~ (\bar{s}_{i},\bar{x})} \Hat N(x;S_{i}(s)). \end{array} $$

When S ₁ and S ₂ are defined, via Ω₁ and Ω₂, by (12), the extremal principle for multifunctions reduces to the classical extremal principle for sets: Let $\bar {x}$ be a local extremal point to the set system {Ω₁, Ω₂} in an Asplund space X, where both sets Ω₁ and Ω₂ are locally closed around $\bar {x}$. Then there is a nonzero dual element x ^∗∈X ^∗ such that $x^{*} \in N(\bar {x};{\Omega }_{1})\cap (- N(\bar {x};{\Omega }_{2}))$ provided that either Ω₁ is SNC at $\bar {x}$ or Ω₂ is SNC at $\bar {x}$. The SNC condition can be relaxed in product spaces. In this paper, we need the Exact Extremal Principle for Sets in Product Spaces: Let $\bar {x}\in {\Omega }_{1}\cap {\Omega }_{2}$ be a local extremal point of the product sets Ω₁, Ω₂⊂X ₁ × X ₂ that are supposed to be locally closed around $\bar {x}$, and let J ₁, J ₂⊂{1,2} with J ₁∪J ₂ = {1, 2}. Assume that both spaces X ₁ and X ₂ are Asplund, and that Ω₁ is PSNC at $\bar {x}$ with respect to J ₁ while Ω₂ is strongly PSNC at $\bar {x}$ with respect to J ₂. Then there exists x ^∗ ≠ 0 satisfying

$$x^{*} \in N(\bar{x};{\Omega}_{1})\cap (-N(\bar{x};{\Omega}_{2})). $$

The extremal principle and its variants can be viewed as variational counterparts of the convex separation theorem for sets in nonconvex settings. In fact, it plays a fundamental role in variational analysis similar to that played by the separation theorem and equivalent results in convex analysis as well as in its outgrowths and applications; see the books by Mordukhovich [16, 17], which fully revolve around the extremal principle for sets and its modi?cations. In this paper, we develop new applications of the extremal principle to multiobjective optimization with respect to variable ordering structures.

3 Extremal Systems in Constrained Multiobjective Nondomination Problems

This section addresses the relationships between nondomination and extremality at nondominated solutions for the class of problems (P)

$$\mathrm{nondominate } \;\; F(x)\;\; \mathrm{ subject\; to }\;\; x\in {\Omega}\;\;\mathrm{ with\; respect\; to }\;\; \mathbf{D} $$

since the most crucial step in establishing necessary optimality conditions for Pareto efficient solutions in the variational dual-space approach is to construct an extremal system at a solution under consideration. Let us recall several useful extremal systems at efficient solutions before constructing new extremal systems of sets and multifunctions at nondominated solutions of problem (P).

Proposition 1

(Extremal set systems for Pareto efficient solutions) Assume that $(\bar {x},\bar {z})$ is a Pareto efficient solution of problem ( P ) with respect to the constant ordering structure, i.e., D(z) ≡ Θ for some convex and nonsubspace cone of the image space Z in the sense of ( 3 ). Then it is a (local) extremal point of the set system {Ω ₁ , Ω ₂ }, where Ω ₁ :=gph F and ${\Omega }_{2} := {\Omega }\times (\bar {z} - {\Theta })$.

Proof

See, e.g., [2, Theorem 5.1]. □

Proposition 2

(Extremal multifunction systems for Pareto efficient solutions) Assume that $(\bar {x},\bar {z})$ is a Pareto efficient solution of problem ( P ) with respect to the variable ordering structure D in the sense of ( 3 ). Then $(\bar {x},\bar {z})$ is a (local) extremal point at $(\bar {z},\mathbf {0})$ for the system of multifunctions $S_{i} : M_{i} \rightrightarrows X\times Z$ for i=1,2 defined by

$$\begin{array}{@{}rcl@{}} S_{1}(s_{1}) &:=& {\Omega} \times \left(s_{1} - \mathrm{cl\,} \mathbf{D}(s_{1})\right) \quad \text{with} \quad M_{1} := \left(\bar{z} - \mathbf{D}(\bar{z})\right)\cup\{\bar{z}\},\\ S_{2}(s_{2}) &\equiv& S_{2} := \mathrm{gph\,} F \quad \text{with} \quad M_{2} := \{\mathbf{0}\}, \end{array} $$

provided that the preference deduced from D enjoys the almost transitivity property.

Proof

See, e.g., [17, Example 5.56]. □

Since the necessary conditions for Pareto efficient solutions of constrained multiobjective optimization problems with respect to ordering cones or constant ordering structures are particular cases of those with respect to variable ordering structures, we do not present necessary conditions derived from the extremal systems above. The reader can find some variants being not the ‘best’ necessary results below; otherwise, see [2, 4].

Next, we introduce several extremal systems in multiobjective optimization with respect to variable ordering structures. To the best of our knowledge, Bao and Mordukhovich constructed for the first time the extremal system of three sets Ω₁, Ω₂, and Ω₃ in the product space W = X × Z × Z:=X × Z ₁ × Z ₂ in the recent paper [6], where Ω _i for i = 1, 2, 3 are described by

$$\begin{array}{@{}rcl@{}} \left\{\begin{array}{ll} {\Omega}_{1} :=\left\{(x,z_{1},z_{2})\in W\left|\right.\;(x,z_{1})\in\mathrm{gph\,} f\right\},\\ {\Omega}_{2} :=\left\{(x,z_{1},z_{2})\in W\left|\right.\;(z_{1},z_{2})\in\mathrm{gph\,}\mathbf{P}=\mathrm{gph\,}(I+\mathbf{D})\right\},\\ {\Omega}_{3} :={\Omega}\times Z\times\{\bar{z}\}\;\mathrm{ with }\;\bar{z}=f(\bar{x}) \end{array}\right. \end{array} $$

in order to establish some necessary nondomination conditions for problems (P). They assumed that the cost function f is single-valued and the ordering structure D enjoys conditions (A)–(C) in the introduction section. Observe that it is possible to combine these three sets into two sets since Ω₁ is independent on Z ₂, Ω₂ is independent on X, and Ω₃ is independent on Z ₁; see Proposition 3 for full justifications. Our main goal is to weaken the convex-valuedness and the cone-valuedness of ordering structures to the so-called asymptotic closedness property defined in [3, Definition 3.2]; see also [4, 5]. For the sake of self-contentedness, we recall the definition of the latter property and list several sufficient conditions for sets while referring the reader to the cited references for more results, discussions, and applications.

Definition 5

(Asymptotic closedness property) Let Ξ⊂Z be a subset in a Banach space Z, and let $\bar {z}\in \mathrm {cl\,}{\Xi }$. We say that Ξ is asymptotically closed at $\bar {z}$ if there is a sequence {c _k }⊂Z with ∥c _k ∥→0 as k → ∞ satisfying

$$\begin{array}{@{}rcl@{}} \mathrm{cl\,} {\Xi}+c_{k} \subset{\Xi}\backslash\{\bar{z}\}. \end{array} $$

Note that the asymptotic closedness of a set is fully independent on the local closedness of the same set and that it holds in many rather general settings:

• every proper convex subcone Ξ ⊂ Z with nonempty interior and its nonconvex complement Z∖Ξ have the asymptotic closedness property at the origin;

• every closed and convex cone Ξ ⊂ Z with Ξ∖(− Ξ) ≠ ∅ has the asymptotic closedness property at the origin;

• the epigraph of an extended-real-valued function $\varphi \colon X\to \mathbb {R}\cup \{\infty \}$ has the asymptotic closedness property at $(\bar {x},\varphi (\bar {x}))$ provided that φ is lower semicontinuous around $\bar {x}$.

Proposition 3

(Extremal set systems for nondominated solutions) Assume that $(\bar {x},\bar {z})$ is a nondominated solution of problem ( P ) with respect to the variable ordering structure D in the sense of Definition 1, where gph D is a closed set in Z × Z. Then, the triple $(\bar {x},\bar {z},\bar {z})$ is a (local) extremal point for the system of two sets Ω ₁ and Ω ₂ with

$$ {\Omega}_{1} := \mathrm{gph\,} F\times \{\bar{z}\} \quad \text{and}\quad {\Omega}_{2} := {\Omega}\times \mathrm{gph\,} (\mathbf{P}), $$

(14)

where P = I + D is the upper level-set mapping of the ordering structure D provided that gph P is asymptotically closed at $(\bar {z},\bar {z})$ which is equivalent to that gph D is asymptotically closed at $(\bar {z},\mathbf {0})$.

Proof

Since the graph of the upper level-set mapping P enjoys the asymptotic closedness property there exists, by Definition 5, a sequence {(b _k , c _k )}⊂Z × Z with ∥(b _k , c _k )∥→0 as k → ∞ satisfying

$$\begin{array}{@{}rcl@{}} \mathrm{cl\,} \mathrm{gph\,} \mathbf{P} + (b_{k},c_{k}) \subset \mathrm{gph\,} \mathbf{P}\setminus\{(\bar{z},\bar{z})\} \quad \mathrm{ for\; all }~ k\in \mathbb{N}. \end{array} $$

(15)

It is easy to check that $(\bar {x},\bar {z},\bar {z})\in {\Omega }_{1}\cap {\Omega }_{2}$. To justify the extremality of the set system {Ω₁, Ω₂} at this common point, it suffices to show the validity of the extremality condition (13) which, by taking into account the set structures of Ω₁ and Ω₂ in (14), reduces to

$$\begin{array}{@{}rcl@{}} \left(\mathrm{gph\,} F \times \{\bar{z}\}\right)\cap\left(({\Omega}\times \mathrm{gph\,} \mathbf{P}) + a_{k}\right)=\emptyset\quad\mathrm{ for\; all }~k\in \mathbb{N}, \end{array} $$

(16)

where the sequence {a _k }⊂X × Z × Z with a _k :=(0, b _k , c _k ) for all $k\in \mathbb {N}$ converges to zero as k → ∞. Arguing by contradiction, suppose that (16) does not hold for some $k\in \mathbb {N}$. Then, we can find (x, z ₁, z ₂) in the intersection on the left-hand side of (16) ensuring that x∈Ω, z ₁∈F(x), $z_{2} = \bar {z}$ satisfying

$$\begin{array}{@{}rcl@{}} (z_{1}, \bar{z}) \in \mathrm{gph\,} \mathbf{P} + (b_{k},c_{k}) = \mathrm{cl\,} \mathrm{gph\,} \mathbf{P} + (b_{k},c_{k}) \subset \mathrm{gph\,} \mathbf{P}\setminus\{(\bar{z},\bar{z})\}, \end{array} $$

where the equality holds due to the closedness assumption and the inclusion does due to the asymptotic closedness assumption (15). This implies that $z_{1}\neq \bar {z}$ and $\bar {z} \in \mathbf {P}(z_{1}) = z_{1} + \mathbf {D}(z_{1})$ contradicting the nondomination of $(\bar {x},\bar {z})$ to problem (P). The obtained contradiction verifies the fulfillment of (16) and thus the extremality of the set system {Ω₁, Ω₂} at the common point $(\bar {x},\bar {z},\bar {z})$. The proof is complete. □

Let us illustrate by an example the importance of the asymptotic closedness requirement imposed on ordering structures in Proposition 3.

Example 1

Consider the following nonconstrained nondomination problem (P)

$$\text{nondominate} \quad \varphi(x) = x^{3} \quad \mathrm{ with\; respect\; to\; the\; ordering\; structure\; \mathbf{D}\; in\;(2)}. $$

It is easy to check that the origin $\bar {x} = 0$ is a nondominated solution of this problem, and that $(0,0,0)\in \mathbb {R}^{3}$ is not a local extremal point to the set system {Ω₁, Ω₂} with

$${\Omega}_{1} := \mathrm{gph\,} x^{3} \times \{0\}\quad \text{and}\quad {\Omega}_{2} := \mathbb{R} \times \mathrm{gph\,} \mathbf{P}, $$

where $\mathrm {gph\,} \mathbf {P} = \{(z_{1}, z_{2})\in \mathbb {R}^{2}\;|\; z_{1} \geq 0\;\mathrm { and }\; z_{2} \geq z_{1}\}\cup \{(z_{1}, z_{2})\in \mathbb {R}^{2}\;|\; z_{1} \leq 0\mathrm { and } z_{2} \leq z_{1}\}$. The extremality does not occur at $(\bar {x},\bar {z},\bar {z}) = (0,0,0) \in \mathbb {R}^{3}$ since the graph of D is not asymptotically closed at (0, 0) ∈ gph D and thus neither is the graph of P at (0, 0) ∈ gph P.

Remark 1

(Alternative conditions)

• (a) The extremality for the set system {Ω₁, Ω₂} at $(\bar {x},\bar {z},\bar {z})$ in Proposition 3 is still valid by replacing the asymptotic closedness property for D with a stronger asymptotic closedness condition for F: there is a sequence {(b _k , c _k )}⊂Z × Z with ∥(b _k , c _k )∥→0 as k → ∞ such that

  $$\begin{array}{@{}rcl@{}} \mathrm{cl\,} \mathrm{gph\,} F + (b_{k},c_{k}) \subset \mathrm{gph\,} F\setminus(F^{-1}(\bar{z})\times \{\bar{z}\}) \quad \mathrm{ for\; all }~ k\in \mathbb{N}, \end{array} $$

  (17)

  where $F^{-1}(\bar {z}) := \{x\in X \;|\; \bar {z}\in F(x)\}$. Define a new sequence a _k :=(b _k , c _k , 0) for all $k\in \mathbb {N}$. Obviously, ∥a _k ∥→0 as k → ∞. We claim that the extremality condition (13) for Ω₁ and Ω₂

  $$\begin{array}{@{}rcl@{}} \left((\mathrm{gph\,} F \times \{\bar{z}\}) + a_{k}\right)\cap\left({\Omega}\times \mathrm{gph\,} \mathbf{P}\right)=\emptyset\quad\mathrm{ for\; all }~k\in \mathbb{N} \end{array} $$

  (18)

  holds for the chosen sequence {a _k }. Again, arguing by contradiction assume that (18) does not hold for some $k\in \mathbb {N}$. There is a triple (x, z ₁, z ₂) in the intersection on the left-hand side of (18) with x∈Ω, $z_{2} = \bar {z}$, $\bar {z}\in \mathbf {P}(z_{1})$, and

  $$\begin{array}{@{}rcl@{}} (x, z_{1}) \in \mathrm{gph\,} F + (b_{k},c_{k}) \subset\mathrm{gph\,} F\setminus(F^{-1}(\bar{z})\times \{\bar{z}\}), \end{array} $$

  where the inclusion holds due to (17). Thus, $z_{1}\neq \bar {z}$. This together with $\bar {z}\in \mathbf {P}(z_{1}) = z_{1} + \mathbf {D}(z_{1})$ contradicts the nondomination of $(\bar {x},\bar {z})$ to problem (P). The contradiction verifies the validity of (18) and thus the extremality of the set system {Ω₁, Ω₂} at the point $(\bar {x},\bar {z},\bar {z})$.

• (b) We can also show that the assertion in Proposition 3 is still valid when Ω is asymptotically closed at $\bar {x}$, F = f:X → Z is a single-valued function, and the preimage of f at $\bar {z}$, i.e., $f^{-1}(\bar {z})$ is singleton.

• (c) Observe that for any variable ordering structure D satisfying conditions (A)–(C), the upper level-set mapping P is asymptotically closed at $(\bar {z},\bar {z})$. Indeed, we have

  $$\begin{array}{@{}rcl@{}} \mathrm{gph\,}\mathbf{P} + (\mathbf{0}, k^{-1}\mathbf{e}) \subset \mathrm{gph\,} \mathbf{P}\setminus\{(\bar{z},\bar{z})\} \end{array} $$

  (19)

  where the nonzero vector e∈Θ _D is taken from condition (C). Obviously, gph P+(0, k ⁻¹ e)⊂gph P due to the convexity of D(z) for all z∈Z. Arguing by contradiction, assume that (19) does not hold, i.e., $(\bar {z},\bar {z}) \in \mathrm {gph\,} \mathbf {P} + (\mathbf {0}, k^{-1}\mathbf {e})$. This implies that $\bar {z} - k^{-1}\mathbf {e} \in \mathbf {P}(\bar {z}) = \bar {z} + \mathbf {D}(\bar {z})$ or $\mathbf {e} \in -\mathbf {D}(\bar {z})$ contradicting condition (B), ${\Theta }_{\mathbf {D}}\cap (-\mathbf {D}(\bar {z})) = \{\mathbf {0}\}$. The contradiction justifies the validity of (19) and thus the asymptotic closedness of gph D at $(\bar {z},\bar {z})$ in the sense of Definition 5.

The next theorem provides an extension of necessary nondomination conditions for nondominated solutions of problems (P) obtained in [6, Theorem 4.2] for set-valued costs. In contrast to the technical tools used in establishing necessary results in the previous paper, we use the extremal system of two sets in (14) instead of three sets defined in the proof of [6, Theorem 4.2] and employ the exact extremal principle for sets in product spaces instead of the approximate extremal principle to simplify our work.

Theorem 1

(Necessary nondomination condition of multiobjective problems) Let $(\bar {x},\bar {z})$ be a nondominated solution of problem ( P ) with respect to a general variable ordering structure $\mathbf {D}:Z\rightrightarrows Z$ satisfying the closedness and asymptotic closedness assumptions in Proposition 3. Assume that gph F and Ω are locally closed around $(\bar {x},\bar {z})$ and $\bar {x}$ , respectively. Assume also the validity of the two SNC conditions:

• (a) D is SNC at $(\bar {z},\mathbf {0})$ , which is equivalent to P being SNC at $(\bar {z},\bar {z})$.

• (b) Either Ω is SNC at $\bar {x}$ , or F is PSNC at $(\bar {x}, \bar {z})$.

Suppose, finally, that the mixed qualification condition

$$\begin{array}{@{}rcl@{}}D^{*}_{M} F(\bar{x},\bar{z})(\mathbf{0})\cap(-N(\bar{z};{\Omega}))=\{\mathbf{0}\} \end{array} $$

(20)

and the fixed point condition

$$\begin{array}{@{}rcl@{}} -z^{*}\in D^{*}_{N}\mathbf{D}(\bar{z},\mathbf{0})(z^{*})\Longrightarrow z^{*}=\mathbf{0}; \end{array} $$

(21)

are satisfied. The fixed point condition for D is equivalent to that the coderivative of the upper level-set mapping $D^{*}_{N}\mathbf {P}(\bar {z},\bar {z})$ has a trivial kernel, i.e., $\mathrm {Ker\,}D^{*}_{N}\mathbf {P}(\bar {z},\bar {z})=\{\mathbf {0}\}$ . Then, there are $\mathbf {0} \neq w^{*} \in D^{*}_{N} \mathbf {P}(\bar {z},\bar {z})(z^{*}) = z^{*} + D^{*}_{N}\mathbf {D}(\bar {z},\mathbf {0})(z^{*})$ for some z ^∗ ∈Z ^∗ such that

$$\begin{array}{@{}rcl@{}} \mathbf{0}\in D^{*}_{N} F(\bar{x},\bar{z})(w^{*}) + N(\bar{x};{\Omega}). \end{array} $$

(22)

Proof

By Proposition 3, we get from the nondomination of $(\bar {x},\bar {z})$ to problem (P) that the triple $(\bar {x},\bar {z},\bar {z})$ is a local extremal point to the system of sets Ω₁ and Ω₂ in (14).

By denoting the product space W:=X ₁ × X ₂ = X×(Z×Z) and using the notation J ₁ and J ₂ in the exact extremal principle for product spaces, the imposed SNC conditions (a) and (b) imply the corresponding following SNC conditions:

• (a) Ω₂ = Ω × gph P is SNC at $(\bar {x},\bar {z},\bar {z})$, i.e., J ₁ = ∅ and J ₂ = {1,2}.

• (b) Ω₂ is strongly PSNC with respect to J ₂ = {2} at $(\bar {x}, \bar {z},\bar {z})$ and ${\Omega }_{1} = \mathrm {gph\,} F\times \{\bar {z}\}$ is PSNC with respect to J ₁ ={1} at $(\bar {x},\bar {z},\bar {z})$.

Therefore, the aforementioned principle applied to the extremal set system {Ω₁, Ω₂} at $(\bar {x},\bar {z},\bar {z})$ ensures the existence of (x ^∗, w ^∗, z ^∗) ≠ 0 satisfying

$$(x^{*}, - w^{*}, z^{*}) \in N \left((\bar{x},\bar{z},\bar{z});\mathrm{gph\,} F\times \{\bar{z}\}\right) \cap \left(- N \left((\bar{x},\bar{z},\bar{z});{\Omega}\times \mathrm{gph\,} \mathbf{P}\right)\right).$$

By taking into account the normal cones to product sets and the normal coderivative definition (7), we get the limiting relationships

$$\begin{array}{@{}rcl@{}} \left\{ \begin{array}{lll} x^{*} \in D_{N}^{*}F(\bar{x},\bar{z})(w^{*}),\;\;- x^{*} \in N(\bar{x};{\Omega}) \;\mathrm{ and }\\[1ex] w^{*} \in D_{N}^{*}\mathbf{P}(\bar{z},\bar{z})(z^{*}) = z^{*} + D_{N}^{*}\mathbf{D}(\bar{z},\mathbf{0})(z^{*}) \end{array}\right\}, \end{array} $$

which clearly verify the necessary nondomination condition

$$\begin{array}{@{}rcl@{}} \mathbf{0}\in D_{N}^{*}F(\bar{x},\bar{z})(w^{*}) + N(\bar{x};{\Omega}) \quad \mathrm{ with }~ w^{*} = z^{*} + D_{N}^{*} \mathbf{D}(\bar{z},\mathbf{0})(z^{*}) \end{array} $$

(23)

provided that w ^∗ ≠ 0.

Let us now show that w ^∗ ≠ 0. Arguing by contradiction, we assume that w ^∗ = 0 and we get from (23) that $\mathbf {0} = z^{*}+D^{*}_{N}\mathbf {D}(\bar {z},\mathbf {0})(z^{*})$. By the fixed point condition (21), z ^∗ = 0. Under the imposed SNC assumptions (a) and (b), the inclusion in (23) reduces to

$$x^{*} \in D^{*}_{M} F(\bar{x},\bar{z})(\mathbf{0})\cap\left(- N(\bar{x};{\Omega})\right). $$

By the mixed qualification condition (20), x ^∗ = 0 and thus (x ^∗, w ^∗, z ^∗) = 0. This contradicts the nontriviality condition of the extremal principle for (x ^∗, w ^∗, z ^∗) and thus justifies the nonzero value of w ^∗. We complete the proof of the theorem. □

We conclude this section by discussing the extremality property for systems of multifunctions in optimization with variable ordering structures.

Proposition 4

(Extremal multifunction systems for nondominated solutions) Assume that $(\bar {x},\bar {z})$ is a nondominated solution of problem ( P ) with respect to the variable ordering structure D in the sense of Definition 1. Then, the pair $(\bar {x},\bar {z})$ is a (local) extremal point at $(\bar {z},\mathbf {0})$ for the system of multifunctions $S_{i} : M_{i} \rightrightarrows X\times Z\times Z$ for i=1,2 defined by

$$\begin{array}{@{}rcl@{}} \left\{\begin{array}{ll} S_{1}(s_{1}) \equiv \mathrm{gph\,} F \times \{\bar{z}\} \quad {with} \quad M_{1} := (Z, \|\cdot\|),\\ S_{2}(s_{2}) := {\Omega}\times \{s_{2}\} \times \mathbf{P}(s_{2}) \quad {with} \quad M_{2} := (Z, \|\cdot\|), \end{array}\right. \end{array} $$

(24)

where P = I + D is the upper level-set mapping of the ordering structure D.

Proof

First observe that $(\bar {x},\bar {z},\bar {z})\in S_{1}(\bar {z})\cap S_{2}(\bar {z})$. To show that $\bar {z}$ is a local extremal point for the system of the multifunctions S ₁ and S ₂ in (24) at $(\bar {x},\bar {z},\bar {z})$, it suffices to check the fulfillment of the extremality condition (11) in the following form:

$$ S_{1}(s_{1}) \cap S_{2}(s_{2}) = \emptyset\quad\mathrm{ for\; all }~ s_{1}\in Z,\; s_{2} \in Z\setminus\{\bar{z}\}. $$

(25)

Arguing by contradiction, suppose that (25) does not hold for some s ₁∈Z, and $s_{2} \in Z\setminus \{\bar {z}\}$. Then, we can find (x, z ₁, z ₂) in the intersection on the left-hand side of (25) ensuring by the set structures in (24) that

$$\begin{array}{@{}rcl@{}} x\in {\Omega},\quad z_{1} = s_{2} \in F(x), \quad \mathrm{ and }\quad z_{2} = \bar{z} \in \mathbf{P}(s_{2}) = \mathbf{P}(z_{1}) = z_{1} + \mathbf{D}(z_{1}), \end{array} $$

which contradicts the nondomination condition (1). The obtained contradiction verifies the fulfillment of (25) and the extremality of the multifunction system {S ₁, S ₂} at $(\bar {x},\bar {z},\bar {z})$. The proof is complete. □

Unfortunately, this extremal system provides a trivial necessary nondomination condition for problem (P). Precisely, employing the exact extremal principle for multifunctions to the extremal system {S ₁, S ₂} in Proposition 4, we get a triple (z ^∗,−w ^∗, z ^∗) ≠ g0 such that

$$\begin{array}{@{}rcl@{}} (z^{*},-w^{*},z^{*}) & \in & N_{+}\left((\bar{x},\bar{z},\bar{z}); S_{1}(\bar{z})\right)\cap \left(-N_{+}\left((\bar{x},\bar{z},\bar{z}); S_{2}(\bar{z})\right)\right)\\ & = & \left(N\left((\bar{x},\bar{z}); \mathrm{gph\,} F\right)\times Z^{*}\right)\cap \left(-N(\bar{x};{\Omega})\times -N_{+}\left((\bar{z},\bar{z}); \{\bar{z}\}\times \mathbf{P}(\bar{z})\right)\right)\\ & = & \left(N\left((\bar{x},\bar{z}); \mathrm{gph\,} F\right)\times Z^{*}\right)\cap \left(-N(\bar{x};{\Omega})\times Z^{*}\times -N_{+}(\bar{z}; \mathbf{P}(\bar{z}))\right). \end{array} $$

It is clear from the last expression that the nonzero triple (x ^∗, w ^∗, z ^∗) = (0, 0, z ^∗) for some $\mathbf {0} \neq z^{*} \in -N_{+}(\bar {z}; \mathbf {P}(\bar {z}))$ satisfies the necessary condition

$$\mathbf{0} \in D^{*}F(\bar{x},\bar{z})(\mathbf{0}) + N(\bar{x};{\Omega}) $$

in a trivial way. Note also that if we impose certain condition such that the dual element w ^∗ is nonzero, the result is still not significant since it does not relate to the ordering structure in question. A natural question arises: is it possible to construct an extremal system for multifunctions such that we can derive from it a necessary nondomination condition for problem (P)? Observe that Example 1 does not enjoy the extremality property of the system of sets Ω₁ and Ω₂ in (14), but the necessary nondomination condition (22) is satisfied with w ^∗ = − 1 and z ^∗ = 0 due to the following

$$-w^{*} = -1 \in z^{*} + D^{*}_{N} \mathbf{D}(0,0)(z^{*}) = 0 + D^{*}_{N} \mathbf{D}(0,0)(0) = \mathbb{R}\quad \mathrm{ and }\quad D^{*}\varphi(0)(1) = \{0\}. $$

The question will be further addressed in our future research.