Optimality and duality in nonsmooth composite vector optimization and applications

Abstract

This article is devoted to the study of a nonsmooth composite vector optimization problem (P for brevity). We apply some advanced tools of variational analysis and generalized differentiation to establish necessary conditions for (weakly) efficient solutions of (P). Sufficient conditions for the existence of such solutions to (P) are also provided by means of proposing the use of (strictly) generalized convex composite vector functions with respect to a cone. We also state a dual problem to (P) and explore weak, strong and converse duality relations. In addition, applications to a multiobjective approximation problem and a composite multiobjective problem with linear operators are deployed.

1 Introduction

Let $F:X\rightarrow W$ and $f:W\rightarrow Y$ be vector functions between finite-dimensional spaces. Given a pointed (i.e., $K\cap (-K)=\{0\}$) closed convex cone $K\subset Y$, we consider a composite vector optimization problem of the form:

$$\begin{aligned} \mathrm{min}_K\;\big \{(f\circ F)(x)\,|\,x\in C\big \}. \end{aligned}$$

(P)

Here, the feasible set C is defined by

$$\begin{aligned} C:=\big \{x\in X\mid \; (g\circ G)(x)\in -S\big \}, \end{aligned}$$

(1.1)

where $G:X\rightarrow V$, $g:V\rightarrow Z$ are vector functions between finite-dimensional spaces and $S\subset Z$ is a nonempty closed convex cone. From now on, we always assume that F, G, f, g are locally Lipschitz at the corresponding points under consideration.

The problem (P) has a quite general formulation, which provides a unified framework for examining various multiobjective/vector optimization problems (see e.g., Jeyakumar and Yang 1993; Tang and Zhao 2013). For example, when $X=W=V$ and F and G are identical maps, the problem (P) reduces to the following conic vector optimization problem:

$$\begin{aligned} \mathrm{min}_K\;\big \{f(x)\,|\, x\in X,\; g(x)\in -S\big \}. \end{aligned}$$

(CP)

In particular, if $X=W=V$, F and G are identical maps, K is the nonnegative orthant in $\mathbb {R}^p$ (i.e., $K:=\mathbb {R}^p_+$),

$$\begin{aligned} S:=\big \{(\alpha _1,\ldots ,\alpha _m,\beta _1,\ldots ,\beta _n)\in \mathbb {R}^{m+n}\mid \alpha _i\ge 0, \; i=1,\ldots ,m, \;\beta _j=0, \;j=1,\ldots , n\big \}\end{aligned}$$

(1.2)

and $f:X\rightarrow Y:=\mathbb {R}^p$ and $g:X\rightarrow Z:=\mathbb {R}^{m+n}$ are given respectively by

$$\begin{aligned} f(x):=(f_1(x),\ldots ,f_p(x)),\; g(x):=(g_1(x),\ldots ,g_m(x), h_1(x),\ldots ,h_n(x)),\; x\in X,\end{aligned}$$

(1.3)

then the problem (P) collapses to a (standard) multiobjective/vector optimization problem of the form:

$$\begin{aligned}&\mathrm{min}_{\mathbb {R}^p_+}\;\big \{(f_1(x),\ldots ,f_p(x))\,\mid \,x\in X, \; g_i(x)\le 0,\; i=1,\ldots ,m,\\&\qquad \qquad \quad h_j(x)=0,\; j=1,\ldots ,n\big \}, \end{aligned}$$

(MP)

where $f_k,\;k=1,\ldots ,p,\; g_i,\; i=1,\ldots , m,\; h_j,\; j=1,\ldots , n$ are real-valued functions on X that are locally Lipschitz at the point in question.

Definition 1.1

We say that an element ${\bar{x}}\in C$ is an efficient solution of problem (P), and write ${\bar{x}}\in \mathcal {S}(P)$, if

$$\begin{aligned} \forall x\in C, \quad (f\circ F)(x)-(f\circ F)({\bar{x}})\notin -K{\setminus } \{0\}. \end{aligned}$$

When the topological interior of K is nonempty (i.e., $\mathrm{int\,}K\ne \emptyset$), an element ${\bar{x}}\in C$ is called a weakly efficient solution of problem (P), and write ${\bar{x}}\in \mathcal {S}^{w}(P)$, if

$$\begin{aligned} \forall x\in C, \quad (f\circ F)(x)-(f\circ F)({\bar{x}})\notin -\mathrm{int\,} K. \end{aligned}$$

In what follows, we always assume that $\mathrm{int\,}K\ne \emptyset$.

Optimality conditions and duality for (weakly) efficient solutions of the model problem (P) have been investigated by many researchers (see e.g., Bot et al. 2007, 2008; Jeyakumar and Yang 1993; Mishra 1996; Mishra and Mukherjee 1995; Reddy and Mukherjee 1999 and the references therein). Most of these works have dealt with problem types of (P), where functions F and G are either convex with respect to a cone or Gâteaux differentiable. The results about necessary conditions for the problem type (P) given in some of the aforementioned papers have been obtained through scalarization processes. More precisely, the authors in that papers have employed the separation theorem of convex sets (cf. Rockafellar 1970) to provide necessary conditions in terms of the Clarke subdifferential (cf. Clarke 1983) for (weakly) efficient solutions of problem (P) via its associated scalar problems. While the results about sufficient conditions and duality of the previous works have been established thanks to convexity or some kinds of invexity of the related functions. The interested reader is referred to Suneja et al. (2018), Sach et al. (2005), Lee and Lee (2018), La Torre (2003), Chuong and Kim (2017), Chuong (2018), Lee and Jiao (2018), Chuong (2017), Chinchuluun and Pardalos (2007), Chuong and Kim (2016) for more information on other optimality conditions and duality in both smooth and nonsmooth multiobjective/vector optimization problems involving convex functions or generalized convex/invex functions.

In this paper, we focus on the problem (P) involving nonsmooth functions; especially, functions F and G are not necessarily G${\hat{a}}$teaux differentiable. We apply some advanced tools of variational analysis and generalized differentiation to establish necessary conditions for (weakly) efficient solutions of problem (P). These formulations are expressed in terms of the limiting/Mordukhovich subdifferential or the normal/Mordukhovich coderivative. It is achieved by (1) applying a fuzzy necessary condition (cf. Chuong 2019), which was established based on the approximate extremal principle (cf. Mordukhovich 2006) and the fuzzy sum rule for the Fréchet subdifferential, and (2) employing the sum rule for the limiting subdifferential and the scalarization formulae of the coderivatives (cf. Mordukhovich et al. 2006). This allows us to establish these necessary conditions directly; meaning that we do not utilize any scalarization process. Sufficient conditions for the existence of such solutions to problem (P) are also provided by means of proposing the use of (strictly) generalized convex composite vector functions with respect to a cone. Along with optimality conditions, we propose a dual problem to (P) and explore weak, strong and converse duality relations between them. In addition, applications to a multiobjective approximation problem and a composite multiobjective problem with linear operators are undertaken.

The rest of the paper is organized as follows. Section 2 contains some basic definitions from variational analysis and several auxiliary results. In Sect. 3, we first establish necessary conditions for (weakly) efficient solutions of problem (P). We then provide sufficient conditions for the existence of such solutions. Section 4 is devoted to describing duality relations in composite vector optimization problems. Applications are given in Sect. 5. Section 6 summarises the obtained results.

2 Preliminaries

Throughout the paper, we use the standard notation of variational analysis (see e.g., Mordukhovich 2006). All spaces under consideration are finite-dimensional spaces equipped with the Euclidean norm $\Vert \cdot \Vert$ and the inner product $\langle \cdot \,,\cdot \rangle .$ $B_X(x,r)$ stands for the closed ball in a space X with center $x\in X$ and radius $r>0$, while the closed unit ball in X is denoted by $B_X$ for simplicity. The topological closure and the topological interior of a set $\Omega \subset X$ are denoted by $\mathrm{cl\;}\Omega$ and $\mathrm{int\;}\Omega$, respectively. As usual, the dual cone of $\Omega \subset X$ is the set

$$\begin{aligned} \Omega ^+:=\{x^*\in X \mid \langle x^*,x\rangle \ge 0,\; \forall x\in \Omega \}. \end{aligned}$$

For $n\in \mathbb {N}:=\{1,2,\ldots \},$ we denote by $\mathbb {R}^n_+$ the nonnegative orthant of $\mathbb {R}^n.$ The symbol $A^\top$ denotes the adjoint operator (or conjugate transpose) of the linear operator A on a finite-dimensional space.

Given a multifunction $H:X\rightrightarrows X$, we denote by

$$\begin{aligned} \begin{array}{ll} \displaystyle \mathop {\mathrm{Lim}\,\mathrm{sup}}_{x\rightarrow \bar{x}}H(x):=\Big \{x^*\in X\big |&{}\exists \; \text{ sequences } \;x_n\rightarrow \bar{x}\; \text{ and } \; x^*_n\rightarrow x^*\\ &{}\text{ with }\;x^*_n\in H(x_n)\; \text{ for } \text{ all } \;n\in \mathbb {N}\Big \} \end{array} \end{aligned}$$

the sequential Painlevé-Kuratowski upper/outer limit of H as $x\rightarrow {\bar{x}}$.

A set $\Omega \subset X$ is called closed around $\bar{x}\in \Omega$ if there is a neighborhood U of ${\bar{x}}$ such that $\Omega \cap \mathrm{cl\,}U$ is closed. We say that $\Omega$ is locally closed if $\Omega$ is closed around x for every $x\in \Omega$.

Let $\Omega \subset X$ be locally closed. The Fréchet normal cone to $\Omega$ at $\bar{x}\in \Omega$ is defined by

$$\begin{aligned} \widehat{N}({\bar{x}};\Omega ):= \Big \{x^*\in X\Big |\limsup _ {x\xrightarrow {\Omega }{\bar{x}}}\frac{\langle x^*, x-{\bar{x}}\rangle }{\Vert x-\bar{x}\Vert }\le 0\Big \}, \end{aligned}$$

where $x\xrightarrow {\Omega }\bar{x}$ means that $x\rightarrow \bar{x}$ with $x\in \Omega$. If $x\notin \Omega ,$ we put $\widehat{N}(x;\Omega ):=\emptyset .$

The limiting/Mordukhovich normal cone $N({\bar{x}};\Omega )$ to $\Omega$ at $\bar{x}\in \Omega$ is obtained from Fréchet normal cones by taking the sequential Painlevé-Kuratowski upper limits as

$$\begin{aligned} N({\bar{x}};\Omega ):=\mathop {\mathop {\mathrm{Lim}\,\mathrm{sup}}}\limits _{x\xrightarrow {\Omega }{\bar{x}}} \widehat{N}(x;\Omega ). \end{aligned}$$

(2.1)

If $x\notin \Omega ,$ we put $N(x;\Omega ):=\emptyset .$

For an extended real-valued function $\varphi :X\rightarrow \overline{\mathbb {R}}:=\mathbb {R}\cup \{\infty \}$, we set

$$\begin{aligned} \text{ dom }\,\varphi :=\{x\in X\mid \varphi (x)<\infty \},\quad \text{ epi }\,\varphi :=\{(x,\mu )\in X\times \mathbb R\; |\; \mu \ge \varphi (x)\}. \end{aligned}$$

The limiting/Mordukhovich subdifferential and the Fréchet subdifferential of $\varphi$ at $\bar{x}\in X$ with $|\varphi ({\bar{x}})|<\infty$ are defined, respectively, by

$$\begin{aligned} \partial \varphi ({\bar{x}})&:=\{ x^*\in X\; |\; (x^*,-1)\in N(({\bar{x}},\varphi ({\bar{x}}));\text{ epi }\,\varphi )\} \end{aligned}$$

(2.2)

and

$$\begin{aligned} {{\widehat{\partial }}}\varphi ({\bar{x}})&:=\{ x^*\in X\; |\; (x^*,-1)\in {\widehat{N}}(({\bar{x}},\varphi ({\bar{x}}));\text{ epi }\,\varphi )\}. \end{aligned}$$

(2.3)

If $|\varphi (\bar{x})|=\infty ,$ then one puts $\partial \varphi ({\bar{x}}) ={{\widehat{\partial }}}\varphi ({\bar{x}})=\emptyset .$

We get by definitions of (2.1), (2.2) and (2.3) that

$$\begin{aligned} {{\widehat{\partial }}}\varphi ({\bar{x}}) \subset \partial \varphi ({\bar{x}})\quad \forall {\bar{x}}\in X. \end{aligned}$$

Let $H:X \rightrightarrows Y$ be a multifunction with the graph

$$\begin{aligned} \text{ gph }\,H:= \big \{(x,y)\in X\times Y\big |\;y\in H(x)\big \}. \end{aligned}$$

The normal/Mordukhovich coderivative of H at $({\bar{x}},{\bar{y}})\in \mathrm{gph}\,H$ is a multifunction $D^*H({\bar{x}},{\bar{y}}):Y\rightrightarrows X$ defined by

$$\begin{aligned} D^*H({\bar{x}},{\bar{y}})(y^*):=\big \{x^*\in X\ |\ (x^*,-y^*)\in N\big (({\bar{x}},{\bar{y}});\mathrm{gph}\,H)\big )\big \},\quad y^*\in Y. \end{aligned}$$

If H is a single-valued map, to simplify the notation, one writes $D^*H({\bar{x}})(y^*)$ instead of $D^*H({\bar{x}}, H({\bar{x}}))(y^*).$

For any vector function $f:X\rightarrow Y$ we can associate f with a scalarization function with respect to some $y^*\in Y$ defined by

$$\begin{aligned} \langle y^*,f\rangle (x)=\langle y^*, f(x)\rangle ,\quad x\in X. \end{aligned}$$

Since our space setting is finite-dimensional, there is a relationship between the normal/Mordukhovich coderivative of a locally Lipschitzian vector function and the limiting subdifferential of its scalarization ones that is formulated as follows (see Mordukhovich 2006, Theorem 3.28): If f is locally Lipschitz at ${\bar{x}}\in X$, then one has

$$\begin{aligned} D^*f({\bar{x}})(y^*)=\partial \langle y^*, f\rangle ({\bar{x}}), \quad y^*\in Y. \end{aligned}$$

(2.4)

In particular, when f is strictly differentiable at ${\bar{x}}$ with the derivative operator $\nabla f({\bar{x}})$, (2.4) reduces to the following one (see Mordukhovich 2006, Theorem 1.38):

$$\begin{aligned} D^*f(\bar{x})(y^*)=\{\nabla f({\bar{x}})^\top y^*\},\quad y^*\in Y, \end{aligned}$$

where we should recall that a function $f:X\rightarrow Y$ is strictly differentiable at ${\bar{x}}\in X$ if there is a linear continuous operator $\nabla f({\bar{x}}):X\rightarrow Y$ such that

$$\begin{aligned} \displaystyle \lim _{x,u\rightarrow {\bar{x}}}\frac{f(x)-f(u)-\langle \nabla f({\bar{x}}),x-u\rangle }{\Vert x-u\Vert }=0. \end{aligned}$$

Another expression of formula (2.4) can be found in the next lemma.

Lemma 2.1

Let $y^*\in \mathbb {R}^n,$ and let $f:X\rightarrow \mathbb {R}^n$ be Lipschitz continuous around ${\bar{x}}\in X.$ We have

• (i) (See Mordukhovich et al. 2006, Proposition 3.5) $x^*\in {{\widehat{\partial }}} \langle y^*,f\rangle ({\bar{x}})\Leftrightarrow (x^*,-y^*)\in {\widehat{N}}(({\bar{x}},f({\bar{x}}));\mathrm{gph}\, f).$

• (ii) (See Mordukhovich 2006, Theorem 1.90) $x^*\in \partial \langle y^*,f\rangle ({\bar{x}})\Leftrightarrow (x^*,-y^*)\in N(({\bar{x}},f({\bar{x}}));\mathrm{gph}\, f).$

The next lemma gives a chain rule for the limiting subdifferential.

Lemma 2.2

(See Mordukhovich 2006, Corollary 3.43) Let $f:X\rightarrow Y$ be locally Lipschitz at ${\bar{x}}\in X$, and let $\varphi : Y\rightarrow \mathbb {R}$ be locally Lipschiz at $f({\bar{x}}).$ Then one has

$$\begin{aligned} \partial (\varphi \circ f)({\bar{x}})\subset \bigcup _{y^*\in \partial \varphi (f({\bar{x}}))}\partial \langle y^*,f\rangle ({\bar{x}}).\end{aligned}$$

(2.5)

The sum rule for the limiting subdifferential is needed for our study.

Lemma 2.3

(See Mordukhovich 2006, Theorem 3.36) Let $\varphi _i: X\rightarrow \overline{\mathbb {R}}, i=1,2,\ldots ,n, n\ge 2,$ be lower semicontinuous around ${\bar{x}}\in X,$ and let all but one of these functions be Lipschitz continuous around ${\bar{x}}.$ Then one has

$$\begin{aligned} \partial (\varphi _1+\varphi _2+\cdots +\varphi _n)({\bar{x}})\subset \partial \varphi _1({\bar{x}})+\partial \varphi _2({\bar{x}})+\cdots +\partial \varphi _n({\bar{x}}).\end{aligned}$$

(2.6)

The following lemma is concerning with the subdifferential of a norm.

Lemma 2.4

(See Gopfert et al. 2003, Lemma 4.1.11) Let $x\in X$ and $\beta >1.$ Then we have

$$\begin{aligned} \partial ||x||={\left\{ \begin{array}{ll}\{x^*\in X\mid \langle x^*,\, x\rangle =||x||, ||x^*||=1\} &{}\quad \text{ if }\ x\ne 0,\\ \{x^*\in X\mid ||x^*||\le 1\} &{}\quad \text{ if }\ x=0,\end{array}\right. } \end{aligned}$$

and

$$\begin{aligned} \partial \left( \frac{1}{\beta }||x||^\beta \right) =\{x^*\in X\mid \langle x^*,\, x\rangle =||x||^\beta , ||x^*||=||x||^{\beta -1} \}. \end{aligned}$$

3 Optimality conditions in composite vector optimization

This section is devoted to studying optimality conditions in composite vector optimization problems. More precisely, we first establish necessary conditions expressed in terms of the limiting subdifferential or the Mordukhovich coderivative for (weakly) efficient solutions of problem (P). We then provide sufficient conditions for the existence of such solutions under assumptions of (strictly) generalized convexity for composite vector functions.

The first theorem provides a necessary optimality condition for (weakly) efficient solutions of problem (P). This condition is of the Fritz-John type and is expressed in terms of the limiting subdifferential. To prove this theorem, we need a fuzzy necessary optimality condition for the conic vector optimization problem (CP) as follows.

Lemma 3.1

(See Chuong 2019, Theorem 3.1) Let ${\bar{x}}$ be a weakly efficient solution of problem (CP). Then for each $k\in \mathbb {N}$ there exist $x^{1k}\in B_{X}({\bar{x}},\frac{1}{k}),x^{2k}\in B_{X}({\bar{x}},\frac{1}{k}), y^*_k\in K^+$ with $||y^*_k||=1$ and $z^*_k\in S^+$ such that

$$\begin{aligned} 0\in&{{\widehat{\partial }}} \langle y^*_k, f\rangle (x^{1k})+{{\widehat{\partial }}}\langle z^*_k,g\rangle (x^{2k})+\frac{1}{k} B_{X}, \\&|\langle z^*_k,g(x^{2k})\rangle |\le \frac{1}{k}. \end{aligned}$$

Theorem 3.2

Let ${\bar{x}}\in \mathcal {S}^w(P).$ Then there exist $y^*\in K^+$ and $z^*\in S^+$ with $||y^*||+||z^*||=1$, such that

$$\begin{aligned}&0\in \bigcup _{w^*\in \partial \langle y^*, f\rangle (F({\bar{x}}))} \partial \langle w^*, F\rangle ({\bar{x}})+ \bigcup _{v^*\in \partial \langle z^*,g\rangle (G({\bar{x}}))}\partial \langle v^*,G\rangle ({\bar{x}}), \nonumber \\&\langle z^*,g(G({\bar{x}}))\rangle =0. \end{aligned}$$

(3.1)

Proof

Let ${\tilde{f}}:= f\circ F$ and ${\tilde{g}}:=g\circ G.$ Then, the problem (P) becomes the following one

Applying the fuzzy necessary condition given in Lemma 3.1 to the problem (${\tilde{\hbox {P}}}$), we arrive at the conclusion that for each $k\in \mathbb {N}$ there exist $x^{1k}\in B_{X}({\bar{x}},\frac{1}{k}),x^{2k}\in B_{X}({\bar{x}},\frac{1}{k}), y^*_k\in K^+$ with $||y^*_k||=1$ and $z^*_k\in S^+$ such that

$$\begin{aligned} 0\in&{{\widehat{\partial }}} \langle y^*_k, {\tilde{f}} \rangle (x^{1k})+{{\widehat{\partial }}}\langle z^*_k,{\tilde{g}} \rangle (x^{2k})+\frac{1}{k} B_{X}, \\&|\langle z^*_k,{\tilde{g}}(x^{2k})\rangle |\le \frac{1}{k}. \end{aligned}$$

Then, we find sequences $\{x^{1k}\}\subset X, \{x^{2k}\}\subset X, \{x^{*}_{1k}\}\subset X, \{x^{*}_{2k}\}\subset X$, $\{y^*_k\}\subset K^+$ with $||y^*_k||=1$ and $\{z^*_k\}\subset S^+$ such that $x^{*}_{1k}\in {{\widehat{\partial }}} \langle y^*_k, f\circ F\rangle (x^{1k}),\quad x^{*}_{2k}\in {{\widehat{\partial }}}\langle z^*_k,g\circ G\rangle (x^{2k}),$

$$\begin{aligned} 0\in x^{*}_{1k}+x^{*}_{2k}+\frac{1}{k} B_X, \end{aligned}$$

(3.2)

and

$$\begin{aligned} x^{1k}\rightarrow {\bar{x}},\quad x^{2k}\rightarrow {\bar{x}},\quad \langle z^*_k,(g\circ G)(x^{2k})\rangle \rightarrow 0 \quad \text{ as } \quad k\rightarrow \infty . \end{aligned}$$

(3.3)

By our assumptions, we suppose that $f\circ F$ is locally Lipschitz at ${\bar{x}}$ with a modulus $\ell >0.$ One has $||x^*_{1k}||\le \ell ||y^{*}_k||\le \ell$ for all $k\in \mathbb {N}$ (see e.g., Mordukhovich 2006, Proposition 1.85). In addition, since X is a finite-dimensional space, we may assume by taking a subsequence if necessary that $x^*_{1k}\rightarrow x^*_1\in X$ as $k\rightarrow \infty .$ Therefore, it stems from (3.2) that $x^{*}_{2k}\rightarrow -x^*_1$ as $k\rightarrow \infty .$ Letting $x^*_2:=-x^*_1$, it holds that $x^{*}_{2k}\rightarrow x^*_2$ as $k\rightarrow \infty ,$ and

$$\begin{aligned} x^{*}_{1}+x^{*}_{2}=0. \end{aligned}$$

(3.4)

Besides, as Y is a finite-dimensional space, we may assume by passing to subsequences if necessary that $y^*_k\rightarrow {\tilde{y}}^*\in K^+$ with $||{\tilde{y}}^*||=1$ as $k\rightarrow \infty$. We now consider the following possibilities:

Case 1: If $\{z^*_k\}$ is unbounded, then we may assume by passing to subsequences if necessary that $||z^*_k||\rightarrow \infty$ and ${\tilde{z}}^*_k:=\frac{z^*_k}{||z^*_k||}\rightarrow z^*\in S^+$ with $||z^*||=1$ as $k\rightarrow \infty .$ Then, it follows from (3.3) that $\langle {\tilde{z}}^*_k,(g\circ G)(x^{2k})\rangle \rightarrow 0$ as $k\rightarrow \infty ,$ i.e., we obtain that

$$\begin{aligned} \langle z^*,g(G({\bar{x}}))\rangle =0. \end{aligned}$$

According to Lemma 2.1, the inclusion

$$\begin{aligned} x^*_{2k}\in {{\widehat{\partial }}} \langle z^*_k,g\circ G\rangle (x^{2k}) \end{aligned}$$

amounts to the following one:

$$\begin{aligned} (x^*_{2k},-z^*_k)\in {\widehat{N}}\big ((x^{2k},(g\circ G)(x^{2k}));\mathrm{gph\,}(g\circ G)\big ),\quad k\in \mathbb {N}.\end{aligned}$$

Hence,

$$\begin{aligned} \left( \frac{x^*_{2k}}{||z^*_k||},\frac{-z^*_k}{ ||z^*_k||}\right) \in {\widehat{N}}\big ((x^{2k},(g\circ G)(x^{2k}));\mathrm{gph\,}(g\circ G)\big ),\quad k\in \mathbb {N}. \end{aligned}$$

(3.5)

Letting $k\rightarrow \infty$ in (3.5) and noticing (2.1), we get the relation

$$\begin{aligned} \left( 0, -z^*\right) \in N\big (({\bar{x}},(g\circ G)({\bar{x}}));\mathrm{gph\,}(g\circ G)\big ), \end{aligned}$$

which is equivalent to

$$\begin{aligned} 0\in \partial \langle z^*,g\circ G\rangle ({\bar{x}}) \end{aligned}$$

(3.6)

due to Lemma 2.1. So, by taking $y^*:=0\in K^+,$ we arrive at

$$\begin{aligned} 0\in \partial \langle y^*,f\circ F\rangle ({\bar{x}})+\langle z^*,g\circ G\rangle ({\bar{x}}). \end{aligned}$$

(3.7)

Case 2: If $\{z^*_k\}$ is bounded, then we may assume without loss of generality that $z^*_k\rightarrow {\tilde{z}}^*\in S^+$ as $k\rightarrow \infty .$ As above, we get from the inclusion $x^*_{1k}\in {{\widehat{\partial }}} \langle y^*_k,f\circ F\rangle (x^{1k})$ that

$$\begin{aligned} (x^*_{1k},-y^*_k)\in {\widehat{N}}\big ((x^{1k},(f\circ F)(x^{1k}));\mathrm{gph\,}(f\circ F)\big ),\quad k\in \mathbb {N}. \end{aligned}$$

Hence,

$$\begin{aligned} \left( \frac{x^*_{1k}}{||y^*_k||+||z^*_k||},\frac{-y^*_k}{||y^*_k||+||z^*_k||}\right) \in {\widehat{N}}\big ((x^{1k},(f\circ F)(x^{1k}));\mathrm{gph\,}(f\circ F)\big ),\quad k\in \mathbb {N}. \end{aligned}$$

(3.8)

Letting $k\rightarrow \infty$ in (3.8) and noticing (2.1), we get the relation

$$\begin{aligned} \left( \frac{x^{*}_1}{||{\tilde{y}}^*||+||{\tilde{z}}^*||},\frac{-{\tilde{y}}^*}{||{\tilde{y}}^*||+||{\tilde{z}}^*||}\right) \in N\big (({\bar{x}},(f\circ F)({\bar{x}}));\mathrm{gph\,}(f\circ F)\big ), \end{aligned}$$

which is equivalent to

$$\begin{aligned} \frac{x^*_1}{||{\tilde{y}}^*||+||{\tilde{z}}^*||}\in \partial \langle y^*,f\circ F\rangle ({\bar{x}}), \end{aligned}$$

(3.9)

where $y^*\!\!:\,=\frac{{\tilde{y}}^*}{||{\tilde{y}}^*||+||{\tilde{z}}^*||}.$ Similarly, we obtain

$$\begin{aligned} \frac{x^*_2}{||{\tilde{y}}^*||+||{\tilde{z}}^*||}\in \partial \langle z^*,g\circ G\rangle ({\bar{x}})\quad \text{ and }\quad \langle z^*,(g\circ G)({\bar{x}})\rangle =0, \end{aligned}$$

(3.10)

where $z^*:=\frac{{\tilde{z}}^*}{||{\tilde{y}}^*||+||{\tilde{z}}^*||}.$ Combining this with (3.9) and (3.4) gives the relation

$$\begin{aligned} 0\in \partial \langle y^*,f\circ F\rangle ({\bar{x}})+\langle z^*,g\circ G\rangle ({\bar{x}}), \end{aligned}$$

i.e., (3.7) holds, too.

Now, letting $\varphi :=\langle y^*,f\rangle$ and $\psi :=\langle z^*,g\rangle$, one can rewrite (3.7) as follow:

$$\begin{aligned} 0\in \partial (\varphi \circ F)({\bar{x}})+\partial (\psi \circ G) ({\bar{x}}). \end{aligned}$$

(3.11)

By our assumptions, F and G are locally Lipschitz at ${\bar{x}},$ and $\varphi$ and $\psi$ are locally Lipschitz at $F({\bar{x}})$ and $G({\bar{x}})$, respectively. So, applying the chain rule (2.5) for (3.11), we obtain (3.1). This completes the proof of the theorem. $\square$

Remark 3.3

The result in Theorem 3.2 develops the corresponding one in Jeyakumar and Yang (1993, Theorem 2.1) and Tang and Zhao (2013, Theorem 3.1). More precisely, Jeyakumar and Yang (1993, Theorem 2.1) and Tang and Zhao (2013, Theorem 3.1) were established in terms of the Clarke subdifferential with F, G being G${\hat{a}}$teaux differentiable, and K, S of the former were considered to be nonegative orthant cones, while F, G of the latter were additionally assumed to be generalized invex with respect to cone $K\times S$. Note further that the proof of Tang and Zhao (2013, Theorem 3.1) relied mainly on the use of the Gordon-type theorem (Craven and Jeyakumar 1986), which always requires some kind of (generalized) convexity.

Based on the scalarization formula of the Mordukhovich coderivative (2.4), we can transform the above result established in terms of subdifferentials into another equivalent form, which is expressed by way of coderivatives.

Corollary 3.4

Let ${\bar{x}}\in \mathcal {S}^w(P).$ Then there exist $y^*\in K^+$ and $z^*\in S^+$ with $||y^*||+||z^*||=1$, such that

$$\begin{aligned} 0\in&\bigcup _{w^*\in D^* f(F({\bar{x}}))(y^*)} D^* F({\bar{x}})(w^*)+ \bigcup _{v^*\in D^*g(G({\bar{x}}))(z^*)} D^*G ({\bar{x}})(v^*), \\&\langle z^*,g(G({\bar{x}}))\rangle =0. \end{aligned}$$

Proof

The proof follows from Theorem 3.2 by using the scalarization formula of the Mordukhovich coderivative given in (2.4). $\square$

The following corollary presents a Fritz-John optimality condition (see also, Chuong 2019, Corollary 3.2) for (weakly) efficient solutions of the multiobjective/vector optimization problem (MP).

Corollary 3.5

Let ${\bar{x}}$ be a weakly efficient solution of problem (MP). Then there exist $\lambda _k\ge 0,\, k=1,\ldots ,p,\; \mu _i\ge 0,\, i=1,\ldots , m$ and $\sigma _j\in \mathbb {R}, \, j=1,\ldots ,n$ with $\sum _{k=1}^p\lambda _k+\sum _{i=1}^m\mu _i+ \sum _{j=1}^n|\sigma _j|\ne 0$ such that

$$\begin{aligned} 0\in&\sum _{k=1}^p\lambda _k\partial f_k({\bar{x}})+\sum _{i=1}^m\mu _i\partial g_i({\bar{x}})+\sum _{j=1}^n \partial (\sigma _j h_j)({\bar{x}}),\nonumber \\&\mu _i g_i({\bar{x}})=0,\quad i=1,\ldots ,m, \end{aligned}$$

(3.12)

Proof

Observe that the multiobjective optimization problem (MP) is a particular case of the composite vector optimization problem (P), where $W:=V:=X$, F and G are identical maps (i.e., $F(x):=x, G(x):=x, x\in X$), $K:=\mathbb {R}^p_+$, S is given as in (1.2), $f:X\rightarrow Y:=\mathbb {R}^p$ and $g:X\rightarrow Z:=\mathbb {R}^{m+n}$ are given respectively as in (1.3). So, invoking Theorem 3.2, we find $y^*\in K^+$ and $z^*\in S^+$ with $||y^*||+||z^*||=1$, such that

$$\begin{aligned}&0\in \bigcup _{w^*\in \partial \langle y^*, f\rangle (F({\bar{x}}))} \partial \langle w^*, F\rangle ({\bar{x}})+ \bigcup _{v^*\in \partial \langle z^*,g\rangle (G({\bar{x}}))}\partial \langle v^*,G\rangle ({\bar{x}}), \nonumber \\&\langle z^*,g(G({\bar{x}}))\rangle =0. \end{aligned}$$

(3.13)

In this setting, we see that $K^+=\mathbb {R}^p_+$ and

$$\begin{aligned} S^+=\{(\mu _1,\ldots ,\mu _m,\sigma _1,\ldots ,\sigma _n)\in \mathbb {R}^{m+n}\mid \mu _i\ge 0, \; i=1,\ldots ,m, \;\sigma _j\in \mathbb {R}, \;j=1,\ldots , n\}. \end{aligned}$$

(3.14)

Thus, taking $y^*:=(\lambda _1,\ldots ,\lambda _p)\in K^+$ and $z^*:=(\mu _1,\ldots ,\mu _m,\sigma _1,\ldots ,\sigma _n)\in S^+,$ (3.13) reduces to the following one

$$\begin{aligned}&0\in \partial \left( \sum _{k=1}^p\lambda _kf_k\right) ({\bar{x}})+\partial \left( \sum _{i=1}^m\mu _ig_i+\sum _{j=1}^n\sigma _jh_j\right) ({\bar{x}}),\nonumber \\&\sum _{i=1}^m\mu _i g_i({\bar{x}})+\sum _{j=1}^n \sigma _j h_j({\bar{x}})=0. \end{aligned}$$

(3.15)

Note here that $\mu _ig_i({\bar{x}})\le 0, i=1,\ldots ,m$ and $h_j({\bar{x}})=0, j=1,\ldots ,n.$ Invoking the sum rule (2.6), we get by (3.15) that

$$\begin{aligned}&0\in \sum _{k=1}^p\lambda _k\partial f_k({\bar{x}})+ \sum _{i=1}^m\mu _i\partial g_i({\bar{x}})+\sum _{j=1}^n\partial (\sigma _j h_j)({\bar{x}}),\\&\mu _i g_i({\bar{x}})=0,\quad \; i=1,\ldots ,m, \end{aligned}$$

Clearly, $\sum _{k=1}^p\lambda _k+\sum _{i=1}^m\mu _i+ \sum _{j=1}^n|\sigma _j|\ne 0$ and so, the proof is complete. $\square$

We refer the interested reader to Minami (1983, Theorem 3.1) for a Fritz-John optimality condition formulated in terms of the Clarke subdifferential that is similar to the one obtained in Corollary 3.5 for a multiobjective optimization problem involving a geometric constraint set and local Lipschitz functions on a Banach space.

To formulate sufficient conditions for the existence of (weakly) efficient solutions of problem (P), let us first define a Karush–Kuhn–Tucker (KKT) type point for this problem.

Definition 3.6

A point ${\bar{x}}\in C$ is termed a (KKT) point of problem (P) if there exist $y^*\in K^+{\setminus }\{0\}$ and $z^*\in S^+$ such that

$$\begin{aligned} 0\in&\bigcup _{w^*\in \partial \langle y^*, f\rangle (F({\bar{x}}))} \partial \langle w^*, F\rangle ({\bar{x}})+ \bigcup _{v^*\in \partial \langle z^*,g\rangle (G({\bar{x}}))}\partial \langle v^*,G\rangle ({\bar{x}}), \\&\langle z^*,g(G({\bar{x}}))\rangle =0. \end{aligned}$$

It can be seen from Theorem 3.2 that a weakly efficient solution of problem (P) becomes a (KKT) point under the fulfilment of the following constraint qualification.

Definition 3.7

We say that the constraint qualification (CQ) is satisfied at ${\bar{x}}\in C$ if there does not exist $z^*\in S^+$ with $||z^*||=1$ and $\langle z^*,g(G({\bar{x}}))\rangle =0$, such that

$$\begin{aligned} 0\in \bigcup _{v^*\in \partial \langle z^*,g\rangle (G({\bar{x}}))}\partial \langle v^*,G\rangle ({\bar{x}}). \end{aligned}$$

(3.16)

In order to better understand the above-defined qualification condition, let us examine a particular case when $V=X$, G is an identical map (i.e., $G(x):=x,\; x\in X$), $g:V\rightarrow Z:=\mathbb {R}^{m+n}$ is given as in (1.3) and S is given as in (1.2). In this case, the dual cone $S^+$ is given as in (3.14). Then, taking $z^*:=(\mu _1,\ldots ,\mu _m,\sigma _1,\ldots ,\sigma _n)\in S^+$ and arguing as in the proof of Corollary 3.5, we arrive at the conclusion that (3.16) reduces to the following one:

$$\begin{aligned} 0\in \sum _{i=1}^m\mu _i\partial g_i({\bar{x}})+\sum _{j=1}^n\gamma _j\big (\partial h_j({\bar{x}})\cup \partial (-h_j)({\bar{x}})\big ), \end{aligned}$$

where $\gamma _j:=|\sigma _j|$ for $j=1,\ldots ,n.$

In this way, we can restate the above-defined (CQ) as follows: There do not exist $\mu _i\ge 0,\, i=1,\ldots , m,$ and $\gamma _j\ge 0, \, j=1,\ldots ,n$, with $\sum _{i=1}^m\mu _i+ \sum _{j=1}^n\gamma _j\ne 0$ such that

$$\begin{aligned} 0\in&\sum _{i=1}^m\mu _i\partial g_i({\bar{x}})+\sum _{j=1}^n\gamma _j\big (\partial h_j({\bar{x}})\cup \partial (-h_j)({\bar{x}})\big ),\\&\mu _i g_i({\bar{x}})=0,\quad i=1,\ldots ,m, \end{aligned}$$

which can be found in Chuong and Kim (2014a), Mordukhovich (2006), and is guaranteed by the Mangasarian-Fromovitz constraint qualification when considering it in the smooth setting. The interested reader is referred to the monographs (Mordukhovich 2006, 2018) for a comprehensive study of various qualification/regularity conditions by way of variational analysis and generalized differentiation that guarantee the fulfilment of the above-defined (CQ).

In general, a (KKT) point of problem (P) is not necessarily to be (weakly) efficient; see e.g., Chuong and Kim (2014a, Example 3.5) for a simple case. Thus, in order for a (KKT) point of problem (P) to be an efficient (weakly efficient) solution, we need to employ concepts of (strictly) generalized convexity (with respect to a cone) for composite vector functions.

Definition 3.8

• (i) We say that $(f\circ F, g\circ G)$ is $(K\times S)$-generalized convex at ${\bar{x}}\in X$ if for any $x\in X, \; \;y^*\in K^+,\; z^*\in S^+$, $w^*\in \partial \langle y^*,f\rangle (F({\bar{x}})),\; x_1^*\in \partial \langle w^*,F\rangle ({\bar{x}}), \; v^*\in \partial \langle z^*,g\rangle (G({\bar{x}})),$ and $x_2^*\in \partial \langle v^*,G\rangle ({\bar{x}}),$ there exists $\nu \in X$ such that

  $$\begin{aligned} \langle y^*,f\circ F\rangle (x)-\langle y^*,f\circ F\rangle ({\bar{x}})\ge&\langle x_1^*,\nu \rangle ,\\ \langle z^*,g\circ G\rangle (x)-\langle z^*,g\circ G\rangle ({\bar{x}})\ge&\langle x_2^*,\nu \rangle . \end{aligned}$$

• (ii) We say that $(f\circ F,g\circ G)$ is K-strictly $(K\times S)$-generalized convex at ${\bar{x}}\in X$ if for any $x\in X\backslash \{{\bar{x}}\}, \; \;y^*\in K^+{\setminus }\{0\},\; z^*\in S^+$, $w^*\in \partial \langle y^*,f\rangle (F({\bar{x}})), \; x_1^*\in \partial \langle w^*,F\rangle ({\bar{x}}), \; v^*\in \partial \langle z^*,g\rangle (G({\bar{x}})),$ and $x_2^*\in \partial \langle v^*,G\rangle ({\bar{x}}),$ there exists $\nu \in X$ such that

  $$\begin{aligned}\langle y^*,f\circ F\rangle (x)-\langle y^*,f\circ F\rangle ({\bar{x}})>&\langle x_1^*,\nu \rangle ,\\ \langle z^*,g\circ G\rangle (x)-\langle z^*,g\circ G\rangle ({\bar{x}})\ge&\langle x_2^*,\nu \rangle . \end{aligned}$$

The following proposition shows that the class of (strict) generalized convex composite vector functions defined above includes (strict) convex vector functions.

Proposition 3.9

Let ${\bar{x}}\in X$, and let f, F and g, G be such that $\langle y^*,f\rangle$ is convex for every $y^*\in K^+$, $\langle z^*,g\rangle$ is convex for every $z^*\in S^+$, $\langle w^*,F \rangle$ is convex for every $w^*\in \partial \langle y^*,f\rangle (F({\bar{x}}))$ and $\langle v^*,G \rangle$ is convex for every $v^*\in \partial \langle z^*,g\rangle (G({\bar{x}})).$

• (i) It holds that $(f\circ F, g\circ G)$ is $(K\times S)$-generalized convex at ${\bar{x}}.$

• (ii) If assume in addition that $\langle y^*,f\rangle$ is strictly convex for every $y^*\in K^+\backslash \{0\}$ and F is injective (i.e., $F(x_1)=F(x_2)\Rightarrow x_1=x_2)$, then $(f\circ F, g\circ G)$ is K-strictly $(K\times S)$-generalized convex at ${\bar{x}}.$

Proof

(i) Let $x\in X, \; \;y^*\in K^+,\; z^*\in S^+$, $w^*\in \partial \langle y^*,f\rangle (F({\bar{x}})),\; x_1^*\in \partial \langle w^*,F\rangle ({\bar{x}}), \; v^*\in \partial \langle z^*,g\rangle (G({\bar{x}}))$ and $x_2^*\in \partial \langle v^*,G\rangle ({\bar{x}}).$ Choose $\nu :=x-{\bar{x}}.$ On the one side, since $\langle w^*,F \rangle$ is convex, it follows that

$$\begin{aligned} \langle x^*_1,\nu \rangle =\langle x^*_1, x-{\bar{x}}\rangle \le \langle w^*, F\rangle (x)- \langle w^*, F\rangle ({\bar{x}})= \langle w^*, F(x)-F({\bar{x}})\rangle . \end{aligned}$$

(3.17)

On the other side, since $\langle y^*,f\rangle$ is a convex function, it holds that

$$\begin{aligned} \langle w^*, F(x)-F({\bar{x}})\rangle \le \langle y^*,f\rangle (F(x))- \langle y^*, f \rangle (F({\bar{x}}))= \langle y^*, f\circ F\rangle (x)-\langle y^*, f\circ F\rangle ({\bar{x}}). \end{aligned}$$

(3.18)

Combining now (3.17) with (3.18) gives us

$$\begin{aligned} \langle x^*_1, \nu \rangle \le \langle y^*, f\circ F\rangle (x)-\langle y^*, f\circ F\rangle ({\bar{x}}). \end{aligned}$$

Similarly, since $\langle z^*,g\rangle$ is convex for every $z^*\in S^+$ and $\langle v^*,G \rangle$ is convex for every $v^*\in \partial \langle z^*,g\rangle (G({\bar{x}}))$, we obtain

$$\begin{aligned} \langle x^*_2, \nu \rangle \le \langle z^*, g\circ G\rangle (x)-\langle z^*, g\circ G\rangle ({\bar{x}}). \end{aligned}$$

So, $(f\circ F, g\circ G)$ is $(K\times S)$-generalized convex at ${\bar{x}}.$

(ii) Let $x\in X\backslash \{{\bar{x}}\}, \; \;y^*\in K^+\backslash \{0\},\; z^*\in S^+$, $w^*\in \partial \langle y^*,f\rangle (F({\bar{x}})),\; x_1^*\in \partial \langle w^*,F\rangle ({\bar{x}}), \; v^*\in \partial \langle z^*,g\rangle (G({\bar{x}}))$ and $x_2^*\in \partial \langle v^*,G\rangle ({\bar{x}}).$ Taking $\nu :=x-{\bar{x}},$ it is shown in the proof of (i) that

$$\begin{aligned} \langle x^*_2, \nu \rangle \le \langle z^*, g\circ G\rangle (x)-\langle z^*, g\circ G\rangle ({\bar{x}}) \end{aligned}$$

and

$$\begin{aligned} \langle x^*_1,\nu \rangle \le \langle w^*, F(x)-F({\bar{x}})\rangle . \end{aligned}$$

(3.19)

Note by the injectivity of F that $F(x)\ne F({\bar{x}})$. Since $\langle y^*,f\rangle$ is a strict convex function, it holds (cf. Hiriart-Urruty and Lemarechal 1993, Proposition 6.1.3, p. 281) that

$$\begin{aligned} \langle w^*, F(x)-F({\bar{x}})\rangle < \langle y^*,f\rangle (F(x))- \langle y^*, f \rangle (F({\bar{x}}))= \langle y^*, f\circ F\rangle (x)-\langle y^*, f\circ F\rangle ({\bar{x}}). \end{aligned}$$

This together with (3.19) entails that

$$\begin{aligned} \langle x^*_1, \nu \rangle < \langle y^*, f\circ F\rangle (x)-\langle y^*, f\circ F\rangle ({\bar{x}}). \end{aligned}$$

Consequently, $(f\circ F, g\circ G)$ is K-strictly $(K\times S)$-generalized convex at ${\bar{x}}.$ $\square$

Remark 3.10

When $W:=V:=X$ and F, G are identical maps (i.e., $F(x)=G(x):=x,\; x\in X$), the notions in Definition 3.8 reduce to the corresponding ones in Chuong (2019, Definition 3.3). In this case, if f is K-convex on X (cf. Luc 1989, Definition 6.1) and g is S-convex on X, then $(f\circ F, g\circ G)=(f,g)$ is $(K\times S)$-generalized convex at any ${\bar{x}}\in X$ with $\nu :=x-{\bar{x}}$ for each $x\in X.$ Moreover, as shown by Chuong (2019, Example 3.1), the class of generalized convex vector functions contains some nonconvex functions.

We are now ready to provide sufficient conditions for (weakly) efficient solutions of problem (P).

Theorem 3.11

Let ${\bar{x}}$ be a (KKT) point of problem (P).

• (i) If $(f\circ F,g\circ G)$ is $(K\times S)$-generalized convex at ${\bar{x}},$ then ${\bar{x}}\in \mathcal {S}^w(P).$

• (ii) If $(f\circ F,g\circ G)$ is K-strictly $(K\times S)$-generalized convex at ${\bar{x}},$ then ${\bar{x}}\in \mathcal {S}(P).$

Proof

As ${\bar{x}}$ is a (KKT) point of problem (P), there exist $y^*\in K^+{\setminus }\{0\}$, $z^*\in S^+$, $w^*\in \partial \langle y^*, f\rangle (F({\bar{x}})),\; x^*_1\in \partial \langle w^*, F\rangle ({\bar{x}}),\; v^*\in \partial \langle z^*, g\rangle (G({\bar{x}})),$ and $x^*_2\in \partial \langle v^*, G\rangle ({\bar{x}})$ such that

$$\begin{aligned} x^*_1+x^*_2&=0, \end{aligned}$$

(3.20)

$$\begin{aligned} \langle z^*,g(G({\bar{x}}))\rangle&=0. \end{aligned}$$

(3.21)

We first justify (i). Assume by contradiction that ${\bar{x}} \notin \mathcal {S}^w(P).$ This means that there is ${\hat{x}}\in C$ such that

$$\begin{aligned}(f\circ F)({\hat{x}})-(f\circ F)({\bar{x}})\in -\mathrm{int\,} K. \end{aligned}$$

Then it holds (cf. Jahn 2004, Lemma 3.21) that

$$\begin{aligned} \langle y^*,(f\circ F)({\hat{x}})-(f\circ F)({\bar{x}})\rangle <0. \end{aligned}$$

(3.22)

Since $(f\circ F,g\circ G)$ is $(K\times S)$-generalized convex at ${\bar{x}},$ we deduce from (3.20) that for ${\hat{x}}$ above there is $\nu \in X$ such that

$$\begin{aligned} 0&= \langle x^*_1,\nu \rangle +\langle x^*_2,\nu \rangle \le \langle y^*,f\circ F\rangle ({\hat{x}})-\langle y^*,f\circ F\rangle ({\bar{x}})\\&\quad +\langle z^*,g\circ G\rangle ({\hat{x}})-\langle z^*,g\circ G\rangle ({\bar{x}}). \end{aligned}$$

In addition, $\langle z^*,g(G({\hat{x}}))\rangle \le 0$ due to $(g\circ G)({\hat{x}})\in -S.$ So, noting (3.21), we conclude that

$$\begin{aligned} 0\le \langle y^*,(f\circ F)({\hat{x}})-(f\circ F)({\bar{x}})\rangle , \end{aligned}$$

which contradicts (3.22), and thus the proof of (i) is finished.

We now prove (ii). Argue by contradiction that ${\bar{x}} \notin \mathcal {S}(P).$ There is ${\tilde{x}}\in C$ such that

$$\begin{aligned} (f\circ F)({\tilde{x}})-(f\circ F)({\bar{x}})\in -K\backslash \{0\}. \end{aligned}$$

Then ${\tilde{x}}\ne {\bar{x}}$ and

$$\begin{aligned} \langle y^*,(f\circ F)({\tilde{x}})-(f\circ F)({\bar{x}})\rangle \le 0. \end{aligned}$$

(3.23)

Since $(f\circ F,g\circ G)$ is K-strictly $(K\times S)$-generalized convex at ${\bar{x}},$ we deduce from (3.20) that for ${\tilde{x}}$ above there is $\nu \in X$ such that

$$\begin{aligned} 0&= \langle x^*_1,\nu \rangle +\langle x^*_2,\nu \rangle < \langle y^*,f\circ F\rangle ({\tilde{x}})-\langle y^*,f\circ F\rangle ({\bar{x}})\\&\quad +\langle z^*,g\circ G\rangle ({\tilde{x}})-\langle z^*,g\circ G\rangle ({\bar{x}}). \end{aligned}$$

By arguing as in the proof of (i), we arrive at

$$\begin{aligned} 0< \langle y^*,(f\circ F)({\tilde{x}})-(f\circ F)({\bar{x}})\rangle , \end{aligned}$$

which is a contradiction to (3.23). So, the proof is complete. $\square$

The next corollary shows how Theorem 3.11 can be used to reobtain (classical) sufficient optimality conditions for (weakly) efficient solutions of a convex multiobjective optimization problem.

Corollary 3.12

For the multiobjective optimization problem (MP), let $f_k,\;k=1,\ldots ,p,\; g_i, i=1,\ldots , m$ be convex functions and $h_j,\; j=1,\ldots , n$ be affine functions. Assume that ${\bar{x}}$ is a (KKT) point of problem (MP), i.e., (3.12) holds with $(\lambda _1,\ldots ,\lambda _p)\ne 0.$ Then, ${\bar{x}}$ is a weakly efficient solution of problem (MP). If assume in addition that $f_k,\;k=1,\ldots ,p,$ are strict convex functions, then ${\bar{x}}$ is an efficient solution of problem (MP).

Proof

Observe first as in the proof of Corollary 3.5 that the multiobjective optimization problem (MP) is a particular case of the composite vector optimization problem (P), where $W:=V:=X$, F and G are identical maps (i.e., $F(x):=x, G(x):=x, x\in X$), $K:=\mathbb {R}^p_+$, S is given as in (1.2), $f:X\rightarrow Y:=\mathbb {R}^p$ and $g:X\rightarrow Z:=\mathbb {R}^{m+n}$ are given respectively as in (1.3).

Since $f_k,\;k=1,\ldots ,p,\; g_i,\; i=1,\ldots , m$ are convex functions and $h_j,\; j=1,\ldots , n$ are affine functions, $(f\circ F, g\circ G)$ is $(K\times S)$-generalized convex at ${\bar{x}}$ as shown by Proposition 3.9(i). If assume in addition that $f_k,\;k=1,\ldots ,p,$ are strict convex functions, then $(f\circ F, g\circ G)$ is K-strictly $(K\times S)$-generalized convex at ${\bar{x}}$ by virtue of Proposition 3.9(ii). So, the desired conclusions follow directly from Theorem 3.11. $\square$

It is worth mentioning here that some related sufficient optimality criteria that also develop the ones in Corollary 3.12 were given in Chuong and Kim (2014a, Theorem 3.7) in terms of L-invex-infine functions on a set for a multiobjective optimization problem involving a geometric constraint set and in Chuong and Kim (2014b, Theorem 3.2) by way of a family of generalized convex functions for a semi-infinite multiobjective optimization problem.

We close this section with a remark that the application of our concepts of (strictly) generalized convex composition vector functions in deriving sufficient conditions for (weakly) efficient solutions of problem (P) enables us to avoid employing the so-called $\eta$-generalized null space condition ($\eta$-GNC) (Tang and Zhao 2013), which is somewhat hard to be verified in the setting under consideration; see Tang and Zhao (2013, Theorems 4.1–4.7) for more details on the use of the $\eta$-GNC.

4 Duality in composite vector optimization

In this section, we address a dual problem to the composite vector optimization problem (P) and explore weak, strong and converse duality relations. The dual problem below is formulated in the sense of Mond and Weir (1981), one can similarly deal with another dual problem in the sense of Wolfe (1961).

Let $z\in X, y^*\in K^+{\setminus }\{0\},$ and $z^*\in S^+.$ In connection with the problem (P), we consider a dual problem of the form:

$$\begin{aligned} \mathrm{max}_{K}\;\big \{L(z,y^*,z^*):=(f\circ F)(z)\mid (z,y^*,z^*)\in C_D\big \}, \end{aligned}$$

(D)

where the feasible set $C_D$ is defined by

$$\begin{aligned} C_D:&=\Big \{(z,y^*,z^*)\in X\times (K^+{\setminus }\{0\})\times S^+\mid \; 0\in \bigcup _{w^*\in \partial \langle y^*, f\rangle (F(z))} \partial \langle w^*, F\rangle (z)\nonumber \\&\quad +\bigcup _{v^*\in \partial \langle z^*,g\rangle (G(z))}\partial \langle v^*,G\rangle (z), \langle z^*, g(G(z))\rangle \ge 0\Big \}. \end{aligned}$$

(4.1)

It should be mentioned here that an efficient solution (resp., a weakly efficient solution) of the dual problem (D) is similarly defined as in Definition 1.1 by replacing $-K$ (resp., $\mathrm{-int\,} K$) by K (resp., $\mathrm{int\,} K$). Besides, we denote the set of efficient solutions (resp., weakly efficient solutions) of problem (D) by $\mathcal {S}(D)$ [resp., $\mathcal {S}^w(D)$].

In what follows, we use the following notation for convenience.

$$\begin{aligned} u\prec v&\Leftrightarrow u-v\in -\mathrm{int\,}K,\quad u\nprec v \text{ is } \text{ the } \text{ negation } \text{ of } u\prec v,\\ u\preceq v&\Leftrightarrow u-v\in -K\backslash \{0\},\quad u\npreceq v \text{ is } \text{ the } \text{ negation } \text{ of } u\preceq v. \end{aligned}$$

Weak duality relations between the primal problem (P) and the dual problem (D) read as follows.

Theorem 4.1

(Weak Duality) Let $x\in C$ and let $(z,y^*,z^*)\in C_D$.

• (i) If $(f\circ F,g\circ G)$ is $(K\times S)$-generalized convex at z,  then

  $$\begin{aligned} (f\circ F)(x)\nprec L(z,y^*,z^*).\end{aligned}$$

• (ii) If $(f\circ F,g\circ G)$ is K-strictly $(K\times S)$-generalized convex at z,  then

  $$\begin{aligned} (f\circ F)(x)\npreceq L(z,y^*,z^*).\end{aligned}$$

Proof

Since $(z,y^*,z^*)\in C_D$, there exist $y^*\in K^+{\setminus }\{0\}$, $z^*\in S^+$, $w^*\in \partial \langle y^*, f\rangle (F(z)),$ $x^*_1\in \partial \langle w^*, F\rangle (z),\; v^*\in \partial \langle z^*, g\rangle (G(z)),$ and $x^*_2\in \partial \langle v^*, G\rangle (z)$ such that

$$\begin{aligned}&x_1^*+x_2^*=0, \end{aligned}$$

(4.2)

$$\begin{aligned}&\langle z^*, g(G(z))\rangle \ge 0. \end{aligned}$$

(4.3)

We first justify (i). Assume to the contrary that

$$\begin{aligned} (f\circ F)(x)\prec L(z,y^*,z^*). \end{aligned}$$

Equivalently,

$$\begin{aligned} (f\circ F)(x)-(f\circ F)(z)\in -\mathrm{int\,}K. \end{aligned}$$

Hence,

$$\begin{aligned} \langle y^*,(f\circ F)(x)-(f\circ F)(z)\rangle <0 \end{aligned}$$

(4.4)

by virtue of Jahn (2004, Lemma 3.21). Since $(f\circ F,g\circ G)$ is $(K\times S)$-generalized convex at z,  we deduce from (4.2) that for x above there is $\nu \in X$ such that

$$\begin{aligned} 0= \langle x_1^*,\nu \rangle +\langle x_2^*,\nu \rangle \le \langle y^*,f\circ F\rangle ( x)-\langle y^*,f\circ F\rangle (z)+\langle z^*,g\circ G\rangle (x)-\langle z^*,g\circ G\rangle (z). \end{aligned}$$

(4.5)

In addition, $\langle z^*,g(G(x))\rangle \le 0$ owing to $(g\circ G)(x)\in -S.$ Thus, taking (4.3) into account, we get by (4.5) that

$$\begin{aligned} 0\le \langle y^*,(f\circ F)(x)-(f\circ F)(z)\rangle , \end{aligned}$$

which contradicts (4.4), and therefore finishes the proof of (i).

Now, we prove (ii). Assume by contradiction that

$$\begin{aligned} (f\circ F)(x)\preceq L(z,y^*,z^*), \end{aligned}$$

which amounts to

$$\begin{aligned} (f\circ F)(x)-(f\circ F)(z)\in -K\backslash \{0\}. \end{aligned}$$

(4.6)

Hence,

$$\begin{aligned} \langle y^*,(f\circ F)(x)-(f\circ F)(z)\rangle \le 0. \end{aligned}$$

(4.7)

In addition, (4.6) infers that $x\ne z.$ As $(f\circ F,g\circ G)$ is K-strictly $(K\times S)$-generalized convex at z,  we deduce from (4.2) that for x above there is $\nu \in X$ such that

$$\begin{aligned} 0= \langle x_1^*,\nu \rangle +\langle x_2^*,\nu \rangle < \langle y^*,f\circ F\rangle (x)-\langle y^*,f\circ F\rangle (z)+\langle z^*,g\circ G\rangle (x)-\langle z^*,g\circ G\rangle (z). \end{aligned}$$

(4.8)

Since $\langle z^*,g(G(x))\rangle \le 0$ as shown above, we deduce from (4.3) and (4.8) that

$$\begin{aligned} 0<\langle y^*,(f\circ F)(x)-(f\circ F)(z)\rangle , \end{aligned}$$

which contradicts (4.7), and thus completes the proof. $\square$

The following example shows that the generalized convexity (with respect to a cone) of $(f\circ F,g\circ G)$ imposed in the above theorem cannot be removed.

Example 4.2

Let $F:\mathbb {R}:\rightarrow \mathbb {R}^2,\; f: \mathbb {R}^2\rightarrow \mathbb {R}^2,\; G:\mathbb {R}\rightarrow \mathbb {R},$ and $g: \mathbb {R}\rightarrow \mathbb {R}$

$$\begin{aligned}&F(x):=(x^3,x^3),\; x\in \mathbb {R},\quad f(w):=(w_1+1, w_2+1),\; w:=(w_1,w_2)\in \mathbb {R}^2,\\&G(x):=x+1,\; x\in \mathbb {R},\quad g(v):=-|v-1|,\; v\in \mathbb {R}. \end{aligned}$$

Consider the problem (P) with $K:=\mathbb {R}^2_+\subset \mathbb {R}^2$ and $S:=[0,+\infty )\subset \mathbb {R}$. It is clear that $C=\mathbb {R}$, and let us select ${\bar{x}}:=-1\in C$. Now, consider the dual problem (D). By taking ${\bar{z}}:=0\in \mathbb {R}, \;{\bar{y}}^*:=(1,1)\in K^+,$ and ${\bar{z}}^*:=0\in S^+,$ we have $({\bar{z}},{\bar{y}}^*,{\bar{z}}^*)\in C_D.$ It can be checked that

$$\begin{aligned} (f\circ F)({\bar{x}})=(0,0) \prec (1,1)= L({\bar{z}},{\bar{y}}^*,{\bar{z}}^*), \end{aligned}$$

showing that the conclusion of Theorem 4.1 fails to hold for this case. The reason is that $(f\circ F,g\circ G)$ is not $(K\times S)$-generalized convex at ${\bar{z}}.$

The next theorem describes strong duality relations between the primal problem (P) and the dual problem (D).

Theorem 4.3

(Strong Duality) Let ${\bar{x}}\in \mathcal {S}^w(P)$ be such that the (CQ) defined in (3.16) is satisfied at this point. Then there exists $(y^*,z^*)\in K^+\times S^+$ such that $({\bar{x}},y^*,z^*)\in C_D$.

• (i) If in addition $(f\circ F,g\circ G)$ is $(K\times S)$-generalized convex at any ${\tilde{x}}\in X,$ then $({\bar{x}},y^*,z^*)\in \mathcal {S}^w(D).$

• (ii) If in addition $(f\circ F,g\circ G)$ is K-strictly $(K\times S)$-generalized convex at any ${\tilde{x}}\in X,$ then $({\bar{x}},y^*,z^*)\in \mathcal {S}(D).$

Proof

According to Theorem 3.2, there exist $y^*\in K^+$ and $z^*\in S^+$ with $||y^*||+||z^*||=1$ such that

$$\begin{aligned} 0\in&\bigcup _{w^*\in \partial \langle y^*, f\rangle (F({\bar{x}}))} \partial \langle w^*, F\rangle ({\bar{x}})+ \bigcup _{v^*\in \partial \langle z^*,g\rangle (G({\bar{x}}))}\partial \langle v^*,G\rangle ({\bar{x}}), \nonumber \\&\langle z^*,g(G({\bar{x}}))\rangle =0. \end{aligned}$$

(4.9)

Since the (CQ) is satisfied at ${\bar{x}}$, it follows that $y^*\ne 0$. So, we conclude that $({\bar{x}},y^*, z^*)\in C_D$.

• (i) Let $(f\circ F,g\circ G)$ be $(K\times S)$-generalized convex at any ${\tilde{x}}\in X.$ For each $({\tilde{x}},{\tilde{y}}^*,{\tilde{z}}^*)\in C_D,$ invoking (i) of Theorem 4.1, we obtain

  $$\begin{aligned} L({\bar{x}}, y^*,z^*)=(f\circ F)({\bar{x}})\nprec L({\tilde{x}}, {\tilde{y}}^*, {\tilde{z}}^*). \end{aligned}$$

  Hence, $({\bar{x}},y^*,z^*)\in \mathcal {S}^w(D).$

• (ii) Let $(f\circ F,g\circ G)$ be K-strictly $(K\times S)$-generalized convex at any ${\tilde{x}}\in X.$ Similarly, invoking (ii) of Theorem 4.1, we assert that

  $$\begin{aligned} L({\bar{x}},y^*,z^*)\npreceq L({\tilde{x}},{\tilde{y}}^*,{\tilde{z}}^*) \end{aligned}$$

  for any $({\tilde{x}},{\tilde{y}}^*,{\tilde{z}}^*)\in C_D.$ It means that $({\bar{x}},y^*,z^*)\in \mathcal {S}(D).$ $\square$

It is worth noting here that the (CQ) imposed in Theorem 4.3 plays an important role. Namely, if ${\bar{x}}$ is a weakly efficient solution of the primal problem at which the (CQ) is not satisfied, then we may not find out a pair $({\bar{y}}^*,{\bar{z}}^*)\in K^+\times S^+$ such that $({\bar{x}},{\bar{y}}^*,{\bar{z}}^*)\in C_D$, where $C_D$ is the feasible set of the dual problem. In this case, of course, we do not have strong duality relations. Let us illustrate this with the following simple example.

Example 4.4

Let $F:\mathbb {R}:\rightarrow \mathbb {R}^2,\; f: \mathbb {R}^2\rightarrow \mathbb {R}^2,\; G:\mathbb {R}\rightarrow \mathbb {R},$ and $g: \mathbb {R}\rightarrow \mathbb {R}$

$$\begin{aligned}&F(x):=(x,x),\; x\in \mathbb {R},\quad f(w):=(w_1+1, w_2+1),\; w:=(w_1,w_2)\in \mathbb {R}^2,\\&G(x):=x,\; x\in \mathbb {R},\quad g(v):=v^2,\; v\in \mathbb {R}. \end{aligned}$$

We consider the problem (P) with $K:=\mathbb {R}^2_+\subset \mathbb {R}^2$ and $S:=[0,+\infty )\subset \mathbb {R}$. Then $C=\{0\}$, and thus ${\bar{x}}:=0\in \mathcal {S}^w(P)(=\mathcal {S}(P)).$ It is easy to verify that the (CQ) defined in (3.16) is not satisfied at ${\bar{x}}.$ Now, consider the dual problem (D). There does not exist $({\bar{y}}^*,{\bar{z}}^*)\in K^+\times S^+$ such that $({\bar{x}},{\bar{y}}^*,{\bar{z}}^*)\in C_D.$ This means that in this case the conclusion of Theorem 4.3 is no longer valid.

Now, we present converse-like duality relations between the primal problem (P) and the dual problem (D).

Theorem 4.5

(Converse Duality) Let $({\bar{x}}, y^*, z^*)\in C_D$ be such that ${\bar{x}}\in C.$

• (i) If $(f\circ F,g\circ G)$ is $(K\times S)$-generalized convex at ${\bar{x}},$ then ${\bar{x}}\in \mathcal {S}^w(P).$

• (ii) If $(f\circ F,g\circ G)$ is K-strictly $(K\times S)$-generalized convex at ${\bar{x}},$ then ${\bar{x}}\in \mathcal {S}(P).$

Proof

By $({\bar{x}},y^*,z^*)\in C_D$, there exist $y^*\in K^+{\setminus }\{0\}$ and $z^*\in S^+$ such that

$$\begin{aligned} 0\in&\bigcup _{w^*\in \partial \langle y^*, f\rangle (F({\bar{x}}))} \partial \langle w^*, F\rangle ({\bar{x}})+ \bigcup _{v^*\in \partial \langle z^*,g\rangle (G({\bar{x}}))}\partial \langle v^*,G\rangle ({\bar{x}}), \\&\langle z^*,g(G({\bar{x}}))\rangle \ge 0. \end{aligned}$$

In addition, by ${\bar{x}}\in C$, i.e., $(g\circ G)({\bar{x}})\in -S$, it follows that $\langle z^*,g(G({\bar{x}}))\rangle \le 0.$ Hence, $\langle z^*,g(G({\bar{x}}))\rangle = 0$, and we conclude by Definition 3.6 that ${\bar{x}}$ is a (KKT) point of problem (P). To finish the proof, it remains to apply Theorem 3.11. $\square$

It should be noted, similarly to Corollary 3.12, that we can employ Theorem 4.5 to derive (classical) converse duality between a convex multiobjective optimization problem and its corresponding dual problem.

The following example shows that the conclusion of Theorem 4.5 may go awry if the generalized convexity (with respect to a cone) of $(f\circ F,g\circ G)$ is omitted.

Example 4.6

Let $F:\mathbb {R}:\rightarrow \mathbb {R}^2,\; f: \mathbb {R}^2\rightarrow \mathbb {R}^2,\; G:\mathbb {R}\rightarrow \mathbb {R},$ and $g: \mathbb {R}\rightarrow \mathbb {R}$

$$\begin{aligned}&F(x):=(x^5,x^7),\; x\in \mathbb {R},\quad f(w):=(w_1+1, w_2+1),\; w:=(w_1,w_2)\in \mathbb {R}^2,\\&G(x):=x+1,\; x\in \mathbb {R},\quad g(v):=-|v-1|,\; v\in \mathbb {R}. \end{aligned}$$

Consider the problem (P) with $K:=\mathbb {R}^2_+\subset \mathbb {R}^2$ and $S:=[0,+\infty )\subset \mathbb {R}$. Clearly, $C=\mathbb {R}$ and let us select ${\bar{x}}:=0\in C$. Now, consider the dual problem (D). By choosing $y^*:=(1,1)\in K^+$ and $z^*:=0\in S^+,$ we see that $({\bar{x}},y^*,z^*)\in C_D.$

Taking ${\hat{x}}:=-1\in C,$ it follows that

$$\begin{aligned} (f\circ F)({\hat{x}})-(f\circ F)({\bar{x}})=(-1,-1) \in -\mathrm{int}\mathbb {R}^2_+, \end{aligned}$$

showing that ${\bar{x}}\notin \mathcal {S}^w(P).$ This means that the conclusion of Theorem 4.5 fails to hold for this setting. The reason is that $(f\circ F,g\circ G)$ is not $(K\times S)$-generalized convex at ${\bar{x}}.$

5 Applications

In this section, we apply some of the obtained results to derive necessary optimality conditions for a multiobjective approximation problem involving equality and inequality constraints and for a composite multiobjective problem with linear operators.

It is known that the class of (vector) multiobjective approximation problems plays a crucial role in optimization theory and in fact many practical problems can be formulated as (vector) multiobjective approximation problems (see e.g., Gopfert et al. 2003; Jahn 2004). For example, location problems, surrogate problems, inverse Stefan type problems and (vector) optimal control problems can be treated as particular models of (multiobjective) approximation problems. We refer the reader to Gopfert (2003) for a comprehensive study involving theory, numerical analysis and applications of multiobjective approximation problems.

Let X and $Y_k,\; k=1,\ldots , m,$ be finite-dimensional spaces, let $l:X\rightarrow \mathbb {R}^m$ be a vector function, and let $A_k:X\rightarrow Y_k,\; k=1,\ldots , m,$ be linear operators. Given $a^k\in Y_k, \;\alpha _k\ge 0,\;\beta _k\ge 1, \; k=1,\ldots , m,$ we consider the following multiobjective approximation problem:

$$\begin{aligned} \mathrm{min}_{\mathbb {R}^m_+}\;\Big \{l(x)+\big (\alpha _1 ||A_1x-a^1||^{\beta _1},\ldots , \alpha _m ||A_mx-a^m||^{\beta _m}\big )\,\big |\,x\in C\Big \}. \end{aligned}$$

(AP)

Here, the feasible set C is defined by

$$\begin{aligned} C:=\Big \{x\in X\mid \;&g_i(x)\le 0,\; i=1,\ldots ,p,\\&h_j(x)=0,\; j=1,\ldots ,q\Big \}, \end{aligned}$$

where $g_i,\; i=1,\ldots , p,$ and $h_j,\; j=1,\ldots , q,$ are real-valued functions on X. The component functions of l will be denoted by $l_k, \;k=1,\ldots , m.$ As before, we always assume that $l_k,\; k=1,\ldots , m,\; g_i, \; i=1,\ldots , p,$ and $h_j,\; j=1,\ldots q,$ are locally Lipschitz at the point under discussion.

We are now in a position to derive a Fritz-John necessary condition for (weakly) efficient solutions of problem (AP), where the notions of (weakly) efficient solutions are understood as in Definition 1.1.

Theorem 5.1

Let ${\bar{x}}$ be a weakly efficient solution of problem (AP). Then there exist $\lambda _k\ge 0,\,k=1,\ldots , m,\, \mu _i\ge 0,\, i=1,\ldots , p,\, \gamma _j\ge 0, \, j=1,\ldots ,q$, not all zero, and $y^*_k\in Y_k, \; k=1,\ldots ,m$ with

$$\begin{aligned} \langle y^*_k, A_k{\bar{x}}-a^k\rangle =||A_k{\bar{x}}-a^k||^{\beta _k}, \;{\left\{ \begin{array}{ll} ||y^*_k||\le 1 &{}\quad \text{ if }\ \beta _k=1,\;A_k{\bar{x}}=a^k,\\ ||y^*_k||=||A_k{\bar{x}}-a^k||^{\beta _k-1} &{}\quad \text{ otherwise },\end{array}\right. } \end{aligned}$$

such that

$$\begin{aligned} 0\in&\sum _{k=1}^m\lambda _k\partial l_k({\bar{x}})+\sum _{k=1}^m\lambda _k\alpha _k\beta _k A^\top _k y^*_k+\sum _{i=1}^p\mu _i\partial g_i({\bar{x}})+\sum _{j=1}^q\gamma _j\big (\partial h_j({\bar{x}})\cup \partial (-h_j)({\bar{x}})\big ), \end{aligned}$$

(5.1)

$$\begin{aligned}&\mu _i g_i({\bar{x}})=0,\quad i=1,\ldots ,p. \end{aligned}$$

(5.2)

Proof

Put ${\tilde{l}}_k(x):= \alpha _k ||A_kx-a^k||^{\beta _k},\;x\in X$ for $k=1,\ldots , m,$ and define the vector functions $F:X\rightarrow \mathbb {R}^m\times \mathbb {R}^m, f:\mathbb {R}^m\times \mathbb {R}^m\rightarrow \mathbb {R}^m, G:X\rightarrow X$ and $g:X\rightarrow \mathbb {R}^p\times \mathbb {R}^q$ as follows:

$$\begin{aligned} F(x):=&\big (l_1(x),\ldots ,l_m(x),{\tilde{l}}_1(x),\ldots , {\tilde{l}}_m(x)\big ),\; x\in X,\\ f(w):=&(w_1+{\tilde{w}}_1,\ldots , w_m+{\tilde{w}}_m), \; w:=(w_1,\ldots , w_m,{\tilde{w}}_1,\ldots , {\tilde{w}}_m)\in \mathbb {R}^m\times \mathbb {R}^m,\\ G(x):=&x,\; x\in X,\\ g(v):=&\big (g_1(v),\ldots ,g_p(v), h_1(v),\ldots ,h_q(v)\big ), \; v\in X. \end{aligned}$$

Then problem (AP) can be viewed as a particular case of problem (P) with $K:=\mathbb {R}^m_+$ and

$$\begin{aligned} S:=\big \{(\zeta _1,\ldots ,\zeta _p,\eta _1,\ldots ,\eta _q)\in \mathbb {R}^{p}\times \mathbb {R}^{q} \mid \zeta _i\ge 0, \; i=1,\ldots ,p, \;\eta _j=0, \;j=1,\ldots , q\big \}. \end{aligned}$$

In this case, it holds that $K^+=\mathbb {R}^m_+$ and

$$\begin{aligned} S^+&=\{(\mu _1,\ldots ,\mu _p,\sigma _1,\ldots ,\sigma _q)\in \mathbb {R}^{p}\nonumber \\&\quad \times \mathbb {R}^{q}\mid \mu _i\ge 0, \; i=1,\ldots ,p, \;\sigma _j\in \mathbb {R}, \;j=1,\ldots , q\}. \end{aligned}$$

(5.3)

Applying Theorem 3.2 to our problem, we find $y^*:=(\lambda _1,\ldots ,\lambda _m)\in K^+$ and $z^*:=(\mu _1,\ldots ,\mu _p,\sigma _1,\ldots ,\sigma _q)\in S^+$ with $||(\lambda _1,\ldots ,\lambda _m)||+||(\mu _1,\ldots ,\mu _p,\sigma _1,\ldots ,\sigma _q)||=1$, such that

$$\begin{aligned} 0\in&\bigcup _{w^*\in \partial \langle y^*, f\rangle (F({\bar{x}}))} \partial \langle w^*, F\rangle ({\bar{x}})+ \bigcup _{v^*\in \partial \langle z^*,g\rangle (G({\bar{x}}))}\partial \langle v^*,G\rangle ({\bar{x}}), \end{aligned}$$

(5.4)

$$\begin{aligned}&\langle z^*,g(G({\bar{x}}))\rangle =0. \end{aligned}$$

(5.5)

By definition, we have for each $w:=(w_1,\ldots , w_m,{\tilde{w}}_1,\ldots , {\tilde{w}}_m)\in \mathbb {R}^m\times \mathbb {R}^m$,

$$\begin{aligned} \langle y^*,f\rangle (w)=\sum _{k=1}^m\lambda _k(w_k+{\tilde{w}}_k), \end{aligned}$$

and thus,

$$\begin{aligned} \partial \langle y^*,f\rangle (w)=\{(\lambda _1,\ldots ,\lambda _m,\lambda _1,\ldots ,\lambda _m)\}. \end{aligned}$$

This infers that

$$\begin{aligned} \partial \langle y^*,f\rangle (F({\bar{x}}))=\{w^*:=(\lambda _1,\ldots ,\lambda _m,\lambda _1,\ldots ,\lambda _m)\}. \end{aligned}$$

By definition, it holds that

$$\begin{aligned} \langle w^*,F\rangle (x)=\sum _{k=1}^m\lambda _k l_k(x)+\sum _{k=1}^m\lambda _k {\tilde{l}}_k(x),\; x\in X. \end{aligned}$$

Hence, taking the sum rule (2.6) into account, we have

$$\begin{aligned} \partial \langle w^*,F\rangle ({\bar{x}})=\partial \left( \sum _{k=1}^m\lambda _k l_k+\sum _{k=1}^m\lambda _k {\tilde{l}}_k\right) ({\bar{x}})\subset \sum _{k=1}^m\lambda _k\partial l_k({\bar{x}})+ \sum _{k=1}^m\lambda _k\partial {\tilde{l}}_k({\bar{x}}). \end{aligned}$$

In addition, by letting for each $k=1,\ldots ,m,$

$$\begin{aligned} u_k(x):=A_kx-a^k, \; x\in X,\quad v_k(y):=\alpha _k||y||^{\beta _k},\; y\in Y_k, \end{aligned}$$

we deduce from Lemmas 2.2 and 2.4 that

$$\begin{aligned} \partial {\tilde{l}}_k({\bar{x}})&=\partial (v_k\circ u_k)({\bar{x}})\subset \{\alpha _k\beta _k A^\top _k y^*_k\mid y^*_k\in Y_k\} \end{aligned}$$

with

$$\begin{aligned} {\left\{ \begin{array}{ll} ||y^*_k||\le 1 &{}\quad \text{ if }\,\quad \beta _k=1, A_k{\bar{x}}=a^k, \\ \langle y^*_k, A_k{\bar{x}}-a^k\rangle =||A_k{\bar{x}}-a^k||^{\beta _k}, ||y^*_k||=||A_k{\bar{x}}-a^k||^{\beta _k-1} &{}\quad \text{ otherwise }.\end{array}\right. }\end{aligned}$$

(5.6)

So, we obtain the estimate

$$\begin{aligned} \bigcup _{w^*\in \partial \langle y^*, f\rangle (F({\bar{x}}))} \partial \langle w^*, F\rangle ({\bar{x}})\subset \sum _{k=1}^m\lambda _k\partial l_k({\bar{x}})+ \left\{ \sum _{k=1}^m\lambda _k\alpha _k\beta _k A^\top _k y^*_k\,\Big |\, y^*_k\in Y_k \right\} , \end{aligned}$$

(5.7)

where $y^*_k$ satisfies (5.6) for $k=1,\ldots ,m.$ Similarly, we get

$$\begin{aligned} \bigcup _{v^*\in \partial \langle z^*,g\rangle (G({\bar{x}}))}\partial \langle v^*,G\rangle ({\bar{x}})\subset \sum _{i=1}^p\mu _i\partial g_i({\bar{x}})+\sum _{j=1}^q|\sigma _j|\big (\partial h_j({\bar{x}})\cup \partial (-h_j)({\bar{x}})\big ) \end{aligned}$$

(5.8)

by noting further that $\partial (\sigma _jh_j)({\bar{x}})\subset |\sigma _j|(\partial h_j({\bar{x}})\cup \partial (-h_j)({\bar{x}}))$ for $j=1,\ldots ,q.$ Now, letting $\gamma _j:=|\sigma _j|$ for $j=1,\ldots ,q,$ and then combining (5.4) with (5.7) and (5.8), we arrive at (5.1). In this setting, (5.5) amounts to the following one

$$\begin{aligned} \sum _{i=1}^p\mu _ig_i({\bar{x}})+\sum _{j=1}^q\sigma _jh_j({\bar{x}})=0. \end{aligned}$$

(5.9)

Moreover, we have $g_i({\bar{x}})\le 0$ for $i=1,\ldots ,p$, and $h_j({\bar{x}})=0$ for $j=1,\ldots ,q.$ Hence, (5.2) follows from (5.9). This completes the proof of the theorem. $\square$

The following simple example illustrates the Fritz-John necessary condition obtained in Theorem 5.1.

Example 5.2

Consider a multiobjective approximation problem of the form:

$$\begin{aligned} \mathrm{min}_{\mathbb {R}^2_+}\;\Big \{&(x_1^3+2x_2,x_1-x_2^2)+\big (2||A_1x+(1,0)||, ||A_2x-(0,2)||\big )\,\big |\,x:=(x_1,x_2)\in \mathbb {R}^2,\\&x_2-|x_1-1|\le 0, -|x_2+1|\le x_1, x_1-2x_2=3, x_1+2x_2=-1\Big \}, \end{aligned}$$

(EP)

where $A_1=\begin{pmatrix} 1 &{} 2 \\ 0&{} 0 \end{pmatrix}$ and $A_2=\begin{pmatrix} 0 &{} 0 \\ 3 &{} 1\end{pmatrix}$. The problem (EP) can be viewed in the form of (AP), where $l:\mathbb {R}^2\rightarrow \mathbb {R}^2$ is given by the component functions $l_1(x):=x_1^3+2x_2$ and $l_2(x):=x_1-x_2^2$ for $x:=(x_1,x_2)\in \mathbb {R}^2$, $a^1:=(-1,0)\in \mathbb {R}^2, a^2:=(0,2)\in \mathbb {R}^2, \alpha _1:=2, \alpha _2:=1, \beta _1:=\beta _2:=1$ and

$$\begin{aligned} C:=\{x\in \mathbb {R}^2\mid&g_1(x):=x_2-|x_1-1|\le 0, g_2(x):=-|x_2+1|-x_1\le 0,\\&h_1(x):=x_1-2x_2-3=0, h_2(x):=x_1+2x_2+1=0\}. \end{aligned}$$

It is easy to see that ${\bar{x}}:=(1,-1)$ is a (weakly) efficient solution of problem (EP). Direct calculation shows that

$$\begin{aligned}&\partial l_1({\bar{x}})=\{(3,2)\}, \; \partial l_2({\bar{x}})=\{(1,2)\}, \; \partial g_1({\bar{x}})=\{(-1,1),(1,1)\},\\&\partial g_2({\bar{x}})=\{(-1,-1),(-1,1)\},\; \partial h_1({\bar{x}})=\{(1,-2)\},\; \partial (-h_1)({\bar{x}})=\{(-1,2)\},\\&\partial h_2({\bar{x}})=\{(1,2)\},\; \partial (-h_2)({\bar{x}})=\{(-1,-2)\}. \end{aligned}$$

Now, we take $\lambda _1:=1, \lambda _2:=0, \mu _1:=\mu _2:=0, \gamma _1:=1, \gamma _2:=0, y_1^*:=(-1,0)$ and $y^*_2:=(1,0)$, we have $||y^*_k||\le 1, \;A_k{\bar{x}}=a^k,\; k=1,2$ and

$$\begin{aligned} 0\in&\sum _{k=1}^2\lambda _k\partial l_k({\bar{x}})+\sum _{k=1}^2\lambda _k\alpha _k\beta _k A^\top _k y^*_k+\sum _{i=1}^2\mu _i\partial g_i({\bar{x}})+\sum _{j=1}^2\gamma _j\big (\partial h_j({\bar{x}})\cup \partial (-h_j)({\bar{x}})\big ),\\&\mu _i g_i({\bar{x}})=0,\; i=1,2, \end{aligned}$$

which shows that the conclusion of Theorem 5.1 holds.

Next, we consider a composite multiobjective problem with linear operators:

$$\begin{aligned} \mathrm{min}_{\mathbb {R}^p_+}\;\Big \{\big (f_1(A_1x),\ldots , f_p(A_px)\big )\,\big |\,x\in C\Big \}, \end{aligned}$$

(ML)

where $f_k:Y_k\rightarrow \mathbb {R}, k=1,\ldots ,p$ are local Lipschitz functions, $A_k:X\rightarrow Y_k, k=1,\ldots , p$ are linear operators and X and $Y_k,\; k=1,\ldots , p,$ are finite-dimensional spaces. Here, the feasible set C is given by

$$\begin{aligned} C:=\Big \{x\in X\mid g(x)\in -S\Big \}, \end{aligned}$$

where $g:X\rightarrow Z$ is a local Lipschitz vector function between finite-dimensional spaces and $S\subset Z$ is a nonempty closed convex cone. The model of (MLP) with $f_k, k=1,\ldots ,p$ being convex functions and g being S-convex vector function was examined in Bot et al. (2008) by way of the Fenchel–Lagrange-type scalar approach.

Let us now derive a Fritz-John necessary condition for (weakly) efficient solutions of problem (MLP), which can be viewed as a nonconvex generalized version of Bot et al. (2008, Theorem 4.1a) if we impose a certain qualification condition.

Theorem 5.3

Let ${\bar{x}}$ be a weakly efficient solution of problem (MLP). Then there exist $\lambda _k\ge 0,\,k=1,\ldots , p$ and $z^* \in S^+$ with $||(\lambda _1,\ldots ,\lambda _p)||+||z^*||= 1$ such that

$$\begin{aligned} 0\in&\sum _{k=1}^p\lambda _k A^\top _k\partial f_k({\bar{y}}_k)+\partial \langle z^*,g\rangle ({\bar{x}}), \end{aligned}$$

(5.10)

$$\begin{aligned}&\langle z^*,g({\bar{x}})\rangle =0, \end{aligned}$$

(5.11)

where ${\bar{y}}_k=A_k{\bar{x}}$ for $k=1,\ldots ,p.$

Proof

We see that the problem (MLP) is a particular case of the composite vector optimization problem (P), where $V:=X$, G is an identical map (i.e., $G(x):=x, x\in X$), $F:X\rightarrow W:=Y_1\times \cdots \times Y_p$ and $f:W\rightarrow Y:=\mathbb {R}^p$ are given respectively by

$$\begin{aligned}&F(x):=\big (A_1x,\ldots ,A_px\big ),\; x\in X,\\&f(w):=\big (f_1(w_1),\ldots ,f_p(w_p)\big ), \; w:=(w_1,\ldots ,w_p)\in Y_1\times \cdots \times Y_p \end{aligned}$$

and $K:=\mathbb {R}^p_+$. So, invoking Theorem 3.2, we find $y^*:=(\lambda _1,\ldots ,\lambda _p)\in K^+=\mathbb {R}^p_+$ and $z^*\in S^+$ with $||(\lambda _1,\ldots ,\lambda _p)||+||z^*||=1$ such that

$$\begin{aligned} 0\in&\bigcup _{w^*\in \partial \langle y^*, f\rangle (F({\bar{x}}))} \partial \langle w^*, F\rangle ({\bar{x}})+ \bigcup _{v^*\in \partial \langle z^*,g\rangle (G({\bar{x}}))}\partial \langle v^*,G\rangle ({\bar{x}}), \end{aligned}$$

(5.12)

$$\begin{aligned}&\langle z^*,g(G({\bar{x}}))\rangle =0. \end{aligned}$$

(5.13)

Consider $\Phi _k: Y_1\times \cdots \times Y_p\rightarrow Y_k, k=1,\ldots ,p$ given by $\Phi _k(w):=w_k$ for $w:=(w_1,\ldots , w_p)\in W:=Y_1\times \cdots \times Y_p$. We see that $\langle y^*,f\rangle (w)=\sum _{k=1}^p\lambda _k(f_k\circ \Phi _k)(w)$ for all $w\in W$ and thus, by Lemmas 2.2 and 2.3,

$$\begin{aligned} \partial \langle y^*,f\rangle (w)&\subset \sum _{k=1}^p\lambda _k(f_k\circ \Phi _k)(w)\subset \lambda _1\partial f_1(w_1)\\&\quad \times \cdots \times \lambda _p\partial f_p(w_p), \; w:=(w_1,\ldots ,w_p)\in W. \end{aligned}$$

This shows that

$$\begin{aligned} \partial \langle y^*,f\rangle (F(x))\subset \lambda _1\partial f_1(y_1)\times \cdots \times \lambda _p\partial f_p(y_p),\; x\in X, \end{aligned}$$

(5.14)

where $y_k=A_kx$ for $k=1,\ldots ,p$.

Take any $w^*\in \partial \langle y^*,f\rangle (F({\bar{x}})).$ Then, by (5.14), there exist $w^*_k\in \partial f_k({\bar{y}}_k), k=1,\ldots ,p$, where ${\bar{y}}_k=A_k{\bar{x}}$ such that $w^*=(\lambda _1w^{*\top }_1,\ldots ,\lambda _p w^{*\top }_p)$. Then, it holds that

$$\begin{aligned} \langle w^*,F\rangle (x)=\sum _{k=1}^p\lambda _k w^{*\top }_kA_kx,\; x\in X \end{aligned}$$

and hence, $\partial \langle w^*,F\rangle ({\bar{x}})=\sum _{k=1}^p\lambda _k A^\top _k w^*_k.$ So, we obtain the estimate

$$\begin{aligned} \bigcup _{w^*\in \partial \langle y^*, f\rangle (F({\bar{x}}))} \partial \langle w^*, F\rangle ({\bar{x}})\subset \sum _{k=1}^p\lambda _kA^\top _k\partial f_k({\bar{y}}_k). \end{aligned}$$

(5.15)

Similarly, we get

$$\begin{aligned} \bigcup _{v^*\in \partial \langle z^*,g\rangle (G({\bar{x}}))}\partial \langle v^*,G\rangle ({\bar{x}})= \partial \langle z^*,g\rangle ({\bar{x}}) \end{aligned}$$

(5.16)

as G is an identical map. Now, combining (5.12) with (5.15) and (5.16), we arrive at (5.10). In this setting, (5.11) is nothing else but (5.13) and so, the proof of is complete. $\square$

6 Conclusions

In this paper, we have employed some advanced tools of variational analysis and generalized differentiation to provide necessary optimality conditions for the nonsmooth composite vector optimization problem (P). These necessary optimality conditions are expressed in terms of the limiting/Mordukhovich subdifferential and the Mordukhovich coderivative of the related functions. Some sufficient optimality conditions for (P) have been also provided by means of (strictly) generalized convex composite vector functions with respect to a cone. Moreover, we have proposed a dual problem to (P) and examined weak, strong and converse duality relations.

It would be of great interest to see how the proposed approach can be applied to reformulate and solve practical problems such as location problems, surrogate problems and control problems.