1 Introduction

Let \(f,g: \mathbb {R}^{n}\rightarrow \mathbb {R}^{n}\) be two continuous functions, and let K be a nonempty, closed and convex subset of \(\mathbb {R}^n\). The so-called generalized variational inequality, denoted by GVI(g, f, K), is to find an \(\mathbf {x} \in \mathbb {R}^n\) such that

$$\begin{aligned} g(\mathbf {x})\in K,\quad \langle f(\mathbf {x}), \mathbf {y}-g(\mathbf {x}) \rangle \ge 0\quad \mathrm{for~all}\; \mathbf {y}\in K, \end{aligned}$$

which was introduced by Noor [20] in 1988. When \(g(\mathbf {x})=\mathbf {x}\), GVI(g, f, K) reduces to the standard finite-dimensional variational inequality [5, 9]. Furthermore, when \(K=\mathbb {R}^n_+\), GVI(g, f, K) further reduces to the standard complementarity problem [7].

The existence and uniqueness of solutions to the variational inequality is an important issue in the studies of the theory, algorithms and applications for the variational inequality. It is known that a vector \(\mathbf {x}\in \mathbb {R}^n\) solves GVI(g, f, K) if and only if it is a solution of the generalized normal equation:

$$\begin{aligned} g(\mathbf {x})=P_K[g(\mathbf {x})-f(\mathbf {x})], \end{aligned}$$

where \(P_K(\mathbf {u})\) denotes the projection of the vector \(\mathbf {u}\in \mathbb {R}^n\) onto the set K. A well-known result on the existence and uniqueness for GVI(g, f, K), achieved by Pang and Yao in [21, Proposition 3.9], is described as follows.

Theorem 1

Let K be a nonempty, closed and convex subset of \(\mathbb {R}^n\), and let f and g be two continuous functions from \(\mathbb {R}^n\) into itself with g being injective. Suppose

  1. (a)

    there exists a vector \(\mathbf {u}\in g^{-1}(K)\) and positive scalars \(\alpha \) and L such that

    $$\begin{aligned} \Vert g(\mathbf {x})-g(\mathbf {u})\Vert \le L\Vert \mathbf {x}-\mathbf {u}\Vert \end{aligned}$$

    holds for all \(\mathbf {x}\in g^{-1}(K)\) with \(\Vert \mathbf {x}\Vert \ge \alpha \);

  2. (b)

    f is strongly monotone with respect to g on K, i.e., there is a scalar \(c > 0\) such that

    $$\begin{aligned} {[}f(\mathbf {x})-f(\mathbf {y}){]}^\top [g(\mathbf {x})-g(\mathbf {y})]\ge c\Vert \mathbf {x}-\mathbf {y}\Vert ^2 \end{aligned}$$
    (1)

    holds for all \(g(\mathbf {x}),g(\mathbf {y})\in K\) with \(\mathbf {x}\ne \mathbf {y}\).

Then, there exists a unique vector \(\bar{\mathbf {x}}\in \mathbb {R}^n\) satisfying \(g(\mathbf {x})=P_K[g(\mathbf {x})-f(\mathbf {x})]\).

When f and g are both polynomials, the corresponding GVI(g, f, K) is called the generalized polynomial variational inequality in this paper, and we denote it by the GPVI. In recent years, several subclasses of the GPVI have been studied extensively. These subclasses include tensor complementarity problems [1, 3, 11, 14, 19, 23,24,25] (also see survey papers [12, 13, 22]), polynomial complementarity problems [6, 15], generalized polynomial complementarity problems [16], tensor variational inequalities [26], and polynomial variational inequalities [10]. Many interesting results for these several subclasses were achieved by using the special properties of polynomials. It is natural to ask how to study the GPVI by using special properties of polynomials? In particular, we find that for general polynomials f and g, it is impossible that the function f is strongly monotone with respect to g on the set K (see Proposition 1 in the next section). This implies that the result of Theorem 1 cannot be generally applied to the GPVI. A natural question is that how to investigate the existence and uniqueness of solutions to the GPVI?

In this paper, our main purpose is to establish an existence and uniqueness theorem for the GPVI by making use of properties of the involved polynomials. In the next section, we introduce some symbols and concepts, and discuss the existence of solutions to the GPVI by using the concept of the exceptional family of elements and degree theory. In Sect. 3, we develop an existence and uniqueness theorem for the GPVI, and derive several corollaries for subclasses of the GPVI. Moreover, we give an example in support of our theoretical findings. Conclusions are given in the last section.

2 Preliminaries

We first give an introduction of tensors, which plays an important role in our analysis. A tensor is a natural extension of a matrix. For any given positive integers m and n with \(m,n\ge 2\), we call \(\mathcal {A}=(a_{i_1i_2\cdots i_m})\), where \(a_{i_1i_2\cdots i_m}\in \mathbb {R}\) for \(i_j\in \{1,2,\ldots ,n\}\) and \(j\in \{1,2,\ldots ,m\}\), an m-th order n-dimensional real square tensor; and denote the space of m-th order n-dimensional real square tensors by \(\mathbb {R}^{[m,n]}\).

For any \(\mathcal {A}=(a_{i_1i_2\cdots i_m})\in \mathbb {R}^{[m,n]}\) and \(\mathbf {x}=(x_1,x_2,\ldots ,x_n)^\top \in \mathbb {R}^n\), \(\mathcal {A} \mathbf {x}^{m-1}\) is an n-dimensional vector whose ith component is given by

$$\begin{aligned} (\mathcal {A}\mathbf {x}^{m-1})_i= \sum _{i_2,i_3,\ldots , i_m=1}^n a_{ii_2\cdots i_m}x_{i_2}x_{i_3}\cdots x_{i_m}\quad \text{ for } \text{ all }\; i\in \{1,2,\ldots ,n\}, \end{aligned}$$

and \(\mathcal {A} \mathbf {x}^{m}\) is a homogeneous polynomial of degree m, defined by

$$\begin{aligned} \mathcal {A}\mathbf {x}^{m}= \sum _{i_1,i_2,\ldots , i_m=1}^n a_{i_1i_2\cdots i_m}x_{i_1}x_{i_2}\cdots x_{i_m}\quad \text{ for } \text{ all }\; i\in \{1,2,\ldots ,n\}. \end{aligned}$$

\(\mathcal {A}=(a_{i_1i_2\cdots i_m})\in \mathbb {R}^{[m,n]}\) is said to be a positive definite tensor if and only if \(\mathcal {A}\mathbf {x}^m > 0\) for all \(\mathbf {x}\in \mathbb {R}^n{\setminus }\{\mathbf {0}\}\).

Let

$$\begin{aligned} f(\mathbf {x}):=\sum _{k=1}^{m-1}\mathcal {A}^{(k)}\mathbf {x}^{m-k}+\mathbf {a}\quad \text{ and }\quad g(\mathbf {x}):=\sum _{p=1}^{l-1}\mathcal {B}^{(p)}\mathbf {x}^{l-p}+\mathbf {b}, \end{aligned}$$
(2)

where

$$\begin{aligned} \left\{ \begin{array}{ll} \Lambda :=\Big ( \mathcal {A}^{(1)},\ldots ,\mathcal {A}^{(m-1)}\Big ) \in \mathcal {F}_{m,n}:=\mathbb {R}^{[m,n]}\times \cdots \times \mathbb {R}^{[2,n]}, &{}\quad \mathbf {a}\in \mathbb {R}^n;\\ \Theta :=\Big ( \mathcal {B}^{(1)},\ldots ,\mathcal {B}^{(l-1)}\Big ) \in \mathcal {F}_{l,n}:=\mathbb {R}^{[l,n]}\times \cdots \times ~\mathbb {R}^{[2,n]}, &{}\quad \mathbf {b}\in \mathbb {R}^n, \end{array}\right. \end{aligned}$$
(3)

then the corresponding GVI(g, f, K) is a generalized polynomial variational inequality, denoted by \(GPVI(\Lambda ,\mathbf {a},\Theta ,\mathbf {b},K)\).

We have the following observation.

Proposition 1

Let K be a nonempty, closed and convex subset of \(\mathbb {R}^n\), and let \(f,g:\mathbb {R}^n\rightarrow \mathbb {R}^n\) be two polynomials defined by

$$\begin{aligned} f(\mathbf {x}):=\sum _{k=1}^{m^\prime }\mathcal {A}^{(k)}\mathbf {x}^{m-k}+\mathbf {a}\quad \text{ and }\quad g(\mathbf {x}):=\sum _{p=1}^{l^\prime }\mathcal {B}^{(p)}\mathbf {x}^{l-p}+\mathbf {b} \end{aligned}$$
(4)

for some \(m^\prime \in \{1,2,\ldots ,m-1\}\) and \(l^\prime \in \{1,2,\ldots ,l-1\}\), where

$$\begin{aligned} \left\{ \begin{array}{ll} \Big (\mathcal {A}^{(1)},\ldots ,\mathcal {A}^{(m-m^\prime +1)}\Big ) \in \mathbb {R}^{[m,n]}\times \cdots \times \mathbb {R}^{[m-m^\prime +1,n]};\\ \Big (\mathcal {B}^{(1)},\ldots ,\mathcal {B}^{(l-l^\prime +1)}\Big ) \in \mathbb {R}^{[l,n]}\times \cdots \times ~\mathbb {R}^{[l-l^\prime +1,n]}. \end{array}\right. \end{aligned}$$

If \(g(\mathbf {0})\in K\) and \(m-m^\prime +l-l^\prime >2\), then the function f is not strongly monotone with respect to g on K.

Proof

Suppose that f is strongly monotone with respect to g on K, i.e., there exists a constant \(c >0\) such that (1) holds for any \(g(\mathbf {x}), g(\mathbf {y})\in K\) with \(\mathbf {x}\ne \mathbf {y}\). Take \(\mathbf {y}=\mathbf {0}\), then we can get from (1) that

$$\begin{aligned} \left\langle \sum _{p=1}^{l^\prime }\mathcal {B}^{(p)}\mathbf {x}^{l-p}, \sum _{k=1}^{m^\prime }\mathcal {A}^{(k)}\mathbf {x}^{m-k}\right\rangle =\sum _{q=2}^{m^\prime +l^\prime }\mathcal {C}^{(q-1)}\mathbf {x}^{m+l-q}\ge c\Vert \mathbf {x}\Vert ^2, \end{aligned}$$
(5)

where \(\mathcal {C}^{(q-1)}\in \mathbb {R}^{[m+l-q,n]}\) for all \(q\in \{2,3,\ldots ,m^\prime +l^\prime \}\). By dividing both sides of (5) by \(\Vert \mathbf {x}\Vert ^{m+l-m^\prime -l^\prime }\), we get

$$\begin{aligned} \mathcal {C}^{(m^\prime +l^\prime -1)}\left( \frac{\mathbf {x}}{\Vert \mathbf {x}\Vert }\right) ^{m+l-m^\prime -l^\prime }+ \sum _{q=2}^{m^\prime +l^\prime -1}\frac{\mathcal {C}^{(q-1)}\mathbf {x}^{m+l-q}}{\Vert \mathbf {x}\Vert ^{m+l-m^\prime -l^\prime }} \ge c\frac{\Vert \mathbf {x}\Vert ^2}{\Vert \mathbf {x}\Vert ^{m+l-m^\prime -l^\prime }}. \end{aligned}$$
(6)

Let \(\Vert \mathbf {x}\Vert \rightarrow \mathbf {0}\), it follows by the condition \(m-m^\prime +l-l^\prime >2\) that the left-hand side of the inequality (6) is bounded; but the right-hand side of (6) tends to \(\infty \), which leads to a contradiction. \(\square \)

Proposition 1 demonstrates that if \(g(\mathbf {0})\in K\), the degrees of both f and g defined by (2) are greater or equal to 1, and at least one of f and g has no term of degree 1, then f is not strongly monotone with respect to g on K. This implies that the result of Theorem 1 for GVI(g, f, K) cannot be generally applied to \(GPVI(\Lambda ,\mathbf {a},\Theta ,\mathbf {b},K)\). In order to study the existence and uniqueness of solutions to \(GPVI(\Lambda ,\mathbf {a},\Theta ,\mathbf {b},K)\), we need to use the concept of exceptional family of elements. In fact, the concept of exceptional family of elements has been extensively used to investigate the existence of the standard variational inequalities and complementarity problems. The following definition is similar to the those in [8, 27, 28].

Definition 1

Let f, g be two continuous functions. A set of points \(\{\mathbf {x}^r\}\subseteq \mathbb {R}^n\) is called an exceptional family of elements for the pair (f, g) with respect to \(\hat{\mathbf {x}}\in \mathbb {R}^n\), if \(\Vert \mathbf {x}^r\Vert \rightarrow \infty \) as \(r\rightarrow \infty \); and for each \(\mathbf {x}^r\), there exists a positive scalar \(\alpha ^r\) such that

$$\begin{aligned} \pi _r:=\alpha _r(\mathbf {x}^r-\hat{\mathbf {x}})+g(\mathbf {x}^r)\in K\quad \text{ and }\quad -\alpha _r(\mathbf {x}^r-\hat{\mathbf {x}})-f(\mathbf {x}^r)\in \mathcal{N}_K(\pi _r), \end{aligned}$$
(7)

where \(\mathcal{N}_K(\pi _r)\) denotes the normal cone of the convex set K at the point the point \(\pi _r\).

In order to establish the existence result for \(GPVI(\Lambda ,\mathbf {a},\Theta ,\mathbf {b},K)\) by using the concept of exceptional family of elements, we need to use the degree theory. We now review some basic concepts and results of degree theory [18]. Suppose \(\Omega \) is a bounded open set in a finite-dimensional Hilbert space H, we use \(\bar{\Omega }\) and \(\partial \Omega \) to denote its closure and boundary in H, respectively. Let \(\phi : \bar{\Omega }\rightarrow H\) be continuous, and \(\mathbf {p}\in H\) with \(\mathbf {p} \notin \phi (\partial \Omega )\). Then, the topological degree of \(\phi \) over \(\Omega \) with respect to \(\mathbf {p}\) can be defined, which is denoted by deg\((\phi ,\Omega , \mathbf {p})\). Let \(\mathcal {H}(\mathbf {x},t): H \times [0,1]\rightarrow H\) be continuous. Suppose that for some bounded open set \(\Omega \) in H, \(\mathbf {0}\notin H(\partial \Omega ,t)\) for all \(t\in \) [0,1]. Then, the homotopy invariance property of degree says that deg(\(\mathcal {H}(\cdot ,\)t\(),\Omega ,\mathbf {0})\) is independent of t. In particular, if the set

$$\begin{aligned} \Psi :=\{\mathbf {x}:\mathcal {H}(\mathbf {x},t)=\mathbf {0}\; \text{ for } \text{ some }\; t\in [0,1]\} \end{aligned}$$

is bounded, then for any bounded open set \(\Omega \) in H that contains the set \(\Psi \), it follows that

$$\begin{aligned} \mathrm{deg }(\mathcal {H}(\cdot ,1), \Omega , \mathbf {0})=\mathrm{deg }(\mathcal {H}(\cdot ,0),\Omega ,\mathbf {0}). \end{aligned}$$

The following result is very useful for us to prove the main result in this paper.

Lemma 1

For two continuous mappings \(f,g: \mathbb {R}^n\rightarrow \mathbb {R}^n\) and a nonempty, closed and convex set K in \(\mathbb {R}^n\), there exists either a solution of GVI(g, f, K) or an exceptional family of elements with respect to any given \(\hat{\mathbf {x}}\in \mathbb {R}^n\) for the pair (f, g).

Proof

The proof is similar to the one in [27]. For the sake of completeness, we give it here. Suppose that GVI(g, f, K) has no solution, we need to show there exists an exceptional family of elements with respect to any given \(\hat{\mathbf {x}}\in \mathbb {R}^n\) for the pair (f, g). Let \(\Phi : \mathbb {R}^n\rightarrow \mathbb {R}^n\) be defined by

$$\begin{aligned} \Phi (\mathbf {x}):=g(\mathbf {x})-P_K[g(\mathbf {x})-f(\mathbf {x})]\quad \text{ for } \text{ any }\; \mathbf {x}\in \mathbb {R}^n. \end{aligned}$$

Then, \(\Phi \) is a continuous function; and \(\mathbf {x}\in \mathbb {R}^n\) solves GVI(g, f, K) if and only if it is a solution of \(\Phi (\mathbf {x})=\mathbf {0}\). For any \(\mathbf {x},~\hat{\mathbf {x}}\in \mathbb {R}^n\), we define

$$\begin{aligned} H(\mathbf {x},t):=t(\mathbf {x}-\hat{\mathbf {x}})+(1-t)g(\mathbf {x})-(1-t)P_K[g(\mathbf {x})-f(\mathbf {x})]\quad \text{ for } \text{ any }\; t\in [0,1]. \end{aligned}$$

We claim that the set \(\mathcal {F}:=\{\mathbf {x}\in \mathbb {R}^n: H(\mathbf {x},t)=\mathbf {0}\;\text{ for } \text{ some }\; t\in [0,1]\}\) is unbounded. Suppose by the way of contradiction that \(\mathcal {F}\) is bounded. Then there exists a bounded open set \(\Omega \) in \(\mathcal {F}\) such that \(\mathbf {x}\notin \mathcal {F}\) for all \(\mathbf {x}\in \partial \Omega \). Thus, by the homotopy invariance theorem of degree, it follows that

$$\begin{aligned} 1=\text{ deg }(\mathbf {x}-\hat{\mathbf {x}},\Omega ,\mathbf {0})=\text{ deg }(\Phi (\mathbf {x}),\Omega ,\mathbf {0}), \end{aligned}$$

which implies that \(\Phi (\mathbf {x})=\mathbf {0}\) has a solution in \(\Omega \). This is a contradiction with the assumption at the beginning of the proof. So, the set \(\mathcal {F}\) is unbounded. Thus, there exists an unbounded sequence \(\{\mathbf {x}^r\}\subseteq \mathcal {F}\). Without loss of generality, we may assume that \(\Vert \mathbf {x}^r\Vert > \Vert {\hat{\mathbf {x}}}\Vert \) for all r. By the definition of \(\mathcal {F}\), for each \(\mathbf {x}^r\) there is a scalar \(t^r\in [0,1]\) such that

$$\begin{aligned} \mathbf {0}=H(\mathbf {x}^r,t^r)=t^r(\mathbf {x}^r-\hat{\mathbf {x}})+(1-t^r)g(\mathbf {x}^r)-(1-t^r)P_K[g(\mathbf {x}^r)-f(\mathbf {x}^r)]. \end{aligned}$$
(8)

Since GVI(g, f, K) has no solution, we have \(\Phi (\mathbf {x}^r)\ne \mathbf {0}\). We deduce from (8) that \(t^r\ne 0,1\). Thus, in the rest of the proof, it is sufficient to consider the case of \(t^r\in (0,1)\). In this case, we show that GVI(g, f, K) has an exceptional family with respect to \(\hat{\mathbf {x}}\).

From (8) it follows that

$$\begin{aligned} \frac{t^r}{1-t^r}(\mathbf {x}^r-\hat{\mathbf {x}})+g(\mathbf {x}^r)=P_K[g(\mathbf {x}^r)-f(\mathbf {x}^r)]. \end{aligned}$$

Denote \(\alpha _r:=\frac{t^r}{1-t^r}\) and \(\pi _r:=\alpha _r(\mathbf {x}^r-\hat{\mathbf {x}})+g(\mathbf {x}^r)\). Then, \(\pi _r=P_K[g(\mathbf {x}^r)-f(\mathbf {x}^r)]\), which implies that \(\pi _r\in K\), and that \(\pi _r\) is the unique solution to the following convex program:

$$\begin{aligned} \underset{\mathbf {y} \in K}{\min }\; h(\mathbf {y}):=\frac{1}{2}\Vert \mathbf {y}-[g(\mathbf {x}^r)-f(\mathbf {x}^r)]\Vert ^2. \end{aligned}$$

Obviously, the function h is locally Lipschitz continuous, and hence, by the corollary of Proposition 2.4.3 of [4] we have

$$\begin{aligned} \nabla h(\pi _r)=\pi _r-[g(\mathbf {x}^r)-f(\mathbf {x}^r)]=\alpha _r(\mathbf {x}^r-\hat{\mathbf {x}})+f(\mathbf {x}^r)\in -\mathcal {N}_K(\pi _r). \end{aligned}$$

Therefore, \(\{\mathbf {x}^r\}\) is an exceptional family with respect to \(\hat{\mathbf {x}}\) for GVI(g, f, K).

The proof is complete. \(\square \)

3 Main result

In this section, by using Lemma 1 we establish an existence and uniqueness result for \(GPVI(\Lambda ,\mathbf {a},\Theta ,\mathbf {b},K)\); and then, we give an example in support of our theoretical findings.

Theorem 2

Let f and g be defined by (2) and (3), and let \(K\subseteq \mathbb {R}^n\) be a nonempty, closed and convex set with \(g(\mathbf {0})\in K\). Suppose that

  1. (a)

    f is strictly monotone with respect to g on K, i.e.,

    $$\begin{aligned}{}[f(\mathbf {x})-f(\mathbf {y})]^\top [g(\mathbf {x})-g(\mathbf {y})]>0\quad \text{ for } \text{ all }\; g(\mathbf {x}),g(\mathbf {y})\in K; \end{aligned}$$
  2. (b)

    for any nonzero vector \(\mathbf {x}\in \mathbb {R}^n\), \(\langle \mathcal {A}^{(1)}\mathbf {x}^{m-1},\mathcal {B}^{(1)}\mathbf {x}^{l-1}\rangle \ne 0\);

  3. (c)

    one of \(\mathcal {A}^{(1)}\) and \(\mathcal {B}^{(1)}\), whose order is even and is larger than or equal to another, is positive definite,

then for any given vectors \(\mathbf {a}\in \mathbb {R}^n\) and \(\mathbf {b}\in \) K, \(GPVI(\Lambda ,\mathbf {a},\Theta ,\mathbf {b},K)\) has a unique solution.

Proof

Without loss of generality, we assume that \(m\ge l\). In this case, it follows from the condition (c) that \(\mathcal {A}^{(1)}\) is positive definite.

We first prove the nonemptyness of \(SOL(\Lambda ,\mathbf {a},\Theta ,\mathbf {b},K)\). Suppose, on the contrary, that \(SOL(\Lambda ,\mathbf {a},\Theta ,\mathbf {b},K)=\emptyset \). Then, it follows from Lemma 1 that there exists an exceptional family of elements for the pair (f, g) with respect to \(\mathbf {0}\in \mathbb {R}^n\), that is to say, there exists a sequence \(\{\mathbf {x}^r\}_{r=1}^\infty \subseteq \mathbb {R}^n\) satisfying \(\Vert \mathbf {x}^r\Vert \rightarrow \infty ~as~r\rightarrow \infty \), and scalars \(\alpha _r>0\) such that (7) holds, and hence, from the definition of the normal cone, we have

$$\begin{aligned} \langle \alpha _r\mathbf {x}^r+f(\mathbf {x}^r),\mathbf {y}-\alpha _r\mathbf {x}^r-g(\mathbf {x}^r)\rangle \ge 0\quad \text{ for } \text{ any }\; \mathbf {y}\in K. \end{aligned}$$
(9)

By dividing both sides of (9) by \(\Vert \mathbf {x}^r\Vert ^{m+l-2}\) , we get

$$\begin{aligned} \begin{array}{l} \frac{\alpha _r}{\Vert \mathbf {x}^r\Vert ^{l-2}} \left\langle \frac{\mathbf {x}^r}{\Vert \mathbf {x}^r\Vert }, \frac{\mathbf {y} -\sum \nolimits _{k=1}^{m-1}\mathcal {A}^{(k)}({\mathbf {x}^r})^{m-k} -\sum \nolimits _{p=1}^{l-1}\mathcal {B}^{(p)}({\mathbf {x}^r})^{l-p}-\mathbf {a}-\mathbf {b}}{\Vert \mathbf {x}^r\Vert ^{m-1}}\right\rangle \\ \quad +\left\langle \frac{\sum \nolimits _{k=1}^{m-1}\mathcal {A}^{(k)}({\mathbf {x}^r})^{m-k}+\mathbf {a}}{\Vert \mathbf {x}^r\Vert ^{m-1}}, \frac{\mathbf {y}-\sum \nolimits _{p=1}^{l-1}\mathcal {B}^{(p)}({\mathbf {x}^r})^{l-p}-\mathbf {b}}{\Vert \mathbf {x}^r\Vert ^{l-1}}\right\rangle - \frac{\alpha _r^2}{\Vert \mathbf {x}^r\Vert ^{m+l-4}}\ge 0, \end{array} \end{aligned}$$

that is,

$$\begin{aligned} \begin{array}{l} \left\langle \frac{\mathbf {x}^r}{\Vert \mathbf {x}^r\Vert },\frac{\mathbf {y} -\sum \nolimits _{k=1}^{m-1}\mathcal {A}^{(k)}({\mathbf {x}^r})^{m-k} -\sum \nolimits _{p=1}^{l-1}\mathcal {B}^{(p)}({\mathbf {x}^r})^{l-p}-\mathbf {a}-\mathbf {b}}{\Vert \mathbf {x}^r\Vert ^{m-1}}\right\rangle \\ \quad +\frac{\Vert \mathbf {x}^r\Vert ^{l-2}}{\alpha _r} \left\langle \frac{\sum \nolimits _{k=1}^{m-1}\mathcal {A}^{(k)}({\mathbf {x}^r})^{m-k}+\mathbf {a}}{\Vert \mathbf {x}^r\Vert ^{m-1}}, \frac{\mathbf {y}-\sum \nolimits _{p=1}^{l-1}\mathcal {B}^{(p)}({\mathbf {x}^r})^{l-p}-\mathbf {b}}{\Vert \mathbf {x}^r\Vert ^{l-1}}\right\rangle - \frac{\alpha _r}{\Vert \mathbf {x}^r\Vert ^{m-2}}\ge 0. \end{array} \end{aligned}$$
(10)

For any r, we let \( \bar{\mathbf {x}}^r=\frac{\mathbf {x}^r}{\Vert \mathbf {x}^r\Vert }\). Without loss of generality, we may assume \(\bar{\mathbf {x}}^r\rightarrow \bar{\mathbf {x}}\). Since \(\mathcal {A}^{(1)}\) is positive definite and \(\bar{\mathbf {x}}\in \mathbb {R}^n{\setminus }\{\mathbf {0}\}\), it holds from (10) that

$$\begin{aligned} \begin{array}{l} \lim \limits _{r\rightarrow \infty }\left\langle \frac{\mathbf {x}^r}{\Vert \mathbf {x}^r\Vert }, \frac{\mathbf {y}-\sum \nolimits _{k=1}^{m-1}\mathcal {A}^{(k)}({\mathbf {x}^r})^{m-k}-\mathbf {a}-\sum \nolimits _{p=1}^{l-1}\mathcal {B}^{(p)}(\mathbf {{x}^r})^{l-p}-\mathbf {b}}{\Vert \mathbf {x}^r\Vert ^{m-1}}\right\rangle \\ \quad =-\langle \bar{\mathbf {x}},\mathcal {A}^{(1)}(\bar{\mathbf {x}})^{m-1}\rangle <0. \end{array} \end{aligned}$$
(11)

Moreover, since f is strictly monotone with respect to g on K and \(g(\mathbf {0})\in K\), by taking \(\mathbf {y}=\mathbf {0}\), it follows that

$$\begin{aligned} \left\langle \sum _{p=1}^{l-1}\mathcal {B}^{(p)}({\mathbf {x}^r})^{l-p},\sum _{k=1}^{m-1}\mathcal {A}^{(k)}({\mathbf {x}^r})^{m-k}\right\rangle >0. \end{aligned}$$

By dividing both sides by \(\Vert \mathbf {x}^r\Vert ^{m+l-2}\) and let \(r\rightarrow \infty \), we have

$$\begin{aligned}\langle \mathcal {A}^{(1)}(\bar{\mathbf {x}})^{m-1},\mathcal {B}^{(1)}(\bar{\mathbf {x}})^{l-1}\rangle \ge 0, \end{aligned}$$

which, together with the condition (b), implies that

$$\begin{aligned} \langle \mathcal {A}^{(1)}(\bar{\mathbf {x}})^{m-1},\mathcal {B}^{(1)}(\bar{\mathbf {x}})^{l-1}\rangle > 0. \end{aligned}$$

Furthermore, we have

$$\begin{aligned} \begin{array}{l} \lim \limits _{r\rightarrow \infty }\left\langle \frac{\sum _{k=1}^{m-1}\mathcal {A}^{(k)}({\mathbf {x}^r})^{m-k}+\mathbf {a}}{\Vert \mathbf {x}^r\Vert ^{m-1}}, \frac{\mathbf {y}-\sum _{p=1}^{l-1}\mathcal {B}^{(p)}({\mathbf {x}^r})^{l-p}-\mathbf {b}}{\Vert \mathbf {x}^r\Vert ^{l-1}}\right\rangle \\ \quad =-\langle \mathcal {A}^{(1)}(\bar{\mathbf {x}})^{m-1},\mathcal {B}^{(1)}(\bar{\mathbf {x}})^{l-1}\rangle <0. \end{array} \end{aligned}$$

Thus, for all sufficiently large r, we deduce that

$$\begin{aligned} \left\langle \frac{\sum _{k=1}^{m-1}\mathcal {A}^{(k)}({\mathbf {x}^r})^{m-k}+\mathbf {a}}{\Vert \mathbf {x}^r\Vert ^{m-1}}, \frac{\mathbf {y}-\sum _{p=1}^{l-1}\mathcal {B}^{(p)}({\mathbf {x}^r})^{l-p}-\mathbf {b}}{\Vert \mathbf {x}^r\Vert ^{l-1}}\right\rangle <0. \end{aligned}$$

that is, for all sufficiently large r,

$$\begin{aligned} \begin{array}{l} u^r:=\frac{\Vert \mathbf {x}^r\Vert ^{l-2}}{\alpha _r} \left\langle \frac{\sum \nolimits _{k=1}^{m-1}\mathcal {A}^{(k)}({\mathbf {x}^r})^{m-k}+\mathbf {a}}{\Vert \mathbf {x}^r\Vert ^{m-1}}, \frac{\mathbf {y}-\sum \nolimits _{p=1}^{l-1}\mathcal {B}^{(p)}({\mathbf {x}^r})^{l-p}-\mathbf {b}}{\Vert \mathbf {x}^r\Vert ^{l-1}}\right\rangle - \frac{\alpha _r}{\Vert \mathbf {x}^r\Vert ^{m-2}}\\ <0. \end{array} \end{aligned}$$

Now, we consider the following two cases.

  1. (i)

    If \(\{u^r\}\) is unbounded, then, without loss of generality, we assume that \(u^r\rightarrow -\infty \) as \(r\rightarrow \infty \). Thus, it follows from (10) and (11) that

    $$\begin{aligned} \begin{array}{rcl} 0&{}\le &{}\left\langle \frac{\mathbf {x}^r}{\Vert \mathbf {x}^r\Vert },\frac{\mathbf {y}-\sum \nolimits _{k=1}^{m-1}\mathcal {A}^{(k)}({\mathbf {x}^r})^{m-k}-\sum \nolimits _{p=1}^{l-1}\mathcal {B}^{(p)}({\mathbf {x}^r})^{l-p}-\mathbf {a}-\mathbf {b}}{\Vert \mathbf {x}^r\Vert ^{m-1}}\right\rangle +u^r\\ &{}\rightarrow &{} -\infty , \end{array} \end{aligned}$$

    which is a contradiction.

  2. (ii)

    If \(\{u^r\}\) is bounded, then there exists a convergent subsequence of \(\{u^r\}\), and we denote it as \(\{u^r\}\). We assume that \(u^r\rightarrow -\beta \) as \(r\rightarrow \infty \), then \(\beta \ge 0\). So, we have

    $$\begin{aligned} \begin{array}{rcl} 0&{}\le &{}\lim _{r\rightarrow \infty }\left\langle \frac{\mathbf {x}^r}{\Vert \mathbf {x}^r\Vert },\frac{\mathbf {y}-\sum \nolimits _{k=1}^{m-1}\mathcal {A}^{(k)}({\mathbf {x}^r})^{m-k}-\sum \nolimits _{p=1}^{l-1}\mathcal {B}^{(p)}({\mathbf {x}^r})^{l-p}-\mathbf {a}-\mathbf {b}}{\Vert \mathbf {x}^r\Vert ^{m-1}}\right\rangle +\lim \limits _{r\rightarrow \infty }u_r\\ &{}=&{}-\langle (\bar{\mathbf {x}}),\mathcal {A}^{(1)}(\bar{\mathbf {x}})^{m-1}\rangle -\beta <0, \end{array} \end{aligned}$$

    which is also a contradiction.

Therefore, \(SOL(\Lambda ,\mathbf {a},\Theta ,\mathbf {b},K)\) is nonempty.

Now, we show the uniqueness of solutions to \(GPVI(\Lambda ,\mathbf {a},\Theta ,\mathbf {b},K)\). Suppose that \(\mathbf {x}^*\) and \(\bar{\mathbf {x}}\) are two different solutions to \(GPVI(\Lambda ,\mathbf {a},\Theta ,\mathbf {b},K)\). Then,

$$\begin{aligned} \langle f(\mathbf {x}^*)-f(\bar{\mathbf {x}}),g(\bar{\mathbf {x}})\rangle \ge 0\quad \text{ and }\quad \langle f(\bar{\mathbf {x}})-f(\mathbf {x}^*),g(\mathbf {x}^*)\rangle \ge 0, \end{aligned}$$

which yields that

$$\begin{aligned} \langle f(\mathbf {x}^*)-f(\bar{\mathbf {x}}),g(\mathbf {x}^*)-g(\bar{\mathbf {x}})\rangle \le 0. \end{aligned}$$

This is a contradiction with the condition (a). Therefore, \(GPVI(\Lambda ,\mathbf {a},\Theta ,\mathbf {b},K)\) has a unique solution.\(\square \)

In the following, we give an example to show Theorem 2 cannot be covered by Theorem 1.

Example 1

Consider GVI(g, f, K), where

$$\begin{aligned} g(\mathbf {x})=(x_1^3+1,x_2^3)^\top ,\quad f(\mathbf {x})=(x_1^3+x_1+2,x_2^3)^\top ,\quad K=\{(k,0)^\top : k \ge 1\}. \end{aligned}$$

We show that f and g satisfy all conditions in Theorem 2, but do not satisfy the conditions of Theorem 1. We also give the unique solution of GPVI(g, f, K).

(i) Consider the condition (a) of Theorem 2. We show that function f is strictly monotone with respect to g on the set K. It is easy to see that

$$\begin{aligned} g^{-1}(K)=:\left\{ (x_1,x_2)^\top |x_1\ge 0,x_2=0 \right\} . \end{aligned}$$

For any \(g(\mathbf {x}),g(\mathbf {y})\in K\), it follows that \(x_1\ge 0\), \(y_1\ge 0\), \(x_1\ne y_1\), and hence,

$$\begin{aligned}{}[g(\mathbf {x})-g(\mathbf {y})]^\top [f(\mathbf {x})-f(\mathbf {y})]= & {} (x_1^3-y_1^3)(x_1^3-y_1^3+x_1-y_1)\\= & {} (x_1^3-y_1^3)^2+(x_1-y_1)^2(x_1^2+x_1y_1+y_1^2)\\> & {} 0. \end{aligned}$$

Therefore, the function f is strictly monotone with respect to g on the set K.

(ii) Consider the condition (b) of Theorem 2. In this example, the leading tensors of f and g are the same, i.e., \(\mathcal {A}^{(1)}=\mathcal {B}^{(1)}=(d_{i_1i_2i_3i_4})\in \mathbb {R}^{[4,2]}\) with \(d_{1111}=1\), \(d_{2222}=1\), and other entries being zero. Thus, it is obvious that

$$\begin{aligned} \langle \mathcal {A}^{(1)}\mathbf {x}^{3},\mathcal {B}^{(1)}\mathbf {x}^{3}\rangle =x_1^6+x_2^6\ne 0 \end{aligned}$$

for any nonzero vector \(\mathbf {x}\in \mathbb {R}^2\), i.e., the condition (b) of Theorem 2 holds.

(iii) Consider the condition (c) of Theorem 2. For any \(\mathbf {x}\in \mathbb {R}^2{\setminus }\{\mathbf {0}\}\), it is obvious that

$$\begin{aligned} \mathcal {A}^{(1)}\mathbf {x}^4=\mathcal {B}^{(1)}\mathbf {x}^4=x_1^4+x_2^4 >0, \end{aligned}$$

which implies that both the leading tensors of f and g are positive definite, i.e., the condition (c) of Theorem 2 holds.

(iv) Consider the condition (a) of Theorem 1. By Proposition 1, it is easy to see that f is not strongly monotone with respect to g on K, i.e., the condition (a) of Theorem 1 is not satisfied.

(v) Consider the condition (b) of Theorem 1. Suppose the condition (b) of Theorem 1 holds, i.e., there exists a vector \(\mathbf {u}\in g^{-1}(K)\) and positive scalars \(\alpha \) and L such that

$$\begin{aligned} \Vert g(\mathbf {x})-g(\mathbf {u})\Vert \le L\Vert \mathbf {x}-\mathbf {u}\Vert \quad \text{ for } \text{ all }\; \mathbf {\mathbf {x}}\in g^{-1}(K)\; \text{ with }\; \Vert \mathbf {x}\Vert \ge \alpha . \end{aligned}$$

Then, \(x_1\ge 0\), \(u_1\ge 0\), and

$$\begin{aligned} \Vert g(\mathbf {x})-g(\mathbf {u})\Vert = |x_1^3-u_1^3|=|x_1-u_1||x_1^2+x_1u_1+u_1^2| \le L|x_1-u_1|, \end{aligned}$$

and hence,

$$\begin{aligned} |x_1^2+x_1u_1+u_1^2| \le L. \end{aligned}$$
(12)

Since \(u_1\ge 0\), it follows when \(x_1\rightarrow +\infty \) that the left-hand side of inequality (12) tends to +\(\infty \), which leads to a contradiction.

(vi) We give the unique solution of GPVI(g, f, K). In this example, we need to find \(x_1\in \mathbb {R}\) such that

$$\begin{aligned} x_1^3+1\ge 1,\quad ( x_1^3+x_1+2)(y_1-x_1^3-1)\ge 0\quad \text{ for } \text{ any }\; y_1\ge 1. \end{aligned}$$
(13)

Obviously, (13) has a unique solution \(x_1^*=0\). Thus, \(\mathbf {x}^*=(0,0)^\top \) is the unique solution of GPVI(g, f, K).

When \(K=\mathbb {R}^n_+\), \(GPVI(\Lambda ,\mathbf {a},\Theta ,\mathbf {b},K)\) becomes the generalized polynomial complementarity problem investigated in [16], denoted by \(GPCP(\Lambda ,\mathbf {a},\Theta ,\mathbf {b})\). The following result is an immediate consequence of Theorem 2, which gives an existence and uniqueness result for \(GPCP(\Lambda ,\mathbf {a},\Theta ,\mathbf {b})\).

Corollary 1

Let f and g be defined by (2) and (3). Suppose that f is strictly monotone with respect to g on \(\mathbb {R}^n_+\), \(\langle \mathcal {A}^{(1)}(\mathbf {x})^{m-1},\mathcal {B}^{(1)}(\mathbf {x})^{l-1}\rangle \ne 0\) for any nonzero vector \(\mathbf {x}\in \mathbb {R}^n\), and one of \(\mathcal {A}^{(1)}\) and \(\mathcal {B}^{(1)}\), whose order is even and is lager than or equal to another, is positive definite, then for any given vectors \(\mathbf {a}\in \mathbb {R}^n\) and \(\mathbf {b}\in \mathbb {R}^n_+\), \(GPCP(\Lambda ,\mathbf {a},\Theta ,\mathbf {b})\) has a unique solution.

When \(g(\mathbf {x})=\mathbf {x}\), \(GPVI(\Lambda ,\mathbf {a},\Theta ,\mathbf {b},K)\) becomes the polynomial variational inequality investigated in [10], denoted by \(PVI(K,\Lambda ,\mathbf {a})\). The following result is an immediate consequence of Theorem 2, which gives an existence and uniqueness result for \(PVI(K,\Lambda ,\mathbf {a})\).

Corollary 2

Let \(K\subseteq \mathbb {R}^n\) be a nonempty, closed and convex set with \(\mathbf {0} \in K\). Suppose that the function \(\sum _{k=1}^{m-1}\mathcal {A}^{(k)}\mathbf {x}^{m-k}\) is strictly monotone on K where \(\Lambda :=\Big (\mathcal {A}^{(1)},\ldots ,\mathcal {A}^{(m-1)}\Big )\in \mathcal {F}_{m,n}\) defined by (3), and the tensor \(\mathcal {A}^{(1)}\) is positive definite on K. Then for any given \(\mathbf {a} \in \mathbb {R}^n\), \(PVI(K,\Lambda ,\mathbf {a})\) has a unique solution.

We would like to emphasize that the results in Corollaries 1 and 2 are new ones on the existence and uniqueness for the generalized polynomial complementarity problem and the polynomial variational inequality, respectively.

When \(g(\mathbf {x})=\mathbf {x}\) and \(f(\mathbf {x})=\mathcal {A}\mathbf {x}^{m-1}+\mathbf {a}\) with \(\mathcal {A}\in \mathbb {R}^{[m,n]}\) and \(\mathbf {a}\in \mathbb {R}^n\), \(GPVI(\Lambda ,\mathbf {a},\Theta ,\mathbf {b},K)\) reduces to the tensor variational inequality investigated in [26]. In this case, Corollary 2 reduces to [26, Theorem 4.1], since \(\mathcal {A}\in \mathbb {R}^{[m,n]}\) is positive definite on K under the assumptions that \(\mathbf {0}\in K\) and f is strictly monotone on K. Furthermore, when \(K=\mathbb {R}^n_+\), this problem reduces to the tensor complementarity problem, and in this case, Corollary 2 can further reduce to [2, Corollary 1(b)] if the involved tensor is a Gram tensor. Moreover, for the tensor complementarity problem, some existence and uniqueness results were established in [1, 17].

4 Conclusions

The generalized polynomial variational inequality is a generalized variational inequality with the involved functions being polynomials, however, the known existence and uniqueness theorem for the generalized variational inequality cannot be generally applied to the generalized polynomial variational inequality, since the hypotheses of Theorem 1 do not hold for the general polynomial functions. In this paper, in terms of a characterization theorem related to exceptional family of elements and by using properties of polynomial functions, we established an existence and uniqueness theorem for the generalized polynomial variational inequality, by which we also obtained new existence and uniqueness theorems for the generalized polynomial complementarity problem and the polynomial variational inequality. They can be seen as extensions of the corresponding results for the tensor variational inequality and the tensor complementarity problem. By exploiting properties of polynomials, more theoretical results for the generalized polynomial variational inequality can be further developed.