Double phase problems with competing potentials: concentration and multiplication of ground states

Abstract

In this paper, we establish concentration and multiplicity properties of ground state solutions to the following perturbed double phase problem with competing potentials:

$$\begin{aligned} \left\{ \begin{array}{ll} -\epsilon ^{p}\Delta _{p} u-\epsilon ^{q}\Delta _{q} u +V(x)(|u|^{p-2}u+|u|^{q-2}u)=K(x)f(u),&{} \quad \hbox {in}~\mathbb {R}^{N},\\ u\in W^{1,p}(\mathbb {R}^{N})\cap W^{1,q}(\mathbb {R}^{N}),\ u>0, &{} \quad \hbox {in}~\mathbb {R}^{N},\\ \end{array} \right. \end{aligned}$$

where $1<p<q<N$, $\Delta _{s}u=\hbox {div}(|\nabla u|^{s-2}\nabla u)$, with $s\in \{p,q\}$, is the s-Laplacian operator, and $\epsilon$ is a small positive parameter. We assume that the potentials V, K and the nonlinearity f are continuous but are not necessarily of class $C^{1}$. Under some natural hypotheses, using topological and variational tools from Nehari manifold analysis and Ljusternik–Schnirelmann category theory, we study the existence of positive ground state solutions and the relation between the number of positive solutions and the topology of the set where V attains its global minimum and K attains its global maximum. Moreover, we determine two concrete sets related to the potentials V and K as the concentration positions and we describe the concentration of ground state solutions as $\epsilon \rightarrow 0$. The asymptotic convergence and the exponential decay of positive solutions are also explored. Finally, we establish a sufficient condition for the non-existence of ground state solutions.

1 Introduction

In this paper, we are concerned with the following perturbed double phase problem with competing potentials

$$\begin{aligned} \left\{ \begin{array}{ll} -\epsilon ^{p}\Delta _{p} u-\epsilon ^{q}\Delta _{q} u +V(x)(|u|^{p-2}u+|u|^{q-2}u)=K(x)f(u)&{}\quad \hbox {in}~\mathbb {R}^{N},\\ u\in W^{1,p}(\mathbb {R}^{N})\cap W^{1,q}(\mathbb {R}^{N}),\ u>0 &{}\quad \hbox {in}~\mathbb {R}^{N},\\ \end{array} \right. \end{aligned}$$

(1.1)

where $1<p<q<N$, $\Delta _{s}u=\hbox {div}(|\nabla u|^{s-2}\nabla u)$, with $s\in \{p,q\}$, is the usual s-Laplace operator, $\epsilon$ is small positive parameter, V and K are potential functions and f is the reaction term with subcritical growth. We are interested in the qualitative and asymptotic analysis of solutions to problem (1.1) and we are mainly concerned with existence and multiplicity properties of solutions, as well as with concentration phenomena as $\epsilon \rightarrow 0$.

The features of this paper are the following:

• (1) the presence of several differential operators with different growth, which generates a double phase associated energy;

• (2) the problem combines the multiple effects generated by two variable potentials;

• (3) there exists a competition effect between the absorption potential and the reaction potential, which implies more complex phenomena to locate the concentration positions;

• (4) the main concentration phenomenon creates a bridge between the global maximum point of the solution versus the global maximum of the reaction potential and the global minimum of the absorption potential;

• (5) due to the unboundedness of the domain, the Palais–Smale sequences do not have the compactness property;

• (6) the proofs combine refined analysis techniques, including topological and variational tools.

Problems like (1.1) arise when one looks for the stationary solutions of reaction-diffusion systems of the form

$$\begin{aligned} u_{t}=\hbox {div}[D(u)\nabla u]+c(x,u),~x\in \mathbb {R}^N~\hbox {and}~t>0, \end{aligned}$$

where $D(u)=|\nabla u|^{p-2}+|\nabla u|^{q-2}$. This system has a wide range of applications in physics and related sciences, such as biophysics, plasma physics, and chemical reaction design (see [12]). In such applications, the function u is a state variable and describes density or concentration of multi-component substances, $\hbox {div}[D(u)\nabla u]$ corresponds to the diffusion with a diffusion coefficient D(u), and c(x, u) is the reaction and relates to source and loss processes. Typically, in chemical and biological applications, the reaction term c(x, u) has a polynomial form with respect to the unknown concentration denoted by u.

Since the content of the paper is closely concerned with double phase problems, we start with a short description on the background and applications. An interesting phenomenon is that the operator involved in (1.1) is the so-called double phase operator whose behavior switches between two different elliptic situations. Originally, the idea to treat such operators comes from Zhikov [43] who introduced such classes to provide models of strongly anisotropic materials; see also the monograph of Zhikov, Kozlov and Oleinik [44].

Moreover, the double phase problem (1.1) is also motivated by numerous models arising in mathematical physics. For example, we can refer to the following Born–Infeld equation [11] that appears in electromagnetism, electrostatics and electrodynamics as a model based on a modification of Maxwell’s Lagrangian density:

$$\begin{aligned} -\mathrm{div}\,\left( \frac{\nabla u}{(1-2|\nabla u|^2)^{1/2}}\right) =h(u) \quad \text{ in }\ \Omega . \end{aligned}$$

Indeed, by the Taylor formula, we have

$$\begin{aligned} (1-x)^{-1/2}=1+\frac{x}{2}+\frac{3}{2\cdot 2^2}x^2+\frac{5!!}{3!\cdot 2^3}x^3+\cdots +\frac{(2n-3)!!}{(n-1)!2^{n-1}}x^{n-1}+\cdots \ \text{ for }\ |x|<1. \end{aligned}$$

Taking $x=2|\nabla u|^2$ and adopting the first order approximation, we can obtain problem (1.1) for $p=2$ and $q=4$. Furthermore, the n-th order approximation problem is driven by the multi-phase differential operator

$$\begin{aligned} -\Delta u-\Delta _4u-\frac{3}{2}\Delta _6u-\cdots -\frac{(2n-3)!!}{(n-1)!}\Delta _{2n}u. \end{aligned}$$

We also refer to the following fourth-order relativistic operator

$$\begin{aligned} u\mapsto \mathrm{div}\, \left( \frac{|\nabla u|^2}{(1-|\nabla u|^4)^{3/4}}\,\nabla u\right) , \end{aligned}$$

which describes large classes of phenomena arising in relativistic quantum mechanics. Again, by Taylor’s formula, we have

$$\begin{aligned} x^2(1-x^4)^{-3/4}=x^2+\frac{3x^6}{4}+\frac{21x^{10}}{32}+\cdots . \end{aligned}$$

This shows that the fourth-order relativistic operator can be approximated by the following autonomous double phase operator

$$\begin{aligned} u\mapsto \Delta _4u+\frac{3}{4}\Delta _8u. \end{aligned}$$

For more details in the physical backgrounds and other applications, we refer to Bahrouni, Rădulescu and Repovš [7] (for phenomena associated with transonic flows) and to Benci, D’Avenia, Fortunato and Pisani [10] (for models arising in quantum physics).

In the past few decades, problem (1.1) has been the subject of extensive mathematical studies. For the case $\epsilon =1$, using various variational and topological arguments, many authors studied the existence and multiplicity results of nontrivial solutions, ground state solutions, nodal solutions and some qualitative properties of solutions, respectively. See for example [19, 28, 30,31,32] for the case of bounded domains. In this classical setting we recall here the seminal papers by Ni and Wei [29], Li and Nirenberg [23], del Pino and Felmer [15], del Pino, Kowalczyk and Wei [16], and we refer to Ambrosetti and Malchiodi [4] for detailed discussions.

We point out that the problem settled on the whole space has been considered recently by several authors. In [21], He and Li established some regularity results for problem (1.1). By combining the concentration-compactness principle with the mountain pass arguments, an existence result has been established by He and Li [22] when the nonlinearity has an asymptotic $(p-1)$-linear growth at infinity. Figueiredo [18] proved the existence of positive ground state solutions for the critical growth case. We also point out that the existence of infinitely many solutions was obtained in [8] by using linking theory and the symmetric mountain pass theorem.

For the case when $\epsilon >0$ sufficiently small, the solutions of problem (1.1) are often referred to as semiclassical states, which possess many significant physical insights for $\epsilon$ small. More precisely, the concentration phenomenon of semiclassical states, as $\epsilon \rightarrow 0$, reflects the transformation process between quantum mechanics and classical mechanics. For such problem, some asymptotic behaviors of semiclassical states, such as concentration, convergence and exponential decay, are very interesting research topics in mathematics and physics. In this framework, from a mathematical viewpoint, it is worth to study not only the existence of semiclassical solutions but also their asymptotic behavior as $\epsilon \rightarrow 0$. Typically, solutions tend to concentrate around critical points of the potentials functions: such solutions are called spikes. To put our result in perspective, we review briefly some related results in this direction.

In [3], Alves and Figueiredo studied the multiplicity and concentration of solutions for the following problem with linear potential

$$\begin{aligned} \left\{ \begin{array}{ll} -\Delta _{p} u-\Delta _{q} u +V(\epsilon x)(|u|^{p-2}u+|u|^{q-2}u)=f(u),&{}\quad \hbox {in}~\mathbb {R}^{N},\\ u\in W^{1,p}(\mathbb {R}^{N})\cap W^{1,q}(\mathbb {R}^{N}),u>0, &{}\quad \hbox {in}~\mathbb {R}^{N},\\ \end{array} \right. \end{aligned}$$

(1.2)

which is equivalent to the problem under the action of variable substitution

$$\begin{aligned} \left\{ \begin{array}{ll} -\epsilon ^{p}\Delta _{p} u-\epsilon ^{q}\Delta _{q} u +V(x)(|u|^{p-2}u+|u|^{q-2}u)=f(u),&{}\quad \hbox {in}~\mathbb {R}^{N},\\ u\in W^{1,p}(\mathbb {R}^{N})\cap W^{1,q}(\mathbb {R}^{N}),u>0, &{}\quad \hbox {in}~\mathbb {R}^{N}.\\ \end{array} \right. \end{aligned}$$

(1.3)

Here the authors assumed that the potential V satisfies the following global condition introduced by Rabinowitz [33]

$$\begin{aligned} 0<\inf _{x\in \mathbb {R}^N}V(x)<\liminf _{|x|\rightarrow \infty }V(x)<\infty , \end{aligned}$$

(1.4)

and the nonlinear term f has $C^{1}$-smoothness with superlinear and subcritical growth. Using mountain pass arguments combined with the Ljusternik–Schnirelmann category theory, they proved the existence and multiplicity of positive solutions which concentrate at global minimum points of V for problem (1.2). Subsequently, the multiplicity result in [3] has been improved in [6] by considering continuous nonlinearities. It should be noted that Ambrosio and Repovš [6] proved a useful splitting lemma in detail under the continuous condition. Recall the following local hypothesis introduced by del Pino and Felmer [14]: there exists a bounded domain $\Omega$ such that

$$\begin{aligned} 0<\inf _{x\in \Omega }V(x)<\inf _{x\in \partial \Omega }V(x). \end{aligned}$$

(1.5)

Under this assumption, Alves and da Silva [2] obtained multiplicity and concentration properties depending on $\Omega$ by penalization method and Lusternik–Schnirelmann theory. We also mention the recent papers [5] and [42] in which the multiplicity and concentration of positive solutions for a class of fractional double phase problems with local condition (1.5) are established.

We would like to emphasize that, in all the works mentioned above, the authors focused only on the linear potential case for the double phase problem. In other words, the multiplicity and concentration results depend only on the properties of the linear potential V. Furthermore, the problem is autonomous in the reaction, in the sense that the nonlinearities on the right-hand side of the equation do not depend on the variable x. That is why it is quite natural to ask how the appearance of nonlinear potential and linear potential will affect the existence, multiplicity and concentration of solutions to problem (1.1)? This is an interesting question which motivates the present work.

Inspired by the above facts, in the present paper, we are going to study the existence, multiplicity and concentration phenomena of positive solutions to problem (1.1) with both absorption potential (linear potential) V and reaction potential (nonlinear potential) K. To the best of our knowledge, it seems that such a problem was not considered in the literature before. Here, it is worth pointing out that the combination of linear potential and nonlinear potential is called the competing potentials (see [39]), which makes difficulties in determining the concentration positions of solutions.

More precisely, we first establish the existence result of positive ground state solutions for small $\epsilon$. Secondly, we investigate the concentration phenomena of ground state solutions as $\epsilon \rightarrow 0$. For this purpose, we determine two concrete sets related to the potentials V and K as the concentration positions of these solutions. Roughly speaking, the ground state solutions concentrate at such points $x_{0}$ where $V(x_{0})$ is small or $K(x_{0})$ is large. As a special case, we can show that these ground state solutions concentrate around such points which are both the minima points of the potential V and the maximum points of the potential K. Furthermore, we analyze the asymptotic convergence of ground state solutions under scaling and translation and the exponential decay estimate. Finally, we investigate the relation between the number of positive solutions and the topology of the set where V attains its global minimum and K attains its global maximum. To the best of our knowledge, the present paper is the first work dealing with concentration properties for double phase problems in the presence of two competing potentials.

1.1 Main results

We start with the following basic notations:

$$\begin{aligned} V_{\min }=\min V,~{\mathscr {V}}=\{x\in \mathbb {R}^{N}: V(x)=V_{\min }\}~\hbox {and}~V_{\infty }=\liminf \limits _{|x|\rightarrow \infty }V(x) \end{aligned}$$

and

$$\begin{aligned} K_{\max }=\max K,~{\mathscr {K}}=\{x\in \mathbb {R}^{N}: K(x)=K_{\max }\}~\hbox {and}~K_{\infty }=\limsup \limits _{|x|\rightarrow \infty }K(x). \end{aligned}$$

We assume that V and K satisfy the following conditions:

($A_{0}$):

$V, K\in C(\mathbb {R}^N, \mathbb {R})$ are bounded, $V_{\min }:=\inf V>0$ and $K_{\min }:=\inf K>0$;

($A_{1}$):

$V_{\min }<V_{\infty }$ and there is $x_{v}\in {\mathscr {V}}$ such that $K(x_{v})\ge K(x)$ for all $|x|\ge R$ and some large $R>0$;

($A_{2}$):

$K_{\max }>K_{\infty }$ and there is $x_{k}\in {\mathscr {K}}$ such that $V(x_{k})\le V(x)$ for all $|x|\ge R$ and some large $R>0$;

($A_{3}$):

$V, K\in C(\mathbb {R}^N, \mathbb {R})$ are bounded functions such that $0<V^{\infty }:=\lim _{|x|\rightarrow \infty }V(x)\le V(x)$ and $0<K(x)\le K^{\infty }:=\lim _{|x|\rightarrow \infty }K(x)$, and $|\mathcal {V}|>0$ or $|\mathcal {K}|>0$, where

$$\begin{aligned} \mathcal {V}=\{x\in \mathbb {R}^{N}: V^{\infty }<V(x)\}\quad \hbox {and}\quad \mathcal {K}=\{x\in \mathbb {R}^{N}: K^{\infty }>K(x)\}. \end{aligned}$$

Note that, for the case $(A_{1})$, we can assume $K(x_{v})=\max _{x\in {\mathscr {V}}}K(x)$; and for the case $(A_{2})$, we can assume $V(x_{k})=\min _{x\in {\mathscr {K}}}V(x)$. To describe some concentration phenomena of positive ground state solutions, we define two concrete sets related to the potentials V and K:

$$\begin{aligned} {\mathscr {A}}_{v}:=\{x\in {\mathscr {V}}: K(x)=K(x_{v})\}\cup \{x\notin {\mathscr {V}}: K(x)>K(x_{v})\}, \end{aligned}$$

and

$$\begin{aligned} {\mathscr {A}}_{k}:=\{x\in {\mathscr {K}}: V(x)=V(x_{k})\}\cup \{x\notin {\mathscr {K}}: V(x)<V(x_{k})\}. \end{aligned}$$

This kind of structure was recently introduced by Ding and Liu [17] which generalized the condition (1.4) introduced by Rabinowitz [33].

We observe that $x_{v}\in {\mathscr {A}}_{v}$ and $x_{k}\in {\mathscr {A}}_{k}$, which shows that ${\mathscr {A}}_{v}$ and ${\mathscr {A}}_{k}$ are non-empty and bounded sets. Moreover, if $(A_{1})$ holds and ${\mathscr {V}}\cap {\mathscr {K}}\ne \emptyset$, we can set $K(x_{v})=\max _{x\in {\mathscr {V}}\cap {\mathscr {K}}}K(x)$ and

$$\begin{aligned} {\mathscr {A}}_{v}:=\{x\in {\mathscr {V}}\cap {\mathscr {K}}: K(x)=K(x_{v})\}, \end{aligned}$$

which implies that ${\mathscr {A}}_{v}={\mathscr {V}}\cap {\mathscr {K}}$. Similarly, if $(A_{2})$ holds and ${\mathscr {V}}\cap {\mathscr {K}}\ne \emptyset$, then ${\mathscr {A}}_{k}={\mathscr {V}}\cap {\mathscr {K}}$.

We assume that the nonlinearity f satisfies the following hypotheses:

($f_{1}$):

$f\in C(\mathbb {R}, \mathbb {R})$ and $f(s)=0$ for all $s<0$;

($f_{2}$):

$f(s)=o(|s|^{p-1})$ as $s\rightarrow 0$;

($f_{3}$):

there exist $c_{0}>0$ and $r\in (q,q^*)$, with $q^*=\frac{Nq}{N-q}$, such that $|f(s)|\le c_{0}(1+|s|^{r-1})$ for all s;

($f_{4}$):

there exists $\theta \in (q,q^*)$ such that

$$\begin{aligned} 0<\theta F(s)=\theta \int _{0}^{s}f(t)dt\le sf(s)\quad \hbox {for all}~s>0; \end{aligned}$$

($f_{5}$):

$f(s)/s^{q-1}$ is increasing for all $s\in (0,\infty )$.

We state in what follows the main results of this paper.

Theorem 1.1

Suppose that $(A_{0})$, $(A_{1})$ and $(f_{1})$–$(f_{5})$ hold, then for all small $\epsilon >0$

(i):

problem (1.1) has at least a positive ground state solution $u_{\epsilon }$;

(ii):

${\mathscr {L}}_{\epsilon }$ is compact, where ${\mathscr {L}}_{\epsilon }$ denotes the set of all positive ground state solutions;

(iii):

$u_{\epsilon }(x)$ possesses a maximum point $x_{\epsilon }$ such that, up to a subsequence, $x_{\epsilon }\rightarrow x_0$ as $\epsilon \rightarrow 0$, and $\lim _{\epsilon \rightarrow 0}dist(x_{\epsilon },{\mathscr {A}}_{v})=0$, and $v_{\epsilon }(x):=u_{\epsilon }(\epsilon x+x_{\epsilon })$ converges to a ground state solution of

$$\begin{aligned} -\Delta _{p} u-\Delta _{q} u +V(x_{0})(|u|^{p-2}u+|u|^{q-2}u)=K(x_{0})f(u)\quad \hbox {in}~\mathbb {R}^{N}. \end{aligned}$$

In particular, if ${\mathscr {V}}\cap {\mathscr {K}}\ne \emptyset$, then $\lim _{\epsilon \rightarrow 0}dist(x_{\epsilon },{\mathscr {V}}\cap {\mathscr {K}})=0$, and up to a subsequence, $v_{\epsilon }$ converges to a ground state solution of

$$\begin{aligned} -\Delta _{p} u-\Delta _{q} u +V_{\min }(|u|^{p-2}u+|u|^{q-2}u)=K_{\max }f(u)\quad \hbox {in}~\mathbb {R}^{N}. \end{aligned}$$

(iv):

We have $\lim _{|x|\rightarrow \infty }u_{\epsilon }(x)=0$ and $u_{\epsilon }\in C_{loc}^{1,\sigma }(\mathbb {R}^{N})$ with $\sigma \in (0,1)$. Furthermore, there exist positive constants c, C such that

$$\begin{aligned} u_{\epsilon }(x)\le C\exp \left( -\frac{c}{\epsilon }|x-x_{\epsilon }|\right) . \end{aligned}$$

Theorem 1.2

Suppose that $(A_{0})$, $(A_{2})$ and $(f_{1})$–$(f_{5})$ hold, and we replace ${\mathscr {A}}_{v}$ by ${\mathscr {A}}_{k}$, then all the conclusions of Theorem 1.1 remain true.

To obtain the multiplicity result of positive solutions for problem (1.1), we first recall the definition of Ljusternik–Schnirelmann category. If Y is a given closed subset of a topological space X, the Ljusternik–Schnirelmann category cat$_{X}(Y)$ is the least number of closed and contractible sets in X which cover Y.

To prove the multiplicity result, in the following we assume ${\mathscr {V}}\cap {\mathscr {K}}\ne \emptyset$. Let us denote by

$$\begin{aligned} \Lambda :={\mathscr {V}}\cap {\mathscr {K}}\quad \hbox {and}\quad \Lambda _{\delta }=\{x\in \mathbb {R}^{N}: \hbox {dist}(x,\Lambda )\le \delta \}\quad \hbox {for}~\delta >0. \end{aligned}$$

Clearly, by $(A_0)$, $(A_1)$ and $(A_2)$, we know that the set $\Lambda$ is compact. We establish the following multiplicity property of positive solutions.

Theorem 1.3

Suppose that $(A_{0})$, $(A_{1})$ (or $(A_{2})$) and $(f_{1})$–$(f_{5})$ hold and $\Lambda \ne \emptyset$. Then for any $\delta >0$ there exists $\epsilon _{\delta }>0$ such that, for any $\epsilon \in (0,\epsilon _{\delta })$, problem (1.1) has at least cat$_{\Lambda _{\delta }}(\Lambda )$ positive solutions.

Finally, we give the non-existence result of ground state solutions as follows.

Theorem 1.4

Assume that $(A_{3})$ and $(f_{1})$–$(f_{5})$ hold, then for each $\epsilon >0$, problem (1.1) has no positive ground state solutions.

Additionally, according to the above observations, we can obtain some consequences of the above results. More precisely, we consider the double phase problem with nonlinear potential

$$\begin{aligned} \left\{ \begin{array}{ll} -\epsilon ^{p}\Delta _{p} u-\epsilon ^{q}\Delta _{q} u +|u|^{p-2}u+|u|^{q-2}u=K(x)f(u),&{}\quad \hbox {in}~\mathbb {R}^{N},\\ u\in W^{1,p}(\mathbb {R}^{N})\cap W^{1,q}(\mathbb {R}^{N}),u>0, &{}\quad \hbox {in}~\mathbb {R}^{N}.\\ \end{array} \right. \end{aligned}$$

(1.6)

and assume that the reaction potential K satisfies the following condition:

($K_{1}$):

$0<\inf _{x\in \mathbb {R}^{N}}K(x)$ and $\limsup _{|x|\rightarrow \infty }K(x)<\max _{x\in \mathbb {R}^{N}}K(x)$.

($K_{2}$):

$K\in C(\mathbb {R}^N, \mathbb {R})$ is bounded such that $0<K(x)\le K^{\infty }:=\lim _{|x|\rightarrow \infty }K(x)$ and $|\mathcal {K}|>0$, where $\mathcal {K}=\{x\in \mathbb {R}^{N}: K^{\infty }>K(x)\}$.

The main results are the following theorems for problem (1.6).

Theorem 1.5

Assume that $(K_{1})$ and $(f_{1})$–$(f_{5})$ hold, then for all small $\epsilon >0$

(i):

problem (1.6) has at least a positive ground state solution $u_{\epsilon }$;

(ii):

${\mathscr {L}}_{\epsilon }$ is compact, where ${\mathscr {L}}_{\epsilon }$ denotes the set of all positive ground state solutions;

(iii):

$u_{\epsilon }(x)$ possesses a maximum point $x_{\epsilon }$ such that, up to a subsequence, $x_{\epsilon }\rightarrow x_0$ as $\epsilon \rightarrow 0$, and $\lim _{\epsilon \rightarrow 0}dist(x_{\epsilon },{\mathscr {K}})=0$, and $v_{\epsilon }(x):=u_{\epsilon }(\epsilon x+x_{\epsilon })$ converges to a ground state solution of

$$\begin{aligned} -\Delta _{p} u-\Delta _{q} u +|u|^{p-2}u+|u|^{q-2}u=K_{\max }f(u)~ \quad \hbox {in}~\mathbb {R}^{N}. \end{aligned}$$

(iv):

We have $\lim _{|x|\rightarrow \infty }u_{\epsilon }(x)=0$ and $u_{\epsilon }\in C_{loc}^{1,\sigma }(\mathbb {R}^{N})$ with $\sigma \in (0,1)$. Furthermore, there exist positive constants c, C such that

$$\begin{aligned} u_{\epsilon }(x)\le C\exp \left( -\frac{c}{\epsilon }|x-x_{\epsilon }|\right) . \end{aligned}$$

Theorem 1.6

Assume that $(K_{1})$ and $(f_{1})$–$(f_{5})$ hold. Then for any $\delta >0$ there exists $\epsilon _{\delta }>0$ such that, for any $\epsilon \in (0,\epsilon _{\delta })$, problem (1.6) has at least cat$_{{\mathscr {K}}_{\delta }}({\mathscr {K}})$ positive solutions.

Theorem 1.7

Assume that $(K_{2})$ and $(f_{1})$–$(f_{5})$ hold, then for each $\epsilon >0$, problem (1.6) has no positive ground state solutions.

Compared with the existing issues in [3, 6] where the authors studied the relationship between the linear potential and multiplicity and concentration properties of positive solutions, a novelty of the present paper is that we will study how the behavior of competing potentials will affect the multiplicity and concentration of positive solutions of problem (1.1). For problem (1.1), there exists competition between the linear potential V and the nonlinear potential K. This happens because V has the tendency to attract solutions to its minimum points, while the potential K tends to attract solutions to its maximum points. Furthermore, the existence, multiplicity and some properties of positive solutions depend not only on the linear potential but also on the nonlinear potential. Therefore, the study of problem (1.1) is much more complicated due to the multiple effects of the competing potentials. That is why we will discuss carefully in this paper the interactions of the two potentials, and the present arguments seem to be more delicate. Besides, another novelty of this paper is that we obtain a non-existence result of positive ground state solutions under suitable conditions. So our results can apply to more general situations. From the comments above, the results obtained in this paper cover, improve and extend the relevant results in [3, 6].

On the other hand, our main motivation of this paper also comes from the study of Schrödinger equations

$$\begin{aligned} -\epsilon ^2\Delta u+V(x)u=K(x)f(u),~~u\in H^{1}(\mathbb {R}^{N}). \end{aligned}$$

(1.7)

Note that, if $p=q=2$, problem (1.1) reduces to problem (1.7). Many scholars investigated the existence, concentration and multiplicity of semiclassical solutions for problem (1.7). In particular, in the pioneering work by Rabinowitz [33], the author proved the existence of positive ground state solution of problem (1.7) with $K\equiv 1$ under the global condition (1.4) by using mountain pass theorem. Based on the work of [33], Wang [37] studied the concentration phenomena around the global minimum points of potential V. Using the penalization method, del Pino and Felmer in [14] studied the localized concentration phenomena: the solutions concentrating around local minima of potential V. Later, Wang and Zeng [39] first constructed a ground state solutions concentrating at a special set characterized by the competing potential functions V and K. For related results on coupled nonlinear Schrödinger system with competing potentials, we refer to [41]. Applying the Ljusternik–Schnirelmann category theory, a multiplicity result depending on the topology properties of the competing potentials was obtained by Cingolani and Lazzo [13]. Compared with the semilinear Schrödinger equations, the lack of homogeneity caused by the (p, q)-Laplacian operator makes our analysis more delicate and intriguing with respect to the above mentioned works, and some refined estimates will be carried out to implement our variational machinery.

In order to complete the proofs of main results, we need to use some suitable variational and topological arguments and refined analysis techniques. More precisely, due to the fact that f is only continuous, the Nehari manifold ${\mathscr {N}}_{\epsilon }$ is not differentiable and some well-known arguments for $C^{1}$-Nehari manifolds are not applicable in our situation. To overcome this obstacle, in the spirit of [6], we intend to make use of the method developed by Szulkin and Weth [34] to deal with our problem. The main idea of this method is to find a homeomorphism mapping between the Nehari manifold ${\mathscr {N}}_{\epsilon }$ and the unit sphere of working space $E_{\epsilon }$. Then, one can construct a reduction functional $I_{\epsilon }$ on the unit sphere such that critical points of $I_{\epsilon }$ are in one-to-one correspondence with critical points of the original functional $\Phi _{\epsilon }$, where $E_{\epsilon }$, ${\mathscr {N}}_{\epsilon }$, $\Phi _{\epsilon }$ and $I_{\epsilon }$ will be defined in Sect. 2. On the other hand, due to the interactions of the competing potentials, we need to characterize the comparison relationships of the ground state energy value between the original problem and certain auxiliary problems, which will play a crucial role in our arguments. To obtain the multiplicity result, we use a technique introduced by Benci and Cerami [9], which consists of making precise comparisons between the category of some sublevel sets of the energy functional $\Phi _{\epsilon }$ and the category of the set $\Lambda$. Since we want to use Ljusternik–Schnirelmann category theory, we need to prove certain compactness properties for the functional $\Phi _{\epsilon }$. This will be done by applying suitable energy comparison methods. In particular, we will see that the levels of compactness are strongly related to the behaviors of the potentials V and K at infinity.

Finally, it should be noted that we only give the detailed proofs for problem (1.1) because the arguments for problem (1.6) are similar.

The structure of the present is the following. In Sect. 2, we establish the variational framework of problem (1.1) and give some useful preliminary results which will be used later. In Sect. 3, we present some necessary results for the constant coefficients problem. In Sect. 4, we analyze the Palais–Smale compactness condition to overcome the lack of compactness. In Sect. 5, we prove the existence and concentration of positive ground state solutions and we complete the proofs of Theorems 1.1 and 1.2. In Sect. 6, we are devoted to the multiplicity result and we give the proof of Theorem 1.3. Finally, we prove the non-existence result of ground state solutions in Sect. 7.

2 Variational framework and preliminary results

Throughout this paper, for convenience we will use the following notations:

– $\Vert \cdot \Vert _s$ denotes the usual norm of the space $L^s(\mathbb {R}^N)$, $1\le s\le \infty$;

– c, C, $c_i$, $C_{i}$ denote some different positive constants.

In what follows, we recall some facts about the Sobolev spaces and introduce some lemmas which we will use later.

For $p\in (1,\infty )$ and $N>p$, we define $\mathcal {D}^{1,p}(\mathbb {R}^{N})$ as the closure of $C_{0}^{\infty }(\mathbb {R}^{N})$ with respect to

$$\begin{aligned} \Vert \nabla u\Vert _{p}^{p}=\int _{\mathbb {R}^N}|\nabla u|^p\mathrm {d}x. \end{aligned}$$

Let $W^{1,p}(\mathbb {R}^{N})$ be the usual Sobolev space endowed with the standard norm

$$\begin{aligned} \Vert u\Vert ^{p}=\int _{\mathbb {R}^N}(|\nabla u|^p+|u|^p)\mathrm {d}x. \end{aligned}$$

According to [1], we have the following embedding theorem for Sobolev spaces.

Lemma 2.1

Let $N>p$, then there exists a constant $S_{*}>0$ such that, for any $u\in \mathcal {D}^{1,p}(\mathbb {R}^{N})$,

$$\begin{aligned} \Vert u\Vert _{p^*}^{p}\le S_{*}^{-1}\Vert \nabla u\Vert _{p}^{p}. \end{aligned}$$

Furthermore, $W^{1,p}(\mathbb {R}^{N})$ is embedded continuously into $L^{s}(\mathbb {R}^{N})$ for any $s\in [p,p^{*}]$ and compactly into $L_{loc}^{s}(\mathbb {R}^{N})$ for any $s\in [1,p^{*})$.

We recall the following Lions compactness lemma, see [24].

Lemma 2.2

Let $N>p$ and $r\in [p,p^*)$. If $\{u_n\}$ is a bounded sequence in $W^{1,p}(\mathbb {R}^N)$ and if

$$\begin{aligned} \lim _{n\rightarrow \infty }\sup _{y\in \mathbb {R}^N}\int _{B_{R}(y)}|u|^{r}\mathrm {d}x=0, \end{aligned}$$

where $R>0$, then $u_n\rightarrow 0$ in $L^{s}(\mathbb {R}^N)$ for all $s\in (p,p^*)$.

In order to prove the main results, we do not deal with the problem (1.1) directly, but instead we study an equivalent problem to problem (1.1). In fact, after the change of variable $x\mapsto \epsilon x$, we can rewrite problem (1.1) as the following equivalent problem

$$\begin{aligned} \left\{ \begin{array}{ll} -\Delta _{p} u-\Delta _{q} u +V(\epsilon x)(|u|^{p-2}u+|u|^{q-2}u)=K(\epsilon x)f(u),&{}\quad \hbox {in}~\mathbb {R}^{N},\\ u\in W^{1,p}(\mathbb {R}^{N})\cap W^{1,q}(\mathbb {R}^{N}),u>0, &{}\quad \hbox {in}~\mathbb {R}^{N}.\\ \end{array} \right. \end{aligned}$$

(2.1)

We observe that if u is a solution of problem (2.1), then $v(x):=u(x/\epsilon )$ is a solution of problem (1.1). Thus, to study the original problem (1.1), next we will study the equivalent problem (2.1).

Now we establish the variational framework of problem (2.1). For any fixed $\epsilon >0$, we introduce the following working space

$$\begin{aligned} E_{\epsilon }=\left\{ u\in W^{1,p}(\mathbb {R}^N)\cap W^{1,q}(\mathbb {R}^N): \int _{\mathbb {R}^N}V(\epsilon x)(|u|^p+|u|^q)\mathrm {d}x<\infty \right\} \end{aligned}$$

endowed with the norm

$$\begin{aligned} \Vert u\Vert _{\epsilon }=\Vert u\Vert _{V_{\epsilon },p}+\Vert u\Vert _{V_{\epsilon },q}, \end{aligned}$$

where

$$\begin{aligned} \Vert u\Vert _{V_{\epsilon },s}^{s}=\int _{\mathbb {R}^N}(|\nabla u|^{s}+V(\epsilon x)|u|^{s})\mathrm {d}x\quad \hbox {for all}~s>1. \end{aligned}$$

Since V is a bounded function, then $\Vert \cdot \Vert _{\epsilon }$ and the norm of $W^{1,p}(\mathbb {R}^N)\cap W^{1,q}(\mathbb {R}^N)$ are equivalent. So, $E_{\epsilon }=E:=W^{1,p}(\mathbb {R}^N)\cap W^{1,q}(\mathbb {R}^N)$.

Now, let us recall the following embedding property proved by Alves and Figueiredo, see [3] for more details.

Lemma 2.3

The space $E_{\epsilon }$ is embedded continuously into $L^s(\mathbb {R}^N)$ for $s\in [p,q^*]$ and compactly into $L_{loc}^s(\mathbb {R}^N)$ for $s\in [1,q^*)$. Moreover, there is a positive constant $\pi _{s}$ such that

$$\begin{aligned} \pi _{s}\Vert u\Vert _{s}\le \Vert u\Vert _{\epsilon },\quad \hbox {for all}~s\in [p,q^{*}]. \end{aligned}$$

(2.2)

In order to study problem (2.1), we define the energy functional on $E_\epsilon$

$$\begin{aligned} \Phi _{\epsilon }(u)=\frac{1}{p}\Vert \nabla u\Vert _{p}^{p}+\frac{1}{q}\Vert \nabla u\Vert _{q}^{q} +\int _{\mathbb {R}^N}V(\epsilon x)\left( \frac{1}{p}| u|^{p}+\frac{1}{q}| u|^{q}\right) \mathrm {d}x-\int _{\mathbb {R}^N}K(\epsilon x)F(u)\mathrm {d}x. \end{aligned}$$

Furthermore, using Lemma 2.3 and some standard arguments, we know that $\Phi _{\epsilon }\in C^{1}(E_{\epsilon },\mathbb {R})$ under the conditions $(f_1)$, $(f_2)$ and $(f_3)$, and critical points of $\Phi _{\epsilon }$ are weak solutions of problem (2.1). Also, for any $u,v\in E_{\epsilon }$, we have

$$\begin{aligned} \begin{aligned} \langle \Phi '_{\epsilon }(u),v\rangle&=\int _{\mathbb {R}^N}|\nabla u|^{p-2}\nabla u\cdot \nabla v\mathrm {d}x+\int _{\mathbb {R}^N}|\nabla u|^{q-2}\nabla u\cdot \nabla v\mathrm {d}x\\&\quad +\int _{\mathbb {R}^N}V(\epsilon x)(|u|^{p-2}u+|u|^{q-2}u)v\mathrm {d}x-\int _{\mathbb {R}^N}K(\epsilon x)f(u)v\mathrm {d}x. \end{aligned} \end{aligned}$$

According to $(f_{1})$, $(f_{2})$ and $(f_{3})$, for any $\varepsilon >0$, there exists $C_{\varepsilon }>0$ such that

$$\begin{aligned} |f(s)|\le \varepsilon |s|^{p-1}+C_{\varepsilon }|s|^{r-1}~\hbox {and}~|F(s)|\le \varepsilon |s|^p+C_{\varepsilon }|s|^r~\hbox {for any}~s\in \mathbb {R}. \end{aligned}$$

(2.3)

Moreover, from $(f_{4})$ and $(f_{5})$ we can infer that

$$\begin{aligned} F(s)>0\quad \hbox {and}\quad \frac{1}{q}f(s)s-F(s)>0,~\forall s>0. \end{aligned}$$

(2.4)

To prove the positive ground state solutions of problem (2.1), we consider the Nehari manifold related to $\Phi _{\epsilon }$

$$\begin{aligned} {\mathscr {N}}_{\epsilon }:=\{u\in E_{\epsilon }\setminus \{0\}: \langle \Phi _{\epsilon }'(u),u\rangle =0\}, \end{aligned}$$

deeply studied by Szulkin and Weth [34]. Obviously, ${\mathscr {N}}_{\epsilon }$ contains all nontrivial critical points of $\Phi _{\epsilon }$.

Let

$$\begin{aligned} c_{\epsilon }:=\inf _{{\mathscr {N}}_{\epsilon }}\Phi _{\epsilon }. \end{aligned}$$

If $c_{\epsilon }$ is attained by $u_{\epsilon }\in {\mathscr {N}}_{\epsilon }$, then $u_{\epsilon }$ is a critical point of $\Phi _{\epsilon }$. Since $c_{\epsilon }$ is the lowest level for $\Phi _{\epsilon }$, then $u_{\epsilon }$ is called a ground state solution of problem (2.1).

Employing Lemma 2.3 and using some standard arguments [6], one can check easily some elementary properties of the functional $\Phi _{\epsilon }$.

Lemma 2.4

Assume that $(f_1)$–$(f_5)$ hold, then $\Phi _{\epsilon }$ satisfies the following conditions:

(i):

$\Phi '_{\epsilon }$ maps bounded sets of $E_{\epsilon }$ into bounded sets of $E_\epsilon$;

(ii):

$\Phi '_{\epsilon }$ is weakly sequentially continuous in $E_{\epsilon }$.

Lemma 2.5

(mountain pass geometry) Assume that $(f_1)$–$(f_5)$ hold, then

(i):

there exist $\alpha$, $\varrho >0$ such that $\Phi _{\epsilon }(u)\ge \alpha$ with $\Vert u\Vert _{\epsilon }=\varrho$;

(ii):

there exist $u_0\in E_\epsilon$ and $R>0$ with $\Vert u_0\Vert _{\epsilon }>R$ such that $\Phi _\epsilon (u_0)<0$.

Proof

(i) Let $u\in E_\epsilon$, from (2.2) and (2.3), it is easy to obtain

$$\begin{aligned} \begin{aligned} \Phi _{\epsilon }(u)&=\frac{1}{p}\Vert u\Vert _{V_{\epsilon },p}^{p}+\frac{1}{q}\Vert u\Vert _{V_{\epsilon ,}q}^{q}-\int _{\mathbb {R}^N}K(\epsilon x)F(u)\mathrm {d}x\\&\ge \frac{1}{p}\Vert u\Vert _{V_{\epsilon },p}^{p}+\frac{1}{q}\Vert u\Vert _{V_{\epsilon ,}q}^{q}-\varepsilon K_{\max }\Vert u\Vert _{p}^{p}-C_{\varepsilon }K_{\max }\Vert u\Vert _{r}^{r}\\&\ge c_{1}\Vert u\Vert _{V_{\epsilon },p}^{p}+\frac{1}{q}\Vert u\Vert _{V_{\epsilon ,}q}^{q}-C_{\varepsilon }K_{\max }\Vert u\Vert _{r}^{r}\\&\ge c_{1}\Vert u\Vert _{V_{\epsilon },p}^{p}+\frac{1}{q}\Vert u\Vert _{V_{\epsilon ,}q}^{q}-C_{\varepsilon }K_{\max }\pi _{r}^{-r}\Vert u\Vert _{\epsilon }^{r}. \end{aligned} \end{aligned}$$

(2.5)

We take $\varrho \in (0,1)$ with $\Vert u\Vert _{\epsilon }=\varrho$, then $\Vert u\Vert _{V_{\epsilon },p}<1$. Therefore, $\Vert u\Vert _{V_{\epsilon },p}^{p}\ge \Vert u\Vert _{V_{\epsilon },p}^{q}$ since $1<p<q$. According to (2.5) and the inequality: $a^{s}+b^{s}\ge c_{s}(a+b)^{s}$ for any $a,b\ge 0$ and $s>1$, we get

$$\begin{aligned} \begin{aligned} \Phi _{\epsilon }(u)&\ge c_{1}\Vert u\Vert _{V_{\epsilon },p}^{p}+\frac{1}{q}\Vert u\Vert _{V_{\epsilon ,}q}^{q}-C_{\varepsilon }K_{\max }\pi _{r}^{-r}\Vert u\Vert _{\epsilon }^{r}\\&\ge c_{1}\Vert u\Vert _{V_{\epsilon },p}^{q}+\frac{1}{q}\Vert u\Vert _{V_{\epsilon ,}q}^{q}-C_{\varepsilon }K_{\max }\pi _{r}^{-r}\Vert u\Vert _{\epsilon }^{r}\\&\ge c_{2}(\Vert u\Vert _{V_{\epsilon },p}+\Vert u\Vert _{V_{\epsilon ,}q})^{q}-C_{\varepsilon }K_{\max }\pi _{r}^{-r}\Vert u\Vert _{\epsilon }^{r}\\&=c_{2}\Vert u\Vert _{\epsilon }^{q}-C_{\varepsilon }K_{\max }\pi _{r}^{-r}\Vert u\Vert _{\epsilon }^{r}. \end{aligned} \end{aligned}$$

Since $q<r$, then there exist $\alpha , \varrho >0$ such that $\Phi _{\epsilon }(u)\ge \alpha >0$ for $\Vert u\Vert _{\epsilon }=\varrho$.

(ii) Let $e\in E_\epsilon \backslash \{0\}$, using $(f_4)$ we have

$$\begin{aligned} \Phi _{\epsilon }(te)=\frac{t^{p-q}}{p}\Vert e\Vert ^{p}_{V_\epsilon ,p}+\frac{1}{p}\Vert e\Vert ^{q}_{V_\epsilon ,q} -\int _{\mathbb {R}^N}K(\epsilon x)\frac{F(te)}{t^{q}}\mathrm {d}x\rightarrow -\infty \end{aligned}$$

as $t\rightarrow \infty$. Hence, there exist $R>\varrho$ and $t_0>0$ with $u_0=t_0e$ and $\Vert u_0\Vert _{\epsilon }>R$ such that $\Phi _{\epsilon }(u_{0})<0$. The proof is completed. $\square$

According to Lemma 2.5, we can use a version of mountain pass theorem without the Palais–Smale condition [40] to deduce the existence of a Palais–Smale sequence $\{u_n\}$ at level ${\tilde{c}}_{\epsilon }$, namely

$$\begin{aligned} \Phi _{\epsilon }(u_n)\rightarrow {\tilde{c}}_{\epsilon }\quad \hbox {and}\quad \Phi '_{\epsilon }(u_n)\rightarrow 0, \end{aligned}$$

where ${\tilde{c}}_{\epsilon }$ is the mountain pass level of $\Phi _{\epsilon }$ defined as

$$\begin{aligned} {\tilde{c}}_{\epsilon }=\inf _{\ell \in \Gamma }\max _{t\in [0,1]}\Phi _{\epsilon }(\ell (t)), \end{aligned}$$

and

$$\begin{aligned} \Gamma =\{\ell \in C([0,1], E_{\epsilon }): \ell (0)=0, \Phi _{\epsilon }(\ell (1))<0\}. \end{aligned}$$

Next, we are ready to prove some properties for ${\mathscr {N}}_{\epsilon }$, which will be used frequently in the sequel of the paper.

Lemma 2.6

${\mathscr {N}}_\epsilon$ is bounded away from 0. Moreover, ${\mathscr {N}}_\epsilon$ is closed in $E_\epsilon$.

Proof

We follow the idea of [34]. For any $u\in {\mathscr {N}}_{\epsilon }$, we infer from Lemma 2.1, (2.2) and (2.3) that

$$\begin{aligned} \begin{aligned} \Vert u\Vert _{V_\epsilon ,p}^{p}+\Vert u\Vert _{V_\epsilon ,q}^{q}&=\int _{\mathbb {R}^{N}}K(\epsilon x)f(u)u\mathrm {d}x\\&\le \varepsilon K_{\max }\Vert u\Vert _{p}^{p}+C_{\epsilon }K_{\max }\Vert u\Vert _{r}^{r}\\&\le \varepsilon c_{3}\Vert u\Vert _{V_\epsilon ,p}^{p}+C_{\epsilon }K_{\max }\pi _{r}^{-r}\Vert u\Vert _{\epsilon }^{r}. \end{aligned} \end{aligned}$$

According to the arbitrariness of $\varepsilon$, we can deduce that

$$\begin{aligned} c_{4}\Vert u\Vert _{V_\epsilon ,p}^{p}+\Vert u\Vert _{V_\epsilon ,q}^{q}\le c_{5}\Vert u\Vert _{\epsilon }^{r}. \end{aligned}$$

If $\Vert u\Vert _{\epsilon }\ge 1$, the conclusion is obvious. If $\Vert u\Vert _{\epsilon }<1$, then $\Vert u\Vert _{V_\epsilon ,p}^{p}\ge \Vert u\Vert _{V_\epsilon ,p}^{q}$ and we have

$$\begin{aligned} c_{5}\Vert u\Vert _{\epsilon }^{r}\ge c_{4}\Vert u\Vert _{V_\epsilon ,p}^{p}+\Vert u\Vert _{V_\epsilon ,q}^{q} \ge c_{4}\Vert u\Vert _{V_\epsilon ,p}^{q}+\Vert u\Vert _{V_\epsilon ,q}^{q}\ge c_{6}\Vert u\Vert _{\epsilon }^{q}, \end{aligned}$$

which implies that $\Vert u\Vert _{\epsilon }\ge \alpha _{0}$ for some $\alpha _{0}>0$.

Next, we verify that ${\mathscr {N}}_{\epsilon }$ is closed in $E_{\epsilon }$. Let $\{u_n\}\subset {\mathscr {N}}_{\epsilon }$ be a sequence such that $u_n\rightarrow u$ in $E_\epsilon$. According to Lemma 2.4, $\Phi '_{\epsilon }(u_n)$ is bounded. Then we can get

$$\begin{aligned} \langle \Phi '_{\epsilon }(u_n),u_n\rangle -\langle \Phi '_{\epsilon }(u),u\rangle =\langle \Phi '_{\epsilon }(u_n)-\Phi '_{\epsilon }(u_n),u\rangle +\langle \Phi '_{\epsilon }(u_n),u_n-u\rangle \rightarrow 0, \end{aligned}$$

this shows that $\langle \Phi '_{\epsilon }(u),u\rangle =0$. Using the above conclusion we deduce that $\Vert u\Vert _{\epsilon }\ge \alpha _{0}$, hence $u\in {\mathscr {N}}_{\epsilon }$. This completes the proof. $\square$

Lemma 2.7

Let $u\in E_{\epsilon }\backslash \{0\}$, then there exists a unique $t_u>0$ such that $t_uu\in {\mathscr {N}}_\epsilon$. Moreover, ${\hat{m}}_{\epsilon }(u)=t_uu$ is the unique global maximum of $\Phi _{\epsilon }$ on $\mathbb {R}^{+} u$. In particular, if $u\in {\mathscr {N}}_{\epsilon }$, then

$$\begin{aligned} \Phi _{\epsilon }(u)=\max _{t\ge 0}\Phi _{\epsilon }(tu)\ge \Phi _{\epsilon }(tu)~\hbox {for all}~t\ge 0. \end{aligned}$$

Proof

Let $u\in E_{\epsilon }\backslash \{0\}$, we define the function $g(t)=\Phi _{\epsilon }(tu)$ for $t>0$. Following the same arguments as in the proof of Lemma 2.5, we infer that $g(0)=0$, $g(t)>0$ for t sufficiently small and $g(t)<0$ for t sufficiently large. Therefore, there is $t=t_u$ such that $\max _{t>0}g(t)$ is achieved at $t_u$, so $g'(t_u)=0$ and $t_uu\in {\mathscr {N}}_\epsilon$.

Next, we claim that $t_u$ is the unique critical point of g. Assume by contradiction that there exist $t_1$ and $t_2$ with $0<t_1<t_2$ such that $t_1u,t_2u\in {\mathscr {N}}_\epsilon$, then it follows that

$$\begin{aligned} t_{1}^{p-q}\Vert u\Vert _{V_{\epsilon },p}^{p}+\Vert u\Vert _{V_{\epsilon },q}^{q}=\int _{\mathbb {R}^N}K(\epsilon x)\frac{f(t_1u)}{(t_1u)^{q-1}}u^q\mathrm {d}x \end{aligned}$$

and

$$\begin{aligned} t_{2}^{p-q}\Vert u\Vert _{V_{\epsilon },p}^{p}+\Vert u\Vert _{V_{\epsilon },q}^{q}=\int _{\mathbb {R}^N}K(\epsilon x)\frac{f(t_2u)}{(t_2u)^{q-1}}u^q\mathrm {d}x. \end{aligned}$$

Subtracting term by term in the above equalities, we get

$$\begin{aligned} (t_{1}^{p-q}-t_{2}^{p-q})\Vert u\Vert _{V_{\epsilon },p}^{p} =\int _{\mathbb {R}^N}K(\epsilon x)\left( \frac{f(t_1u)}{(t_1u)^{q-1}}-\frac{f(t_2u)}{(t_2u)^{q-1}}\right) u^q\mathrm {d}x. \end{aligned}$$

Using $(f_5)$ and recalling that $p<q$, we can deduce that

$$\begin{aligned} 0<(t_{1}^{p-q}-t_{2}^{p-q})\Vert u\Vert _{V_{\epsilon },p}^{p} =\int _{\mathbb {R}^N}K(\epsilon x)\left( \frac{f(t_1u)}{(t_1u)^{q-1}}-\frac{f(t_2u)}{(t_2u)^{q-1}}\right) u^q\mathrm {d}x<0, \end{aligned}$$

which implies a contradiction. The proof is completed. $\square$

Lemma 2.8

There exists $\lambda >0$ such that $t_u\ge \lambda$ for each $u\in S_{\epsilon }$, and for each compact subset $\mathcal {S}\subset S_{\epsilon }$, there exists $C_{\mathcal {S}}>0$ such that $t_u\le C_{\mathcal {S}}$ for all $u\in \mathcal {S}$, where $S_{\epsilon }=\{u\in E_\epsilon : \Vert u\Vert _{\epsilon }=1\}$.

Proof

For each $u\in S_{\epsilon }$, by using Lemmas 2.6 and 2.7, there exists $t_u>0$ such that $t_uu\in {\mathscr {N}}_{\epsilon }$, and $t_u=\Vert t_uu\Vert _{\epsilon }\ge \alpha _{0}$. It remains we prove that $t_u\le C_{\mathcal {S}}$ for all $u\in \mathcal {S}\subset S_{\epsilon }$. Arguing by contradiction we assume that there exist a sequence $\{u_n\}\subset \mathcal {S}\subset S_{\epsilon }$ and $\{t_n\}$ such that $t_n\rightarrow \infty$. Note that, since $\mathcal {S}$ is compact, there exists $u\in \mathcal {S}$ such that $u_n\rightarrow u$ in $E_\epsilon$. Similar to the proof of Lemma 2.5, we can see that $\Phi _{\epsilon }(t_nu_n)\rightarrow -\infty$. However, for any $u\in {\mathscr {N}}_{\epsilon }$, using (2.4) we have

$$\begin{aligned} \begin{aligned} \Phi _{\epsilon }(u)&=\Phi _{\epsilon }(u)-\frac{1}{q}\langle \Phi '_{\epsilon }(u),u\rangle \\&=\left( \frac{1}{p}-\frac{1}{q}\right) \Vert u\Vert _{V_{\epsilon },p}^{p}+\int _{\mathbb {R}^N}K(\epsilon x)\left( \frac{1}{q}f(u)u-F(u)\right) \mathrm {d}x>0. \end{aligned} \end{aligned}$$

So, the conclusion $\Phi _{\epsilon }(t_nu_n)\rightarrow -\infty$ is impossible, a contradiction. $\square$

Combining with Lemmas 2.5, 2.7 and 2.8, the ground state energy value $c_{\epsilon }$ has a minimax characterization given by

$$\begin{aligned} c_{\epsilon }={\tilde{c}}_{\epsilon }=\inf _{u\in E_{\epsilon }\backslash \{0\}}\max _{t\ge 0}\Phi _{\epsilon }(tu) =\inf _{u\in S_{\epsilon }}\max _{t\ge 0}\Phi _{\epsilon }(tu). \end{aligned}$$

(2.6)

The proof can be found in [6, 34] and [40], here we omit the details.

Lemma 2.9

(i) There exists $\alpha >0$ independent of $\epsilon$ such that $c_{\epsilon }\ge \alpha >0$.

(ii) The mapping $\Phi _{\epsilon }$ is coercive on ${\mathscr {N}}_{\epsilon }$, i.e., $\Phi _{\epsilon }(u)\rightarrow \infty$ as $\Vert u\Vert _{\epsilon }\rightarrow \infty$, $u\in {\mathscr {N}}_{\epsilon }$.

Proof

(i) Let $u\in {\mathscr {N}}_{\epsilon }$, according to the proof of Lemma 2.5-(i) we infer that there exists $\alpha >0$ independent of $\epsilon$ such that $\Phi _{\epsilon }(tu)\ge \alpha >0$ for $t>0$ small. Moreover, in view of (2.6) we can get $c_{\epsilon }\ge \alpha >0$. So the conclusion (i) holds.

(ii) Arguing by contradiction, we assume that there exists a sequence $\{u_n\}\subset {\mathscr {N}}_{\epsilon }$ such that $\Vert u_n\Vert _{\epsilon }\rightarrow \infty$ and $\Phi _\epsilon (u_n)\le d$ for some $d\in [\alpha , \infty )$. Let $v_n=u_n/\Vert u_n\Vert _{\epsilon }$, then $\Vert v_n\Vert _{\epsilon }=1$ and $\max \{\Vert v_n\Vert _{V_{\epsilon },p}, \Vert v_n\Vert _{V_{\epsilon },q}\}<1$. Passing to a subsequence, we can assume that $v_n\rightharpoonup v$ in $E_{\epsilon }$, and $v_n(x)\rightarrow v(x)$ a.e. in $\mathbb {R}^N$.

There are two cases need to discuss: $\{v_n\}$ either is vanishing, i.e.,

$$\begin{aligned} \lim _{n\rightarrow \infty }\sup _{y\in \mathbb {R}^N}\int _{B_{R}(y)}|v_{n}|^{q}\mathrm {d}x=0,\quad \forall R>0, \end{aligned}$$

or non-vanishing, i.e., there exist $R_{0}, \delta >0$ and a sequence $y_{n}\in \mathbb {R}^N$ such that

$$\begin{aligned} \lim _{n\rightarrow \infty }\int _{B_{R_{0}}(y_{n})}|v_{n}|^{q}\mathrm {d}x\ge \delta >0. \end{aligned}$$

If $\{v_n\}$ is vanishing, then Lemma 2.2 implies that $v_n\rightarrow 0$ in $L^s(\mathbb {R}^N)$ for $s\in (q,q^*)$. From (2.3), for any t satisfying $t^{q-p}>q/p$, we have

$$\begin{aligned} \int _{\mathbb {R}^N}K(\epsilon x)F(tv_n)\mathrm {d}x\rightarrow 0. \end{aligned}$$

(2.7)

Observe that, according to Lemma 2.7 and (2.7) we get

$$\begin{aligned} \begin{aligned} d\ge \Phi _{\epsilon }(u_n)\ge \Phi _{\epsilon }(tv_n)&=\frac{t^{p}}{p}\Vert v_n\Vert _{V_{\epsilon },p}^{p}+\frac{t^{q}}{q}\Vert v_n\Vert _{V_{\epsilon },q}^{q} -\int _{\mathbb {R}^N}K(\epsilon x)F(tv_n)\mathrm {d}x\\&\ge \frac{t^{p}}{p}\left( \Vert v_n\Vert _{V_{\epsilon },p}^{p}+\Vert v_n\Vert _{V_{\epsilon },q}^{q}\right) -\int _{\mathbb {R}^N}K(\epsilon x)F(tv_n)\mathrm {d}x\\&\ge \frac{t^{p}}{p}\left( \Vert v_n\Vert _{V_{\epsilon },p}^{q}+\Vert v_n\Vert _{V_{\epsilon },q}^{q}\right) -\int _{\mathbb {R}^N}K(\epsilon x)F(tv_n)\mathrm {d}x\\&\ge \frac{c_{q}t^{p}}{p}\left( \Vert v_n\Vert _{V_{\epsilon },p}+\Vert v_n\Vert _{V_{\epsilon },q}\right) ^{q}-\int _{\mathbb {R}^N}K(\epsilon x)F(tv_n)\mathrm {d}x\\&\ge \frac{c_{q}t^{p}}{p}\Vert v_n\Vert _{\epsilon }^{q}-\int _{\mathbb {R}^N}K(\epsilon x)F(tv_n)\mathrm {d}x\rightarrow \frac{c_{q}t^{p}}{p}. \end{aligned} \end{aligned}$$

Evidently, this is a contradiction if t is large enough. So the vanishing case does not occur.

Assume that $\{v_n\}$ is non-vanishing. Let us define ${\tilde{v}}_{n}(x)=v_n(x+y_n)$, then

$$\begin{aligned} \int _{B_{R_{0}}(0)}|{\tilde{v}}_{n}|^{q}\mathrm {d}x\ge \frac{\delta }{2}. \end{aligned}$$

Passing to a subsequence, we may assume that ${\tilde{v}}_{n}\rightarrow {\tilde{v}}$ in $L_{loc}^{q}(\mathbb {R}^N)$ with ${\tilde{v}}\ne 0$. Set $\Omega :=\{x\in \mathbb {R}^{N}: {\tilde{v}}(x)\ne 0\}$. Then $|\Omega |>0$ and for each $x\in \Omega$, $|u_{n}(x+y_{n})|=|{\tilde{v}}_{n}(x)|\Vert u_{n}\Vert _{\epsilon }\rightarrow \infty$. Hence, using $(A_{0})$, $(f_{4})$, Fatou’s lemma and conclusion-(i) we can deduce that

$$\begin{aligned} \begin{aligned} 0\le \frac{\Phi _{\epsilon }(u_n)}{\Vert u_n\Vert _{\epsilon }^{q}}&=\frac{1}{p}\frac{\Vert u_n\Vert _{V_{\epsilon },p}^{p}}{\Vert u_n\Vert _{\epsilon }^{q}} +\frac{1}{q}\frac{\Vert u_n\Vert _{V_{\epsilon },q}^{q}}{\Vert u_n\Vert _{\epsilon }^{q}}-\int _{\mathbb {R}^N}K(\epsilon x)\frac{F(u_n)}{\Vert u_n\Vert _{\epsilon }^{q}}\mathrm {d}x\\&\le \frac{1}{p}+\frac{1}{q}-K_{\min }\int _{\mathbb {R}^N}\frac{F(u_n)}{\Vert u_n\Vert _{\epsilon }^{q}}\mathrm {d}x\\&\le \frac{1}{p}+\frac{1}{q}-K_{\min }\int _{\mathbb {R}^N}\frac{F(u_n(x+y_n))}{|u_n(x+y_n)|^{q}}|{\tilde{v}}_n|^{q}\mathrm {d}x\\&\rightarrow -\infty . \end{aligned} \end{aligned}$$

This yields a concentration. So the non-vanishing case does not occur, and we finish the proof. $\square$

We introduce the following crucial results from [34], with which we can use the Nehari method developed by Szulkin and Weth [34].

Lemma 2.10

The map ${\hat{m}}_{\epsilon }: E_\epsilon \backslash \{0\}\rightarrow {\mathscr {N}}_{\epsilon }$ is continuous, and the map $m_{\epsilon }:={\hat{m}}_{\epsilon }|_{S_{\epsilon }}:S_\epsilon \rightarrow {\mathscr {N}}_{\epsilon }$ is a homeomorphism between $S_{\epsilon }$ and ${\mathscr {N}}_{\epsilon }$ with inverse given by

$$\begin{aligned} {\check{m}}_{\epsilon }:{\mathscr {N}}_{\epsilon }\rightarrow S_{\epsilon },\quad {\check{m}}_{\epsilon }(u)=u/\Vert u\Vert . \end{aligned}$$

Proof

We adapt some ideas found in [34]. Suppose that $u_n\rightarrow u\ne 0$. Since ${\hat{m}}_{\epsilon }(su)={\hat{m}}_{\epsilon }(u)$ for each $s>0$, we may assume $u_n\in S_\epsilon$ for all n and it suffices to show that ${\hat{m}}_{\epsilon }(u_{n})\rightarrow {\hat{m}}_{\epsilon }(u)$ after passing to a subsequence. According to Lemma 2.7, ${\hat{m}}_{\epsilon }(u_{n})=t_{u_n}u_n$. It follows from Lemma 2.8 that $\{t_{u_n}\}$ is bounded and bounded away from 0, hence, taking a subsequence, $t_{u_n} \rightarrow t_0>0$. By Lemma 2.6, ${\mathscr {N}}_{\epsilon }$ is closed and ${\hat{m}}_{\epsilon }(u_{n})\rightarrow t_0u$ and $t_0u\in {\mathscr {N}}_{\epsilon }$. Hence $t_0u=t_uu={\hat{m}}_{\epsilon }(u)$. From the above proof, the second conclusion is an immediate consequence. $\square$

Based on Lemma 2.10, we now consider the functional ${\hat{I}}_{\epsilon }:E_{\epsilon }\backslash \{0\}\rightarrow \mathbb {R}$ and the restriction $I_{\epsilon }: S_{\epsilon }\rightarrow \mathbb {R}$ as follows

$$\begin{aligned} {\hat{I}}_{\epsilon }(u)=\Phi _{\epsilon }({\hat{m}}_{\epsilon }(u))\quad \hbox {and}\quad I_{\epsilon }={\hat{I}}_{\epsilon }|_{S_{\epsilon }}. \end{aligned}$$

Moreover, Lemma 2.10 shows that ${\hat{I}}_{\epsilon }$ is continuous.

Lemma 2.11

The following conclusions hold:

(i):

${\hat{I}}_{\epsilon }\in C^{1}(E_{\epsilon }\backslash \{0\},\mathbb {R})$ and for $u,v\in E_{\epsilon }$ and $u\ne 0$,

$$\begin{aligned} \langle {\hat{I}}'_{\epsilon }(u),v\rangle =\frac{\Vert {\hat{m}}_{\epsilon }(u)\Vert _{\epsilon }}{\Vert u\Vert _{\epsilon }} \langle \Phi '_{\epsilon }({\hat{m}}_{\epsilon }(u)),v\rangle . \end{aligned}$$

(ii):

$I_{\epsilon }\in C^{1}(S_{\epsilon },\mathbb {R})$ and $\langle I'_{\epsilon }(u),v\rangle =\Vert {\hat{m}}_{\epsilon }(u)\Vert \langle \Phi '_{\epsilon }({\hat{m}}_{\epsilon }(u)),v\rangle$ for $v\in T_{u}(S_{\epsilon })$, where $T_{u}(S_{\epsilon })$ is the tangent space of $S_{\epsilon }$ at u.

(iii):

$\{u_n\}$ is a Palais–Smale sequence for $I_{\epsilon }$ if and only if $\{{\hat{m}}_{\epsilon }(u_n)\}$ is a Palais–Smale sequence for $\Phi _{\epsilon }$.

(iv):

$u\in S_{\epsilon }$ is a critical point of $I_{\epsilon }$ if and only if ${\hat{m}}_{\epsilon }(u)\in {\mathscr {N}}_{\epsilon }$ is a critical point of $\Phi _{\epsilon }$. Moreover, the corresponding values of $I_{\epsilon }$ and $\Phi _{\epsilon }$ coincide and

$$\begin{aligned} \inf _{S_{\epsilon }}I_{\epsilon }=\inf _{{\mathscr {N}}_{\epsilon }}\Phi _{\epsilon }=c_{\epsilon }. \end{aligned}$$

3 The constant coefficients problem

For our scope, we shall also investigate the corresponding limit problem. To this end, we first discuss in this section the existence and some properties of the positive ground state solutions of the constant coefficients problem.

For any $\mu >0$ and $\kappa >0$, we consider the constant coefficients problem

$$\begin{aligned} \left\{ \begin{array}{ll} -\Delta _{p} u-\Delta _{q} u +\mu (|u|^{p-2}u+|u|^{q-2}u)=\kappa f(u),&{}\quad \hbox {in}~\mathbb {R}^{N},\\ u\in W^{1,p}(\mathbb {R}^{N})\cap W^{1,q}(\mathbb {R}^{N}),u>0, &{}\quad \hbox {in}~\mathbb {R}^{N}.\\ \end{array} \right. \end{aligned}$$

(3.1)

Now we define the following working space

$$\begin{aligned} E_{\mu }=\left\{ u\in W^{1,p}(\mathbb {R}^N)\cap W^{1,q}(\mathbb {R}^N): \int _{\mathbb {R}^N}\mu (|u|^p+|u|^q)\mathrm {d}x<\infty \right\} \end{aligned}$$

endowed with the norm

$$\begin{aligned} \Vert u\Vert _{\mu }=\Vert u\Vert _{\mu ,p}+\Vert u\Vert _{\mu ,q}, \end{aligned}$$

where

$$\begin{aligned} \Vert u\Vert _{\mu ,s}^{s}=\int _{\mathbb {R}^N}(|\nabla u|^{s}+\mu |u|^{s})\mathrm {d}x\quad \hbox {for all}~s>1. \end{aligned}$$

It is well known that the solutions of problem (3.1) are critical points of the functional

$$\begin{aligned} \mathcal {J}_{\mu \kappa }(u)=\frac{1}{p}\Vert \nabla u\Vert _{p}^{p}+\frac{1}{q}\Vert \nabla u\Vert _{q}^{q} +\mu \int _{\mathbb {R}^N}\left( \frac{1}{p}|u|^{p}+\frac{1}{q}|u|^{q}\right) \mathrm {d}x-\kappa \int _{\mathbb {R}^N}F(u)\mathrm {d}x. \end{aligned}$$

Clearly, $\mathcal {J}_{\mu \kappa }\in C^{1}(E_{\mu },\mathbb {R})$ and its differential is given by

$$\begin{aligned} \begin{aligned} \langle \mathcal {J}'_{\mu \kappa }(u),v\rangle =&\int _{\mathbb {R}^N}|\nabla u|^{p-2}\nabla u\cdot \nabla v\mathrm {d}x+\int _{\mathbb {R}^N}|\nabla u|^{q-2}\nabla u\cdot \nabla v\mathrm {d}x\\&+\mu \int _{\mathbb {R}^N}(|u|^{p-2}u+|u|^{q-2}u)v\mathrm {d}x-\kappa \int _{\mathbb {R}^N}f(u)v\mathrm {d}x \end{aligned} \end{aligned}$$

for any $u,v\in E_{\mu }$. The corresponding Nehari manifold is defined by

$$\begin{aligned} {\mathscr {N}}_{\mu \kappa }:=\{u\in E_{\mu }\setminus \{0\}: \langle \mathcal {J}_{\mu \kappa }'(u),u\rangle =0\}. \end{aligned}$$

Similarly, we define the ground state energy level on ${\mathscr {N}}_{\mu \kappa }$

$$\begin{aligned} c_{\mu \kappa }=\inf _{{\mathscr {N}}_{\mu \kappa }}\mathcal {J}_{\mu \kappa }. \end{aligned}$$

Moreover, as before, define the mapping

$$\begin{aligned} {\hat{m}}_{\mu \kappa }:E_{\mu }\backslash \{0\}\rightarrow {\mathscr {N}}_{\mu \kappa } \quad \hbox {and} \quad m_{\mu \kappa }={\hat{m}}_{\mu \kappa }|_{S}: S\rightarrow {\mathscr {N}}_{\mu \kappa }, \end{aligned}$$

and the inverse of $m_{\mu \kappa }$ is given by

$$\begin{aligned} {\check{m}}_{\mu \kappa }:{\mathscr {N}}_{\mu \kappa }\rightarrow S,\quad {\check{m}}_{\mu \kappa }(u)=u/\Vert u\Vert _{\mu }. \end{aligned}$$

According to arguments in Sect. 2, we can verify that $\mathcal {J}_{\mu \kappa }$, ${\mathscr {N}}_{\mu \kappa }$ and $c_{\mu \kappa }$ have properties similar to those of $\Phi _{\epsilon }$, ${\mathscr {N}}_{\epsilon }$ and $c_{\epsilon }$. Moreover, all related Lemmas in Sect. 2 still hold for the constant coefficients problem (3.1). So we collect some relevant results for problem (3.1).

Lemma 3.1

The following conclusions hold:

(a):

${\mathscr {N}}_{\mu \kappa }$ is bounded away from 0. Moreover, ${\mathscr {N}}_{\mu \kappa }$ is closed in $E_{\mu }$.

(b):

For all $u\in E_{\mu }\backslash \{0\}$, there exists a unique $t_u>0$ such that $t_uu\in {\mathscr {N}}_{\mu \kappa }$. Moreover, ${\hat{m}}_{\mu \kappa }(u)=t_uu$ is the unique global maximum of $\mathcal {J}_{\mu \kappa }$ on $\mathbb {R}^{+} u$.

(c):

There exists $\lambda >0$ such that $t_u\ge \lambda$ for each $u\in S$, and for each compact subset $\mathcal {S}\subset S$, there exists $C_{\mathcal {S}}>0$ such that $t_u\le C_{\mathcal {S}}$ for all $u\in \mathcal {S}$.

(d):

$c_{\mu \kappa }>0$ and $\mathcal {J}_{\mu \kappa }$ has positive bounded below on ${\mathscr {N}}_{\mu \kappa }$.

(e):

$\mathcal {J}_{\mu \kappa }$ is coercive on ${\mathscr {N}}_{\mu \kappa }$, i.e., $\mathcal {J}_{\mu \kappa }(u)\rightarrow \infty$ as $\Vert u\Vert _{\mu }\rightarrow \infty$, $u\in {\mathscr {N}}_{\mu \kappa }$.

Define the functional ${\hat{I}}_{\mu \kappa }:E_{\mu }\backslash \{0\}\rightarrow \mathbb {R}$ and the restriction $I_{\mu \kappa }: S\rightarrow \mathbb {R}$ as follows

$$\begin{aligned} {\hat{I}}_{\mu \kappa }(u)=\mathcal {J}_{\mu \kappa }({\hat{m}}_{\mu \kappa }(u))\quad \hbox {and}\quad I_{\mu \kappa }={\hat{I}}_{\mu \kappa }|_{S}. \end{aligned}$$

According to (2.6), we also have

$$\begin{aligned} c_{\mu \kappa }=\inf _{u\in E_{\mu }\backslash \{0\}}\max _{t\ge 0}\mathcal {J}_{\mu \kappa }(tu) =\inf _{u\in S}\max _{t\ge 0}\mathcal {J}_{\mu \kappa }(tu). \end{aligned}$$

Moreover, we also have the following results.

Lemma 3.2

The following conclusions hold:

(a):

$I_{\mu \kappa }\in C^{1}(S,\mathbb {R})$ and $\langle I'_{\mu \kappa }(u),v\rangle =\Vert {\hat{m}}_{\mu \kappa }(u)\Vert \langle \mathcal {J}'_{\mu \kappa }({\hat{m}}_{\mu \kappa }(u)),v\rangle$ for $v\in T_{u}(S)$, where $T_{u}(S)$ is the tangent space of S at u.

(b):

$\{u_n\}$ is a Palais–Smale sequence for $I_{\mu \kappa }$ if and only if $\{{\hat{m}}_{\mu \kappa }(u_n)\}$ is a Palais–Smale sequence for $\mathcal {J}_{\mu \kappa }$.

(c):

$u\in S$ is a critical point of $I_{\mu \kappa }$ if and only if ${\hat{m}}_{\mu \kappa }(u)\in {\mathscr {N}}_{\mu \kappa }$ is a critical point of $\mathcal {J}_{\mu \kappa }$. Moreover, the corresponding values of $I_{\mu \kappa }$ and $\mathcal {J}_{\mu \kappa }$ coincide and

$$\begin{aligned} c_{\mu \kappa }=\inf _{{\mathscr {N}}_{\mu \kappa }}\mathcal {J}_{\mu \kappa }=\inf _{S}I_{\mu \kappa }. \end{aligned}$$

We now state the main result for problem (3.1).

Lemma 3.3

Assume that $\mu , \kappa >0$ and $(f_{1})$–$(f_{5})$ hold. Then problem (3.1) has at least one positive ground state solution u such that $\mathcal {J}_{\mu \kappa }(u)=c_{\mu \kappa }$.

Proof

Firstly, in view of Lemma 3.1-(d) , we can derive $c_{\mu \kappa }>0$. If $u\in {\mathscr {N}}_{\mu \kappa }$ satisfies $\mathcal {J}_{\mu \kappa }(u)=c_{\mu \kappa }$, then ${\check{m}}_{\mu \kappa }(u)\in S$ is a minimizer of $I_{\mu \kappa }$, and hence a critical point of $I_{\mu \kappa }$. Therefore, Lemma 3.2 shows that u is a critical point of $\mathcal {J}_{\mu \kappa }$. It remains to show that there exists a minimizer ${\tilde{u}}\in {\mathscr {N}}_{\mu \kappa }$ such that $\mathcal {J}_{\mu \kappa }({\tilde{u}})=c_{\mu \kappa }$. In fact, using Ekeland’s variational principle [40], there exists a sequence $\{v_{n}\}\subset S$ such that $I_{\mu \kappa }(v_n)\rightarrow c_{\mu \kappa }$ and $I'_{\mu \kappa }(v_n)\rightarrow 0$ as $n\rightarrow \infty$. Let $u_n={\hat{m}}_{\mu \kappa }(v_n)\in {\mathscr {N}}_{\mu \kappa }$ for all $n\in \mathbb {N}$. Then using Lemma 3.2 again, we can get $\mathcal {J}_{\mu \kappa }(u_n)\rightarrow c_{\mu \kappa }$ and $\mathcal {J}'_{\mu \kappa }(u_n)\rightarrow 0$. By Lemma 3.1-(e), we know that $\{u_n\}$ is bounded in $E_{\mu }$, and $\Vert u_{n}\Vert _{\mu }\ge \alpha _{0}$ for some $\alpha _{0}>0$ by Lemma 3.1-(a). Moreover, we have

$$\begin{aligned} \varliminf _{n\rightarrow \infty }\sup \limits _{y\in {{\mathbb {R}}}^{N}} \int _{B_{R}(y)}|u_{n}|^{q}\mathrm {d}x>0. \end{aligned}$$

Otherwise, from Lemma 2.2 we have $u_n\rightarrow 0$ in $L^{s}(\mathbb {R}^N)$ for any $s\in (q,q^{*})$. Using the fact that $u_n\in {\mathscr {N}}_{\mu \kappa }$ and (2.3) we can infer that

$$\begin{aligned} \begin{aligned} 0=\langle \mathcal {J}_{\mu \kappa }'(u_{n}),u_{n}\rangle&=\Vert \nabla u_n\Vert _{p}^{p}+\Vert \nabla u_n\Vert _{q}^{q}+\mu (\Vert u_n\Vert _{p}^{p}+\Vert u_n\Vert _{q}^{q})- \kappa \int _{\mathbb {R}^N}f(u_n)u_n\mathrm {d}x\\&\ge \Vert \nabla u_n\Vert _{p}^{p}+\Vert \nabla u_n\Vert _{q}^{q}+\mu (\Vert u_n\Vert _{p}^{p}+\Vert u_n\Vert _{q}^{q})- \varepsilon \kappa \Vert u_n\Vert _{p}^{p}-C_{\varepsilon }\kappa \Vert u_n\Vert _{r}^{r}\\&\ge c_{7}\Vert u_n\Vert _{\mu ,p}^{p}+c_{8}\Vert u_n\Vert _{\mu ,q}^{q}-c_{9}\Vert u_n\Vert _{r}^{r}, \end{aligned} \end{aligned}$$

and consequently

$$\begin{aligned} c_{7}\Vert u_n\Vert _{\mu ,p}^{p}+c_{8}\Vert u_n\Vert _{\mu ,q}^{q}\le c_{9}\Vert u_n\Vert _{r}^{r}\rightarrow 0 \end{aligned}$$

since $r\in (q,q^*)$. Then, we have $\Vert u_{n}\Vert _{\mu }\rightarrow 0$, which contradicts with $\Vert u_{n}\Vert _{\mu }\ge \alpha _{0}$. Therefore, there are $\delta >0$ and $\{k_n\}\subset \mathbb {Z}^{N}$ such that

$$\begin{aligned} \int _{B_{R}(k_{n})}|u_n|^{q}\mathrm {d}x\ge \delta . \end{aligned}$$

Let us define ${\tilde{u}}_n(x)=u_{n}(x+k_n)$ so that

$$\begin{aligned} \int _{B_{R}(0)}|{\tilde{u}}_n|^{q}\mathrm {d}x\ge \delta . \end{aligned}$$

(3.2)

Since problem (3.1) is autonomous, then $\mathcal {J}_{\mu \kappa }$ possesses translation invariance, and we can obtain $\Vert {\tilde{u}}_n\Vert _{\mu }=\Vert u_n\Vert _{\mu }$ and

$$\begin{aligned} \mathcal {J}_{\mu \kappa }({\tilde{u}}_n)\rightarrow c_{\mu \kappa }~~\hbox {and}~~\mathcal {J}_{\mu \kappa }'({\tilde{u}}_n)\rightarrow 0. \end{aligned}$$

(3.3)

Passing to a subsequence, we assume that ${\tilde{u}}_n \rightharpoonup {\tilde{u}}$ in $E_{\mu }$, ${\tilde{u}}_n \rightarrow {\tilde{u}}$ in $L_{loc}^{s}(\mathbb {R}^N)$ for $s\in (q,q^{*})$, and ${\tilde{u}}_n(x) \rightarrow {\tilde{u}}(x)$ a.e. in $\mathbb {R}^{N}$. Therefore, from Lemma 2.4, (3.2) and (3.3) we infer that ${\tilde{u}}\ne 0$ and $\mathcal {J}_{\mu \kappa }'({\tilde{u}})=0$. This shows that ${\tilde{u}}\in {\mathscr {N}}_{\mu \kappa }$ and $\mathcal {J}_{\mu \kappa }({\tilde{u}})\ge c_{\mu \kappa }$. On the other hand, using Fatou’s lemma and (2.4), we get

$$\begin{aligned} \begin{aligned} c_{\mu \kappa }&= \lim _{n\rightarrow \infty }\left( \mathcal {J}_{\mu \kappa }({\tilde{u}}_n)-\frac{1}{q}\langle \mathcal {J}_{\mu \kappa }'({\tilde{u}}_n),{\tilde{u}}_n\rangle \right) \\&=\lim _{n\rightarrow \infty }\left[ \left( \frac{1}{p}-\frac{1}{q}\right) \Vert {\tilde{u}}_n\Vert _{\mu ,p}^{p} +\kappa \int _{\mathbb {R}^{N}}\left( \frac{1}{q}f({\tilde{u}}_n){\tilde{u}}_n-F({\tilde{u}}_n)\right) \mathrm {d}x\right] \\&\ge \left( \frac{1}{p}-\frac{1}{q}\right) \Vert {\tilde{u}}\Vert _{\mu ,p}^{p} +\kappa \int _{\mathbb {R}^{N}}\left( \frac{1}{q}f({\tilde{u}}){\tilde{u}}-F({\tilde{u}})\right) \mathrm {d}x\\&=\mathcal {J}_{\mu \kappa }({\tilde{u}})-\frac{1}{q}\langle \mathcal {J}_{\mu \kappa }'({\tilde{u}}),{\tilde{u}}\rangle =\mathcal {J}_{\mu \kappa }({\tilde{u}}), \end{aligned} \end{aligned}$$

which implies that $\mathcal {J}_{\mu \kappa }({\tilde{u}})\le c_{\mu \kappa }$. Hence $\mathcal {J}_{\mu \kappa }({\tilde{u}})= c_{\mu \kappa }$ and ${\tilde{u}}$ is critical point of $\mathcal {J}_{\mu \kappa }$, which shows that ${\tilde{u}}$ is a ground state solution of problem (3.1).

Next, we verify that the ground state solution ${\tilde{u}}$ is positive. In fact, taking ${\tilde{u}}^{-}=\min \{{\tilde{u}},0\}$ as test function in problem (3.1), and applying $(f_1)$ and the following inequality

$$\begin{aligned} |a-b|^{s-2}(a-b)(a^{-}-b^{-})\ge |a^{-}-b^{-}|^{s}\quad \hbox {for all}~s>1, \end{aligned}$$

we can obtain

$$\begin{aligned} \begin{aligned} \Vert {\tilde{u}}^-\Vert _{\mu ,p}^{p}+\Vert {\tilde{u}}^-\Vert _{\mu ,q}^{q}&\le \int _{\mathbb {R}^N}|\nabla {\tilde{u}}|^{p-2}\nabla {\tilde{u}}\nabla {\tilde{u}}^-\mathrm {d}x+\mu \int _{\mathbb {R}^N}|{\tilde{u}}|^{p-2}{\tilde{u}}{\tilde{u}}^-\mathrm {d}x\\&\quad +\int _{\mathbb {R}^N}|\nabla {\tilde{u}}|^{q-2}\nabla {\tilde{u}}\nabla {\tilde{u}}^-\mathrm {d}x+\mu \int _{\mathbb {R}^N}|{\tilde{u}}|^{q-2}{\tilde{u}}{\tilde{u}}^-\mathrm {d}x\\&=\kappa \int _{\mathbb {R}^N}f({\tilde{u}}){\tilde{u}}^-\mathrm {d}x=0. \end{aligned} \end{aligned}$$

This shows that ${\tilde{u}}^-=0$, that is ${\tilde{u}}\ge 0$ in $\mathbb {R}^N$. By the regularity results in [21], we know that ${\tilde{u}}\in L^{\infty }(\mathbb {R}^N)\cap C_{loc}^{1,\sigma }(\mathbb {R}^N)$. Now we can apply Harnack’s inequality in [36] to conclude that ${\tilde{u}}>0$ in $\mathbb {R}^N$. $\square$

At the end of this section, we establish the following lemma which describes a comparison between the ground state energy values for different parameters and is crucial in the process of seeking for the existence and concentration of ground state solutions.

Lemma 3.4

Assume that $\mu _{i}>0$ and $\kappa _{i}>0$ for $i=1,2$, with $\min \{\mu _{2}-\mu _{1}, \kappa _{1}-\kappa _{2}\}\ge 0$. Then $c_{\mu _{1}\kappa _{1}}\le c_{\mu _{2}\kappa _{2}}$. Additionally, if $\max \{\mu _{2}-\mu _{1}, \kappa _{1}-\kappa _{2}\}>0$, then $c_{\mu _{1}\kappa _{1}}<c_{\mu _{2}\kappa _{2}}$. In particular, we have $c_{\mu _{1}\kappa _{i}}<c_{\mu _{2}\kappa _{i}}$ if $\mu _{1}<\mu _{2}$, and $c_{\mu _{i}\kappa _{1}}>c_{\mu _{i}\kappa _{2}}$ if $\kappa _{1}<\kappa _{2}$.

Proof

Let $u\in {\mathscr {N}}_{\mu _{2}\kappa _{2}}$ with $\mathcal {J}_{\mu _{2}\kappa _{2}}(u)=c_{\mu _{2}\kappa _{2}}$, then, Lemma 3.1-(b) implies that

$$\begin{aligned} c_{\mu _{2}\kappa _{2}}=\mathcal {J}_{\mu _{2}\kappa _{2}}(u)=\max _{t\ge 0}\mathcal {J}_{\mu _{2}\kappa _{2}}(tu). \end{aligned}$$

According to Lemma 3.1-(b) again, there exist $t_{0}>0$ such that $u_0=t_0u\in {\mathscr {N}}_{\mu _{1}\kappa _{1}}$ satisfying

$$\begin{aligned} \mathcal {J}_{\mu _{1}\kappa _{1}}(u_{0})=\max _{t\ge 0}\mathcal {J}_{\mu _{1}\kappa _{1}}(tu_0). \end{aligned}$$

Clearly, from the above fact we get

$$\begin{aligned} \begin{aligned} c_{\mu _{2}\kappa _{2}}&=\mathcal {J}_{\mu _{2}\kappa _{2}}(u)\ge \mathcal {J}_{\mu _{2}\kappa _{2}}(u_{0})\\&=\mathcal {J}_{\mu _{1}\kappa _{1}}(u_{0})+\frac{1}{p}(\mu _{2}-\mu _{1})\Vert u_{0}\Vert _{p}^{p}+\frac{1}{q}(\mu _{2}-\mu _{1})\Vert u_{0}\Vert _{q}^{q} +(\kappa _{1}-\kappa _{2})\int _{\mathbb {R}^{N}}F(u_{0})\mathrm {d}x\\&\ge c_{\mu _{1}\kappa _{1}}+\frac{1}{p}(\mu _{2}-\mu _{1})\Vert u_{0}\Vert _{p}^{p}+\frac{1}{q}(\mu _{2}-\mu _{1})\Vert u_{0}\Vert _{q}^{q} +(\kappa _{1}-\kappa _{2})\int _{\mathbb {R}^{N}}F(u_{0})\mathrm {d}x, \end{aligned} \end{aligned}$$

this implies that $c_{\mu _{2}\kappa _{2}}\ge c_{\mu _{1}\kappa _{1}}$. If $\max \{\mu _{2}-\mu _{1}, \kappa _{1}-\kappa _{2}\}>0$, it is easy to see that $c_{\mu _{2}\kappa _{2}}>c_{\mu _{1}\kappa _{1}}$ by the above formula. This completes the proof of the lemma. $\square$

4 Palais–Smale compactness condition

In this section we analyze the Palais–Smale compactness condition. First we introduce the following splitting lemma due to [6] which will be very useful later.

Lemma 4.1

Let $\{u_n\}$ be a sequence such that $u_n\rightharpoonup u$ in $E_{\epsilon }$, and set $v_n=u_n-u$. Then the following conclusions hold:

$$\begin{aligned} \begin{aligned}&\Vert \nabla v_n\Vert _{p}^{p}+\Vert \nabla v_n\Vert _{q}^{q}=(\Vert \nabla u_{n}\Vert _{p}^{p}+\Vert \nabla u_{n}\Vert _{q}^{q}) -(\Vert \nabla u\Vert _{p}^{p}+\Vert \nabla u\Vert _{q}^{q})+o_n(1),\\&\int _{\mathbb {R}^N}V(\epsilon x)|v_{n}|^{p}\mathrm {d}x=\int _{\mathbb {R}^N}V(\epsilon x)|u_{n}|^{p}\mathrm {d}x-\int _{\mathbb {R}^N}V(\epsilon x)|u|^{p}\mathrm {d}x+o_n(1),\\&\int _{\mathbb {R}^N}V(\epsilon x)|v_{n}|^{q}\mathrm {d}x=\int _{\mathbb {R}^N}V(\epsilon x)|u_{n}|^{q}\mathrm {d}x-\int _{\mathbb {R}^N}V(\epsilon x)|u|^{q}\mathrm {d}x+o_n(1),\\&\int _{\mathbb {R}^N}F(v_n)\mathrm {d}x=\int _{\mathbb {R}^N}F(u_{n})\mathrm {d}x-\int _{\mathbb {R}^N}F(u)\mathrm {d}x+o_n(1),\\&\int _{\mathbb {R}^N}f(v_n)\phi \mathrm {d}x=\int _{\mathbb {R}^N}f(u_{n})\phi \mathrm {d}x-\int _{\mathbb {R}^N}f(u)\phi \mathrm {d}x+o_n(1)~\hbox {uniformly in}~\phi \in E_{\epsilon }. \end{aligned} \end{aligned}$$

In order to obtain the compactness result, we need the following crucial result (see [26]).

Lemma 4.2

Let $\varphi _n:\mathbb {R}^N\rightarrow \mathbb {R}^m$, $m\ge 1$, with $\varphi _n\in L^{s}(\mathbb {R}^N)\times \dots \times L^{s}(\mathbb {R}^N)$ $(s>1)$, $\varphi _n\rightarrow 0$ a.e. in $\mathbb {R}^m$ and $B(y)=|y|^{s-2}y$, $y\in \mathbb {R}^m$. Then, if $\Vert \varphi _{n}\Vert _{s}\le c$ for all $n\in \mathbb {N}$, there holds

$$\begin{aligned} \int _{\mathbb {R}^N}|B(\varphi _{n}+u)-B(\varphi _{n})-B(u)|^{\frac{s}{s-1}}\mathrm {d}x=o_n(1) \end{aligned}$$

for each $u\in L^{s}(\mathbb {R}^N)\times \dots \times L^{s}(\mathbb {R}^N)$.

According to Lemmas 4.1 and 4.2 and using some standard arguments, we can prove the following result holds.

Lemma 4.3

Let $\{u_n\}$ be a sequence such that $u_n\rightharpoonup u$ in $E_{\epsilon }$, and set $v_n=u_n-u$. Then we have

$$\begin{aligned} \Phi _{\epsilon }(v_n)&=\Phi _{\epsilon }(u_n)-\Phi _{\epsilon }(u)+o_n(1),\\ \langle \Phi '_{\epsilon }(v_n),\phi \rangle&=\langle \Phi '_{\epsilon }(u_n),\phi \rangle -\langle \Phi '_{\epsilon }(u),\phi \rangle +o_n(1) \end{aligned}$$

uniformly in $\phi \in E_{\epsilon }$.

Proof

Obviously, the first conclusion is an immediate consequence of Lemma 4.1. Next we show that the second conclusion holds. Indeed, for $s\in \{p,q\}$, Lemma 4.2 implies that

$$\begin{aligned} \int _{\mathbb {R}^N}|B(v_{n})-B(u_{n})+B(u)|^{\frac{s}{s-1}}\mathrm {d}x=o_n(1). \end{aligned}$$

(4.1)

Moreover, according to the proof of Theorem 3.3 in [26], we can see that

$$\begin{aligned} \int _{\mathbb {R}^N}V(\epsilon x)||v_n|^{s-2}v_{n}-|u_n|^{s-2}u_{n}+|u|^{s-2}u|^{\frac{s}{s-1}}\mathrm {d}x=o_n(1). \end{aligned}$$

(4.2)

Using the Hölder inequality, for any $\phi \in E_{\epsilon }$ such that $\Vert \phi \Vert _{\epsilon }\le 1$, we have

$$\begin{aligned} \begin{aligned}&|\langle \Phi '_{\epsilon }(v_n)-\Phi '_{\epsilon }(u_n)+\Phi '_{\epsilon }(u),\phi \rangle |\\&\quad \le \left( \int _{\mathbb {R}^N}|B(v_{n})-B(u_{n})+B(u)|^{\frac{p}{p-1}}\mathrm {d}x\right) ^{\frac{p-1}{p}}\left( \int _{\mathbb {R}^N}|\nabla \phi |^{p}\mathrm {d}x\right) ^{\frac{1}{p}}\\&\qquad \quad +\left( \int _{\mathbb {R}^N}|B(v_{n})-B(u_{n})+B(u)|^{\frac{q}{q-1}}\mathrm {d}x\right) ^{\frac{q-1}{q}}\left( \int _{\mathbb {R}^N}|\nabla \phi |^{q}\mathrm {d}x\right) ^{\frac{1}{q}}\\&\qquad \quad +\left( \int _{\mathbb {R}^N}V(\epsilon x)||v_n|^{p-2}v_{n}-|u_n|^{p-2}u_{n}+|u|^{p-2}u|^{\frac{p}{p-1}}\mathrm {d}x\right) ^{\frac{p-1}{p}} \left( \int _{\mathbb {R}^N}V(\epsilon x)|\phi |^{p}\mathrm {d}x\right) ^{\frac{1}{p}}\\&\qquad \quad +\left( \int _{\mathbb {R}^N}V(\epsilon x)||v_n|^{q-2}v_{n}-|u_n|^{q-2}u_{n}+|u|^{q-2}u|^{\frac{q}{q-1}}\mathrm {d}x\right) ^{\frac{q-1}{q}} \left( \int _{\mathbb {R}^N}V(\epsilon x)|\phi |^{q}\mathrm {d}x\right) ^{\frac{1}{q}}\\&\qquad \quad +\int _{\mathbb {R}^N}K(\epsilon x)|(f(v_n)-f(u_n)+f(u))\phi |\mathrm {d}x. \end{aligned} \end{aligned}$$

By using Lemma 4.1, (4.1) and (4.2), it is easy to see that the second conclusion holds. The proof is completed. $\square$

Consider the limit problem

$$\begin{aligned} \left\{ \begin{array}{ll} -\Delta _{p} u-\Delta _{q} u +V_{\infty }(|u|^{p-2}u+|u|^{q-2}u)=K_{\infty } f(u),&{}\quad \hbox {in}~\mathbb {R}^{N},\\ u\in W^{1,p}(\mathbb {R}^{N})\cap W^{1,q}(\mathbb {R}^{N}),u>0, &{}\quad \hbox {in}~\mathbb {R}^{N}.\\ \end{array} \right. \end{aligned}$$

(4.3)

As before, $\mathcal {J}_{V_{\infty }K_{\infty }}$, ${\mathscr {N}}_{V_{\infty }K_{\infty }}$ and $c_{V_{\infty }K_{\infty }}$ denote the corresponding energy functional, Nehari manifold and ground state energy value of limit problem (4.3), respectively.

Lemma 4.4

Let $\{u_n\}$ be a Palais–Smale sequence at level $c>0$ for $\Phi _{\epsilon }$ with $u_n\rightharpoonup u$ in $E_{\epsilon }$. Then the following alternative holds: either $u_n\rightarrow u$ in $E_{\epsilon }$ along a subsequence, or $c-\Phi _{\epsilon }(u)\ge c_{V_{\infty }K_{\infty }}$.

Proof

Define $v_n=u_n-u$ and assume that $v_n\nrightarrow 0$ in $E_{\epsilon }$. From Lemma 2.7, for each $v_n$ there is a unique $\{t_n\}\subset (0,\infty )$ such that $\{t_nv_n\}\subset {\mathscr {N}}_{V_{\infty }K_{\infty }}$. We divide the proof into three steps.

Step 1. The sequence $\{t_n\}$ satisfies

$$\begin{aligned} \limsup _{n\rightarrow \infty } t_n\le 1. \end{aligned}$$

Indeed, assume by contradiction that the above conclusion does not hold. Then, there exist $\tau >0$ and a subsequence of $\{t_n\}$, still denoted by itself, such that

$$\begin{aligned} t_n\ge 1+\tau \quad \hbox {for all}~n\in \mathbb {N}. \end{aligned}$$

According to Lemma 4.3, it is easy to see that $\langle \Phi '_{\epsilon }(v_n),v_n\rangle =o_n(1)$. Moreover, from the fact that $\{t_nv_n\}\subset {\mathscr {N}}_{V_{\infty }K_{\infty }}$, we can infer that

$$\begin{aligned} \Vert \nabla v_{n}\Vert _{p}^{p}+\Vert \nabla v_{n}\Vert _{q}^{q}+\int _{\mathbb {R}^N}V(\epsilon x)(|v_n|^{p}+|v_n|^{q})\mathrm {d}x -\int _{\mathbb {R}^N}K(\epsilon x)f(v_n)v_n\mathrm {d}x=o_n(1) \end{aligned}$$

and

$$\begin{aligned} t_{n}^{p-q}\Vert \nabla v_{n}\Vert _{p}^{p}+\Vert \nabla v_{n}\Vert _{q}^{q}+V_{\infty }\int _{\mathbb {R}^N}(t_{n}^{p-q}|v_n|^{p}+|v_n|^{q})\mathrm {d}x -K_{\infty }\int _{\mathbb {R}^N}\frac{f(t_nv_n)}{(t_nv_n)^{q-1}}v_n^{q}\mathrm {d}x=0. \end{aligned}$$

Consequently,

$$\begin{aligned} \begin{aligned}&\int _{\mathbb {R}^N}\left( \frac{K_{\infty }f(t_nv_n)}{(t_nv_n)^{q-1}}-\frac{K_{\infty }f(v_n)}{v_n^{q-1}}\right) v_n^{q}\mathrm {d}x +\int _{\mathbb {R}^N}\left( \frac{K_{\infty }f(v_n)}{v_n^{q-1}}-\frac{K(\epsilon x)f(v_n)}{v_n^{q-1}}\right) v_n^{q}\mathrm {d}x\\&\quad =(t_{n}^{p-q}-1)\Vert \nabla v_{n}\Vert _{p}^{p}+\int _{\mathbb {R}^N}(t_{n}^{p-q}V_{\infty }-V(\epsilon x))v_{n}^{p}\mathrm {d}x\\&\qquad + \int _{\mathbb {R}^N}(V_{\infty }-V(\epsilon x))v_{n}^{q}\mathrm {d}x+o_{n}(1). \end{aligned} \end{aligned}$$

(4.4)

By the definition of $V_{\infty }$ and $K_{\infty }$, for any $\varepsilon >0$, there exists $R=R(\varepsilon )>0$ such that

$$\begin{aligned} V(\epsilon x)\ge V_{\infty }-\varepsilon >V_{\infty }/t_n^{q-p}-\varepsilon ~\hbox {and}~K(\epsilon x)\le K_{\infty }+\varepsilon ~\hbox {for any}~|x|\ge R. \end{aligned}$$

(4.5)

Since $v_n\rightharpoonup 0$ in $E_{\epsilon }$, then $v_n\rightarrow 0$ in $L_{loc}^{s}(\mathbb {R}^N)$ for $s\in [1,q^*)$ by Lemma 2.3. Using (2.3) and (4.4) we deduce that

$$\begin{aligned} \begin{aligned}&\int _{\mathbb {R}^N}\left( \frac{K_{\infty }f(t_nv_n)}{(t_nv_n)^{q-1}}-\frac{K_{\infty }f(v_n)}{v_n^{q-1}}\right) v_n^{q}\mathrm {d}x\\&\quad \le \int _{\mathbb {R}^N}(K(\epsilon x)-K_{\infty })f(v_n)v_n\mathrm {d}x\\&\qquad +\int _{\mathbb {R}^N}\left[ (t_{n}^{p-q}V_{\infty }-V(\epsilon x))v_{n}^{p}+(V_{\infty }-V(\epsilon x))v_{n}^{q}\right] \mathrm {d}x+o_{n}(1)\\&\quad \le \varepsilon \int _{|x|\ge R}f(v_n)v_n\mathrm {d}x+\varepsilon \int _{|x|\ge R}(|v_n|^{p}+|v_n|^{q})\mathrm {d}x\\&\qquad + 2K_{\max }\int _{|x|\le R}f(v_n)v_n\mathrm {d}x+2V_{\max }\int _{|x|\le R}(|v_n|^{p}+|v_n|^{q})\mathrm {d}x+o_{n}(1)\\&\quad =c\varepsilon +o_n(1). \end{aligned} \end{aligned}$$

(4.6)

By the fact $v_n\nrightarrow 0$ in $E_{\epsilon }$ and $\Phi '_{\epsilon }(v_n)\rightarrow 0$, there exist ${\bar{R}}$, $\delta >0$ and $y_n\in \mathbb {R}^N$ such that

$$\begin{aligned} \int _{B_{{\bar{R}}}(y_n)}|v_n|^q\mathrm {d}x\ge \delta . \end{aligned}$$

(4.7)

Note that the above claim is true, because otherwise, using again Lemma 2.2, we have $v_n\rightarrow 0$ in $L^{s}(\mathbb {R}^N)$ for $s\in (q,q^*)$. According to $\langle \Phi '_{\epsilon }(v_n),v_n\rangle =o_n(1)$ and (2.3) we can infer that

$$\begin{aligned} \begin{aligned} o_n(1)&=\langle \Phi _{\epsilon }'(v_{n}),v_{n}\rangle \\&=\Vert \nabla v_n\Vert _{p}^{p}+\Vert \nabla v_n\Vert _{q}^{q}+\int _{\mathbb {R}^N}V(\epsilon x)(|v_n|^{p}+|v_n|^{q})\mathrm {d}x- \int _{\mathbb {R}^N}K(\epsilon x)f(v_n)v_n\mathrm {d}x\\&\ge \Vert \nabla v_n\Vert _{p}^{p}+\Vert \nabla v_n\Vert _{q}^{q}\\&\quad +\int _{\mathbb {R}^N}V(\epsilon x)(|v_n|^{p}+|v_n|^{q})\mathrm {d}x- \varepsilon K_{\max }\Vert v_n\Vert _{p}^{p}-C_{\varepsilon }K_{\max }\Vert v_n\Vert _{r}^{r}\\&\ge c_{10}\Vert v_n\Vert _{V_{\epsilon },p}^{p}+c_{11}\Vert v_n\Vert _{V_{\epsilon },q}^{q}-c_{12}\Vert v_n\Vert _{r}^{r}, \end{aligned} \end{aligned}$$

and consequently

$$\begin{aligned} c_{10}\Vert v_n\Vert _{V_{\epsilon },p}^{p}+c_{11}\Vert v_n\Vert _{V_{\epsilon },q}^{q}\le c_{12}\Vert v_n\Vert _{r}^{r}\rightarrow 0 \end{aligned}$$

since $r\in (q,q^*)$. Then, $v_{n}\rightarrow 0$ in $E_{\epsilon }$, which is a contradiction. Setting ${\tilde{v}}_{n}=v_n(x+y_n)$, we may suppose that, passing to a subsequence, ${\tilde{v}}_{n}\rightharpoonup {\tilde{v}}$ in $E_{\epsilon }$ and ${\tilde{v}}_{n}(x)\rightarrow {\tilde{v}}(x)$ a.e. in $\mathbb {R}^N$. Thus,

$$\begin{aligned} \int _{B_{{\bar{R}}}(0)}|{\tilde{v}}_n|^q\mathrm {d}x\ge \delta , \end{aligned}$$

showing that ${\tilde{v}}\ne 0$. Moreover, using the fact that $v_n\ge 0$ for all $n\in \mathbb {N}$, we have that ${\tilde{v}}(x)\ge 0$ a.e. in $\mathbb {R}^N$. Hence, there exists a subset $\Omega _{1}\subset \mathbb {R}^N$ with positive measure such that ${\tilde{v}}(x)>0$ for all $x\in \Omega _{1}$. Consequently, it follows from $(f_5)$ and (4.6) that

$$\begin{aligned} 0<\int _{\Omega _{1}}\left( \frac{K_{\infty }f((1+\tau )v_n)}{((1+\tau )v_n)^{q-1}}-\frac{K_{\infty }f(v_n)}{v_n^{q-1}}\right) v_n^{q}\mathrm {d}x\le c\varepsilon +o_n(1). \end{aligned}$$

Letting $n\rightarrow \infty$ in the last inequality and applying Fatou’s lemma, it follows that

$$\begin{aligned} 0<\int _{\Omega _{1}}\left( \frac{K_{\infty }f((1+\tau ){\tilde{v}})}{((1+\tau ){\tilde{v}})^{q-1}} -\frac{K_{\infty }f({\tilde{v}})}{{\tilde{v}}^{q-1}}\right) {\tilde{v}}^{q}\mathrm {d}x\le c\varepsilon , \end{aligned}$$

which is absurd, since the arbitrariness of $\varepsilon$.

From Step 1, we derive that

$$\begin{aligned} \limsup _{n\rightarrow \infty }t_n=1\quad \hbox {or}\quad \limsup _{n\rightarrow \infty }t_n=t_{0}<1. \end{aligned}$$

Next, we will study each one of these possibilities.

Step 2. The sequence $\{t_n\}$ satisfies

$$\begin{aligned} \limsup _{n\rightarrow \infty } t_n=1. \end{aligned}$$

In this case, there exists a subsequence, such that $t_n\rightarrow 1$. Since $\mathcal {J}_{V_{\infty }K_{\infty }}(t_nv_n)\ge c_{V_{\infty }K_{\infty }}$, by Lemma 4.3 we have

$$\begin{aligned} \begin{aligned} c-\Phi _{\epsilon }(u)+o_n(1)&=\Phi _{\epsilon }(v_n)\\&=\Phi _{\epsilon }(v_n)-\mathcal {J}_{V_{\infty }K_{\infty }}(t_nv_n)+\mathcal {J}_{V_{\infty }K_{\infty }}(t_nv_n)\\&\ge \Phi _{\epsilon }(v_n)-\mathcal {J}_{V_{\infty }K_{\infty }}(t_nv_n)+c_{V_{\infty }K_{\infty }}. \end{aligned} \end{aligned}$$

(4.8)

Observe that,

$$\begin{aligned} \begin{aligned}&\Phi _{\epsilon }(v_n)-\mathcal {J}_{V_{\infty }K_{\infty }}(t_nv_n)\\&\quad =\frac{(1-t_n^p)}{p}\Vert \nabla v_{n}\Vert _{p}^{p}+\frac{(1-t_n^q)}{q}\Vert \nabla v_{n}\Vert _{q}^{q} +\frac{1}{p}\int _{\mathbb {R}^N}(V(\epsilon x)-t_{n}^{p}V_{\infty })v_{n}^{p}\mathrm {d}x\\&\qquad +\frac{1}{q}\int _{\mathbb {R}^N}(V(\epsilon x)-t_{n}^{q}V_{\infty })v_{n}^{q}\mathrm {d}x +\int _{\mathbb {R}^N}(K_{\infty }F(t_nv_n)-K(\epsilon x)F(v_n))\mathrm {d}x. \end{aligned} \end{aligned}$$

(4.9)

It follows from (4.5) that

$$\begin{aligned} V(\epsilon x)-t_{n}^{p}V_{\infty }=(V(\epsilon x)-V_{\infty }) +(1-t_n^{p})V_{\infty }\ge -\varepsilon +(1-t_{n}^{p})V_{\infty }~\hbox {for any}~|x|\ge R, \end{aligned}$$

then by $v_n\rightarrow 0$ in $L_{loc}^{p}(\mathbb {R}^N)$ and $t_n\rightarrow 1$ we get

$$\begin{aligned} \begin{aligned}&\int _{\mathbb {R}^N}(V(\epsilon x)-t_{n}^{p}V_{\infty })v_{n}^{p}\mathrm {d}x\\&\quad =\int _{|x|\le R}(V(\epsilon x)-t_{n}^{p}V_{\infty })v_{n}^{p}\mathrm {d}x+\int _{|x|\ge R}(V(\epsilon x)-t_{n}^{p}V_{\infty })v_{n}^{p}\mathrm {d}x\\&\quad \ge (V_{\min }-t_{n}^{p}V_{\infty })\int _{|x|\le R}v_{n}^{p}\mathrm {d}x -\varepsilon \int _{|x|\ge R}v_{n}^{p}\mathrm {d}x+V_{\infty }(1-t_n^{p})\int _{|x|\ge R}v_{n}^{p}\mathrm {d}x\\&\quad \ge o_n(1)-c\varepsilon . \end{aligned} \end{aligned}$$

(4.10)

Similarly, we can prove that

$$\begin{aligned} \int _{\mathbb {R}^N}(V(\epsilon x)-t_{n}^{q}V_{\infty })v_{n}^{q}\mathrm {d}x\ge o_n(1)-c\varepsilon . \end{aligned}$$

(4.11)

In what follows, we verify that

$$\begin{aligned} \int _{\mathbb {R}^N}(K_{\infty }F(t_nv_n)-K(\epsilon x)F(v_n))\mathrm {d}x\ge o_n(1)-c\varepsilon . \end{aligned}$$

(4.12)

Indeed, note that

$$\begin{aligned} \begin{aligned}&\int _{\mathbb {R}^N}(K_{\infty }F(t_nv_n)-K(\epsilon x)F(v_n))\mathrm {d}x\\&\quad =\int _{\mathbb {R}^N}(K_{\infty }F(t_nv_n)-K_{\infty }F(v_n))\mathrm {d}x+\int _{\mathbb {R}^N}(K_{\infty }F(v_n)-K(\epsilon x)F(v_n))\mathrm {d}x\\&\quad :=T_{1}+T_{2}. \end{aligned} \end{aligned}$$

On the one hand, using the mean value theorem, the fact that $t_n\rightarrow 1$ and the boundedness of $\{v_n\}$ we have

$$\begin{aligned} \begin{aligned} T_{1}&\le K_{\infty }\int _{\mathbb {R}^N}|F(t_nv_n)-F(v_n)|\mathrm {d}x\\&\le c_{13}|t_n-1|\int _{\mathbb {R}^N}|v_n|^{p}\mathrm {d}x +c_{14}|t_n-1|\int _{\mathbb {R}^N}|v_n|^{q}\mathrm {d}x\\&=o_n(1). \end{aligned} \end{aligned}$$

On the other hand, applying the fact $v_n\rightarrow 0$ in $L_{loc}^{s}(\mathbb {R}^N)$ for $s\in [1,q^*)$ and (4.5) we obtain

$$\begin{aligned} \begin{aligned} T_{2}&=\int _{|x|\le R}(K_{\infty }-K(\epsilon x))F(v_n)\mathrm {d}x+\int _{|x|\ge R}(K_{\infty }-K(\epsilon x))F(v_n)\mathrm {d}x\\&\ge o_n(1)-c\varepsilon . \end{aligned} \end{aligned}$$

Combining the above facts, we can see that (4.12) holds. Finally, from the boundedness of $\{v_n\}$ and $t_n\rightarrow 1$ we can conclude that

$$\begin{aligned} \frac{(1-t_n^p)}{p}\Vert \nabla v_{n}\Vert _{p}^{p}=o_n(1)\quad \hbox {and}\quad \frac{(1-t_n^q)}{q}\Vert \nabla v_{n}\Vert _{q}^{q}=o_n(1). \end{aligned}$$

(4.13)

Using (4.8), (4.9) (4.10), (4.11), (4.12) and (4.13) we are led to

$$\begin{aligned} c-\Phi _{\epsilon }(u)\ge o_n(1)-c\varepsilon +c_{V_{\infty }K_{\infty }}, \end{aligned}$$

and taking the limit as $\varepsilon \rightarrow 0$ we get

$$\begin{aligned} c-\Phi _{\epsilon }(u)\ge c_{V_{\infty }K_{\infty }}. \end{aligned}$$

Step 3. The sequence $\{t_n\}$ satisfies

$$\begin{aligned} \limsup _{n\rightarrow \infty } t_n=t_0<1. \end{aligned}$$

We assume that there exists a subsequence, still denoted by $\{t_n\}$, such that $t_n\rightarrow t_0<1$. First, similar to the above arguments, we can get

$$\begin{aligned} \begin{aligned}&\int _{\mathbb {R}^N}(V_{\infty }-V(\epsilon x))v_n^p\mathrm {d}x=o_n(1)\\&\int _{\mathbb {R}^N}(K_{\infty }-K(\epsilon x))\left( \frac{1}{q}f(v_n)v_n-F(v_n)\right) \mathrm {d}x=o_n(1). \end{aligned} \end{aligned}$$

(4.14)

Since $\langle \Phi _{\epsilon }(v_n),v_n\rangle =o_n(1)$, then we have

$$\begin{aligned} \begin{aligned} c-\Phi _{\epsilon }(u)+o_n(1)&=\Phi _{\epsilon }(v_n)-\frac{1}{q}\langle \Phi _{\epsilon }(v_n),v_n\rangle \\&=\left( \frac{1}{p}-\frac{1}{q}\right) \Vert v_n\Vert _{V_\epsilon ,p}^{p}+\int _{\mathbb {R}^N}K(\epsilon x)\left( \frac{1}{q}f(v_n)v_n-F(v_n)\right) \mathrm {d}x. \end{aligned} \end{aligned}$$

(4.15)

Recalling that $t_nv_n\in {\mathscr {N}}_{V_{\infty }K_{\infty }}$, and using $(f_5)$, (4.14) and (4.15) we get

$$\begin{aligned} \begin{aligned} c_{V_{\infty }K_{\infty }}&\le \mathcal {J}_{V_{\infty }K_{\infty }}(t_nv_n)\\&=\mathcal {J}_{V_{\infty }K_{\infty }}(t_nv_n)-\frac{1}{q}\langle \mathcal {J}_{V_{\infty }K_{\infty }}(t_nv_n),t_nv_n\rangle \\&=\left( \frac{1}{p}-\frac{1}{q}\right) \Vert t_nv_n\Vert _{V_{\infty },p}^{p} +\int _{\mathbb {R}^N}K_{\infty }\left( \frac{1}{q}f(t_nv_n)t_nv_n-F(t_nv_n)\right) \mathrm {d}x.\\&\le \left( \frac{1}{p}-\frac{1}{q}\right) \Vert v_n\Vert _{V_{\infty },p}^{p} +\int _{\mathbb {R}^N}K_{\infty }\left( \frac{1}{q}f(v_n)v_n-F(v_n)\right) \mathrm {d}x\\&=\left( \frac{1}{p}-\frac{1}{q}\right) \Vert v_n\Vert _{V_{\epsilon },p}^{p} +\int _{\mathbb {R}^N}K(\epsilon x)\left( \frac{1}{q}f(v_n)v_n-F(v_n)\right) \mathrm {d}x\\&~~+\left( \frac{1}{p}-\frac{1}{q}\right) \int _{\mathbb {R}^N}(V_{\infty }-V(\epsilon x))v_n^p\mathrm {d}x\\&\quad + \int _{\mathbb {R}^N}(K_{\infty }-K(\epsilon x))\left( \frac{1}{q}f(v_n)v_n-F(v_n)\right) \mathrm {d}x\\&=\Phi _{\epsilon }(v_n)-\frac{1}{q}\langle \Phi _{\epsilon }(v_n),v_n\rangle +o_n(1)\\&=c-\Phi _{\epsilon }(u)+o_n(1). \end{aligned} \end{aligned}$$

Taking the limit as $n\rightarrow \infty$, we get

$$\begin{aligned} c-\Phi _{\epsilon }(u)\ge c_{V_{\infty }K_{\infty }}. \end{aligned}$$

The proof is now complete. $\square$

Applying Lemmas 4.3 and 4.4, we have the following compactness result.

Lemma 4.5

Let $\{u_n\}$ be a bounded Palais–Smale sequence at level $c<c_{V_{\infty }K_{\infty }}$ for $\Phi _{\epsilon }$. Then $\{u_n\}$ has a convergent subsequence in $E_{\epsilon }$.

Proof

Let $\{u_n\}$ be a bounded Palais–Smale sequence, up to a subsequence, we may assume that $u_n\rightharpoonup u$ in $E_{\epsilon }$, $u_n\rightarrow u$ in $L_{loc}^{s}(\mathbb {R}^{N})$ for $s\in [1,q^*)$ and $u_n(x)\rightarrow u(x)$ a.e. in $\mathbb {R}^N$. Using Lemma 2.4, we can see that $\Phi '_{\epsilon }(u)=0$. Moreover, it follows from (2.4) that

$$\begin{aligned} \begin{aligned} \Phi _{\epsilon }(u)&=\Phi _{\epsilon }(u)-\frac{1}{q}\langle \Phi '_{\epsilon }(u),u\rangle \\&=\left( \frac{1}{p}-\frac{1}{q}\right) \Vert u\Vert _{V_{\epsilon },p}^{p}+\int _{\mathbb {R}^N}K(\epsilon x)\left( \frac{1}{q}f(u)u-F(u)\right) \mathrm {d}x\ge 0. \end{aligned} \end{aligned}$$

(4.16)

Hence, we have $c-\Phi _{\epsilon }(u)\le c< c_{V_{\infty }K_{\infty }}$. From Lemma 4.4 we can deduce that $u_n\rightarrow u$ in $E_{\epsilon }$. This completes the proof. $\square$

5 Existence and concentration of positive ground state solutions

In this section, we will prove the existence and concentration phenomena of positive ground state solutions to problem (2.1). Moreover, we complete the proofs of Theorems 1.1 and 1.2.

We first consider the situation that ($A_{0}$) and ($A_{1}$) are satisfied. For any $x_{v}\in {\mathscr {V}}$, we set ${\tilde{V}}(\epsilon x)=V(\epsilon x+\epsilon x_{v})$ and ${\tilde{K}}(\epsilon x)=K(\epsilon x+\epsilon x_{v})$. It is clear that if ${\tilde{u}}$ is a solution of

$$\begin{aligned} \left\{ \begin{array}{ll} -\Delta _{p}u-\Delta _{q} u +{\tilde{V}}(\epsilon x)(|u|^{p-2}u+|u|^{q-2}u)={\tilde{K}}(\epsilon x)f(u),&{}\quad \hbox {in}~\mathbb {R}^{N},\\ u\in W^{1,p}(\mathbb {R}^{N})\cap W^{1,q}(\mathbb {R}^{N}),u>0, &{}\quad \hbox {in}~\mathbb {R}^{N},\\ \end{array} \right. \end{aligned}$$

then $u(x)={\tilde{u}}(x-\epsilon x_{v})$ solves problem (2.1). By conditions ($A_{0}$) and ($A_{1}$), without loss of generality, we may assume that $x_{v}=0\in {\mathscr {V}}$ or $x_{v}=0\in {\mathscr {V}}\cap {\mathscr {K}}$ if ${\mathscr {V}}\cap {\mathscr {K}}\ne \emptyset$. Then we have

$$\begin{aligned} V(0)=V_{\min }\quad \hbox {and}\quad \kappa :=K(0)\ge K(x)\quad \hbox {for all}~|x|\ge R. \end{aligned}$$

(5.1)

Consider the following problem

$$\begin{aligned} -\Delta _{p}u-\Delta _{q} u +V_{\min }(|u|^{p-2}u+|u|^{q-2}u)=\kappa f(u)~\hbox {in}~\mathbb {R}^{N}. \end{aligned}$$

(5.2)

In the sequel, we also use the associated notations $\mathcal {J}_{V_{\min }\kappa }$, ${\mathscr {N}}_{V_{\min }\kappa }$ and $c_{V_{\min }\kappa }$ as before, which denote the energy functional, Nehari manifold and ground state energy value of problem (5.2), respectively. Moreover, we deduce from Lemma 3.3 that problem (5.2) possesses at least one positive ground state solution.

Next, we give the comparison relationship of the ground state energy value between problem (2.1) and problem (5.2), which play a significant role in our analysis.

Lemma 5.1

We have $\limsup _{\epsilon \rightarrow 0}c_{\epsilon }\le c_{V_{\min }\kappa }$. In particular, if ${\mathscr {V}}\cap {\mathscr {K}}\ne \emptyset$, then $\lim _{\epsilon \rightarrow 0}c_{\epsilon }=c_{V_{\min }K_{\max }}$.

Proof

Let u be a positive ground state solution of problem (5.2), then from Lemma 3.1-(b) we have

$$\begin{aligned} u\in {\mathscr {N}}_{V_{\min }\kappa }~\hbox {and}~c_{V_{\min }\kappa }=\mathcal {J}_{V_{\min }\kappa }(u)=\max _{t\ge 0}\mathcal {J}_{V_{\min }\kappa }(tu). \end{aligned}$$

(5.3)

It follows from Lemma 2.7 that there exists $t_{\epsilon }>0$ such that $t_{\epsilon }u\in {\mathscr {N}}_{\epsilon }$, and

$$\begin{aligned} c_{\epsilon }\le \Phi _{\epsilon }(t_{\epsilon } u)=\max _{t\ge 0}\Phi _{\epsilon }(tu). \end{aligned}$$

(5.4)

Moreover, it is clear to see that $\{t_{\epsilon }\}$ is bounded. Then, passing to a subsequence, we assume that $t_{\epsilon }\rightarrow t_{0}$. Note that

$$\begin{aligned} \begin{aligned} \Phi _{\epsilon }(t_{\epsilon }u)&=\mathcal {J}_{V_{\min }\kappa }(t_{\epsilon }u)+\int _{\mathbb {R}^N}(\kappa -K(\epsilon x))F(t_{\epsilon }u)\mathrm {d}x\\&~~~+\frac{t_{\epsilon }^{p}}{p}\int _{\mathbb {R}^N}(V(\epsilon x)-V_{\min })u^{p}\mathrm {d}x+\frac{t_{\epsilon }^{q}}{q}\int _{\mathbb {R}^N}(V(\epsilon x)-V_{\min })u^{q}\mathrm {d}x. \end{aligned} \end{aligned}$$

(5.5)

According to the boundedness of $t_{\epsilon }$, $K(\epsilon x)\rightarrow \kappa$ in a bounded domain and the exponential decay of u, we have

$$\begin{aligned} \begin{aligned}&\int _{\mathbb {R}^N}(\kappa -K(\epsilon x))F(t_{\epsilon }u)\mathrm {d}x\\&\quad =\int _{|x|\le R}(\kappa -K(\epsilon x))F(t_{\epsilon }u)\mathrm {d}x +\int _{|x|\ge R}(\kappa -K(\epsilon x))F(t_{\epsilon }u)\mathrm {d}x\\&\quad =o_{\epsilon }(1). \end{aligned} \end{aligned}$$

(5.6)

Similarly, according to the boundedness of $t_{\epsilon }$, $V(\epsilon x)\rightarrow V_{\min }$ in a bounded domain and the exponential decay of u, we get

$$\begin{aligned} \frac{t_{\epsilon }^{p}}{p}\int _{\mathbb {R}^N}(V(\epsilon x)-V_{\min })u^{p}\mathrm {d}x=o_{\epsilon }(1) \quad \hbox {and}\quad \frac{t_{\epsilon }^{q}}{q}\int _{\mathbb {R}^N}(V(\epsilon x)-V_{\min })u^{q}\mathrm {d}x=o_{\epsilon }(1). \end{aligned}$$

(5.7)

Using (5.5) (5.6) and (5.7) we have

$$\begin{aligned} \Phi _{\epsilon }(t_{\epsilon }u)=\mathcal {J}_{V_{\min }\kappa }(t_{\epsilon }u)+o_{\epsilon }(1). \end{aligned}$$

Together with (5.3) and (5.4), as $\epsilon \rightarrow 0$, we can infer that

$$\begin{aligned} c_{\epsilon }\le \Phi _{\epsilon }(t_{\epsilon }u)\rightarrow \mathcal {J}_{V_{\min }\kappa }(t_{0}u)\le \max _{t\ge 0}\mathcal {J}_{V_{\min }\kappa }(tu)=\mathcal {J}_{V_{\min }\kappa }(u)=c_{V_{\min }\kappa }. \end{aligned}$$

Thus,

$$\begin{aligned} \limsup _{\epsilon \rightarrow 0}c_{\epsilon }\le c_{V_{\min }\kappa }. \end{aligned}$$

(5.8)

Now we show that the second conclusion holds. Observe that

$$\begin{aligned} \begin{aligned} \Phi _{\epsilon }(u)&=\mathcal {J}_{V_{\min }K_{\max }}(u)+\int _{\mathbb {R}^N}(K_{\max }-K(\epsilon x))F(u)\mathrm {d}x\\&\quad +\frac{1}{p}\int _{\mathbb {R}^N}(V(\epsilon x)-V_{\min })u^{p}\mathrm {d}x+\frac{1}{q}\int _{\mathbb {R}^N}(V(\epsilon x)-V_{\min })u^{q}\mathrm {d}x. \end{aligned} \end{aligned}$$

It follows that

$$\begin{aligned} c_{V_{\min }K_{\max }}\le c_{\epsilon }. \end{aligned}$$

On the other hand, since ${\mathscr {V}}\cap {\mathscr {K}}\ne \emptyset$, then $\kappa =K_{\max }$. So, according to (5.8) we can get

$$\begin{aligned} \lim _{\epsilon \rightarrow 0}c_{\epsilon }= c_{V_{\min }K_{\max }}. \end{aligned}$$

The proof is now complete. $\square$

Lemma 5.2

Assume that $(A_0)$, $(A_1)$ and $(f_1)$–$(f_5)$ hold. Then for any $\epsilon >0$ small enough, problem (2.1) has a positive ground state solution.

Proof

Observe that if $u_{\epsilon }\in {\mathscr {N}}_{\epsilon }$ satisfies $\Phi _{\epsilon }(u_{\epsilon })=c_{\epsilon }$, then

$$\begin{aligned} I_{\epsilon }({\check{m}}_{\epsilon }(u_{\epsilon }))=\Phi _{\epsilon }({\hat{m}}_{\epsilon }({\check{m}}_{\epsilon }(u_{\epsilon }))) =\Phi _{\epsilon }(u_{\epsilon })=c_{\epsilon }=\inf _{S_{\epsilon }}I_{\epsilon }. \end{aligned}$$

This shows that ${\check{m}}_{\epsilon }(u_{\epsilon })\in S_{\epsilon }$ is a minimizer of $I_{\epsilon }$, and hence a critical point of $I_{\epsilon }$. From Lemma 2.11, we know that $u_{\epsilon }$ is a critical point of $\Phi _{\epsilon }$. Therefore, it suffices to prove that there exists a minimizer $u_{\epsilon }\in {\mathscr {N}}_{\epsilon }$ such that $\Phi _{\epsilon }(u_{\epsilon })=c_{\epsilon }$. In fact, using Ekeland’s variational principle [40], there exists a sequence $\{v_{n}\}\subset S_{\epsilon }$ such that $I_{\epsilon }(v_n)\rightarrow c_{\epsilon }$ and $I'_{\epsilon }(v_n)\rightarrow 0$ as $n\rightarrow \infty$. Let $u_n={\hat{m}}_{\epsilon }(v_n)\in {\mathscr {N}}_{\epsilon }$ for all $n\in \mathbb {N}$. Then using Lemma 2.11 again, we get $\Phi _{\epsilon }(u_n)\rightarrow c_{\epsilon }$ and $\Phi '_{\epsilon }(u_n)\rightarrow 0$. According to Lemma 2.9, we can see that $\{u_n\}$ is bounded in $E_{\epsilon }$. Passing to a subsequence, we can assume that $u_{n}\rightharpoonup u_{\epsilon }$ in $E_{\epsilon }$. From (5.1) and $(A_{1})$, we can see that $V_{\min }< V_{\infty }$ and $\kappa \ge K_{\infty }$. By Lemma 3.4, we have $c_{V_{\min }\kappa }<c_{V_{\infty }K_{\infty }}$, moreover, using Lemma 5.1 we deduce that $c_{\epsilon }\le c_{V_{\min }\kappa }<c_{V_{\infty }K_{\infty }}$ for $\epsilon >0$ small enough. Therefore, Lemma 4.5 shows that $\Phi _{\epsilon }$ satisfies the Palais–Smale condition for $\epsilon >0$ small enough. Using Lemma 2.4 and continuity of $\Phi _{\epsilon }$, we have $\Phi '_{\epsilon }(u_{\epsilon })=0$ and $\Phi _{\epsilon }(u_{\epsilon })=c_{\epsilon }$. Hence, problem (2.1) has a ground state solution $u_{\epsilon }$. The positivity of the ground state solution follows with same arguments as in the proof of Lemma 3.3. $\square$

Let ${\mathscr {L}}_{\epsilon }$ be the set of all positive ground state solutions of problem (2.1). Then we have the following result.

Lemma 5.3

${\mathscr {L}}_{\epsilon }$ is compact in $E_{\epsilon }$ for all small $\epsilon >0$.

Proof

Arguing by contradiction, we assume that ${\mathscr {L}}_{\epsilon _{j}}$ is not compact in $E_{\epsilon }$ for some $\epsilon _{j}\rightarrow 0$. Thus, for each j, there exists a sequence $\{u_{n}^{j}\}\subset {\mathscr {L}}_{\epsilon _{j}}$ such that it does not have convergent subsequence. Nevertheless, we note that $\{u_{n}^{j}\}$ is bounded in $E_{\epsilon }$. So, without loss of generality, we may assume that $u_{n}^{j}\rightharpoonup u$ in $E_{\epsilon }$ as $n\rightarrow \infty$. Finally, as in the proof of Lemma 5.2, we can deduce a contradiction. $\square$

Next, we are devoted to the study of concentration phenomena for the positive ground state solution $u_{\epsilon }$ obtained in Lemma 5.2 as $\epsilon \rightarrow 0$. The following result plays a fundamental role in the study of the behaviors of ground state solutions.

Lemma 5.4

Assume that $u_{\epsilon }\in {\mathscr {L}}_{\epsilon }$. Then $u_{\epsilon }$ possesses a maximum point $y_{\epsilon }$ such that, up to a subsequence, $\epsilon y_{\epsilon }\rightarrow x_0$ as $\epsilon \rightarrow 0$, $\lim _{\epsilon \rightarrow 0}$dist$(\epsilon y_{\epsilon }, {\mathscr {A}}_{v})=0$ and $v_{\epsilon }(x):=u_{\epsilon }(x+y_{\epsilon })$ converges strongly to a positive ground state solution of

$$\begin{aligned} -\Delta _{p} u-\Delta _{q} u +V(x_{0})(|u|^{p-2}u+|u|^{q-2}u)=K(x_{0})f(u)\quad \hbox {in}~\mathbb {R}^{N}. \end{aligned}$$

In particular, if ${\mathscr {V}}\cap {\mathscr {K}}\ne \emptyset$, then $\lim _{\epsilon \rightarrow 0}dist(\epsilon y_{\epsilon },{\mathscr {V}}\cap {\mathscr {K}})=0$, and up to a subsequence, $v_{\epsilon }$ converges strongly to a positive ground state solution of

$$\begin{aligned} -\Delta _{p} u-\Delta _{q} u +V_{\min }(|u|^{p-2}u+|u|^{q-2}u)=K_{\max }f(u)\quad \hbox {in}~\mathbb {R}^{N}. \end{aligned}$$

Proof

Let $u_{\epsilon }\in {\mathscr {L}}_{\epsilon }$, we first show that there exist $\{{\tilde{y}}_{\epsilon }\}\subset \mathbb {R}^{N}$, $R_{0}>0$ and $\sigma _{0}>0$ such that

$$\begin{aligned} \int _{B_{R_{0}}({\tilde{y}}_{\epsilon })}|u_{\epsilon }|^{q}\mathrm {d}x\ge \sigma _{0} \end{aligned}$$

(5.9)

for all small $\epsilon >0$. Arguing indirectly, we assume that there exists a sequence $\epsilon _{j}\rightarrow 0$ as $j\rightarrow \infty$, such that for any $R_{1}>0$,

$$\begin{aligned} \lim _{j\rightarrow \infty }\sup \limits _{y\in {{\mathbb {R}}}^{N}} \int _{B_{R_{1}}(y)}|u_{\epsilon _{j}}|^{q}\mathrm {d}x=0. \end{aligned}$$

Then Lemma 2.2 yields that $u_{\epsilon _{j}}\rightarrow 0$ in $L^{s}(\mathbb {R}^N)$ for $s\in (q,q^*)$. From (2.3) we can infer that

$$\begin{aligned} \begin{aligned} 0&=\langle \Phi _{\epsilon }'(u_{\epsilon _{j}}),u_{\epsilon _{j}}\rangle \\&=\Vert \nabla u_{\epsilon _{j}}\Vert _{p}^{p}+\Vert \nabla u_{\epsilon _{j}}\Vert _{q}^{q}+\int _{\mathbb {R}^N}V(\epsilon _{j} x)(|u_{\epsilon _{j}}|^{p}+|u_{\epsilon _{j}}|^{q})\mathrm {d}x- \int _{\mathbb {R}^N}K(\epsilon _{j} x)f(u_{\epsilon _{j}})u_{\epsilon _{j}}\mathrm {d}x\\&\ge \Vert \nabla u_{\epsilon _{j}}\Vert _{p}^{p}+\Vert \nabla u_{\epsilon _{j}}\Vert _{q}^{q} +\int _{\mathbb {R}^N}V(\epsilon _{j} x)(|u_{\epsilon _{j}}|^{p}+|u_{\epsilon _{j}}|^{q})\mathrm {d}x\\&\quad - \varepsilon K_{\max }\Vert u_{\epsilon _{j}}\Vert _{p}^{p}-C_{\varepsilon }K_{\max }\Vert u_{\epsilon _{j}}\Vert _{r}^{r}\\&\ge c_{15}\Vert u_{\epsilon _{j}}\Vert _{V_{\epsilon },p}^{p}+c_{16}\Vert u_{\epsilon _{j}}\Vert _{V_{\epsilon },q}^{q}-c_{17}\Vert u_{\epsilon _{j}}\Vert _{r}^{r}, \end{aligned} \end{aligned}$$

and consequently

$$\begin{aligned} c_{15}\Vert u_{\epsilon _{j}}\Vert _{V_{\epsilon },p}^{p}+c_{16}\Vert u_{\epsilon _{j}}\Vert _{V_{\epsilon },q}^{q}\le c_{17}\Vert u_{\epsilon _{j}}\Vert _{r}^{r}\rightarrow 0 \end{aligned}$$

since $r\in (q,q^*)$. Then, $u_{\epsilon _{j}}\rightarrow 0$ in $E_{\epsilon }$. However, $\{u_{\epsilon _{j}}\}$ has positive bounded from below by Lemma 2.6, a contradiction. So (5.9) holds.

Let $\{y_{\epsilon }\}\subset \mathbb {R}^{N}$ be maximum point of $u_{\epsilon }$, then $u_{\epsilon }(y_{\epsilon })=\max _{x\in \mathbb {R}^{N}}u_{\epsilon }(x)$. We show that there exist $\eta _{0}>0$ independent of $\epsilon$ such that $u_{\epsilon }(y_{\epsilon })\ge \eta _{0}$ for all small $\epsilon >0$. If not, then we assume that $u_{\epsilon }(y_{\epsilon })\rightarrow 0$ as $\epsilon \rightarrow 0$. Form (5.9) we can deduce that

$$\begin{aligned} 0<\sigma _{0}\le \int _{B_{R_{0}}({\tilde{y}}_{\epsilon })}|u_{\epsilon }|^{q}\mathrm {d}x\le c|u_{\epsilon }(y_{\epsilon })|^{q}\rightarrow 0 \quad \hbox {as}~\epsilon \rightarrow 0, \end{aligned}$$

which is a contradiction. Therefore, there exist $R>R_0>0$ and $\sigma >0$ such that for all small $\epsilon >0$

$$\begin{aligned} \int _{B_{R}(y_{\epsilon })}|u_{\epsilon }|^{q}\mathrm {d}x\ge \sigma >0. \end{aligned}$$

(5.10)

Setting $v_{\epsilon }=u_{\epsilon }(x+y_{\epsilon })$, passing to a subsequence, we may assume that $v_{\epsilon }\rightharpoonup v$ in $E_{\epsilon }$ and $v_{\epsilon }\rightarrow v$ in $L_{loc}^{s}(\mathbb {R}^N)$ for $s\in [1,q^*)$, moreover, $v\ne 0$ by (5.10). It is obvious that $v_{\epsilon }$ is a solution of the problem

$$\begin{aligned} -\Delta _{p} u-\Delta _{q} u +V(\epsilon x+\epsilon y_{\epsilon })(|u|^{p-2}u+|u|^{q-2}u) =K(\epsilon x+\epsilon y_{\epsilon })f(u)~ \hbox {in}~\mathbb {R}^{N}. \end{aligned}$$

(5.11)

and the energy

$$\begin{aligned} \begin{aligned} \mathcal {T}_{\epsilon }(v_{\epsilon })&=\frac{1}{p}\Vert \nabla v_{\epsilon }\Vert ^{p}_{p}+\frac{1}{q}\Vert \nabla v_{\epsilon }\Vert ^{q}_{q} +\frac{1}{p}\int _{\mathbb {R}^{N}}V(\epsilon x+\epsilon y_{\epsilon })|v_{\epsilon }|^{p}\mathrm {d}x\\&\quad +\frac{1}{q}\int _{\mathbb {R}^{N}}V(\epsilon x+\epsilon y_{\epsilon })|v_{\epsilon }|^{q}\mathrm {d}x -\int _{\mathbb {R}^{N}}K(\epsilon x+\epsilon y_{\epsilon })F(v_{\epsilon })\mathrm {d}x\\&=\mathcal {T}_{\epsilon }(v_{\epsilon })-\frac{1}{q}\langle \mathcal {T}'_{\epsilon }(v_{\epsilon }),v_{\epsilon }\rangle \\&=\left( \frac{1}{p}-\frac{1}{q}\right) \Vert v_{\epsilon }\Vert ^{p}_{V_{\epsilon },p}+\int _{\mathbb {R}^N}K(\epsilon x+\epsilon y_{\epsilon })\left( \frac{1}{q}f(v_{\epsilon })v_{\epsilon }-F(v_{\epsilon })\right) \mathrm {d}x\\&=\Phi _{\epsilon }(u_{\epsilon })-\frac{1}{q}\langle \Phi '_{\epsilon }(u_{\epsilon }),u_{\epsilon }\rangle \\&=\Phi _{\epsilon }(u_{\epsilon })=c_{\epsilon }. \end{aligned} \end{aligned}$$

(5.12)

Moreover, for any $\varphi \in C_{0}^{\infty }(\mathbb {R}^N)$ there holds

$$\begin{aligned} \begin{aligned} \langle \mathcal {T}'_{\epsilon }(v_{\epsilon }),\varphi \rangle&=\int _{\mathbb {R}^N}|\nabla v_{\epsilon }|^{p-2}\nabla v_{\epsilon }\nabla \varphi \mathrm {d}x+ \int _{\mathbb {R}^N}|\nabla v_{\epsilon }|^{q-2}\nabla v_{\epsilon }\nabla \varphi \mathrm {d}x\\&\quad +\int _{\mathbb {R}^{N}}V(\epsilon x+\epsilon y_{\epsilon })|v_{\epsilon }|^{p-2}v_{\epsilon }\varphi \mathrm {d}x+ \int _{\mathbb {R}^{N}}V(\epsilon x+\epsilon y_{\epsilon })|v_{\epsilon }|^{q-2}v_{\epsilon }\varphi \mathrm {d}x\\&\quad -\int _{\mathbb {R}^{N}}K(\epsilon x+\epsilon y_{\epsilon })f(v_{\epsilon })\varphi \mathrm {d}x=0. \end{aligned} \end{aligned}$$

(5.13)

According to condition $(A_{0})$, without loss of generality, we can assume that $V(\epsilon y_{\epsilon })\rightarrow V_0$ and $K(\epsilon y_{\epsilon })\rightarrow K_0$ as $\epsilon \rightarrow 0$.

Next we complete our proof by several steps.

Step 1. We show that v is a positive ground state solution of the limit problem

$$\begin{aligned} -\Delta _{p} u-\Delta _{q} u +V_{0}(|u|^{p-2}u+|u|^{q-2}u)=K_{0}f(u)~ \hbox {in}~\mathbb {R}^{N}. \end{aligned}$$

(5.14)

Indeed, since $v_{\epsilon }\rightharpoonup v$ in $E_{\epsilon }$, applying some standard arguments, we can easily check that

$$\begin{aligned} \begin{aligned} \int _{\mathbb {R}^{N}}V(\epsilon x+\epsilon y_{\epsilon })|v_{\epsilon }|^{p-2}v_{\epsilon }\varphi \mathrm {d}x&=\int _{supp\varphi }V(\epsilon x+\epsilon y_{\epsilon })|v_{\epsilon }|^{p-2}v_{\epsilon }\varphi \mathrm {d}x\\&\rightarrow \int _{\mathbb {R}^{N}}V_{0}|v|^{p-2}v\varphi \mathrm {d}x. \end{aligned} \end{aligned}$$

Similarly, we have

$$\begin{aligned} \int _{\mathbb {R}^{N}}V(\epsilon x+\epsilon y_{\epsilon })|v_{\epsilon }|^{q-2}v_{\epsilon }\varphi \mathrm {d}x \rightarrow \int _{\mathbb {R}^{N}}V_{0}|v|^{q-2}v\varphi \mathrm {d}x \end{aligned}$$

and

$$\begin{aligned} \int _{\mathbb {R}^{N}}K(\epsilon x+\epsilon y_{\epsilon })f(v_{\epsilon })\varphi \mathrm {d}x \rightarrow \int _{\mathbb {R}^{N}}K_{0}f(v)\varphi \mathrm {d}x. \end{aligned}$$

According to (5.13), we can see that v is a solution of problem (5.14). Moreover,

$$\begin{aligned} \begin{aligned} \mathcal {J}_{V_{0}K_{0}}(v)&=\mathcal {J}_{V_{0}K_{0}}(v)-\frac{1}{q}\langle \mathcal {J}'_{V_{0}K_{0}}(v),v\rangle \\&=\left( \frac{1}{p}-\frac{1}{q}\right) \Vert v\Vert _{V_{0},p}^{p}+K_{0}\int _{\mathbb {R}^{N}}\left( \frac{1}{q}f(v)v-F(v)\right) \mathrm {d}x\\&\ge c_{V_{0}K_{0}}, \end{aligned} \end{aligned}$$

where $c_{V_{0}K_{0}}$ is the ground state energy value of $\mathcal {J}_{V_{0}K_{0}}$. On the other hand, from Fatou’s lemma and Lemma 5.1, we can deduce that

$$\begin{aligned} \begin{aligned} c_{V_{0}K_{0}}&\le \left( \frac{1}{p}-\frac{1}{q}\right) \Vert v\Vert _{V_{0},p}^{p}+K_{0}\int _{\mathbb {R}^{N}}\left( \frac{1}{q}f(v)v-F(v)\right) \mathrm {d}x\\&\le \liminf _{\epsilon \rightarrow 0}\left[ \left( \frac{1}{p}-\frac{1}{q}\right) \Vert v_{\epsilon }\Vert _{\epsilon ,p}^{p} +\int _{\mathbb {R}^{N}}K(\epsilon x+\epsilon y_{\epsilon })\left( \frac{1}{q}f(v_{\epsilon })v_{\epsilon }-F(v_{\epsilon })\right) \mathrm {d}x\right] \\&=\liminf _{\epsilon \rightarrow 0}\mathcal {T}_{\epsilon }(v_{\epsilon })\le \limsup _{\epsilon \rightarrow 0}\Phi _{\epsilon }(u_{\epsilon })\le c_{V_{0}K_{0}}. \end{aligned} \end{aligned}$$

Therefore, v is a ground state solution of problem (5.14). According to the proof of Lemma 3.3, we can prove that v is positive. Moreover, we also have

$$\begin{aligned} \lim _{\epsilon \rightarrow 0}\mathcal {T}_{\epsilon }(v_{\epsilon }) =\lim _{\epsilon \rightarrow 0}c_{\epsilon }=\mathcal {J}_{V_{0}K_{0}}(v)=c_{V_{0}K_{0}}. \end{aligned}$$

(5.15)

Step 2. We claim that $\{\epsilon y_{\epsilon }\}$ is bounded. Arguing by contradiction we assume that, up to a subsequence, $|\epsilon y_{\epsilon }|\rightarrow \infty$. Since

$$\begin{aligned} V(0)=V_{\min }~\hbox {and}~\kappa :=K(0)\ge K(x)~\hbox {for all}~|x|\ge R, \end{aligned}$$

then using $(A_1)$ we can infer that $V_{0}>V_{\min }$ and $K_{0}\le \kappa$. So it follows from Lemma 3.4 that $c_{V_{0}K_{0}}>c_{V_{\min }\kappa }$. But, according to Lemma 5.1 and (5.15), we can get $c_{\epsilon }\rightarrow c_{V_{0}K_{0}}\le c_{V_{\min }\kappa }$. This is a contradiction. So, $\{\epsilon y_{\epsilon }\}$ is bounded.

It follows from Step 2 that, passing to a subsequence, we can assume that $\epsilon y_{\epsilon }\rightarrow x_{0}$ as $\epsilon \rightarrow 0$, then $V_{0}=V(x_0)$ and $K_0=K(x_0)$. Therefore, according to Step 1, we can see that v is a positive ground state solution of the limit problem

$$\begin{aligned} -\Delta _{p} u-\Delta _{q} u +V(x_0)(|u|^{p-2}u+|u|^{q-2}u)=K(x_0)f(u)~ \hbox {in}~\mathbb {R}^{N}. \end{aligned}$$

Step 3. We prove that $\lim _{\epsilon \rightarrow 0}$dist$(\epsilon y_{\epsilon }, {\mathscr {A}}_{v})=0$. In fact, we just need to prove $x_0\in {\mathscr {A}}_{v}$. Arguing by contradiction, we assume that $x_0\notin {\mathscr {A}}_{v}$, then we get $V_{0}\ge V_{\min }$ and $K_{0}<\kappa$ by condition $(A_1)$ and the definition of ${\mathscr {A}}_{v}$. Moreover, using Lemma 3.4, we have $c_{V_{0}K_{0}}>c_{V_{\min }\kappa }$. Thus, from (5.15) and Lemma 5.1, we can deduce that

$$\begin{aligned} \lim _{\epsilon \rightarrow 0}c_{\epsilon }=c_{V_{0}K_{0}}>c_{V_{\min }\kappa }\ge \lim _{\epsilon \rightarrow 0}c_{\epsilon }, \end{aligned}$$

which is a contradiction.

Step 4. We verify that $v_{\epsilon }\rightarrow v$ in $W^{1,p}(\mathbb {R}^N)\cap W^{1,q}(\mathbb {R}^N)$. We use some ideas developed in [17]. Let $\eta :[0,\infty )\rightarrow [0,1]$ be a smooth function satisfying $\eta (t)=1$ if $t\le 1$, $\eta (t)=0$ if $t\ge 2$. Define ${\tilde{v}}_{\epsilon }(x)=\eta (2\epsilon |x|)v(x)$. By straightforward computation, we have

$$\begin{aligned} \Vert v-{\tilde{v}}_{\epsilon }\Vert _{\epsilon }\rightarrow 0\quad \hbox {and}\quad \Vert v-{\tilde{v}}_{\epsilon }\Vert _{s}\rightarrow 0\quad \hbox {as}~\epsilon \rightarrow 0 \end{aligned}$$

(5.16)

for $s\in [p,q^*]$. Setting $w_{\epsilon }=v_{\epsilon }-{\tilde{v}}_{\epsilon }$, it is easy to verify by applying Lemma 4.1 that up to a subsequence,

$$\begin{aligned} \lim _{\epsilon \rightarrow 0}\left| \int _{\mathbb {R}^N}K(\epsilon x+\epsilon y_{\epsilon })(F(v_{\epsilon })-F(w_{\epsilon })-F({\tilde{v}}_{\epsilon }))\mathrm {d}x\right| =0 \end{aligned}$$

(5.17)

and

$$\begin{aligned} \lim _{\epsilon \rightarrow 0}\left| \int _{\mathbb {R}^N}K(\epsilon x+\epsilon y_{\epsilon })(f(v_{\epsilon })-f(w_{\epsilon })-f({\tilde{v}}_{\epsilon }))\varphi \mathrm {d}x\right| =0 \end{aligned}$$

(5.18)

uniformly in $\varphi \in E_{\epsilon }$ with $\Vert \varphi \Vert _{\epsilon }\le 1$. Using the exponentially decay of v and (5.16) one checks easily the following

$$\begin{aligned} \lim _{\epsilon \rightarrow 0}\int _{\mathbb {R}^N}V(\epsilon x+\epsilon y_{\epsilon })(|{\tilde{v}}_{\epsilon }|^{p}+|{\tilde{v}}_{\epsilon }|^{q})\mathrm {d}x \rightarrow \int _{\mathbb {R}^N}V_{0}(|v|^{p}+|v|^{q})\mathrm {d}x \end{aligned}$$

(5.19)

and

$$\begin{aligned} \lim _{\epsilon \rightarrow 0}\int _{\mathbb {R}^N}K(\epsilon x+\epsilon y_{\epsilon })F({\tilde{v}}_{\epsilon })\mathrm {d}x \rightarrow \int _{\mathbb {R}^N}K_{0}F(v)\mathrm {d}x. \end{aligned}$$

(5.20)

From (5.15), (5.16), (5.17), (5.19) and (5.20), we are lead to

$$\begin{aligned} \begin{aligned} \mathcal {T}_{\epsilon }(w_{\epsilon })&=\mathcal {T}_{\epsilon }(v_{\epsilon })-{{\mathcal {J}}}_{V_{0}K_{0}}(v)\\&\quad +\int _{\mathbb {R}^N}K(\epsilon x+\epsilon y_{\epsilon })(F(v_{\epsilon })-F(w_{\epsilon })-F({\tilde{v}}_{\epsilon }))\mathrm {d}x+o_{\epsilon }(1)\\&= o_{\epsilon }(1), \end{aligned} \end{aligned}$$

which implies that $\mathcal {T}_{\epsilon }(w_{\epsilon })\rightarrow 0$. Similarly, according to (5.18), we get $\mathcal {T}'_{\epsilon }(w_{\epsilon })\rightarrow 0$. Therefore, by the condition $(f_4)$, we obtain

$$\begin{aligned} \begin{aligned} \mathcal {T}_{\epsilon }(w_{\epsilon })-\frac{1}{\theta }\langle \mathcal {T}'_{\epsilon }(w_{\epsilon }),w_{\epsilon }\rangle&=\left( \frac{1}{p}-\frac{1}{\theta }\right) \Vert w_{\epsilon }\Vert _{V_{\epsilon }, p}^{p} +\left( \frac{1}{q}-\frac{1}{\theta }\right) \Vert w_{\epsilon }\Vert _{V_{\epsilon }, q}^{q}\\&\quad +\int _{\mathbb {R}^N}K(\epsilon x+\epsilon y_{\epsilon })\left( \frac{1}{\theta }f(w_\epsilon )w_\epsilon -F(w_\epsilon )\right) \mathrm {d}x\\&\ge \left( \frac{1}{p}-\frac{1}{\theta }\right) \Vert w_{\epsilon }\Vert _{V_{\epsilon }, p}^{p} +\left( \frac{1}{q}-\frac{1}{\theta }\right) \Vert w_{\epsilon }\Vert _{V_{\epsilon }, q}^{q}, \end{aligned} \end{aligned}$$

which shows that $\Vert w_\epsilon \Vert _{\epsilon }\rightarrow 0$. This, together with (5.16), implies that $v_{\epsilon }\rightarrow v$ in $E_{\epsilon }$. But $\Vert \cdot \Vert _{\epsilon }$ and the norm of $W^{1,p}(\mathbb {R}^N)\cap W^{1,q}(\mathbb {R}^N)$ are equivalent. Consequently, the desired conclusion holds.

Finally, if ${\mathscr {V}}\cap {\mathscr {K}}\ne \emptyset$, using condition $(A_1)$, we have ${\mathscr {A}}_{v}={\mathscr {V}}\cap {\mathscr {K}}$. From the above arguments, we can prove that $\lim _{\epsilon \rightarrow 0}\mathrm{dist}\,(\epsilon y_{\epsilon },{\mathscr {V}}\cap {\mathscr {K}})=0$. So, $x_0\in {\mathscr {V}}\cap {\mathscr {K}}$, $V(x_{0})=V_{\min }$ and $K(x_{0})=K_{\max }$. Hence, up to a subsequence, $v_{\epsilon }$ converges in $W^{1,p}(\mathbb {R}^N)\cap W^{1,q}(\mathbb {R}^N)$ to a positive ground state solution v of the limit problem

$$\begin{aligned} -\Delta _{p} u-\Delta _{q} u +V_{\min }(|u|^{p-2}u+|u|^{q-2}u)=K_{\max }f(u)~ \hbox {in}~\mathbb {R}^{N}. \end{aligned}$$

Combining the above steps, we prove all conclusions of Lemma 5.4. $\square$

In the following we study the exponential decay property of solutions.

Lemma 5.5

$v_{\epsilon }\in C_{loc}^{1,\sigma }(\mathbb {R}^{N})$ with $\sigma \in (0,1)$ and $v_{\epsilon }(x)\rightarrow 0$ as $|x|\rightarrow \infty$ uniformly holds for all small $\epsilon >0$.

Proof

We follow some ideas developed in [21, 35]. Using the interior $L^{p}$-estimate and the Sobolev embedding, we get $v_\epsilon \in L^{s}(\mathbb {R}^N)\cap C_{loc}^{1,\sigma }(\mathbb {R}^N)$ for $s\in [p,\infty ]$ and $\sigma \in (0,1)$. Moreover, from the Step 4 of proof of Lemma 5.4, we have $v_{\epsilon }\rightarrow v$ in $W^{1,p}(\mathbb {R}^N)\cap W^{1,q}(\mathbb {R}^N)$. So, applying the Moser iterative method in [6, 27], we can prove that $v_{\epsilon }(x)\rightarrow 0$ as $|x|\rightarrow \infty$ uniformly holds for all small $\epsilon >0$. The details of the proof can be found in [6,  Lemma 7.1], here we omit it. $\square$

Lemma 5.6

There are c, $C>0$ such that for all small $\epsilon >0$, there holds

$$\begin{aligned} u_{\epsilon }(x)\le C\exp \left( -c|x-y_{\epsilon }|\right) . \end{aligned}$$

Proof

We adapt some arguments from [21] (see also [6]). According to Lemma 5.5 and $(f_2)$, there exists $R>0$ such that

$$\begin{aligned} K_{\max }f(v_\epsilon )\le \frac{V_{\min }}{2}(v^{p-1}_{\epsilon }+v^{q-1}_{\epsilon })~\hbox {for all}~|x|\ge R. \end{aligned}$$

Then, for $|x|\ge R$ we get

$$\begin{aligned} \begin{aligned}&-\Delta _{p}v_{\epsilon }-\Delta _{q}v_{\epsilon }+\frac{V_{\min }}{2}(v^{p-1}_{\epsilon }+v^{q-1}_{\epsilon })\\&\quad =K(\epsilon x+\epsilon y_{\epsilon })f(v_\epsilon )-\left( V(\epsilon x+\epsilon y_{\epsilon })-\frac{V_{\min }}{2}\right) (v^{p-1}_{\epsilon }+v^{q-1}_{\epsilon })\\&\quad \le K_{\max }f(v_\epsilon )-\frac{V_{\min }}{2}(v^{p-1}_{\epsilon }+v^{q-1}_{\epsilon })\le 0. \end{aligned} \end{aligned}$$

(5.21)

Let $\psi (x)=C_{0}\exp (-c_{0}|x|)$ with $c_{0}, C_{0}>0$ such that $c_{0}^{p}(p-1)<\frac{V_{\min }}{2}$, $c_{0}^{q}(q-1)<\frac{V_{\min }}{2}$ and $v_{\epsilon }(x)\le C_{0}\exp (-c_{0}R)$ for all $|x|=R$. Then, computing directly, we have

$$\begin{aligned} \begin{aligned}&-\Delta _{p}\psi -\Delta _{q}\psi +\frac{V_{\min }}{2}(\psi ^{p-1}+\psi ^{q-1})\\&\quad = \psi ^{p-1}\left( \frac{V_{\min }}{2}-c_{0}^{p}(p-1)+\frac{N-1}{|x|}c_{0}^{p-1}\right) \\&\qquad +\psi ^{q-1}\left( \frac{V_{\min }}{2}-c_{0}^{q}(q-1)+\frac{N-1}{|x|}c_{0}^{q-1}\right) \\&\quad >0 \end{aligned} \end{aligned}$$

(5.22)

for all $|x|\ge R$. Let $\Sigma =\{|x|\ge R\}\cap \{v_{\epsilon }>\psi \}$. Using the following inequality

$$\begin{aligned} (|x|^{s-2}x-|y|^{s-2}y)\cdot (x-y)\ge 0~\hbox {for all}~s>1~\hbox {and}~x,y\in \mathbb {R}^N \end{aligned}$$

and choosing $\phi =\max \{v_{\epsilon }-\psi ,0\}\in W_{0}^{1,p}(\mathbb {R}^{N}\backslash B_{R})\cap W_{0}^{1,q}(\mathbb {R}^{N}\backslash B_{R})$ as a test function in (5.21) and (5.22), we obtain

$$\begin{aligned} \begin{aligned} 0&\ge \int _{\Sigma }\left[ (|\nabla v_{\epsilon }|^{p-2}\nabla v_{\epsilon }-|\nabla \psi |^{p-2}\psi )\cdot \nabla \phi +(|\nabla v_{\epsilon }|^{q-2}\nabla v_{\epsilon }-|\nabla \psi |^{q-2}\psi )\cdot \nabla \phi \right] \mathrm {d}x\\&\quad +\frac{V_{\min }}{2}\int _{\Sigma }\left[ (v^{p-1}_{\epsilon }-\psi ^{p-1})+(v^{q-1}_{\epsilon }-\psi ^{q-1})\right] \phi \mathrm {d}x\\&\ge 0. \end{aligned} \end{aligned}$$

Therefore, the set $\Sigma$ is empty. From this we can easily conclude that $v_{\epsilon }(x)\le \psi (x)$ for all $|x|\ge R$, and

$$\begin{aligned} v_{\epsilon }(x)\le \psi (x)=C_{0}\exp (-c_{0}|x|)\quad \hbox {for all}~|x|\ge R. \end{aligned}$$

Recalling that $u_{\epsilon }(x)=v_{\epsilon }(x-y_{\epsilon })$, there exist $c, C>0$ such that

$$\begin{aligned} u_{\epsilon }(x)\le C\exp (-c|x-y_{\epsilon }|) \end{aligned}$$

for all $x\in \mathbb {R}^N$ and small $\epsilon >0$. The proof is completed. $\square$

Now we are in a position to finish the proofs of Theorems 1.1 and 1.2.

Proof of Theorem 1.1

From Lemma 5.2, problem (2.1) has a positive ground state solution $v_{\epsilon }$ for $\epsilon >0$ small enough. Hence, $u_{\epsilon }(x)=v_{\epsilon }(\frac{x}{\epsilon })$ is a positive ground state solution of problem (1.1), which shows that conclusion (i) holds in Theorem 1.1. By Lemma 5.3 we deduce that conclusion (ii) holds in Theorem 1.1. Furthermore, the maximum points $x_\epsilon$ and $y_{\epsilon }$ of $u_{\epsilon }$ and $v_{\epsilon }$ satisfy $x_\epsilon =\epsilon y_{\epsilon }$. Setting ${\hat{v}}_{\epsilon }:=u_{\epsilon }(\epsilon x+x_{\epsilon })$. Then by Lemma 5.4 we get

$$\begin{aligned} x_{\epsilon }\rightarrow x_{0}\quad \hbox {and}\quad \lim \limits _{\epsilon \rightarrow 0}\hbox {dist}(x_{\epsilon }, {\mathscr {A}}_{v})=0, \end{aligned}$$

and ${\hat{v}}_{\epsilon }$ converges in $W^{1,p}(\mathbb {R}^N)\cap W^{1,q}(\mathbb {R}^N)$ to a positive ground state solution v of

$$\begin{aligned} -\Delta _{p} u-\Delta _{q} u +V(x_0)(|u|^{p-2}u+|u|^{q-2}u)=K(x_0)f(u)\quad \hbox {in}~\mathbb {R}^{N}. \end{aligned}$$

In particular, if ${\mathscr {V}}\cap {\mathscr {K}}\ne \emptyset$, then ${\mathscr {A}}_{v}={\mathscr {V}}\cap {\mathscr {K}}$ and

$$\begin{aligned} \lim \limits _{\epsilon \rightarrow 0}\hbox {dist}(x_{\epsilon }, {\mathscr {V}}\cap {\mathscr {K}})=0, \end{aligned}$$

and ${\hat{v}}_{\epsilon }$ converges in $W^{1,p}(\mathbb {R}^N)\cap W^{1,q}(\mathbb {R}^N)$ to a positive ground state solution v of

$$\begin{aligned} -\Delta _{p} u-\Delta _{q} u +V_{\min }(|u|^{p-2}u+|u|^{q-2}u)=K_{\max }f(u)\quad \hbox {in}~\mathbb {R}^{N}. \end{aligned}$$

Evidently, conclusion (iii) holds.

Finally, according to Lemmas 5.5 and 5.6, we can get $u_{\epsilon }\in C_{loc}^{1,\sigma }(\mathbb {R}^{N})$ with $\sigma \in (0,1)$ and $\lim _{|x|\rightarrow \infty }u_{\epsilon }(x)=0$. Moreover,

$$\begin{aligned} |u_{\epsilon }(x)|=\left| v_{\epsilon }\left( \frac{x}{\epsilon }\right) \right| \le C\exp \left( -c|\frac{x}{\epsilon }-y_{\epsilon }|\right) =C\exp \left( -\frac{c}{\epsilon }|x-x_{\epsilon }|\right) \end{aligned}$$

for some $c, C>0$. We now complete the proof of Theorem 1.1. $\square$

Proof of Theorem 1.2

Suppose that the potentials V and K satisfy conditions ($A_{0}$) and ($A_{2}$). We may assume without loss of generality that $x_{k}=0\in {\mathscr {K}}$ or $x_{k}=0\in {\mathscr {V}}\cap {\mathscr {K}}$ if ${\mathscr {V}}\cap {\mathscr {K}}\ne \emptyset$. It follows that

$$\begin{aligned} \pi :=V(0)\le V(x)~\hbox {for all}~|x|\ge R~\hbox {and}~ K(0)=K_{\max }. \end{aligned}$$

Analogous to the proofs of Lemma 5.1, we can show that

$$\begin{aligned} \limsup \limits _{\epsilon \rightarrow 0}c_{\epsilon }\le c_{\pi K_{\max }}. \end{aligned}$$

The remaining proofs are similar to the proof of Theorem 1.1 with suitable modification, so we omit the details here. $\square$

6 Multiplicity of positive solutions

In this section we are going to show the multiplicity of positive solutions and study the behavior of their maximum points in relationship with the set $\Lambda$, where $\Lambda$ is defined in Sect. 1. Moreover, we complete the proof of Theorem 1.3.

Let u be a positive ground state solution of problem

$$\begin{aligned} -\Delta _{p} u-\Delta _{q} u +V_{\min }(|u|^{p-2}u+|u|^{q-2}u)=K_{\max }f(u)\quad \hbox {in}~\mathbb {R}^{N}, \end{aligned}$$

(6.1)

and $\zeta$ be a smooth nonincreasing cut-off function in $[0,+\infty )$ such that $\zeta (s)=1$ if $0\le s\le \frac{1}{2}$ and $\zeta (s)=0$ if $s\ge 1$. For any $y\in \Lambda$, we define

$$\begin{aligned} \Psi _{\epsilon ,y}(x)=\zeta (|\epsilon x-y|)u \left( \frac{\epsilon x-y}{\epsilon }\right) . \end{aligned}$$

Then, there exists $t_{\epsilon }>0$ such that

$$\begin{aligned} \max _{t\ge 0}\Phi _{\epsilon }(t\Psi _{\epsilon ,y})=\Phi _{\epsilon }(t_{\epsilon }\Psi _{\epsilon ,y}). \end{aligned}$$

We define $\gamma _{\epsilon }: \Lambda \rightarrow {\mathscr {N}}_\epsilon$ by $\gamma _{\epsilon }(y)=t_{\epsilon }\Psi _{\epsilon ,y}$. By the construction, $\gamma _{\epsilon }(y)$ has compact support for any $y\in \Lambda$.

Lemma 6.1

The function $\gamma _{\epsilon }$ satisfies

$$\begin{aligned} \lim _{\epsilon \rightarrow 0}\Phi _{\epsilon }(\gamma _{\epsilon }(y))=c_{V_{\min }K_{\max }}~\text { uniformly in}~y\in \Lambda . \end{aligned}$$

Proof

Suppose by contradiction that there exist $\varepsilon _{0}>0$, $\{y_n\}\subset \Lambda$ and $\epsilon _{n}\rightarrow 0$ such that

$$\begin{aligned} |\Phi _{\epsilon _{n}}(\gamma _{\epsilon _{n}}(y))-c_{V_{\min }K_{\max }}|\ge \varepsilon _{0}. \end{aligned}$$

(6.2)

Observe that, by Lebesgue’s dominated convergence theorem, we can easily check that

$$\begin{aligned} \Vert \nabla \Psi _{\epsilon _{n},y_{n}}\Vert _{p}^{p}+\int _{\mathbb {R}^N}V(\epsilon _{n}x)|\Psi _{\epsilon _{n},y_{n}}|^{p}\mathrm {d}x\rightarrow \Vert \nabla u\Vert _{p}^{p}+\int _{\mathbb {R}^N}V_{\min }|u|^{p}\mathrm {d}x, \end{aligned}$$

(6.3)

$$\begin{aligned} \Vert \nabla \Psi _{\epsilon _{n},y_{n}}\Vert _{q}^{q}+\int _{\mathbb {R}^N}V(\epsilon _{n}x)|\Psi _{\epsilon _{n},y_{n}}|^{q}\mathrm {d}x\rightarrow \Vert \nabla u\Vert _{q}^{q}+\int _{\mathbb {R}^N}V_{\min }|u|^{q}\mathrm {d}x, \end{aligned}$$

(6.4)

and

$$\begin{aligned} \int _{\mathbb {R}^N}K(\epsilon _{n}x)F(\Psi _{\epsilon _{n},y_{n}})\mathrm {d}x\rightarrow K_{\max }\int _{\mathbb {R}^N}F(u)\mathrm {d}x. \end{aligned}$$

(6.5)

Using the fact that $\langle \Phi '_{\epsilon _{n}}(t_{\epsilon _{n}}\Psi _{\epsilon _{n},y_{n}}),t_{\epsilon _{n}}\Psi _{\epsilon _{n},y_{n}}\rangle =0$ and the change of variable $z=\frac{\epsilon _{n}x-y_{n}}{\epsilon _{n}}$, we obtain

$$\begin{aligned} \begin{aligned}&t^{p}_{\epsilon _{n}}\Vert \nabla \Psi _{\epsilon _{n},y_{n}}\Vert _{p}^{p}+t^{q}_{\epsilon _{n}}\Vert \nabla \Psi _{\epsilon _{n},y_{n}}\Vert _{q}^{q}+ \int _{\mathbb {R}^N}V(\epsilon _{n}x)(|t_{\epsilon _{n}}\Psi _{\epsilon _{n},y_{n}}|^{p}+|t_{\epsilon _{n}}\Psi _{\epsilon _{n},y_{n}}|^{q})\mathrm {d}x\\&\quad =\int _{\mathbb {R}^N}K(\epsilon _{n}x)f(t_{\epsilon _{n}}\Psi _{\epsilon _{n},y_{n}})t_{\epsilon _{n}}\Psi _{\epsilon _{n},y_{n}}\mathrm {d}x\\&\quad =\int _{\mathbb {R}^N}K(\epsilon _{n}z+y_{n})f(t_{\epsilon _{n}}\zeta (|\epsilon _{n}z|)u(z))t_{\epsilon _{n}}\zeta (|\epsilon _{n}z|)u(z)\mathrm {d}z. \end{aligned} \end{aligned}$$

(6.6)

We claim that $t_{\epsilon _{n}}\rightarrow 1$. First we need to prove that $\{t_{\epsilon _{n}}\}$ is bounded. In fact, assume by contradiction that $t_{\epsilon _{n}}\rightarrow \infty$. Using (6.6) and $(f_5)$ we have

$$\begin{aligned} \begin{aligned}&t^{p-q}_{\epsilon _{n}}\Vert \nabla \Psi _{\epsilon _{n},y_{n}}\Vert _{p}^{p}+\Vert \nabla \Psi _{\epsilon _{n},y_{n}}\Vert _{q}^{q}+ \int _{\mathbb {R}^N}V(\epsilon _{n}x)(t^{p-q}_{\epsilon _{n}}|\Psi _{\epsilon _{n},y_{n}}|^{p}+|\Psi _{\epsilon _{n},y_{n}}|^{q})\mathrm {d}x\\&\quad =\int _{\mathbb {R}^N}K(\epsilon _{n}z+y_{n})f(t_{\epsilon _{n}}\zeta (|\epsilon _{n}z|)u(z))t_{\epsilon _{n}}\zeta (|\epsilon _{n}z|)u(z)t^{-q}_{\epsilon _{n}}\mathrm {d}z\\&\quad \ge K_{\min }\int _{B_{\frac{1}{2}}(0)}\frac{f(t_{\epsilon _{n}}u(z))}{(t_{\epsilon _{n}}u(z))^{q-1}}u(z)^{q}\mathrm {d}z\\&\quad \ge K_{\min }\frac{f(t_{\epsilon _{n}}u(z_{0}))}{(t_{\epsilon _{n}}u(z_{0}))^{q-1}}\int _{B_{\frac{1}{2}}(0)}u(z)^{q}\mathrm {d}z, \end{aligned} \end{aligned}$$

(6.7)

where $u(z_0)=\min \{u(z): |z|\le \frac{1}{2}\}>0$ (this is true because $u\in C(\mathbb {R}^N)$ by Lemma 5.5). Since $p<q$, then from $(f_4)$ and (6.7), we can deduce that $\Vert \Psi _{\epsilon _{n},y_{n}}\Vert _{V_{\epsilon },q}^{q}\rightarrow \infty$. Clearly, this contradicts relation (6.4). Hence, $\{t_{\epsilon _{n}}\}$ is bounded. Passing to a subsequence, we may assume that $t_{\epsilon _{n}}\rightarrow t_0\ge 0$. If $t_0=0$, by $(f_2)$, (6.4) and (6.6) we can infer that $\Vert \Psi _{\epsilon _{n},y_{n}}\Vert _{V_{\epsilon },p}^{p}\rightarrow 0$, this contradicts relation (6.3). We conclude that $t_0>0$.

Next, we prove that $t_0=1$. Letting $n\rightarrow \infty$ in (6.6), we obtain

$$\begin{aligned} t_{0}^{p-q}\Vert \nabla u\Vert _{p}^{p}+\Vert \nabla u\Vert _{q}^{q}+V_{\min }\int _{\mathbb {R}^N}(t_{0}^{p-q}|u|^{p}+|u|^{q})\mathrm {d}x= K_{\max }\int _{\mathbb {R}^N}\frac{f(t_0u)}{(t_0u)^{q-1}}u^{q}\mathrm {d}x. \end{aligned}$$

(6.8)

Moreover, since u is a positive ground state solution of problem (6.1)

$$\begin{aligned} \Vert \nabla u\Vert _{p}^{p}+\Vert \nabla u\Vert _{q}^{q}+V_{\min }\int _{\mathbb {R}^N}(|u|^{p}+|u|^{q})\mathrm {d}x= K_{\max }\int _{\mathbb {R}^N}f(u)u\mathrm {d}x. \end{aligned}$$

(6.9)

Combining (6.8) and (6.9), we can conclude that

$$\begin{aligned} (t_{0}^{p-q}-1)\Vert \nabla u\Vert _{p}^{p}+(t_{0}^{p-q}-1)V_{\min }\int _{\mathbb {R}^N}|u|^{p}\mathrm {d}x= K_{\max }\int _{\mathbb {R}^N}\left( \frac{f(t_0u)}{(t_0u)^{q-1}}-\frac{f(u)}{u^{q-1}}\right) u^{q}\mathrm {d}x. \end{aligned}$$

Then, we deduce from $(f_5)$ that $t_0=1$. Therefore, from (6.3), (6.4) and (6.5), we infer that

$$\begin{aligned} \begin{aligned} \Phi _{\epsilon _{n}}(\gamma _{\epsilon _{n}}(y_n))&=\frac{t^{p}_{\epsilon _{n}}}{p}\Vert \nabla \Psi _{\epsilon _{n},y_{n}}\Vert _{p}^{p} +\frac{t^{q}_{\epsilon _{n}}}{q}\Vert \nabla \Psi _{\epsilon _{n},y_{n}}\Vert _{q}^{q}\\ {}&\quad +\int _{\mathbb {R}^N}V(\epsilon _{n}x)\left( \frac{t^{p}_{\epsilon _{n}}}{p}|\Psi _{\epsilon _{n},y_{n}}|^{p} +\frac{t^{q}_{\epsilon _{n}}}{q}|\Psi _{\epsilon _{n},y_{n}}|^{q}\right) \mathrm {d}x\\ {}&\quad -\int _{\mathbb {R}^N}K(\epsilon _{n}x)F(t_{\epsilon _{n}}\Psi _{\epsilon _{n},y_{n}})\mathrm {d}x\\ {}&\quad \rightarrow \frac{1}{p}\Vert \nabla u\Vert _{p}^{p}+\frac{1}{q}\Vert \nabla u\Vert _{q}^{q}+\int _{\mathbb {R}^N}V_{\min }\left( \frac{1}{p}|u|^{p} +\frac{1}{q}|u|^{q}\right) \mathrm {d}x\\ {}&\quad -K_{\max }\int _{\mathbb {R}^N}F(u)\mathrm {d}x\\ {}&= \mathcal {J}_{V_{\min }K_{\max }(u)}=c_{V_{\min }K_{\max }}. \end{aligned} \end{aligned}$$

Obviously, from (6.2) we can see that this is impossible. The proof is completed. $\square$

Now, we are in the position to introduce the barycenter map. For any $\delta >0$, let $\rho =\rho (\delta )>0$ be such that $\Lambda _{\delta }\subset B_{\rho }(0)$. We define $\eta : \mathbb {R}^N\rightarrow \mathbb {R}^N$ as follows

$$\begin{aligned} \eta (x)=x~\hbox {for}~|x|\le \rho \quad \hbox {and}\quad \eta (x)=\frac{\rho x}{|x|}\quad \hbox {for}\quad |x|\ge \rho . \end{aligned}$$

Let us consider $\beta _{\epsilon }: {\mathscr {N}}_{\epsilon }\rightarrow \mathbb {R}^N$ given by

$$\begin{aligned} \beta _{\epsilon }(u)=\frac{\int _{\mathbb {R}^N}\eta (\epsilon x)(|u|^{p}+|u|^{q})\mathrm {d}x}{\int _{\mathbb {R}^N}(|u|^{p}+|u|^{q})\mathrm {d}x}. \end{aligned}$$

Using the above notations, we have the following result.

Lemma 6.2

The function $\beta _{\epsilon }$ satisfies

$$\begin{aligned} \lim _{\epsilon \rightarrow 0}\beta _{\epsilon }(\gamma _{\epsilon }(y))=y\quad \hbox {uniformly in}~y\in \Lambda . \end{aligned}$$

Proof

Arguing by contradiction, we assume that there exist $\sigma _{0}>0$, $\{y_n\}\subset \Lambda$ and $\epsilon _{n}\rightarrow 0$ such that

$$\begin{aligned} |\beta _{\epsilon _{n}}(\gamma _{\epsilon _{n}}(y_{n}))-y_n|\ge \sigma _{0}>0. \end{aligned}$$

(6.10)

According to the definitions of $\gamma _{\epsilon _{n}}$ and $\beta _{\epsilon _{n}}$, and using the change of variable $z=\frac{\epsilon _{n}x-y_n}{\epsilon _n}$ we get

$$\begin{aligned} \beta _{\epsilon _{n}}(\gamma _{\epsilon _{n}}(y_{n}))=y_n+\frac{\int _{\mathbb {R}^N}[\eta (\epsilon _{n}z+y_n)-y_n] (|\zeta (|\epsilon _{n}z|)u(z)|^{p}+|\zeta (|\epsilon _{n}z|)u(z)|^{q})\mathrm {d}z}{\int _{\mathbb {R}^N}(|\zeta (|\epsilon _{n}z|)u(z)|^{p}+|\zeta (|\epsilon _{n}z|)u(z)|^{q})\mathrm {d}z}. \end{aligned}$$

Taking into account $\{y_n\}\subset \Lambda \subset B_{\rho }(0)$ and using the Lebesgue’s dominates convergence theorem, we have

$$\begin{aligned} |\beta _{\epsilon _{n}}(\gamma _{\epsilon _{n}}(y_{n}))-y_n|\rightarrow 0, \end{aligned}$$

which contradicts relation (6.10). $\square$

Lemma 6.3

Let $\epsilon _{n}\rightarrow 0$ and $\{u_n\}\subset {\mathscr {N}}_{\epsilon _{n}}$ be a sequence satisfying $\Phi _{\epsilon _{n}}(u_n)\rightarrow c_{V_{\min }K_{\max }}$. Then there exists $\{{\tilde{y}}_{n}\}\subset \mathbb {R}^N$ such that $v_n=u_n(x+{\tilde{y}}_{n})$ has a convergent subsequence. Moreover, up to a subsequence, $y_{n}\rightarrow y\in \Lambda$, where $y_n=\epsilon _{n}{\tilde{y}}_{n}$.

Proof

Since $u_n\in {\mathscr {N}}_{\epsilon _{n}}$ and $\Phi _{\epsilon _{n}}(u_n)\rightarrow c_{V_{\min }K_{\max }}$, we have that $\{u_n\}$ is bounded. We claim that there are $R_{0},\delta >0$ and ${\tilde{y}}_n\in \mathbb {R}^N$ such that

$$\begin{aligned} \liminf _{n\rightarrow \infty }\int _{B_{R_{0}}({\tilde{y}}_n)}|u_n|^{q}\mathrm {d}x\ge \delta . \end{aligned}$$

(6.11)

Indeed, if relation (6.11) does not hold, Lemma 2.2 implies that $u_n\rightarrow 0$ in $L^{s}(\mathbb {R}^N)$ for $s\in (q,q^*)$. According to (2.3) and the fact $u_n\in {\mathscr {N}}_{\epsilon _{n}}$ it is easy to verify that $u_n\rightarrow 0$ in $E_{\epsilon }$, which is a contradiction, because $\Phi _{\epsilon _{n}}(u_n)\rightarrow c_{V_{\min }K_{\max }}>0$. So, (6.11) holds. Let us define $v_n(x)=u_n(x+{\tilde{y}}_{n})$. Passing to a subsequence, we may assume that $v_n\rightharpoonup v\ne 0$. By virtue of Lemma 2.7, there exists $t_{n}>0$ such that ${\tilde{v}}_{n}=t_{n}v_{n}\in {\mathscr {N}}_{V_{\min }K_{\max }}$. Then we have

$$\begin{aligned} c_{V_{\min }K_{\max }}\le \mathcal {J}_{V_{\min }K_{\max }}({\tilde{v}}_{n})=\mathcal {J}_{V_{\min }K_{\max }}(t_nu_{n}) \le \Phi _{\epsilon _{n}}(t_{n}u_{n})\le \Phi _{\epsilon _{n}}(u_{n})\rightarrow c_{V_{\min }K_{\max }}, \end{aligned}$$

which shows that $\mathcal {J}_{V_{\min }K_{\max }}({\tilde{v}}_{n})\rightarrow c_{V_{\min }K_{\max }}$. According to Lemma 3.1-(e), we know that $\{{\tilde{v}}_n\}$ is bounded. Thus, for some subsequence, ${\tilde{v}}_{n}\rightharpoonup {\tilde{v}}$ with ${\tilde{v}}\ne 0$. Moreover, $\mathcal {J}'_{V_{\min }K_{\max }}({\tilde{v}})=0$. Using Lemma 4.3 we have

$$\begin{aligned} \mathcal {J}_{V_{\min }K_{\max }}({\tilde{v}}_{n}-{\tilde{v}})\rightarrow c_{V_{\min }K_{\max }}-\mathcal {J}_{V_{\min }K_{\max }}({\tilde{v}}) \quad \hbox {and}\quad \mathcal {J}'_{V_{\min }K_{\max }}({\tilde{v}}_{n}-{\tilde{v}})\rightarrow 0. \end{aligned}$$

Observe that, from (2.4) we get

$$\begin{aligned} \begin{aligned} c_{V_{\min }K_{\max }}&=\lim _{n\rightarrow \infty }\left( \mathcal {J}_{V_{\min }K_{\max }}({\tilde{v}}_{n}) -\frac{1}{q}\langle \mathcal {J}'_{V_{\min }K_{\max }}({\tilde{v}}_{n}),{\tilde{v}}_{n}\rangle \right) \\&=\lim _{n\rightarrow \infty }\left[ \left( \frac{1}{p}-\frac{1}{q}\right) \Vert {\tilde{v}}_{n}\Vert _{V_{\min },p}^{p} +K_{\max }\int _{\mathbb {R}^N}\left( \frac{1}{q}f({\tilde{v}}_n){\tilde{v}}_{n}-F({\tilde{v}}_{n})\right) \mathrm {d}x\right] \\&\ge \left( \frac{1}{p}-\frac{1}{q}\right) \Vert {\tilde{v}}\Vert _{V_{\min },p}^{p} +K_{\max }\int _{\mathbb {R}^N}\left( \frac{1}{q}f({\tilde{v}}){\tilde{v}}-F({\tilde{v}})\right) \mathrm {d}x\\&= \mathcal {J}_{V_{\min }K_{\max }}({\tilde{v}})-\frac{1}{q}\langle \mathcal {J}'_{V_{\min }K_{\max }}({\tilde{v}}),{\tilde{v}}\rangle \\&= \mathcal {J}_{V_{\min }K_{\max }}({\tilde{v}})\\&\ge c_{V_{\min }K_{\max }}. \end{aligned} \end{aligned}$$

It follows that

$$\begin{aligned} \mathcal {J}_{V_{\min }K_{\max }}({\tilde{v}}_{n}-{\tilde{v}})\rightarrow 0\quad \hbox {and}\quad \mathcal {J}'_{V_{\min }K_{\max }}({\tilde{v}}_{n}-{\tilde{v}})\rightarrow 0. \end{aligned}$$

(6.12)

Moreover, using $(f_4)$ and (6.12) we have

$$\begin{aligned} \begin{aligned} o_{n}(1)&=\mathcal {J}_{V_{\min }K_{\max }}({\tilde{v}}_{n}-{\tilde{v}}) -\frac{1}{\theta }\langle \mathcal {J}'_{V_{\min }K_{\max }}({\tilde{v}}_{n}-{\tilde{v}}),{\tilde{v}}_{n}-{\tilde{v}}\rangle \\&= \left( \frac{1}{p}-\frac{1}{\theta }\right) \Vert {\tilde{v}}_{n}-{\tilde{v}}\Vert _{V_{\min }, p}^{p} +\left( \frac{1}{q}-\frac{1}{\theta }\right) \Vert {\tilde{v}}_{n}-{\tilde{v}}\Vert _{V_{\min }, q}^{q}\\&\quad +K_{\max }\int _{\mathbb {R}^N}\left( \frac{1}{\theta }f({\tilde{v}}_{n}-{\tilde{v}})({\tilde{v}}_{n}-{\tilde{v}})-F({\tilde{v}}_{n}-{\tilde{v}})\right) \mathrm {d}x\\&\ge \left( \frac{1}{p}-\frac{1}{\theta }\right) \Vert {\tilde{v}}_{n}-{\tilde{v}}\Vert _{V_{\min }, p}^{p} +\left( \frac{1}{q}-\frac{1}{\theta }\right) \Vert {\tilde{v}}_{n}-{\tilde{v}}\Vert _{V_{\min }, q}^{q}, \end{aligned} \end{aligned}$$

which implies that ${\tilde{v}}_n\rightarrow {\tilde{v}}$ in $E_{V_{\min }}$. Since $\{t_n\}$ is bounded, we can assume that $t_n\rightarrow t_0>0$, and so, $v_n\rightarrow v$ in $E_{V_{\min }}$.

Next, we prove that $\{y_n\}=\{\epsilon _{n}{\tilde{y}}_{n}\}$ has a subsequence satisfying $y_n\rightarrow y\in \Lambda$. We first claim that $\{y_n\}$ is bounded. In fact, assume by contradiction that $\{y_n\}$ is not bounded. Then, there exists a subsequence, still denoted by $\{y_n\}$, such that $|y_n|\rightarrow \infty$. From ${\tilde{v}}_n\rightarrow {\tilde{v}}$ in $E_{V_{\min }}$, $V_{\min }< V_{\infty }$ and $K_{\max }>K_{\infty }$, we can deduce that

$$\begin{aligned} \begin{aligned} c_{V_{\min }K_{\max }}&=\frac{1}{p}\Vert {\tilde{v}}\Vert _{V_{\min },p}^{p}+\frac{1}{q}\Vert {\tilde{v}}\Vert _{V_{\min },q}^{q}-K_{\max }\int _{\mathbb {R}^N}F({\tilde{v}})\mathrm {d}x\\&< \frac{1}{p}\Vert {\tilde{v}}\Vert _{V_{\infty },p}^{p}+\frac{1}{q}\Vert {\tilde{v}}\Vert _{V_{\infty },q}^{q}-K_{\infty }\int _{\mathbb {R}^N}F({\tilde{v}})\mathrm {d}x\\&\le \liminf _{n\rightarrow \infty }\bigg [\frac{1}{p}\int _{\mathbb {R}^N}|\nabla {\tilde{v}}_{n}|^{p}\mathrm {d}x+\frac{1}{q}\int _{\mathbb {R}^N}|\nabla {\tilde{v}}_{n}|^{q}\mathrm {d}x\\&\quad +\int _{\mathbb {R}^N}V(\epsilon _{n}x+y_n)\left( \frac{1}{p}|{\tilde{v}}_{n}|^{p}+\frac{1}{q}|{\tilde{v}}_{n}|^{q}\right) \mathrm {d}x -\int _{\mathbb {R}^N}K(\epsilon _{n}x+y_n)F({\tilde{v}}_{n})\mathrm {d}x\bigg ]\\&\le \liminf _{n\rightarrow \infty }\bigg [\frac{1}{p}\int _{\mathbb {R}^N}|\nabla t_nu_{n}|^{p}\mathrm {d}x+\frac{1}{q}\int _{\mathbb {R}^N}|\nabla t_nu_{n}|^{q}\mathrm {d}x\\&\quad +\int _{\mathbb {R}^N}V(\epsilon _{n}x)\left( \frac{1}{p}|t_nu_{n}|^{p}+\frac{1}{q}|t_nu_{n}|^{q}\right) \mathrm {d}x -\int _{\mathbb {R}^N}K(\epsilon _{n}x)F(t_nu_{n})\mathrm {d}x\bigg ]\\&=\liminf _{n\rightarrow \infty }\Phi _{\epsilon _{n}}(t_nu_n)\\&\le \liminf _{n\rightarrow \infty }\Phi _{\epsilon _{n}}(u_n)\\&= c_{V_{\min }K_{\max }}, \end{aligned} \end{aligned}$$

which is a contradiction. Thus, $\{y_n\}$ is bounded and, passing to a subsequence, we may assume that $y_n\rightarrow y$. If $y\not \in \Lambda$, then $V_{\min }<V(y)$ and $K_{\max }>K(y)$, and according to the above steps we get a contradiction. Consequently, we conclude that $y\in \Lambda$. $\square$

Let $\vartheta :\mathbb {R}^+\rightarrow \mathbb {R}^+$ be a positive function given by

$$\begin{aligned} \vartheta (\epsilon )=\max _{y\in \Lambda }|\Phi _{\epsilon }(\gamma _{\epsilon }(y))-c_{V_{\min }K_{\max }}|. \end{aligned}$$

It follows from Lemma 6.1 that $\vartheta (\epsilon )\rightarrow 0$ as $\epsilon \rightarrow 0$. We introduce a subset $\tilde{{\mathscr {N}}}_{\epsilon }$ of ${\mathscr {N}}_{\epsilon }$. Setting

$$\begin{aligned} \tilde{{\mathscr {N}}}_{\epsilon }:=\{u\in {\mathscr {N}}_{\epsilon }: \Phi _{\epsilon }(u)\le c_{V_{\min }K_{\max }}+\vartheta (\epsilon )\}, \end{aligned}$$

Since $\gamma _{\epsilon }(y)\in \tilde{{\mathscr {N}}}_{\epsilon }$ for all $y\in \Lambda$, then we can deduce that $\tilde{{\mathscr {N}}}_{\epsilon }\ne \emptyset$. Moreover, we have the following result.

Lemma 6.4

For any $\delta >0$, then the following holds

$$\begin{aligned} \lim _{\epsilon \rightarrow 0}\sup _{u\in \tilde{{\mathscr {N}}}_{\epsilon }}\inf _{y\in \Lambda _{\delta }}|\beta _{\epsilon }(u)-y|=0. \end{aligned}$$

Proof

Let $\epsilon _{n}\rightarrow 0$ as $n\rightarrow \infty$. For each $n\in \mathbb {N}$, there exists $\{u_n\}\subset \tilde{{\mathscr {N}}}_{\epsilon _{n}}$, such that

$$\begin{aligned} \inf _{y\in \Lambda _{\delta }}|\beta _{\epsilon _{n}}(u_{n})-y| =\sup _{u\in \tilde{{\mathscr {N}}}_{\epsilon _{n}}}\inf _{y\in \Lambda _{\delta }}|\beta _{\epsilon _{n}}(u)-y|+o_n(1). \end{aligned}$$

Hence, it is sufficient to prove that there exists $\{y_n\}\subset \Lambda _{\delta }$ such that

$$\begin{aligned} \lim _{n\rightarrow \infty }|\beta _{\epsilon _{n}}(u_{n})-y_{n}|=0. \end{aligned}$$

Indeed, since $\{u_n\}\subset \tilde{{\mathscr {N}}}_{\epsilon _{n}}$, then we have

$$\begin{aligned} c_{V_{\min }K_{\max }}\le c_{\epsilon _{n}}\le \Phi _{\epsilon _{n}}(u_n)\le c_{V_{\min }K_{\max }}+\vartheta (\epsilon _{n}), \end{aligned}$$

which implies that

$$\begin{aligned} \Phi _{\epsilon _{n}}(u_n)\rightarrow c_{V_{\min }K_{\max }}\quad \hbox {and}\quad \{u_n\}\subset {\mathscr {N}}_{\epsilon _{n}}. \end{aligned}$$

According to Lemma 6.3, there exists $\{{\tilde{y}}_{n}\}\subset \mathbb {R}^N$ such that $v_n(x)=u_n(x+{\tilde{y}}_{n})$ has a convergent subsequence. Moreover, up to a subsequence, $y_n=\epsilon _{n}{\tilde{y}}_{n}\rightarrow y\in \Lambda$. Therefore, we get

$$\begin{aligned} \begin{aligned} \beta _{\epsilon _{n}}(u_n)&=\frac{\int _{\mathbb {R}^N}\eta (\epsilon _{n} x)(|u_{n}|^{p}+|u_{n}|^{q})\mathrm {d}x}{\int _{\mathbb {R}^N}(|u_{n}|^{p}+|u_{n}|^{q})\mathrm {d}x}\\&= \frac{\int _{\mathbb {R}^N}\eta (\epsilon _{n} z+y_n)(|u_{n}(z+{\tilde{y}}_{n})|^{p}+|u_{n}(z+{\tilde{y}}_{n})|^{q})\mathrm {d}z}{\int _{\mathbb {R}^N}(|u_{n}(z+{\tilde{y}}_{n})|^{p}+|u_{n}(z+{\tilde{y}}_{n})|^{q})\mathrm {d}z}\\&= y_n+\frac{\int _{\mathbb {R}^N}[\eta (\epsilon _{n} z+y_n)-y_n](|v_{n}(z)|^{p}+|v_{n}(z)|^{q})\mathrm {d}z}{\int _{\mathbb {R}^N}(|v_{n}(z)|^{p}+|v_{n}(z)|^{q})\mathrm {d}z}\\&\rightarrow y\in \Lambda . \end{aligned} \end{aligned}$$

Consequently, there exists $\{y_n\}\subset \Lambda _{\delta }$ such that

$$\begin{aligned} \lim _{n\rightarrow \infty }|\beta _{\epsilon _{n}}(u_{n})-y_{n}|=0. \end{aligned}$$

The proof is now complete. $\square$

We shall use the Ljusternik–Schnirelmann category theory (see [13,  Theorem 2.1]) and the idea in [9] to prove the multiplicity result of positive solutions. Since ${\mathscr {N}}_{\epsilon }$ is not a $C^1$-submanifold of $E_{\epsilon }$, we cannot directly apply the Ljusternik–Schnirelmann category theory. Fortunately, according to Lemma 2.10, we can know that the mapping $m_{\epsilon }$ is a homeomorphism between ${\mathscr {N}}_{\epsilon }$ and $S_{\epsilon }$, and $S_{\epsilon }$ is a $C^1$-submanifold of $E_{\epsilon }$. So we can apply the Ljusternik–Schnirelmann category theoty to the functional $I_{\epsilon }(u)=\Phi _{\epsilon }({\hat{m}}_{\epsilon }(u))|_{S_{\epsilon }}=\Phi _{\epsilon }(m_{\epsilon }(u))$. Based on the above observations, we are ready to give the proof of Theorem 1.3.

Proof of Theorem 1.3

For any $\epsilon >0$, we define $\omega _{\epsilon }:\Lambda \rightarrow S_{\epsilon }$ as follows

$$\begin{aligned} \omega _{\epsilon }(y)={\check{m}}_{\epsilon }(t_{\epsilon }\Psi _{\epsilon ,y}) ={\check{m}}_{\epsilon }(\gamma _{\epsilon }(y))\quad \hbox {for all}~y\in \Lambda . \end{aligned}$$

Using Lemma 6.1 we get

$$\begin{aligned} \lim _{\epsilon \rightarrow 0}I_{\epsilon }(\omega _{\epsilon }(y))=\lim _{\epsilon \rightarrow 0}\Phi _{\epsilon }(\gamma _{\epsilon }(y)) =c_{V_{\min }K_{\max }}\quad \hbox {uniformly in}~y\in \Lambda . \end{aligned}$$

Moreover, we set

$$\begin{aligned} {\tilde{S}}_{\epsilon }=\{u\in S_{\epsilon }: I_{\epsilon }(u)\le c_{V_{\min }K_{\max }}+\vartheta (\epsilon )\}, \end{aligned}$$

with $\vartheta (\epsilon )=\sup _{y\in \Lambda }|I_{\epsilon }(u)-c_{V_{\min }K_{\max }}|\rightarrow 0$ as $\epsilon \rightarrow 0$. Hence, $\omega _{\epsilon }(y)\in {\tilde{S}}_{\epsilon }$ for all $y\in \Lambda$, and this shows that ${\tilde{S}}_{\epsilon }\ne \emptyset$ for all $\epsilon >0$.

According to Lemmas 2.10, 2.11, 6.1 and 6.4, we can find $\epsilon _{\delta }>0$ such that the diagram

$$\begin{aligned} \Lambda \xrightarrow {\gamma _{\epsilon }}\tilde{{\mathscr {N}}}_{\epsilon }\xrightarrow {{\check{m}}_{\epsilon }}{\tilde{S}}_{\epsilon } \xrightarrow {m_{\epsilon }}\tilde{{\mathscr {N}}}_{\epsilon }\xrightarrow {\beta _{\epsilon }}\Lambda _{\delta } \end{aligned}$$

is well defined for any $\epsilon \in (0,\epsilon _{\delta })$. By Lemma 6.2, there exists a function $l(\epsilon ,y)$ with $|l(\epsilon ,y)|<\frac{\delta }{2}$ uniformly in $y\in \Lambda$ for all $\epsilon \in (0,\epsilon _{\delta })$, such that $\beta _{\epsilon }(\gamma _{\epsilon }(y))=y+l(\epsilon ,y)$ for all $y\in \Lambda$. We define the function $H(t,y)=y+(1-t)l(\epsilon ,y)$. Then, $H:[0,1]\times \Lambda \rightarrow \Lambda _{\delta }$ is continuous. Evidently, $H(0,y)=\beta _{\epsilon }(\gamma _{\epsilon }(y))$ and $H(1,y)=y$ for all $y\in \Lambda$, and $\beta _{\epsilon }\circ \gamma _{\epsilon }=(\beta _{\epsilon }\circ m_{\epsilon })\circ \omega _{\epsilon }$ is homotopic to the inclusion mapping $id:\Lambda \rightarrow \Lambda _{\delta }$. So, making use of Lemma 2.2 of [13] (see also [6,  Lemma 6.4]), we have

$$\begin{aligned} \hbox {cat}_{{\tilde{S}}_{\epsilon }}({\tilde{S}}_{\epsilon })\ge \hbox {cat}_{\Lambda _{\delta }}(\Lambda ). \end{aligned}$$

On the other hand, let us choose a function $\vartheta (\epsilon )>0$ such that $\vartheta (\epsilon )\rightarrow 0$ as $\epsilon \rightarrow 0$ and such that $c_{V_{\min }K_{\max }}+\vartheta (\epsilon )$ is not a critical level for $\Phi _{\epsilon }$. For $\epsilon >0$ small enough, Lemma 4.5 shows that $\Phi _{\epsilon }$ satisfies the Palais–Smale condition in $\tilde{{\mathscr {N}}}_{\epsilon }$. Then, according to Lemma 2.11, we know that $I_{\epsilon }$ satisfies the Palais–Smale condition in ${\tilde{S}}_{\epsilon }$. Therefore, applying the Ljusternik–Schnirelmann category theory [13,  Theorem 2.1], we obtain that $I_{\epsilon }$ has at least $\hbox {cat}_{{\tilde{S}}_{\epsilon }}({\tilde{S}}_{\epsilon })$ critical points on ${\tilde{S}}_{\epsilon }$. Then, using Lemma 2.11 again, we can deduce that $\Phi _{\epsilon }$ has at least $\hbox {cat}_{\Lambda _{\delta }}(\Lambda )$ critical points. We finish the proof of Theorem 1.3. $\square$

7 Nonexistence of positive ground state solutions

In this section, we prove the nonexistence of positive ground state solutions. Consider the following auxiliary problem

$$\begin{aligned} -\Delta _{p} u-\Delta _{q} u +V^{\infty }(|u|^{p-2}u+|u|^{q-2}u)=K^{\infty }f(u)\quad \hbox {in}~\mathbb {R}^{N}, \end{aligned}$$

(7.1)

where $V^{\infty }$ and $K^{\infty }$ are given in condition $(A_{3})$. Moreover, according to the discussion in Sect. 3, we know that problem (7.1) has a positive ground state solution. In the following, we follow the idea of [38] and give the proof of Theorem 1.4.

Proof of Theorem 1.4

First we need to claim that $c_{\epsilon }=c_{V^{\infty }K^{\infty }}$ for each $\epsilon >0$. In fact, according to $(A_3)$, we can see that $V^{\infty }\le V(x)$ and $K(x)\le K^{\infty }$ for all $x\in \mathbb {R}^N$, and $c_{\epsilon }\ge c_{V^{\infty }K^{\infty }}$ by Lemma 3.4. Next, we show that $c_{\epsilon }\le c_{V^{\infty }K^{\infty }}$ for any fixed $\epsilon >0$. Let $u^{\infty }$ be a positive ground state solution of problem (7.1), by Lemma 3.1-(b), we know that $u^{\infty }$ is the unique global maximum of $\mathcal {J}_{V^{\infty }K^{\infty }}(tu^{\infty })$. Set $u_{n}=u^{\infty }(\cdot -y_n)$, where $\{y_n\}\subset \mathbb {R}^N$ is a sequence satisfying $|y_{n}|\rightarrow \infty$ as $n\rightarrow \infty$. As in Lemma 2.7, it follows that there exists $t_n>0$ such that ${\hat{m}}_{\epsilon }(u_n)=t_{n}u_{n}\in {\mathscr {N}}_{\epsilon }$ is the unique global maximum of $\Phi _{\epsilon }(tu_n)$ for each n. Moreover, the sequence $\{t_n\}$ is bounded.

We have

$$\begin{aligned} \begin{aligned} c_{\epsilon }&\le \Phi _{\epsilon }(t_{n}u_n)\\&=\mathcal {J}_{V^{\infty }K^{\infty }}(t_nu_n)+\frac{t_{n}^{p}}{p}\int _{\mathbb {R}^N}(V(\epsilon x)-V^{\infty })|u_{n}|^{p}\mathrm {d}x\\&\quad +\frac{t_{n}^{q}}{q}\int _{\mathbb {R}^N}(V(\epsilon x)-V^{\infty })|u_{n}|^{q}\mathrm {d}x +\int _{\mathbb {R}^N}(K^{\infty }-K(\epsilon x))F(t_{n}u_{n})\mathrm {d}x\\&= \mathcal {J}_{V^{\infty }K^{\infty }}(t_nu^{\infty })+\frac{t_{n}^{p}}{p}\int _{\mathbb {R}^N}(V(\epsilon x+\epsilon y_{n})-V^{\infty })|u^{\infty }|^{p}\mathrm {d}x\\&\quad +\frac{t_{n}^{q}}{q}\int _{\mathbb {R}^N}(V(\epsilon x+\epsilon y_{n})-V^{\infty })|u^{\infty }|^{q}\mathrm {d}x +\int _{\mathbb {R}^N}(K^{\infty }-K(\epsilon x+\epsilon y_{n}))F(t_{n}u^{\infty })\mathrm {d}x\\&\le c_{V^{\infty }K^{\infty }}+\frac{t_{n}^{p}}{p}\int _{\mathbb {R}^N}(V(\epsilon x+\epsilon y_{n})-V^{\infty })|u^{\infty }|^{p}\mathrm {d}x\\&\quad +\frac{t_{n}^{q}}{q}\int _{\mathbb {R}^N}(V(\epsilon x+\epsilon y_{n})-V^{\infty })|u^{\infty }|^{q}\mathrm {d}x +\int _{\mathbb {R}^N}(K^{\infty }-K(\epsilon x+\epsilon y_{n}))F(t_{n}u^{\infty })\mathrm {d}x. \end{aligned} \end{aligned}$$

(7.2)

Using the exponential decay of $u^{\infty }$, it follows that for any $\varepsilon >0$, there exists $R>0$ such that

$$\begin{aligned} \int _{|x|\ge R}(V(\epsilon x+\epsilon y_{n})-V^{\infty })|u^{\infty }|^{s}\mathrm {d}x\le c\varepsilon , \end{aligned}$$

where $s=\{p,q\}$. Moreover, from $(A_3)$ and Lebesgue’s dominated convergence theorem we have

$$\begin{aligned} \lim _{n\rightarrow \infty }\int _{|x|\le R}(V(\epsilon x+\epsilon y_{n})-V^{\infty })|u^{\infty }|^{s}\mathrm {d}x=0. \end{aligned}$$

Thus, we have proved that

$$\begin{aligned} \int _{\mathbb {R}^N}(V(\epsilon x+\epsilon y_{n})-V^{\infty })|u^{\infty }|^{s}\mathrm {d}x=o_{n}(1), \end{aligned}$$

(7.3)

where $s=\{p,q\}$.

Similarly, using the above arguments and (2.3) we have

$$\begin{aligned} \int _{\mathbb {R}^N}(K^{\infty }-K(\epsilon x+\epsilon y_{n}))F(t_{n}u^{\infty })\mathrm {d}x=o_{n}(1). \end{aligned}$$

(7.4)

So, from (7.2), (7.3) and (7.4) we deduce that $c_{\epsilon }=c_{V^{\infty }K^{\infty }}$ for each $\epsilon >0$.

We complete the proof by using a contradiction argument. Assume that for some $\epsilon _{0}>0$ there exists a positive function $u_{0}$ such that $u_0\in {\mathscr {N}}_{\epsilon _{0}}$ and $c_{\epsilon _{0}}=\Phi _{\epsilon _{0}}(u_{0})$. We know that $u_{0}$ is the unique global maximum of $\Phi _{\epsilon _{0}}(tu_{0})$. By Lemma 3.1-(b), there exists $t^{\infty }>0$ such that $t^{\infty }u_{0}\in {\mathscr {N}}_{V^{\infty }K^{\infty }}$, hence

$$\begin{aligned} \displaystyle c_{V^{\infty }K^{\infty }}\le \mathcal {J}_{V^{\infty }K^{\infty }}(t^{\infty }u_0)=\max _{t\ge 0}\mathcal {J}_{V^{\infty }K^{\infty }}(tu_0). \end{aligned}$$

(7.5)

On the other hand, using $(A_3)$ we have $\mathcal {J}_{V^{\infty }K^{\infty }}(u)\le \Phi _{\epsilon _{0}}(u)$ for any u. Thus, combining with (7.5), we have

$$\begin{aligned} c_{V^{\infty }K^{\infty }}\le \mathcal {J}_{V^{\infty }K^{\infty }}(t^{\infty }u_0)\le \Phi _{\epsilon _{0}}(t^{\infty }u_0) \le \Phi _{\epsilon _{0}}(u_0)=c_{\epsilon _{0}}=c_{V^{\infty }K^{\infty }}. \end{aligned}$$

This shows that

$$\begin{aligned} c_{V^{\infty }K^{\infty }}=\mathcal {J}_{V^{\infty }K^{\infty }}(t^{\infty }u_0)=\Phi _{\epsilon _{0}}(t^{\infty }u_0). \end{aligned}$$

(7.6)

Observe that

$$\begin{aligned} \begin{aligned} \mathcal {J}_{V^{\infty }K^{\infty }}(t^{\infty }u_0)&=\Phi _{\epsilon _{0}}(t^{\infty }u_0) +\frac{1}{p}\int _{\mathbb {R}^N}(V^{\infty }-V(\epsilon _{0}x))|t^{\infty }u_0|^{p}\mathrm {d}x\\&\quad +\frac{1}{q}\int _{\mathbb {R}^N}(V^{\infty }-V(\epsilon _{0}x))|t^{\infty }u_0|^{q}\mathrm {d}x\\&\quad +\int _{\mathbb {R}^N}(K(\epsilon _{0}x)-K^{\infty })F(t^{\infty }u_0)\mathrm {d}x. \end{aligned} \end{aligned}$$

(7.7)

We deduce from $(A_3)$ that

$$\begin{aligned} \int _{\mathbb {R}^N}(V^{\infty }-V(\epsilon _{0}x))|t^{\infty }u_0|^{s}\mathrm {d}x =\left( \int _{\mathcal {V}}+\int _{\mathcal {V}^{c}}\right) (V^{\infty }-V(\epsilon _{0}x))|t^{\infty }u_0|^{s}\mathrm {d}x<0, \end{aligned}$$

(7.8)

where $s=\{p,q\}$.

Similarly, we have

$$\begin{aligned} \int _{\mathbb {R}^N}(K(\epsilon _{0}x)-K^{\infty })F(t^{\infty }u_0)\mathrm {d}x =\left( \int _{\mathcal {K}}+\int _{\mathcal {K}^{c}}\right) (K(\epsilon _{0}x)-K^{\infty })F(t^{\infty }u_0)\mathrm {d}x<0. \end{aligned}$$

(7.9)

Combining (7.7), (7.8) and (7.9), we obtain $\mathcal {J}_{V^{\infty }K^{\infty }}(t^{\infty }u_0)<\Phi _{\epsilon _{0}}(t^{\infty }u_0)$, which contradicts relation (7.6). This completes the proof of Theorem 1.4. $\square$