Exponential sums with the Dirichlet coefficients of Rankin–Selberg L-functions

Abstract

We describe a new method to obtain upper bounds for exponential sums with multiplicative coefficients without the Ramanujan conjecture. We verify these hypothesis for (with mild restrictions) the Rankin–Selberg L-functions attached to two cuspidal automorphic representations.

1 Statement of results

Exponential sums with multiplicative coefficients have attracted a lot of attention among mathematicians. In 1974, Daboussi [3] first studied a class of multiplicative functions $f\in \mathcal {F}$, where $\mathcal {F}$ denotes the set of those multiplicative functions f with $|f(n)| \leqslant 1$. He proved that if $|\alpha -a / q| \leqslant 1 / q^2$ for some $(a, q)=1$ and $3 \leqslant q \leqslant (N / \log N)^{1/2}$, then one has

$$\begin{aligned} \sum _{n\le N} f(n)e(n\alpha )\ll \frac{N}{(\log \log N)^{1/2}} \end{aligned}$$

uniformly for $f\in \mathcal {F}$.

Montgomery and Vaughan [16] supposed that a class of multiplicative function f satisfies the following two conditions:

$$\begin{aligned} |f(p)| \leqslant A, \quad \text{ for } \text{ all } \text{ primes } p \end{aligned}$$

(1.1)

and

$$\begin{aligned} \sum _{n \leqslant N}|f(n)|^2 \leqslant A^2 N, \quad \text{ for } \text{ all } \text{ natural } \text{ numbers } N, \end{aligned}$$

(1.2)

where A is an arbitrary constant with $A \geqslant 1$. They proved that if $|\alpha -a / q| \leqslant 1 / q^2$ for some $(a, q)=1$ and $2 \leqslant R \leqslant q \leqslant N / R$, then

$$\begin{aligned} \sum _{n\le N} f(n)e(n\alpha ) \ll \frac{N}{\log N}+\frac{N}{R^{1 / 2}}(\log R)^{3 / 2} \end{aligned}$$

uniformly for f satisfying the conditions (1.1) and (1.2).

Very recently, Jiang et al. [8] generalized the work of Montgomery and Vaughan [16]. They study exponential sums involving a multiplicative function f under milder conditions on the range of f. More precisely, f satisfies the following conditions:

$$\begin{aligned} \sum _{n \leqslant N}|f(n)|^2 \ll N, \end{aligned}$$

(1.3)

$$\begin{aligned} \sum _{p \leqslant N}|f(p)|^2 \log p \ll N \end{aligned}$$

(1.4)

and

$$\begin{aligned} \sum _{\begin{array}{c} p \leqslant N \\ p+h \text{ is } \text{ prime } \end{array}}|f(p) f(p+h)| \ll \frac{h}{\varphi (h)} \cdot \frac{N}{(\log N)^2}, \end{aligned}$$

(1.5)

where h is any positive integer. For f satisfying the conditions (1.3), (1.4) and (1.5), they proved that if $|\alpha -a / q| \leqslant 1 / q^2$ for some $(a, q)=1$ and $1 \leqslant q \leqslant N$, then

$$\begin{aligned} \sum _{n\le N} f(n)e(n\alpha ) \ll \frac{N}{\log N}+\frac{N}{\phi (q)^{1/2}}+(qN)^{1 / 2}\left( \log \left( \frac{N}{q}\right) \right) ^{3 / 2}. \end{aligned}$$

Let $m \geqslant 2$ be an integer and $\pi$ be an automorphic irreducible cuspidal representation of $\textrm{GL}_{m}$ over $\mathbb {Q}$ with unitary central character. Denote by $\lambda _{\pi }(n)$ the Dirichlet coefficients of automorphic L-function $L(s,\pi )$ attached to $\pi$. As an application, they used it together with the analytic theory of automorphic L-functions to prove that for any automorphic cuspidal representation $\pi$ over $\textrm{GL}_{m}$,

$$\begin{aligned} S_{\lambda _{\pi }}(x)=\sum _{n \leqslant x} \lambda _{\pi }(n) e\left( n \alpha \right) \ll _{\pi } \frac{x}{\log x} \end{aligned}$$

for any $\alpha \in \mathbb {R}$. A striking feature of their result is that it applies to the coefficients of automorphic L-functions without the Ramanujan conjecture.

Actually, Jiang et al.’s result [8] can not apply to multiplicative function f the size of the second power-moment of which is more than N. In this paper, we will use a new method to study the exponential sum involving multiplicative function f under milder conditions on the size of the second power-moment of f. Let A be an arbitrary positive constant and $\mathcal {M}$ be the class of all multipticative functions f such that

$$\begin{aligned} \sum _{n\le x} |f(n)|^{2}\ll _{f} x\exp \big ((\log \log x)^{1+\delta }\big ) \end{aligned}$$

(1.6)

and

$$\begin{aligned} \sum _{p\le x}\log (p)|f(p)|\ll _{f} x, \end{aligned}$$

(1.7)

where $\delta$ is a positive constant depending on f. For $f\in \mathcal {M}$, the exponential sum involving multiplicative function f is defined by

$$\begin{aligned} S(N, \alpha ):= \sum _{n\le N}f(n)e(n\alpha ). \end{aligned}$$

Although by the Cauchy–Schwarz inequality and the Chebyshev theorem, we can deduce condition (1.7) from condition (1.4), our results will apply to more classes of L-functions than those in the work of Jiang et al. [8].

Using the theory of smooth numbers, we prove the following result.

Theorem 1.1

Uniformly in $\alpha$ $\in \mathbb {R}$, we suppose that

$$\begin{aligned} \alpha =\frac{a}{q}+\frac{\theta }{q^{2}} \end{aligned}$$

with $|\theta |\le 1$, $2\le y\le q \le x/y$ and (a,q)=1. Then for any multiplicative function $f \in \mathcal {M}$, we have

$$\begin{aligned} \begin{aligned} S(N, \alpha )&\ll _{f} x\big (\exp \big ((\log \log x)^{1+\delta }\big )\big )^{\frac{1}{2}} \exp \bigg (-(1+o(1)) \frac{\log x}{\log y} \bigg ) \\ {}&\quad \quad +x(\log x)\bigg (\frac{\exp \big ((\log \log x)^{1+\delta }\big )}{y}\bigg )^{\frac{1}{2}}+\frac{x\big (\exp \big ((\log \log y)^{1+\delta }\big )\big )^{\frac{1}{2}}\log y}{\log x}. \end{aligned} \end{aligned}$$

Remark 1.2

In Theorem 1.1, we establish a weak upper bound of $S(N,\alpha )$, but with a much milder hypothesis on the size of the second power-moment of f. In particular, our result will apply to all automorphic L-functions and (with mild restrictions) to Rankin–Selberg L-functions attached to two automorphic representations.

In order to make clear the application of our result, we will review some more or less standard facts about L-functions arising from cuspidal automorphic representations and their Rankin–Selberg convolutions in Sect. 3. Let $\mathcal {A}(m)$ be the set of all cuspidal automorphic representations of $\textrm{GL}_{m}$ over $\mathbb {Q}$ with unitary central character. By the general theory (see Sect. 3), each pair of $\pi \in \mathcal {A}(m)$ and $\pi ^{\prime }\in \mathcal {A}(m^{\prime })$ admits a Rankin–Selberg L-function

$$\begin{aligned} L\left( s, \pi \times \pi ^{ \prime }\right) =\sum _{n=1}^{\infty } \frac{\lambda _{\pi \times \pi ^{\prime }}(n)}{n^{s}} \end{aligned}$$

for $\Re (s)>1$. We denote by $\tilde{\pi }$ the contragradient representation of $\pi$ which is also an irreducible cuspidal automorphic representation with unitary central character. Moreover, we say $\pi$ and $\tilde{\pi }^{\prime }$ are not twist equivalent when there exists no primitive character $\chi$ satisfying the property that $\tilde{\pi }^{\prime }=\pi \otimes \chi$. Denote this by $\pi \not \sim \tilde{\pi }^{\prime }$.

In this paper, we are concerned with obtaining upper bounds for exponential sums with the coefficients of (with mild restrictions) Rankin–Selberg L-functions. More precisely, we give a notably milder hypothesis on the size of the second power-moment of $\lambda _{\pi \times \pi ^{\prime }}(n)$. That is

$$\begin{aligned} \sum _{n\le x}|\lambda _{\pi \times \pi ^{\prime }}(n)|^{2}\ll _{\pi ,\pi ^{\prime }}x\exp ((\log \log x)^{1+\delta }), \end{aligned}$$

(1.8)

where $\delta$ is the positive constant depending on $\pi$ and $\pi ^{\prime }$. Under the above hypothesis, we shall apply the Hardy–Littlewood circle method to obtain the following result.

Theorem 1.3

Suppose $\pi \in \mathcal {A}(m)$ and $\pi ^{\prime }\in \mathcal {A}(m^{\prime })$. If $\pi \not \sim \tilde{\pi }^{\prime }$ and $\lambda _{\pi \times \pi ^{\prime }}(n)$ satisfies condition (1.8), then we have

$$\begin{aligned} \begin{aligned} \sum _{n\le x}\lambda _{\pi \times \pi ^{\prime }}(n)e(n\alpha )\ll _{\pi ,\pi ^{\prime }} \frac{x}{(\log x)^{1-\varepsilon }} \end{aligned} \end{aligned}$$

uniformly in $\alpha$ $\in \mathbb {R}$.

Throughout our paper, $\varepsilon$ denotes an arbitrarily small positive constant the value of which may shift in different occurrences.

Remark 1.4

More specifically, we apply Theorem 1.1 to obtain an estimate in the situation where $\alpha$ belongs to so-called minor arcs. In the following proof, we need to discuss how the coefficients $\lambda _{\pi \times \pi ^{\prime }}(n)$ satisfy condition (1.7). When $\alpha$ belongs to the so-called major arcs, we will use a weak subconvexity bound which Soundarajan and Thorner [20] obtained. Their result applies to all automorphic L-functions and (with mild restrictions) the Rankin–Selberg L-functions attached to two cuspidal automorphic representations. For this question we are concerned about, we can also obtain our result by using a convexity bound for Rankin–Selberg L-functions. The principal reason why we use the subconvexity bound is to illustrate if we have a better subconvexty bound, we can obtain a better saving for the result when $\alpha$ belongs to the major arcs.

Remark 1.5

In a number of special situations, condition (1.8) may be dropped, and we give a few such examples.

• Either $\pi$ or $\pi ^{\prime }$ satisfies the Ramanujan conjecture (see Sect. 3). If $\pi$ satisfies the Ramanujan conjecture, by Lemma 3.1 and the Rankin–Selberg theory, we obtain

  $$\begin{aligned} \begin{aligned} \quad \quad \quad \sum _{n\le x}|\lambda _{\pi \times \pi ^{\prime }}(n)|^{2}\le |\lambda _{\pi \times \tilde{\pi }}(n)|\sum _{n\le x}|\lambda _{\pi ^{\prime }\times \tilde{\pi ^{\prime }}}(n)|&\le \max _{n\le x}\{d_{2m}(n)\}\bigg |\sum _{n\le x}\lambda _{\pi ^{\prime }\times \tilde{\pi ^{\prime }}}(n)\bigg | \\ {}&\ll _{\pi ,\pi ^{\prime }} x\exp \big ((\log \log x)^{1+\delta }\big ). \end{aligned} \end{aligned}$$

  Especially, let f, g be newforms and $j_1,j_2\ge 0$. Denote by $\lambda _{{\text {sym}}^{j_{1}}f\times {\text {sym}}^{j_{2}}g}(n)$ the coefficients of the Dirichlet expansion of $L\left( {\text {sym}}^{j_1} f\times {\text {sym}}^{j_{2}}g,s\right)$. Then by the same method, we easily have

  $$\begin{aligned} \begin{aligned} \quad \sum _{n\le x}|\lambda _{{\text {sym}}^{j_{1}}f\times {\text {sym}}^{j_{2}}g}(n)|^{2}&\le \max _{n\le x}\{d_{2j_1+2}(n)\}\bigg |\sum _{n\le x}\lambda _{{\text {sym}}^{j_{2}}g\times \tilde{{\text {sym}}^{j_{2}}g}}(n)\bigg | \\ {}&\ll _{\pi ,\pi ^{\prime }} x\exp \big ((\log \log x)^{1+\delta }\big ). \end{aligned} \end{aligned}$$

• $\pi$ and $\pi ^{\prime }$ are both self-contragredient $\in \mathcal {A}(2)$. It’s known from [12] that

  $$\begin{aligned} \sum _{n\le x}|\lambda _{\pi \times \tilde{\pi }}(n)|^{2}\ll _{\pi } x(\log x)^{4}. \end{aligned}$$

  Thus by Lemma 3.1 and the Cauchy–Schwarz inequality, we obtain that

  $$\begin{aligned} \sum _{n\le x}|\lambda _{\pi \times \pi ^{\prime }}(n)|^{2}\ll _{\pi ,\pi ^{\prime }} x(\log x)^{4}. \end{aligned}$$

• $\pi$ and $\pi ^{\prime }$ are both self-contragredient and $\in \mathcal {A}(3)$. There exists $\pi _{1}\in \mathcal {A}(2)$ such that

  $$\begin{aligned} L(s, \pi \times \pi \times \pi \times \pi )= & {} L\left( s,\left( \textrm{Ad}^4 \pi _{1} \boxplus \textrm{Ad} \pi _{1} \boxplus 1\right) \right. \\{} & {} \left. \times \left( {\text {Ad}}^4 \pi _{1}\boxplus {\text {Ad}} \pi _{1} \boxplus 1\right) \right) , \end{aligned}$$

  where $\textrm{Ad}^4 \pi _{1} \simeq {\text {Sym}}^4 \pi ^{\prime } \otimes \omega ^{-2}$, and $\omega$ is the central character of $\pi _{1}$. Since $\textrm{Ad}^4 \pi _{1}$ and $\textrm{Ad} \pi _{1}$ are cuspidal automorphic representations, then by the (generalized) Ikehara’s theorem, see [21, Chapter II.7, Theorem 15], we have

  $$\begin{aligned} \sum _{n\le x}|\lambda _{\pi \times \tilde{\pi }}(n)|^{2}\ll _{\pi } x(\log x)^{34}. \end{aligned}$$

  Thus using Lemma 3.1 and the Cauchy–Schwarz inequality, we obtain that

  $$\begin{aligned} \sum _{n\le x}|\lambda _{\pi \times \pi ^{\prime }}(n)|^{2}\ll _{\pi ,\pi ^{\prime }} x(\log x)^{34}. \end{aligned}$$

• $\pi$ and $\pi ^{\prime }$ are self-contragredient automorphic cuspidal representations either on $\textrm{GL}_2$ or on $\textrm{GL}_3$. By the same method as above, we deduce that

  $$\begin{aligned} \sum _{n\le x}|\lambda _{\pi \times \pi ^{\prime }}(n)|^{2}\ll _{\pi ,\pi ^{\prime }} x(\log x)^{19}. \end{aligned}$$

Denote by $\mu _{\pi }(n)$ the Dirichlet coefficients of the inverse of $L(s, \pi )$ and $\mu _{\pi \times \pi ^{ \prime }}(n)$ the Dirichlet coefficients of the inverse of $L(s, \pi \times \pi ^{ \prime })$. Another fascinating application of our results is to obtain the upper bound of

$$\begin{aligned} S_{\mu _{\pi \times \pi ^{\prime }}}(x,\alpha )=\sum _{n\le x}\mu _{\pi \times \pi ^{\prime }}(n)e(n\alpha ), \end{aligned}$$

which is uniform in $\alpha$. Jiang and Lü [7] first proved that under Hypothesis H and Hypothesis S,

$$\begin{aligned} S_{\mu _{\pi }}(x,\alpha )=\sum _{n\le x}\mu _{\pi }(n)e(n\alpha )\ll _{\pi } x\frac{\log \log x}{\sqrt{\log x}}. \end{aligned}$$

Very lately, Jiang et al. [9] has proved

$$\begin{aligned} M_{\pi }(x)=\sum _{n\le x}\mu (n)\lambda _{\pi }(n)e(n\alpha )\ll _{\pi } \frac{x}{\log x}, \end{aligned}$$

where $\mu$ denotes the Möbius function. Since in [7], Jiang and Lü found $M_{\pi }(x)$ and $S_{\mu _{\pi }}(x)$ are equivalent by some relation, we easily have

$$\begin{aligned} \sum _{n\le x}\mu _{\pi }(n)e(n\alpha )\ll _{\pi }\frac{x}{\log x}. \end{aligned}$$

In this paper, we will use Theorem 1.1 and a standard zero-free region of Rankin–Selberg L-functions [5, Theorem A.1] to obtain the following result with milder hypothesis on the size of the second power-moment of $\mu _{\pi \times \pi ^{\prime }}(n)$,

$$\begin{aligned} \sum _{n\le x}|\mu _{\pi \times \pi ^{\prime }}(n)|^{2}\ll _{\pi ,\pi ^{\prime }}x\exp ((\log \log x)^{1+\delta }). \end{aligned}$$

(1.9)

Theorem 1.6

Suppose $\pi \in \mathcal {A}(m)$ and $\pi ^{\prime }\in \mathcal {A}(m^{\prime })$. Assume that $\pi$ is not self-dual, $\pi ^{\prime }$ is self-dual and condition (1.9) holds, then we have

$$\begin{aligned} \begin{aligned} \sum _{n\le x}\mu _{\pi \times \pi ^{\prime }}(n)e(n\alpha )\ll _{\pi ,\pi ^{\prime }} \frac{x}{(\log x)^{1-\varepsilon }} \end{aligned} \end{aligned}$$

uniformly in $\alpha$ $\in \mathbb {R}$.

Remark 1.7

According to [10], when $m \ne 2$, the density of self-dual cuspidal automorphic representations is indeed zero. When $m=2$, self-dual cuspidal automorphic representations have positive density due to the fact that $\textrm{SO}_3=\textrm{P G L}_2$—the lifts from this group to $\textrm{G L}_2$ provide for the positive proportion of self-dual representations. It’s well known that $L(s, \pi \times \pi )=L\left( s, \pi , {\text {sym}}^2\right) L\left( s, \pi , \wedge ^2\right)$, where $L\left( s, \pi , {\text {sym}}^2\right)$ are the symmetric square L-functions and $L\left( s, \pi , \wedge ^2\right)$ are the exterior square L-functions. Thus from [14], the representation $\pi$ is self-dual if and only if the symmetric square or exterior square L-function has a pole. Furthermore, following from [17], we know that $\pi$ is a self-dual automorphic representation for $\textrm{GL}_3$ if and only if $\pi$ is a symmetric square lift of a $\textrm{GL}_2$ automorphic representation.

Define

$$\begin{aligned} M_{\pi \times \pi ^{\prime }}(x)=\sum _{n\le x} \mu (n)\lambda _{\pi \times \pi ^{\prime }}(n)e(n\alpha ). \end{aligned}$$

Jiang and Lü established the Möbius randomness principle for the sequence $\{\lambda _{\pi }(n)e(n^{k}\alpha )\}$ in [7]. The Möbius randomness principle asserts that $\mu$ is asymptotically orthogonal to any low-complexity function $\xi : \mathbb {N} \rightarrow \mathbb {C}$ in the sense that

$$\begin{aligned} \sum _{n \leqslant x} \mu (n) \xi (n)=o\left( \sum _{n \leqslant x}|\xi (n)|\right) , \end{aligned}$$

which is advanced by Sarnak [18]. Also, we will find some relation between $M_{\pi \times \pi ^{\prime }}(x)$ and $S_{\mu _{\pi \times \pi ^{\prime }}}(x)$ to prove the sequence $\{\lambda _{\pi \times \pi ^{\prime }}(n)e(n\alpha )\}$ and $\{\mu (n)\}$ are orthogonal.

Corollary 1.8

Suppose $\pi \in \mathcal {A}(m)$ and $\pi ^{\prime }\in \mathcal {A}(m^{\prime })$. Assume that $\pi$ is not self-dual, $\pi ^{\prime }$ is self-dual, condition (1.9) holds and

$$\begin{aligned} |\alpha _{\pi \times \pi ^{\prime },j}(p)| \le p^{\gamma } \text { for }1\le j\le mm^{\prime }, \end{aligned}$$

(1.10)

where $\gamma <1/2$ is a positive constant. Then we have

$$\begin{aligned} \sum _{n \leqslant x} \mu (n)\lambda _{\pi \times \pi ^{\prime }}(n) e(n \alpha ) \ll _{\pi ,\pi ^{\prime }} \frac{x}{(\log x)^{1-\varepsilon }} \end{aligned}$$

uniformly in $\alpha$ $\in \mathbb {R}$.

2 Proof of Theorem 1.1

2.1 Contributions from $\mathcal {N}_{1}(x)$.

For a positive integer m, put P(m) for the largest prime factor of m with $P(1)=1$. Let

$$\begin{aligned} \mathcal {N}_{1}(x)=\{1\leqslant n \leqslant x: P(n) \leqslant y\}. \end{aligned}$$

From the theory of smooth numbers [2], we know that in our range for y versus x,

$$\begin{aligned} \# \mathcal {N}_{1}(x) \ll x \exp (-(1+o(1)) u ), \quad \text{ where } \quad u=\frac{\log x}{\log y} \quad (x \rightarrow \infty ). \end{aligned}$$

(2.1)

By the Cauchy–Schwarz inequality, (1.6) and (2.1), we have

$$\begin{aligned} \begin{aligned} \bigg |\sum _{n \in \mathcal {N}_{1}(x)} f(n)e(n\alpha ) \bigg |&\leqslant \bigg (\sum _{n \in \mathcal {N}_{1}(x)}|f(n)|^{2}\bigg )^{\frac{1}{2}}\bigg (\sum _{n \in \mathcal {N}_{1}(x)}1\bigg )^{\frac{1}{2}}\\ {}&\le x\big (\exp \big ((\log \log x)^{1+\delta }\big )\big )^{1/2} \bigg (\exp \bigg (-(1+o(1)) \frac{\log x}{\log y} \bigg )\bigg )^{1/2}. \end{aligned} \end{aligned}$$

(2.2)

2.2 Contributions from $\mathcal {N}_{2}(x)$.

Define $P(n)=p$. Next let

$$\begin{aligned} \mathcal {N}_{2}(x)=\left\{ n \in [1, x]: p^{2} \mid n \text{ for } p>y / 2\right\} . \end{aligned}$$

Fixing p, the number of $n \in [1, x]$ which are multiples of $p^{2}$ is at most $\left\lfloor x / p^{2}\right\rfloor +1$. Thus,

$$\begin{aligned} \# \mathcal {N}_{2}(x) \leqslant \sum _{y / 2 \leqslant p \leqslant x^{1 / 2}}\left( \left\lfloor x / p^{2}\right\rfloor +1\right) \ll x \sum _{y / 2 \leqslant p\leqslant x ^{1 / 2}} \frac{1}{p^{2}}+\pi (\sqrt{x}) \ll \frac{x}{y}. \end{aligned}$$

Then, by the Cauchy–Schwarz inequality, we have

$$\begin{aligned} \begin{aligned} \left| \sum _{n \in \mathcal {N}_{2}(x)} f(n)e(n\alpha ) \right|&\leqslant \sum _{n \in \mathcal {N}_{2}(x)}\left| f(n)e(n\alpha ) \right| \\ {}&\leqslant (\sharp \mathcal {N}_{2}(x))^{1 / 2}\left( \sum _{n \in \mathcal {N}_{2}(x)}\left| f(n)\right| ^{2}\right) ^{1 / 2}\\&\leqslant x\big (\exp \big ((\log \log x)^{1+\delta }\big )\big )^{1/2}y^{-1/2}, \end{aligned} \end{aligned}$$

(2.3)

as $x \rightarrow \infty$.

2.3 Contributions from $\mathcal {N}_{3}(x)$

Consider $\mathcal {N}_{3}(x) \subseteq [1, x] \backslash \left( \mathcal {N}_{1}(x) \bigcup \mathcal {N}_{2}(x)\right)$. Define $P(n)=p$. Let

$$\begin{aligned} \mathcal {N}_{3}(x)=\left\{ n \in [1, x]: n=pm, p^{2}\not \mid n \text{ where } y<p<\frac{x}{y} \right\} . \end{aligned}$$

For each such $n=pm\in \mathcal {N}_{3}(x)$, let $\mathcal {M}(x)$ be the set of all possible values of m. Then by the multiplicative property of f(n), we consider

$$\begin{aligned} \begin{aligned} S_{\mathcal {N}_{3}(x)}=\sum _{n\in \mathcal {N}_{3}(x)}f(n)e(n\alpha )&=\sum _{y<p<x/y}f(p)\sum _{\begin{array}{c} m\le x/p\\ P(m)<p \end{array}}f(m)e(pm\alpha )\\ {}&=\sum _{m\in \mathcal {M}(x)}f(m)\sum _{P(m)<p\le x/m}f(p)e(pm\alpha )\\ {}&\le \sum _{1\le j < \log \frac{x}{y^{2}}+1}|S_{j}| \end{aligned} \end{aligned}$$

(2.4)

with

$$\begin{aligned} S_{j}= \sum _{\begin{array}{c} 2^{j-1} y<m \leqslant 2^{j} y\\ m\in \mathcal {M}(x) \end{array}} f(m) \sum _{P(m)<p\le x/m}f(p)e(pm\alpha ). \end{aligned}$$

We use the Cauchy–Schwarz inequality and the inequality of arithmetic and geometric means to estimate the sum

$$\begin{aligned} \begin{aligned} S_{j}&\leqslant \bigg (\sum _{\begin{array}{c} 2^{j-1} y<m \leqslant 2^{j} y\\ m\in \mathcal {M}(x) \end{array}}\left| f(m)\right| ^{2}\bigg )^{\frac{1}{2}}\\ {}&\quad \cdot \bigg (\sum _{\begin{array}{c} 2^{j-1} y<m \leqslant 2^{j} y\\ m\in \mathcal {M}(x) \end{array}} \sum _{P(m)<p_{1},p_{2} \leqslant x/m} f(p_{1})\overline{ f(p_{2})} e(m(p_{1}-p_{2}) \alpha )\bigg )^{\frac{1}{2}}\\&\leqslant \bigg (\sum _{\begin{array}{c} 2^{j-1} y<m \leqslant 2^{j} y\\ m\in \mathcal {M}(x) \end{array}} \left| f(m)\right| ^{2}\bigg )^{\frac{1}{2}}\\ {}&\quad \cdot \bigg (\sum _{ \frac{x}{2^{j} y}<p_{1} \leqslant \frac{x}{2^{j-1} y} } |f(p_{1})|^{2}\sum _{|p_{1}-p_{2}|\le \frac{x}{2^{j} y}}\bigg |\sum _{\begin{array}{c} mp_{1}\le x, mp_{2}\le x\\ P(m)<p_{1},P(m)<p_{2} \end{array}}e( m(p_{1}-p_{2})\alpha )\bigg |\bigg )^{\frac{1}{2}}. \end{aligned} \end{aligned}$$

Then taking absolute values, $S_{j}(x)$ is bounded by

$$\begin{aligned} \begin{aligned}&\bigg (\sum _{\begin{array}{c} 2^{j-1} y<m \leqslant 2^{j} y\\ m\in \mathcal {M}(x) \end{array}} \left| f(m)\right| ^{2}\bigg )^{\frac{1}{2}}\bigg (\sum _{ \frac{x}{2^{j} y}<p_{1} \leqslant \frac{x}{2^{j-1} y}} |f(p_{1})|^{2}\\ {}&\qquad \cdot \sum _{|p_{1}-p_{2}|\le \frac{x}{2^{j} y}}\bigg (\bigg |\sum _{\begin{array}{c} m\le 2^{j}y\\ m(p_{1}-p_{2})\le x \end{array}}e( m(p_{1}-p_{2})\alpha )\bigg |+\bigg |\sum _{\begin{array}{c} m\le 2^{j}y\\ P(m)\ge p_{1}orP(m)\ge p_{2} \end{array}}e( m(p_{1}-p_{2})\alpha )\bigg |\bigg )\bigg )^{\frac{1}{2}}\\ {}&\quad \le \bigg (\sum _{\begin{array}{c} 2^{j-1} y<m \leqslant 2^{j} y\\ m\in \mathcal {M}(x) \end{array}} \left| f(m)\right| ^{2}\bigg )^{\frac{1}{2}}\bigg (\sum _{ \frac{x}{2^{j} y}<p_{1} \leqslant \frac{x}{2^{j-1} y} } |f(p_{1})|^{2}\bigg )^{\frac{1}{2}}\\ {}&\quad \cdot \bigg (\sum _{|p_{1}-p_{2}|\le \frac{x}{2^{j} y}}\bigg |\sum _{\begin{array}{c} m\le 2^{j}y\\ m(p_{1}-p_{2})\le x \end{array}}e( m(p_{1}-p_{2})\alpha )\bigg |+\sum _{\frac{x}{2^{j}y} \leqslant p \leqslant p^{\prime }\leqslant x^{1 / 2}}\left( \left\lfloor \frac{x}{pp^{\prime }}\right\rfloor +1\right) \bigg )^{\frac{1}{2}}, \end{aligned} \end{aligned}$$

where $p^{\prime }=P(m)$. If x is sufficiently large, there is a reduced fraction a/q such that $|\alpha -a / q| \leqslant q^{-2}$. Then we use the following estimate from [6, Lemma 13.7] about exponential sums, for any $M, N \geqslant 1$,

$$\begin{aligned} \sum _{|n| \leqslant N}\bigg |\sum _{\begin{array}{c} m \leqslant M \\ m\le x(n) \end{array}} e(\alpha m n)\bigg | \ll \left( M+N+\frac{M N}{q}+q\right) \log q, \end{aligned}$$

to have

$$\begin{aligned} \begin{aligned} S_{j}&\le \bigg (\sum _{\begin{array}{c} 2^{j-1} y<m \leqslant 2^{j} y\\ m\in \mathcal {M}(x) \end{array}} \left| f(m)\right| ^{2}\bigg )^{\frac{1}{2}}\\ {}&\quad \quad \cdot \bigg (\sum _{ \frac{x}{2^{j} y}<p_{1} \leqslant \frac{x}{2^{j-1} y} } |f(p_{1})|^{2}\bigg )^{\frac{1}{2}}\left( \left( 2^{j}y+\frac{x}{2^{j}y}+\frac{x}{q}+q\right) \log q+\frac{2^{2j}y^{2}}{x}\right) ^{\frac{1}{2}}. \end{aligned} \end{aligned}$$

We take $y\leqslant q \leqslant \frac{x}{y}$. By (2.4), (1.6) and the above inequality, we have

$$\begin{aligned} S_{\mathcal {N}_{3}(x)}\ll _{f} x(\log x)\big (\exp \big ((\log \log x)^{1+\delta }\big )\big )^{1/2}y^{-1/2}. \end{aligned}$$

(2.5)

2.4 Contributions from $\mathcal {N}_{4}(x)$

Next let $\mathcal {N}_{4}(x)= [1, x] \backslash \left( \mathcal {N}_{1}(x) \bigcup \mathcal {N}_{2}(x)\bigcup \mathcal {N}_{3}(x)\right)$. We know $n=P(n)m$ and $m\leqslant y$, where $n \in \mathcal {N}_{4}(x)$. Let $P(n)=p$.

By the multiplicative property of f(pm), we have

$$\begin{aligned} \begin{aligned} S_{\mathcal {N}_{4}(x)}&=\sum _{n\le \frac{x}{y}}f(n)e(n\alpha )=\sum _{m\le y} f(m) \sum _{\frac{x}{ym}<p \leqslant \frac{x}{m}} f(p) e(mp\alpha ). \end{aligned} \end{aligned}$$

(2.6)

Due to (1.7), we know

$$\begin{aligned} \log \frac{x}{ym}\sum _{\frac{x}{ym}<p \leqslant \frac{x}{m}} |f(p)|\leqslant \sum _{\frac{x}{ym}<p \leqslant \frac{x}{m}} |f(p)| \log p \ll \frac{x}{m}. \end{aligned}$$

Obviously,

$$\begin{aligned} \sum _{\frac{x}{ym}<p \leqslant \frac{x}{m}} |f(p)| \leqslant \frac{x}{m}\bigg (\log \frac{x}{ym}\bigg )^{-1}. \end{aligned}$$

(2.7)

We use the Cauchy–Schwarz inequality and (1.6) to obtain

$$\begin{aligned} \sum _{n\le x}|f(n)|\ll _{f} x\big (\exp \big ((\log \log x)^{1+\delta }\big )\big )^{1/2}. \end{aligned}$$

(2.8)

Hence it follows from (2.6), (2.7), partial summation and (2.8) that

$$\begin{aligned} S_{\mathcal {N}_{4}(x)}\ll _{f} \frac{x\big (\exp \big ((\log \log y)^{1+\delta }\big )\big )^{1/2}\log y}{\log x}. \end{aligned}$$

(2.9)

Combining (2.2), (2.3), (2.5) with (2.9), we deduce

$$\begin{aligned} \begin{aligned} \sum _{n\le x}f(n)e(n\alpha )&\ll _{f} x\big (\exp \big ((\log \log x)^{1+\delta }\big )\big )^{1/2} \exp \bigg (-(1+o(1)) \frac{\log x}{\log y} \bigg )\\&\quad +x(\log x)\big (\exp \big ((\log \log x)^{1+\delta }\big )\big )^{1/2}y^{-1/2}\\&\quad +\frac{x\big (\exp \big ((\log \log y)^{1+\delta }\big )\big )^{1/2}\log y}{\log x}. \end{aligned} \end{aligned}$$

3 Preliminaries

3.1 Standard L-functions

Let $m \geqslant 2$ be an integer, and let $\mathcal {A}(m)$ be the set of all cuspidal automorphic representations of $\textrm{GL}_{m}$ over $\mathbb {Q}$ with unitary central character. Fix $\pi \in \mathcal {A}(m)$. The standard function $L(s, \pi )$ is given by a Dirichlet series and Euler product

$$\begin{aligned} \begin{aligned} L(s, \pi )&=\sum _{n=1}^{\infty } \frac{\lambda _\pi (n)}{n^s}=\prod _p L_p(s, \pi ), \\ L_p(s, \pi )&=\prod _{j=1}^m\left( 1-\frac{\alpha _{j, \pi }(p)}{p^s}\right) ^{-1}=\sum _{j=0}^{\infty } \frac{\lambda _\pi \left( p^j\right) }{p^{j s}}, \end{aligned} \end{aligned}$$

with both the series and the product converging absolutely for $\Re s>1$. The function $L^{-1}(s, \pi )$ can be written as

$$\begin{aligned} L^{-1}(s, \pi )=\sum _{n=1}^{\infty } \frac{\mu _{\pi }(n)}{n^{s}} \end{aligned}$$

for $\Re s>1$. Then it can be given by

$$\begin{aligned} \mu _{\pi }(n)= {\left\{ \begin{array}{ll}0, &{} p^{m+1} \mid n \text{ for } \text{ some } \text{ prime } p, \\ \prod _{p^{\ell } \Vert n}(-1)^{\ell } \sum _{1 \leqslant j_{1}<\cdots <j_{\ell } \leqslant m} \alpha _{j_{1}, \pi }\left( p \right) \ldots \alpha _{j_{\ell }, \pi }\left( p\right) , &{} \text{ for } \text{ all } \ell \leqslant m.\end{array}\right. } \end{aligned}$$

Clearly, $\mu _{\pi }(n)$ is multiplicative. Taking the logarithmic derivative for $L(s, \pi )$, we define, for $\Re s>1$,

$$\begin{aligned} -\frac{L^{\prime }}{L}(s, \pi )=\sum _{n=1}^{\infty } \frac{a_\pi (n) \Lambda (n)}{n^s}, \end{aligned}$$

where $\Lambda (n)$ is the von Mangoldt function defined by

$$\begin{aligned} \Lambda (n):= {\left\{ \begin{array}{ll}\log p, &{} \text{ if } n=p^{k}, \\ 0, &{} \text{ otherwise. } \end{array}\right. } \end{aligned}$$

Then for $\Re s>1$,

$$\begin{aligned} \quad \log L(s, \pi )=\sum _{n=2}^{\infty } \frac{a_\pi (n) \Lambda (n)}{n^s \log n}, \quad \log L_p(s, \pi )=\sum _{k=1}^{\infty } \frac{a_\pi \left( p^k\right) }{k p^{k s}}, \end{aligned}$$

where

$$\begin{aligned} a_\pi \left( p^k\right) =\sum _{j=1}^m \alpha _{j, \pi }(p)^k. \end{aligned}$$

Let $N_{\pi }$ denote the conductor of $\pi$. At the archimedean place of $\mathbb {Q}$, there exist m complex Langlands parameters $\mu _{\pi }(j)$ from which we define

$$\begin{aligned} L_{\infty }(s, \pi )=N_\pi ^{s / 2} \pi ^{-m s / 2} \prod _{j=1}^m \Gamma \left( \frac{s+\mu _\pi (j)}{2}\right) . \end{aligned}$$

Let $\tilde{\pi }$ denote the contragredient of $\pi \in \mathcal {A}(m)$, which is also an irreducible cuspidal automorphic representation in $\mathcal {A}(m)$. For each $p < \infty$, we have

$$\begin{aligned} \left\{ \alpha _{j, \tilde{\pi }}(p): 1 \leqslant j \leqslant m\right\} =\left\{ \overline{\alpha _{j, \pi }(p)}: 1 \leqslant j \leqslant m\right\} \end{aligned}$$

and

$$\begin{aligned} \left\{ \mu _{\tilde{\pi }}(j): 1 \leqslant j \leqslant m\right\} =\left\{ \overline{\mu _\pi (j)}: 1 \leqslant j \leqslant m\right\} . \end{aligned}$$

The generalized Ramanujan conjecture and Selberg’s conjecture assert that

$$\begin{aligned} \left| \alpha _{j, \pi }(p)\right| =1 \quad \text{ and } \quad \left| \Re \mu _{\pi }(j)\right| =0 \quad (1 \leqslant j \leqslant m). \end{aligned}$$

Due to Kim and Sarnak [11] $(2 \leqslant m \leqslant 4)$ and Luo, Rudnick and Sarnak [13] $(m \geqslant 5)$, the best known record is

$$\begin{aligned} \left| \alpha _{j, \pi }(p)\right| \leqslant p^{\theta _{m}}, \quad \text{ and } \quad -\Re \mu _{\pi }(j)\leqslant \theta _{m} \end{aligned}$$

for all primes p and $1 \leqslant j \leqslant m$, where

$$\begin{aligned} \theta _{2}=\frac{7}{64}, \quad \theta _{3}=\frac{5}{14}, \quad \theta _{4}=\frac{9}{22}, \quad \theta _{m}=\frac{1}{2}-\frac{1}{m^{2}+1}(m \geqslant 5). \end{aligned}$$

(3.1)

The analytic conductor of $\pi$ is defined by

$$\begin{aligned} C(\pi , t)=N_{\pi } \prod _{j=1}^{m}\left( 1+\left| i t+\mu _{\pi }(j)\right| \right) , \quad C(\pi )=C(\pi , 0), \end{aligned}$$

which we need to use in the following proof.

3.2 Rankin–Selberg L-functions

Let $\pi ^{\prime }=\otimes _{p} \pi _{p}^{\prime } \in \mathcal {A}\left( m^{\prime }\right)$ and $\pi =\otimes _{p} \pi _{p} \in \mathcal {A}\left( m\right)$. We define the Rankin–Selberg L-function $L\left( s, \pi \times \pi ^{\ \prime }\right)$ associated to $\pi$ and $\pi ^{\prime }$ to be

$$\begin{aligned} L\left( s, \pi \times \pi ^{ \prime }\right) =\prod _{p} L\left( s, \pi _{p} \times \pi _{p}^{\prime }\right) =\sum _{n=1}^{\infty } \frac{\lambda _{\pi \times \pi ^{\prime }}(n)}{n^{s}} \end{aligned}$$

for $\Re (s)>1$. For each (finite) prime p, the inverse of the local factor $L\left( s, \pi _{p} \times \pi _{p}^{\prime }\right)$ is defined to be a polynomial in $p^{-s}$ of degree $\leqslant m m^{ \prime }$,

$$\begin{aligned} L\left( s, \pi _{p} \times \pi _{p}^{ \prime }\right) ^{-1}=\prod _{j=1}^{m} \prod _{j^{ \prime }=1}^{m^{ \prime }}\left( 1-\frac{\alpha _{j, j^{\prime }, \pi \times \pi ^{ \prime }}(p)}{p^{s}}\right) \end{aligned}$$

(3.2)

for suitable complex numbers $\alpha _{j, j^{ \prime }, \pi \times \pi ^{ \prime }}(p)$. With $\theta _{m}$ as in (3.1), we have the pointwise bound

$$\begin{aligned} \left| \alpha _{j, j^{ \prime }, \pi \times \pi ^{ \prime }}(p)\right| \leqslant p^{\theta _{m}+\theta _{m^{ \prime }}} \leqslant p^{1-\frac{1}{m m^{ \prime }}}. \end{aligned}$$

(3.3)

If $p \not \mid N_{\pi } N_{\pi ^{ \prime }}$, we have the equality of sets

$$\begin{aligned} \left\{ \alpha _{j, j^{ \prime }, \pi \times \pi ^{ \prime }}(p): j \leqslant m, j^{ \prime } \leqslant m^{ \prime }\right\} =\left\{ \alpha _{j, \pi }(p) \overline{\alpha _{j^{\prime }, \pi ^{ \prime }}(p)}: j \leqslant m, j^{ \prime } \leqslant m^{\prime }\right\} .\nonumber \\ \end{aligned}$$

(3.4)

The inverse of $L(s, \pi \times \pi ^{ \prime })$ is

$$\begin{aligned} L^{-1}(s, \pi \times \pi ^{ \prime })=\sum _{n=1}^{\infty } \frac{\mu _{\pi \times \pi ^{\prime }}(n)}{n^{s}}, \end{aligned}$$

where

$$\begin{aligned} \mu _{\pi \times \pi ^{ \prime }}(n)= {\left\{ \begin{array}{ll}0, \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad p^{mm^{\prime }+1} \mid n \text{ for } \text{ some } \text{ prime } p, \\ \prod _{p^{\ell } \Vert n}(-1)^{\ell } \sum _{1 \leqslant j_{1}<\cdots <j_{\ell } \leqslant mm^{\prime }} \alpha _{j_{1}, \pi \times \pi ^{ \prime }}\left( p\right) \ldots \alpha _{j_{\ell }, \pi \times \pi ^{ \prime }}\left( p\right) , \quad \text{ for } \text{ all } \ell \leqslant mm^{\prime }.\end{array}\right. }\nonumber \\ \end{aligned}$$

(3.5)

Taking the logarithmic derivative for $L(s, \pi \times \pi ^{\prime })$, we define, for $\Re s>1$,

$$\begin{aligned} -\frac{L^{\prime }}{L}(s, \pi \times \pi ^{\prime })=\sum _{n=1}^{\infty } \frac{a_{\pi \times \pi ^{\prime }}(n) \Lambda (n)}{n^s}. \end{aligned}$$

Then for $\Re s>1$,

$$\begin{aligned} \quad \log L(s, \pi \times \pi ^{\prime })=\sum _{n=2}^{\infty } \frac{a_{\pi \times \pi ^{\prime }}(n) \Lambda (n)}{n^s \log n}, \quad \log L_p(s, \pi \times \pi ^{\prime })=\sum _{k=1}^{\infty } \frac{a_{\pi \times \pi ^{\prime }}\left( p^k\right) }{k p^{k s}}, \end{aligned}$$

where

$$\begin{aligned} a_{\pi \times \pi ^{\prime }}\left( p^k\right) =\sum _{j=1}^m \sum _{j^{\prime }=1}^{m^{\prime }} \alpha _{j, j^{\prime }, \pi \times \pi ^{\prime }}(p)^k. \end{aligned}$$

(3.6)

At the archimedean place of $\mathbb {Q}$, there are $m m^{\prime }$ complex Langlands parameters $\mu _{\pi \times \pi ^{\prime }}\left( j, j^{\prime }\right)$ from which we define

$$\begin{aligned} L_{\infty }\left( s, \pi \times \pi ^{\prime }\right) =\pi ^{-\frac{m m^{\prime } s}{2}} \prod _{j=1}^{m} \prod _{j^{\prime }=1}^{m^{\prime }} \Gamma \left( \frac{s+\mu _{\pi \times \pi ^{\prime }}(j, j^{\prime })}{2}\right) . \end{aligned}$$

These parameters satisfy the pointwise bound

$$\begin{aligned} \Re \left( \mu _{\pi \times \pi ^{\prime }}\left( j, j^{\prime }\right) \right) \geqslant -\theta _{m}-\theta _{m^{\prime }}. \end{aligned}$$

(3.7)

As with $L(s, \pi )$, we define the analytic conductor of $\pi \times \pi ^{\prime }$ to be

$$\begin{aligned} C\left( \pi \times \pi ^{\prime }, t\right)= & {} N_{\pi \times \pi ^{\prime }} \prod _{j=1}^{m} \prod _{j^{\prime }=1}^{m^{\prime }}\left( 1+\left| i t+\mu _{\pi \times \pi ^{\prime }}\left( j, j^{\prime }\right) \right| \right) , \quad C\left( \pi \times \pi ^{\prime }\right) \\= & {} ssC\left( \pi \times \pi ^{\prime }, 0\right) , \end{aligned}$$

where $N_{\pi \times \pi ^{\prime }}$ is the conductor of $\pi \times \pi ^{\prime }$. Bushnell and Henniart [1] proved that $N_{\pi \times \pi ^{\prime }} \mid N_\pi ^{m^{\prime }} N_{\pi ^{\prime }}^m$. It will be essential to be able to decouple the dependencies of $C\left( \pi \times \pi ^{\prime }, t\right)$ on $\pi , \pi ^{\prime },$ and t. The combined work of Bushnell and Henniart [1, Theorem 1] and Brumley [5, Lemma A.2] yields

$$\begin{aligned} C\left( \pi \times \pi ^{\prime }, t\right) \leqslant C\left( \pi \times \pi ^{\prime }\right) (1+|t|)^{m^{\prime } m}, \quad C\left( \pi \times \pi ^{\prime }\right) \leqslant e^{O\left( m+m^{\prime }\right) } C(\pi )^{m^{\prime }} C\left( \pi ^{\prime }\right) ^m. \end{aligned}$$

The first result is due to Jiang et al. [8]. They proved an inequality between the coefficients of the L-function $L(s,\pi )$ and those of the Rankin–Selberg L-function $L(s,\pi \times \tilde{\pi })$.

Lemma 3.1

Let $\pi \in \mathcal {A}(m)$ and $\pi ^{\prime } \in \mathcal {A}\left( m^{\prime }\right)$. Then the inequality

$$\begin{aligned} \left| \lambda _{\pi \times \pi ^{\prime }}(n)\right| \leqslant \sqrt{\lambda _{\pi \times \tilde{\pi }}(n) \lambda _{\pi ^{\prime } \times \tilde{\pi }^{\prime }}(n)} \end{aligned}$$

holds for any positive integer n. In particular, for any $\pi \in \mathcal {A}(m)$, we have

$$\begin{aligned} \left| \lambda _\pi (n)\right| ^2 \leqslant \lambda _{\pi \times \tilde{\pi }}(n). \end{aligned}$$

In order to prove Theorem 1.3, we need the weak bound of Rankin–Selberg L-functions, which is obtained by Soundarajan and Thorner [20].

Lemma 3.2

If $\pi \in \mathcal {A}\left( m\right)$ and $\pi ^{\prime } \in \mathcal {A}\left( m^{\prime }\right)$ are two cuspidal automorphic representations, then

$$\begin{aligned} \left| L\left( 1 / 2, \pi \times \pi ^{\prime }\right) \right| \ll _{m, m^{\prime }}\left| L\left( 3 / 2, \pi \times \pi ^{\prime }\right) \right| ^{2} \frac{C\left( \pi \times \pi ^{\prime }\right) ^{1 / 4}}{\left( \log C\left( \pi \times \pi ^{\prime }\right) \right) ^{1 /\left( 10^{17} m^{3} m^{\prime 3}\right) }}. \end{aligned}$$

Remark 3.3

The above result for the L-values is at the central point 1/2. In the t-aspect, the results in [20] can apply equally to any point $1 / 2+i t$ on the critical line with trivial modifications. Their work gives the weak subconvexity bound

$$\begin{aligned} \left| L\left( 1 / 2+i t, \pi \times \pi ^{\prime }\right) \right| \ll _{\pi _1, \pi _2} \left| L\left( 3 / 2, \pi \times \pi ^{\prime }\right) \right| ^{2} \frac{C\left( \pi \times \pi ^{\prime },t\right) ^{1 / 4}}{\left( \log C\left( \pi \times \pi ^{\prime },t\right) \right) ^{1 /\left( 10^{17} m^{3} m^{\prime 3}\right) }}. \end{aligned}$$

By the condition (1.8) and partial summation, we know $L\left( 3 / 2, \pi \times \pi ^{\prime }\right)$ is bounded for $\pi$ and $\pi ^{\prime }$ in Theorem 1.3.

In the proof of Theorem 1.6, we need a standard zero-free region whenever at least one of the forms is self-dual. This is the following lemma which is proved by Brumley in [5, Appendix A].

Lemma 3.4

Let $\pi \in \mathcal {A}(m)$ and $\pi ^{\prime } \in \mathcal {A}\left( m^{\prime }\right)$. Assume that $\pi ^{\prime }$ is self-dual. There is an effective absolute constant $c>0$ such that $L\left( s, \pi \times \pi ^{\prime }\right)$ is non-vanishing for all $s=\sigma +i t \in \mathbb {C}$ satisfying

$$\begin{aligned} \sigma \geqslant 1-\frac{c}{\left( m+m^{\prime }\right) ^{3} \log \left( C(\pi ) C\left( \pi ^{\prime }\right) (|t|+3)^{m}\right) } \end{aligned}$$

with the possible exception of one real zero whenever $\pi$ is also self-dual.

3.3 twists

Let $\chi$ be a primitive Dirichlet character with conductor q, $\pi ^{\prime }=\otimes _{p} \pi _{p}^{\prime } \in \mathcal {A}\left( m^{\prime }\right)$ and $\pi =\otimes _{p} \pi _{p} \in \mathcal {A}\left( m\right)$. It’s well known that $\pi \otimes \chi \in \mathcal {A}\left( m\right)$. The twisted Rankin–Selberg L-function is defined by

$$\begin{aligned} L(s, \pi \otimes \chi \times \pi ^{\prime })=\sum _{n=1}^{\infty } \frac{\lambda _{\pi \otimes \chi \times \pi ^{\prime }}(n) }{n^s} =\prod _p \prod _{j=1}^m \prod _{j^{\prime }=1}^{m^{\prime }}\left( 1-\frac{\alpha _{j, j^{\prime }, \pi \otimes \chi \times \pi ^{\prime }}(p)}{p^s}\right) ^{-1}, \end{aligned}$$

in which

$$\begin{aligned} \lambda _{\pi \otimes \chi \times \pi ^{\prime }}(n)=\chi (n)\lambda _{\pi \times \pi ^{\prime }}(n). \end{aligned}$$

Moreover, by (3.4), if $p \not \mid q$, then

$$\begin{aligned} \begin{aligned}&\left\{ \alpha _{j, j^{\prime }, \pi \otimes \chi \times \pi ^{\prime }}(p): 1 \leqslant j \leqslant m, 1 \leqslant j^{\prime } \leqslant m^{\prime }\right\} \\&\quad =\left\{ \chi (p)\alpha _{j_1, j_2, \pi \times \pi ^{\prime }}(p): 1 \leqslant j \leqslant m, 1 \leqslant j^{\prime } \leqslant m^{\prime }\right\} . \end{aligned} \end{aligned}$$

Thus, we have

$$\begin{aligned} \begin{aligned} \sum _{n=1}^{\infty } \frac{\chi (n)\lambda _{\pi \times \pi ^{\prime }}(n) }{n^s}&=\prod _p \prod _{j=1}^m \prod _{j^{\prime }=1}^{m^{\prime }}\left( 1-\frac{\chi (p)\alpha _{j, j^{\prime }, \pi \times \pi ^{\prime }}(p)}{p^s}\right) ^{-1} \\&=L(s, \pi \otimes \chi \times \pi ^{\prime }) \prod _{p \mid q} \prod _{j=1}^m \prod _{j^{\prime }=1}^{m^{\prime }}\left( 1-\frac{\alpha _{j, j^{\prime }, \pi \otimes \chi \times \pi ^{\prime }}(p)}{p^s}\right) . \end{aligned} \end{aligned}$$

Denote $N_{\pi \otimes \chi \times \pi ^{\prime }}$ the conductor of $\pi \otimes \chi \times \pi ^{\prime }$. In fact, due to the work of Bushnell and Henniart [1], the conductor $N_{\pi \otimes \chi \times \pi ^{\prime }}$ has the upper bound

$$\begin{aligned} N_{\pi \otimes \chi \times \pi ^{\prime }}\le N_{\pi \times \pi ^{\prime }}q^{mm^{\prime }}. \end{aligned}$$

Using Lemma 3.4 and the same method in [7, Lemma 4.2], we obtain upper bounds for $\frac{1}{L(s,\pi \otimes \chi \times \pi ^{\prime })}$.

Lemma 3.5

For any Dirichlet character $\chi ({\text {mod}} q)$ and for all $s=\sigma +i t \in \mathbb {C}$, let c be the constant in Lemma 3.4, and suppose that

$$\begin{aligned} \sigma \geqslant 1-\frac{c}{2\left( m+m^{\prime }\right) ^{3} \log \left( C(\pi \otimes \chi ) C(\pi ^{\prime })(|t|+3)^{m}\right) }. \end{aligned}$$

Then

$$\begin{aligned} \frac{1}{L(s,\pi \otimes \chi \times \pi ^{\prime })} \ll _{\pi ,\pi ^{\prime }} \log \left( C(\pi \otimes \chi ) C(\pi ^{\prime }) (|t|+3)\right) . \end{aligned}$$

(3.8)

Proof

In order to derive the estimate for $\frac{1}{L(s,\pi \otimes \chi \times \pi ^{\prime })}$, we need to consider the estimate for $\frac{L^{\prime }}{L}(s, \pi \otimes \chi \times \pi ^{\prime })$.

Firstly, suppose that $\chi$ is a primitive character modulo q. Then $L(s,\pi \otimes \chi \times \pi ^{\prime })$ is an L-function of degree $mm^{\prime }$. By Proposition 5.7 in [6], we know that the number of zeros $\rho =\beta +i \gamma$ such that $|\gamma -T| \leqslant 1$, say $m(T, \pi \otimes \chi \times \pi ^{\prime })$, satisfies

$$\begin{aligned} m(T, \pi \otimes \chi \times \pi ^{\prime }) \ll \log \left( C(\pi \otimes \chi )^{m^{\prime }} C(\pi ^{\prime })^{m} (|T|+3)^{m^{\prime }m}\right) \end{aligned}$$

(3.9)

and for any s in the strip $-1 / 2 \leqslant \sigma \leqslant 2$,

$$\begin{aligned} \begin{aligned} \frac{L^{\prime }}{L}(s,\pi \otimes \chi \times \pi ^{\prime })-\sum _{\left| s+\mu _{\pi \otimes \chi \times \pi ^{\prime }}(j,j^{\prime })\right|<1} \frac{1}{s+\mu _{\pi \otimes \chi \times \pi ^{\prime }}(j,j^{\prime })}&-\sum _{|s-\rho |<1} \frac{1}{s-\rho }&\\ \ll \log \left( C(\pi \otimes \chi )^{m^{\prime }} C(\pi ^{\prime })^{m} (|t|+3)^{m^{\prime }m}\right) . \end{aligned} \end{aligned}$$

So by (3.7), we get

$$\begin{aligned} \frac{L^{\prime }}{L}(s, \pi \otimes \chi \times \pi ^{\prime }){} & {} \ll \sum _{|t-\gamma |<1} \frac{1}{|\sigma -\beta +i(t-\gamma )|}\\{} & {} \quad +\log \left( C(\pi \otimes \chi )^{m^{\prime }} C(\pi ^{\prime })^{m} (|t|+3)^{m^{\prime }m}\right) . \end{aligned}$$

Due to

$$\begin{aligned} \beta <1-\frac{c_{m,m^{\prime }}}{ \log \left( C(\pi \otimes \chi ) C(\pi ^{\prime }) (|t|+3)\right) }, \quad \sigma >1-\frac{c_{m,m^{\prime }}}{2 \log \left( C(\pi \otimes \chi ) C(\pi ^{\prime }) (|t|+3)\right) } \end{aligned}$$

and (3.9), we have

$$\begin{aligned} \begin{aligned} \frac{L^{\prime }}{L}(s, \pi \otimes \chi \times \pi ^{\prime })&\ll _{\pi ,\pi ^{\prime }} \log \left( C(\pi \otimes \chi ) C(\pi ^{\prime }) (|t|+3)\right) \bigg (\sum _{|t-\gamma |<1} 1+1\bigg ) \\&\ll \log ^{2} \left( C(\pi \otimes \chi ) C(\pi ^{\prime }) (|t|+3)\right) . \end{aligned} \end{aligned}$$

(3.10)

Then, suppose $\chi ^{*}$ modulo $q^{*}$ with $q^{*}\mid q$ is the primitive character which includes $\chi ({\text {mod}} q)$. We deduce the following equality between logarithmic derivatives

$$\begin{aligned} -\frac{L^{\prime }}{L}(s, \pi \otimes \chi \times \pi ^{\prime })=-\frac{L^{\prime }}{L}\left( s,\pi \otimes \chi ^{*}\times \pi ^{\prime }\right) +O\left( \sum _{p \mid q, p \not \mid q^{*}}\left| \frac{L^{\prime }}{L}\left( s,\pi _{p} \otimes \chi ^{*}\times \pi _{p}^{\prime }\right) \right| \right) , \end{aligned}$$

using equality

$$\begin{aligned} L(s, \pi ^{\prime }\times \pi \otimes \chi )=L\left( s, \pi \otimes \chi ^{*}\times \pi ^{\prime }\right) \prod _{p \mid q, p \not \mid q^{*}} L\left( s, \pi _{p} \otimes \chi ^{*}\times \pi _{p}^{\prime }\right) . \end{aligned}$$

For the second term on the right hand side, it follows from (3.3) that

$$\begin{aligned} \frac{L^{\prime }}{L}\left( s, \pi _{p} \otimes \chi ^{*}\times \pi _{p}^{\prime }\right) \ll \sum _{1 \leqslant j \leqslant n} \sum _{1 \leqslant j^{\prime } \leqslant n^{\prime }} \frac{\left| \alpha _{\pi ^{\prime }\times \pi , j,j^{\prime }}(p)\right| p^{-\sigma } \log p}{1-\left| \alpha _{\pi ^{\prime }\times \pi , j,j^{\prime }}(p)\right| p^{-\sigma }} \ll 1 \end{aligned}$$

which holds for any $\sigma >1-1/(mm^{\prime })$. Thus, we control the error term by

$$\begin{aligned} \sum _{p \mid q} 1 \ll \log (q+1). \end{aligned}$$

Combining with (3.10), we have, for any character $\chi$,

$$\begin{aligned} \frac{L^{\prime }}{L}(s,\pi \otimes \chi \times \pi ^{\prime }) \ll \log ^{2} \left( C(\pi \otimes \chi ) C(\pi ^{\prime }) (|t|+3)\right) . \end{aligned}$$

(3.11)

Next, consider bounding $L^{-1}(s, \pi \otimes \chi \times \pi ^{\prime })$. Let $s_{1}=1+\frac{1}{\log ^{2} \left( C(\pi \otimes \chi ) C(\pi ^{\prime }) (|t|+3)\right) }+i t$. To get an estimate for the logarithm of $L(s, \pi \otimes \chi \times \pi ^{\prime })$, we integrate the logarithmic derivative along the horizontal line

$$\begin{aligned} \begin{aligned}&\log L(s, \pi \otimes \chi \times \pi ^{\prime })-\log L\left( s_{1}, \pi \otimes \chi \times \pi ^{\prime }\right) \\&\quad =\int _{s_{1}}^{s} \frac{L^{\prime }}{L}(w, \pi \otimes \chi \times \pi ^{\prime }) \textrm{d} w \\&\quad \ll \left| s_{1}-s\right| \log ^{2} \left( C(\pi \otimes \chi ) C(\pi ^{\prime }) (|t|+3)\right) \\&\quad \ll 1, \end{aligned} \end{aligned}$$

(3.12)

where the penultimate inequality is due to the estimate (3.11). It’s known from [20, Lemma 2.2] that

$$\begin{aligned} |a_{\pi \times \pi ^{\prime }}(n)|\leqslant \sqrt{a_{\pi \times \tilde{\pi }}(n)a_{\pi ^{\prime }\times \tilde{\pi ^{\prime }}}(n)}. \end{aligned}$$

(3.13)

Then if $1<\sigma <3 / 2$, by (3.13) and estimating trivially, we obtain

$$\begin{aligned} \begin{aligned} |\log L(s, \pi \otimes \chi \times \pi ^{\prime })|&\leqslant \sum _{n=2}^{\infty } \frac{\Lambda (n)\sqrt{a_{\pi \times \tilde{\pi }}(n)a_{\pi ^{\prime }\times \tilde{\pi ^{\prime }}}(n)} }{n^{\sigma } \log n} \\&\leqslant \sum _{n=2}^{\infty } \frac{\Lambda (n)a_{\pi \times \tilde{\pi }}(n) }{n^{\sigma } \log n}+\sum _{n=2}^{\infty } \frac{\Lambda (n)a_{\pi ^{\prime }\times \tilde{\pi ^{\prime }}}(n) }{n^{\sigma } \log n}. \end{aligned} \end{aligned}$$

Using Shahidi’s non-vanishing result of $L(s, \pi \times \tilde{\pi })$ on $\Re s=1$ (see [19]), we get

$$\begin{aligned} \sum _{n \leqslant x} \Lambda (n) a_{\pi \times \tilde{\pi }}(n) \sim x. \end{aligned}$$

(3.14)

Then we will use the exponential integral formula [4, 3.35(5)]

$$\begin{aligned} \int _{1}^{\infty } \frac{e^{-\mu x}}{x} \textrm{d} x=-{\text {Ei}}(-\mu ) \text{ for } \mu >0 \end{aligned}$$

and the asymptotic representation [4, 8.214(1)] of the function

$$\begin{aligned} {\text {Ei}}(x)=C+\log (-x)+\sum _{k=1}^{\infty } \frac{x^{k}}{k \cdot k !} \quad \text{ for } x<0. \end{aligned}$$

By partial integral and the above two equalities, we have

$$\begin{aligned} \begin{aligned} |\log L(s,\pi \otimes \chi \times \pi ^{\prime })|&\leqslant 2\int _{2}^{\infty } \frac{1}{t^{\sigma } \log t} \textrm{d} t+O(1) \\&\leqslant 2\int _{1}^{\infty } \frac{e^{(1-\sigma ) x}}{x} \textrm{d} x+O(1) \\&\ll |\log (\sigma -1)|+O(1). \end{aligned} \end{aligned}$$

Especially, at the point $s_{1}$, the estimate

$$\begin{aligned} \left| \log L\left( s_{1},\pi \otimes \chi \times \pi ^{\prime }\right) \right| \ll \log \log \left( C(\pi \otimes \chi ) C(\pi ^{\prime }) (|t|+3)\right) \end{aligned}$$

holds. Thus it follows from (3.12) and the above inequality that $\log L(s, \pi ^{\prime }\times \pi \otimes \chi )$ has the same upper bound. Since

$$\begin{aligned} \log \frac{1}{|L(s, \pi \otimes \chi \times \pi ^{\prime })|}=-\Re \log L(s, \pi \otimes \chi \times \pi ^{\prime }), \end{aligned}$$

we get the result (3.8) that we want. $\square$

4 Proof of Theorems 1.3 and 1.6

4.1 The circle method

We shall consider $\alpha \in [0,1)$. Let $1<P<Q$, $PQ=x$, P, Q be parameters to be chosen later. By the Dirichlet approximation theorem, for any $\alpha \in [0,1)$, there exists a rational number a/q such that

$$\begin{aligned} \left| \alpha -\frac{a}{q}\right| \leqslant \frac{1}{q Q},\quad (a, q)=1,\quad 0 \leqslant a<q \leqslant Q. \end{aligned}$$

(4.1)

The initial step of the Hardy–Littlewood circle method would be to divide all $\alpha$ into the major arcs and the minor arcs. For $0 \leqslant a<q \leqslant P$, we first denote the major arcs by

$$\begin{aligned} \mathfrak {M}(a, q)=\left[ \frac{a}{q}-\frac{1}{q Q}, \frac{a}{q}+\frac{1}{q Q}\right] . \end{aligned}$$

Write $\mathfrak {M}$ for the union of all the major arcs

$$\begin{aligned} \mathfrak {M}=\bigcup _{q \leqslant P} \bigcup _{\begin{array}{c} 1 \leqslant a \leqslant q \\ (a, q)=1 \end{array}} \mathfrak {M}(a, q). \end{aligned}$$

Next, define

$$\begin{aligned} \mathfrak {m}=[0,1] \backslash \mathfrak {M}, \end{aligned}$$

which is the complement of $\mathfrak {M}$ in [0, 1).

4.2 Major arcs

Due to (4.1) and partial summation, we obtain that

$$\begin{aligned} \sum _{n \leqslant x} f(n) e(n \alpha ) \ll \left( 1+\frac{x}{q Q}\right) \max _{1 \leqslant t \leqslant x}\left| \sum _{n \leqslant t} f(n) e\left( \frac{a n}{q}\right) \right| . \end{aligned}$$

(4.2)

It follows from the orthogonality of Dirichlet characters that

$$\begin{aligned} \begin{aligned} \sum _{n \leqslant t} f(n) e\left( \frac{a n}{q}\right)&=\sum _{h=1}^{q} e\left( \frac{a h}{q}\right) \sum _{\begin{array}{c} n \leqslant t \\ n \equiv h({\text {mod}} q) \end{array}} f(n) \\&=\sum _{d \mid q} \sum _{\begin{array}{c} h=1 \\ (h, q / d)=1 \end{array}}^{q/d} e\left( \frac{a d h}{q}\right) \sum _{\begin{array}{c} l \leqslant t / d \\ l \equiv h({\text {mod}} q / d) \end{array}} f(d l) \\&=\sum _{d \mid q} \frac{1}{\varphi (q / d)} \sum _{\chi ({\text {mod}} q / d)} \sum _{\begin{array}{c} h=1 \\ (h, q / d)=1 \end{array}}^{q/d} \bar{\chi }(h) e\left( \frac{a d h}{q}\right) \sum _{l \leqslant t / d} f(d l) \chi (l) \\&=\sum _{d \mid q} \frac{1}{\varphi (q / d)} \sum _{\chi ({\text {mod}} q / d)} \chi (a) \tau (\bar{\chi }) \sum _{l \leqslant t / d} f(d l) \chi (l), \end{aligned} \end{aligned}$$

where $\varphi$ is the Euler function. As the Gauss sum $\tau (\bar{\chi })$ has the well-known bound

$$\begin{aligned} \tau (\bar{\chi })=\sum _{\begin{array}{c} h=1 \\ (h, q / d)=1 \end{array}}^{q / d} \bar{\chi }(h) e\left( \frac{d h}{q}\right) \ll (q / d)^{\frac{1}{2}}, \end{aligned}$$

we get

$$\begin{aligned} \sum _{n \leqslant t} f(n) e\left( \frac{a n}{q}\right) \ll q^{1 / 2} \sum _{d \mid q} \frac{1}{d^{1 / 2}} \max _{\chi ({\text {mod}} q / d)}\left| \sum _{l \leqslant x / d} f(d l) \chi (l)\right| . \end{aligned}$$

The multiplicative functions f(dl) need to be factored over l. So let $l=l_{1} l_{2} \quad$ with $l_{1} \mid d^{\infty }$ and $\left( l_{2}, d\right) =1.$ Then by (3.3), we have

$$\begin{aligned} \sum _{n \leqslant t} f(n) e\left( \frac{a n}{q}\right) \ll q^{\frac{1}{2}} \sum _{d \mid q} d^{\frac{1}{2}-\frac{1}{m^{2}}-\frac{1}{m^{\prime 2}}}\sum _{\begin{array}{c} l_{1} \mid d^{\infty } \\ l_{1} \leqslant t \end{array}} l_{1}^{1-\frac{1}{m^{2}}-\frac{1}{m^{\prime 2}}} \max _{\chi ({\text {mod}} q / d)}\left| \sum _{l_{2} \leqslant t /\left( d l_{1}\right) } f\left( l_{2}\right) \chi \left( l_{2}\right) \right| .\nonumber \\ \end{aligned}$$

(4.3)

Now it suffices to estimate the sum of type

$$\begin{aligned} \sum _{n \leqslant X} f(n) \chi (n) \end{aligned}$$

(4.4)

for any $\chi ({\text {mod}} r)$ with $0<r \leqslant q$ and $0<X \leqslant t$. We choose a function $\phi$ supported on $[0, X+Y]$, such that $\phi (z)=1$ if $Y \leqslant z \leqslant X$ and $\phi ^{(j)}(x) \ll _{j} Y^{-j}$ for all $j \geqslant 0$. Here, the parameter Y will be chosen later subject to $1 \leqslant Y \leqslant X$. By partial integration, the Mellin transform of $\phi$ satisfies

$$\begin{aligned} \hat{\phi }(s)=\int _{0}^{X+Y} \phi (z) z^{s-1} \textrm{d} z \ll \frac{Y}{X^{1-\sigma }} \cdot \left( \frac{X}{|s| Y}\right) ^{j} \end{aligned}$$

for any $j \geqslant 1$ and $1 / 2 \leqslant \sigma =\Re s \leqslant 2$.

By the Cauchy–Schwarz inequality and (1.6), we derive that

$$\begin{aligned} \begin{aligned} \sum _{X<n \leqslant X+Y}|f(n)|&\ll \left( \sum _{X<n \leqslant X+Y}\left| f(n)\right| ^{2} \right) ^{\frac{1}{2}}\left( \sum _{X<n \leqslant X+Y}1 \right) ^{\frac{1}{2}}\ll _{f} X^{\frac{1}{2}+\varepsilon }Y^{\frac{1}{2}} \end{aligned} \end{aligned}$$

for $1\leqslant Y \leqslant X$. Thus we can smooth the sum (4.4) by writing

$$\begin{aligned} \sum _{n \leqslant X}f(n)\chi (n) =\sum _{n} f(n) \chi (n) \phi (n)+O(X^{\frac{1}{2}+\varepsilon }Y^{\frac{1}{2}}). \end{aligned}$$

(4.5)

Case 1: $f(n)=\lambda _{\pi \times \tilde{\pi }}(n)$. By Mellin’s inverse transform, we can write

$$\begin{aligned} \sum _{n} \lambda _{\pi \otimes \chi \times \pi ^{\prime }}(n) \phi (n)=\frac{1}{2 \pi i} \int _{(2)} \hat{\phi }(s) L(s, \pi \otimes \chi \times \pi ^{\prime } ) \textrm{d} s. \end{aligned}$$

(4.6)

If $\chi$ is induced by a primitive character $\chi _{1}\left( {\text {mod}} r_{1}\right)$, then $r_{1} \mid r$ and

$$\begin{aligned} L(s,\pi \otimes \chi \times \pi ^{\prime } )=L\left( s, \pi \otimes \chi _{1}\times \pi ^{\prime }\right) \prod _{p \mid \frac{r}{r_{1}}} \prod _{j=1}^{m}\prod _{j^{\prime }=1}^{m^{\prime }}\left( 1-\alpha _{\pi \otimes \chi _{1}\times \pi ^{\prime },j,j^{\prime }}(p) p^{-s}\right) . \end{aligned}$$

Due to the estimate (3.3), for $\Re s=\frac{1}{2}$,

$$\begin{aligned} \prod _{p \mid \frac{r}{r_{1}}} \prod _{j=1}^{m}\prod _{j^{\prime }=1}^{m^{\prime }}\left( 1-\alpha _{\pi \otimes \chi _{1}\times \pi ^{\prime },j,j^{\prime }}(p) \chi _{1}(p) p^{-s}\right) \ll \left( \frac{r}{r_{1}}\right) ^{\frac{m+m^{\prime }}{2}-\frac{m+m^{\prime }}{mm^{\prime }}}. \end{aligned}$$

Moving the vertical line of integration in (4.6) to $\Re s=1/2$, we obtain by Cauchy’s theorem and Lemma 3.2 that

$$\begin{aligned} \begin{aligned}&\sum _{n \leqslant X} \lambda _{\pi \otimes \chi \times \pi ^{\prime }}(n) \phi (n)\\&\quad \ll \left( \frac{r}{r_{1}}\right) ^{\frac{m+m^{\prime }}{2}-\frac{m+m^{\prime }}{mm^{\prime }}}. \int _{(\Re s)}\left| \hat{\phi }(s) L\left( s, \pi \otimes \chi _{1}\times \pi ^{\prime }\right) \right| \textrm{d} s \\&\quad \ll _{\pi ,\pi ^{\prime }}\left( \frac{r}{r_{1}}\right) ^{\frac{m+m^{\prime }}{2}- \frac{m+m^{\prime }}{mm^{\prime }}}\left( \int _{0}^{X / Y} \frac{X^{\sigma }}{t+1}r_{1}^{\frac{mm^{\prime }}{4}} \frac{(2+|t|)^{mm^{\prime } / 4}}{(\log r_1(2+|t|))^{1 /\left( 10^{17} m^3 m^{\prime 3}\right) }}\textrm{d} t\right. \\&\left. \quad \quad +\int _{X / Y}^{\infty } \frac{Y}{X^{1-\sigma }} \cdot \left( \frac{X}{t Y}\right) ^{\frac{mm^{\prime }}{4}+2}r_{1}^{\frac{mm^{\prime }}{4}}\frac{(2+|t|)^{mm^{\prime } / 4}}{(\log r_1(2+|t|))^{1 /\left( 10^{17} m^3 m^{\prime 3}\right) }} \textrm{d} t\right) \\&\quad \ll _{\pi ,\pi ^{\prime }} \left( \frac{r}{r_{1}}\right) ^{\frac{m+m^{\prime }}{2}- \frac{m+m^{\prime }}{mm^{\prime }}}r_{1}^{\frac{mm^{\prime }}{4}}\left( \frac{X}{Y} \right) ^{\frac{mm^{\prime }}{4}}\left( \log \frac{r_1X}{Y}\right) ^{\frac{-1}{10^{17} m^3 m^{\prime 3}}}X^{\frac{1}{2}}.\end{aligned} \end{aligned}$$

We gather the above results to obtain

$$\begin{aligned} \sum _{n \leqslant X} \lambda _{\pi \otimes \chi \times \pi ^{\prime }}(n){} & {} \ll _{\pi ,\pi ^{\prime }}\left( \frac{r}{r_{1}}\right) ^{\frac{m+m^{\prime }}{2}- \frac{m+m^{\prime }}{mm^{\prime }}}r_{1}^{\frac{mm^{\prime }}{4}}\left( \frac{X}{Y} \right) ^{\frac{mm^{\prime }}{4}}\left( \log \frac{r_1X}{Y}\right) ^{\frac{-1}{10^{17} m^3 m^{\prime 3}}}X^{\frac{1}{2}}\\{} & {} \quad +X^{\frac{1}{2}+\varepsilon }Y^{\frac{1}{2}}. \end{aligned}$$

Then choose $Y^{\frac{1}{2}}=\left( \frac{r}{r_{1}}\right) ^{\left( \frac{m+m^{\prime }}{2}- \frac{m+m^{\prime }}{mm^{\prime }}\right) \frac{2}{2+mm^{\prime }}}r_{1}^{\frac{mm^{\prime }}{2(2+mm^{\prime })}}X^{\frac{mm^{\prime }}{2(2+mm^{\prime })}+\varepsilon }$ obtaining

$$\begin{aligned} \begin{aligned} \sum _{n \leqslant X} \lambda _{\pi \otimes \chi \times \pi ^{\prime }}(n)&\ll _{\pi ,\pi ^{\prime }} r^{\frac{m+m^{\prime }}{2+mm^{\prime }}+\frac{1}{2}}X^{\frac{1+mm^{\prime }}{2+mm^{\prime }}+\varepsilon }. \end{aligned} \end{aligned}$$

Inserting this bound into (4.3) yields

$$\begin{aligned} \sum _{n \leqslant t} \lambda _{\pi \times \pi ^{\prime }}(n) e\left( \frac{a n}{q}\right) \ll _{\pi ,\pi ^{\prime }} q^{\frac{m+m^{\prime }}{2+mm^{\prime }}+1}t^{\frac{1+mm^{\prime }}{2+mm^{\prime }}+\varepsilon }. \end{aligned}$$

(4.7)

Finally, it follows from (4.2) and (4.7) that

$$\begin{aligned} \sum _{n \leqslant x} \lambda _{\pi \times \pi ^{\prime }}(n) e(n \alpha ) \ll _{\pi ,\pi ^{\prime },\varepsilon }P^{\frac{m+m^{\prime }}{2+mm^{\prime }}+1} x^{\frac{1+mm^{\prime }}{2+mm^{\prime }}+\varepsilon }, \end{aligned}$$

(4.8)

where $\alpha \in \mathfrak {M}$.

Case 2: $f(n)=\mu _{\pi \times \pi ^{\prime }}(n).$ Also, from Mellin’s inverse transform, we can write

$$\begin{aligned} \sum _{n} \mu _{\pi \otimes \chi \times \pi ^{\prime }}(n) \phi (n)=\frac{1}{2 \pi i} \int _{(2)} \hat{\phi }(s) L^{-1}(s, \pi \otimes \chi \times \pi ^{\prime } ) \textrm{d} s. \end{aligned}$$

Define

$$\begin{aligned} \Omega =\left\{ s=\sigma +i t \mid \sigma \geqslant 1-\frac{c}{2\left( m+m^{\prime }\right) ^{3} \log \left( C(\pi \otimes \chi ) C(\pi ^{\prime })(|t|+3)^{m}\right) }\right\} . \end{aligned}$$

Lemma 3.4 shows that the left edge $\mathcal {Z}$ of $\Omega$ has no zeros. That is to say, we need not care about any pole and only estimate the integral over the left edge $\mathcal {Z}$ of $\Omega$. It follows from Lemma 3.5 that for any $s \in \mathcal {Z}$,

$$\begin{aligned} L^{-1}(s, \pi \otimes \chi \times \pi ^{\prime }) \ll _{\pi ,\pi ^{\prime }}\log \left( C(\pi \otimes \chi ) C(\pi ^{\prime }) (|t|+3)\right) . \end{aligned}$$

Then we obtain by Cauchy’s theorem and Lemma 3.5 that

$$\begin{aligned} \begin{aligned} \sum _{n \leqslant X} \mu _{\pi \otimes \chi \times \pi ^{\prime }}(n) \phi (n)&\ll \int _{(\Re s)}\left| \hat{\phi }(s) L^{-1}\left( s, \pi \otimes \chi _{1}\times \pi ^{\prime }\right) \right| \textrm{d} s \\ {}&\ll \int _{0}^{X / Y} \frac{X^{\sigma (t)}}{t+1}\log \left( C(\pi \otimes \chi ) C(\pi ^{\prime }) (|t|+3)\right) \textrm{d} t\\&\quad \quad +\int _{X / Y}^{\infty } \frac{Y}{X^{1-\sigma (t)}} \cdot \left( \frac{X}{t Y}\right) ^{2}\log \left( C(\pi \otimes \chi ) C(\pi ^{\prime }) (|t|+3)\right) \textrm{d} t \\&\ll _{\pi ,\pi ^{\prime }}X^{\sigma (\frac{X}{Y})} \log ^{2}\left( C(\pi \otimes \chi ) C(\pi ^{\prime })(\frac{X}{Y}+3)\right) , \end{aligned} \end{aligned}$$

where $\sigma (\frac{X}{Y}) =1-\frac{c}{2\left( m+m^{\prime }\right) ^{3} \log \left( C(\pi \otimes \chi ) C(\pi ^{\prime })(\frac{X}{Y}+3)^{m}\right) }$. We know $X^{\varepsilon }\le \exp \big ((\log \log X)^{1+\delta }\big )$ in (4.5). So we choose $\frac{X}{Y}=\exp \left( \frac{\sqrt{\log X}}{2\left( m+m^{\prime }\right) ^{3}} \right)$ getting

$$\begin{aligned} \begin{aligned} \sum _{n \leqslant X} \mu _{\pi ^{\prime }\times \pi }(n) \chi (n)&\ll X \exp \left( -\frac{c_{1} \log X}{\sqrt{\log X}+2\left( m+m^{\prime }\right) ^{3} \log \left( C(\pi \otimes \chi ) C(\pi ^{\prime })\right) }\right) \\ {}&\quad \quad \cdot \log ^{2}\left( C(\pi \otimes \chi ) C(\pi ^{\prime })X\right) +X^{\frac{1}{2}+\varepsilon }Y^{\frac{1}{2}} \\&\ll (C(\pi \otimes \chi ) C(\pi ^{\prime }))^{\frac{1}{2}}X \exp \left( -\frac{c_{2}}{2} \sqrt{\log X}\right) +X^{\frac{1}{2}+\varepsilon }Y^{\frac{1}{2}} \\&\ll _{\pi ,\pi ^{\prime }} r^{\frac{mm^{\prime }}{2}} X \exp \left( -\frac{c_{3}}{2} \sqrt{\log X}\right) , \end{aligned} \end{aligned}$$

where $c_{3}$ depends on $\pi$ and $\pi ^{\prime }$. Inserting this bound into (4.3) yields

$$\begin{aligned} \sum _{n \leqslant t} \mu _{\pi ^{\prime }\times \pi }(n) e\left( \frac{a n}{q}\right) \ll q^{\frac{mm^{\prime }+1}{2}}t \exp \left( -\frac{c_{3}}{2} \sqrt{\log t}\right) . \end{aligned}$$

(4.9)

Finally, for $\alpha \in \mathfrak {M}$, we plug (4.9) back into (4.2) and then obtain

$$\begin{aligned} \sum _{n \leqslant x} \mu _{\pi ^{\prime }\times \pi }(n) e(n \alpha ) \ll _{\pi ,\pi ^{\prime },\varepsilon }\left( P+\frac{x}{Q}\right) P^{\frac{mm^{\prime }-1}{2}}x \exp \left( -\frac{c_{3}}{2} \sqrt{\log x}\right) . \end{aligned}$$

(4.10)

4.3 Minor arcs

We appeal to the following recursion (see [15, Eq. (24)])

$$\begin{aligned} k \lambda _{\pi \times \tilde{\pi }}\left( p^{k}\right) =\sum _{l=1}^{k}\left| a_{\pi }\left( p^{l}\right) \right| ^{2} \lambda _{\pi \times \tilde{\pi }}\left( p^{k-l}\right) \end{aligned}$$

for any $k>0$. Especially, when $k=1$, we note that

$$\begin{aligned} \lambda _{\pi \times \tilde{\pi }}(p)=\left| \lambda _{\pi }(p)\right| ^{2}=\left| a_{\pi }(p)\right| ^{2}. \end{aligned}$$

By the above equation and (3.14), we have

$$\begin{aligned} \sum _{n \leqslant x} \lambda _{\pi \times \tilde{\pi }}(p) \log p \ll x. \end{aligned}$$

Then the multiplicative function $\lambda _{\pi \times \pi ^{\prime }}(n)$ satisfies the second condition (1.7). Due to (3.6) and (3.5), we deduce that

$$\begin{aligned} \mu _{\pi \times \pi ^{\prime }}(p)=a_{\pi \times \pi ^{\prime }}(p). \end{aligned}$$

So Eq. (3.14) implies that the multiplicative function $\mu _{\pi \times \pi ^{\prime }}(n)$ also satisfies the second condition (1.7). Hence applying Theorem 1.1, for $\alpha \in \mathfrak {m}$, we get

$$\begin{aligned} \begin{aligned} \sum _{n\le x}\lambda _{\pi \times \pi ^{\prime }}(n)e(n\alpha )&\ll _{\pi ,\pi ^{\prime }} x\big (\exp \big ((\log \log x)^{1+\delta }\big )\big )^{\frac{1}{2}} \exp \bigg (-(1+o(1)) \frac{\log x}{\log P} \bigg )\\&\quad +x(\log x)\big (\exp \big ((\log \log x)^{1+\delta }\big )\big )^{\frac{1}{2}}P^{-\frac{1}{2}}\\&\quad +\frac{x\big (\exp \big ((\log \log P)^{1+\delta }\big )\big )^{\frac{1}{2}}\log P}{\log x} \end{aligned} \end{aligned}$$

(4.11)

and

$$\begin{aligned} \begin{aligned} \sum _{n\le x}\mu _{\pi \times \pi ^{\prime }}(n)e(n\alpha )&\ll _{\pi ,\pi ^{\prime }} x\big (\exp \big ((\log \log x)^{1+\delta }\big )\big )^{\frac{1}{2}} \exp \bigg (-(1+o(1)) \frac{\log x}{\log P} \bigg )\\&\quad +x(\log x)\big (\exp \big ((\log \log x)^{1+\delta }\big )\big )^{\frac{1}{2}}P^{-\frac{1}{2}}\\&\quad +\frac{x\big (\exp \big ((\log \log P)^{1+\delta }\big )\big )^{\frac{1}{2}}\log P}{\log x}. \end{aligned} \end{aligned}$$

(4.12)

4.4 The choices of parameters

Put $P=\exp \big (4(\log \log x)^{1+\delta }\big )$. Due to (4.8) and (4.10), we deduce that if $\alpha \in \mathfrak {M}$,

$$\begin{aligned} \sum _{n \leqslant x} \lambda _{\pi \times \pi ^{\prime }}(n) e(n \alpha ) \ll _{\pi ,\pi ^{\prime ,\varepsilon }}x^{\frac{1+mm^{\prime }}{2+mm^{\prime }}+\varepsilon } \exp \big (8(\log \log x)^{1+\delta }\big ) \end{aligned}$$

and

$$\begin{aligned} \sum _{n \leqslant x} \mu _{\pi ^{\prime }\times \pi }(n) e(n \alpha ) \ll _{\pi ,\pi ^{\prime }}x\exp \big (4(mm^{\prime })(\log \log x)^{1+\delta }\big ) \exp \left( -\frac{c_{3}}{2} \sqrt{\log x}\right) . \end{aligned}$$

For $\alpha \in \mathfrak {m}$, it follows from (4.11) and (4.12) that

$$\begin{aligned} \begin{aligned} \sum _{n\le x}\lambda _{\pi \times \pi ^{\prime }}(n)e(n\alpha )\ll _{\pi ,\pi ^{\prime }} \frac{x}{(\log x)^{1-\varepsilon }} \end{aligned} \end{aligned}$$

and

$$\begin{aligned} \begin{aligned} \sum _{n\le x}\mu _{\pi \times \pi ^{\prime }}(n)e(n\alpha )\ll _{\pi ,\pi ^{\prime }} \frac{x}{(\log x)^{1-\varepsilon }}. \end{aligned} \end{aligned}$$

(4.13)

By the above results, we complete the proof of Theorems 1.3 and 1.6.

5 Proof of Corollary 1.8

Define the Dirichlet series

$$\begin{aligned} D(s)=\sum _{n=1}^{\infty } \mu (n) \lambda _{\pi \times \pi ^{\prime }}(n) n^{-s}. \end{aligned}$$

Moreover, admit an Euler product

$$\begin{aligned} D(s)=\prod _p\left( 1-\frac{\lambda _{\pi \times \pi ^{\prime }}(p)}{p^s}\right) \end{aligned}$$

(5.1)

which converges absolutely for $\Re s>1$. By (3.2), we have

$$\begin{aligned} \lambda _{\pi \times \pi ^{\prime }}\left( p\right) =\sum _{j=1}^m \sum _{j^{\prime }=1}^{m^{\prime }} \alpha _{j, j^{\prime }, \pi \times \pi ^{\prime }}(p). \end{aligned}$$

(5.2)

It’s known that $L(s, \pi \times \pi ^{\prime })$ converges absolutely for $\Re s>1$. Thus the product $L(s, \pi \times \pi ^{\prime })D(s)$ is also given by a Dirichlet series, namely we have

$$\begin{aligned} L(s, \pi \times \pi ^{\prime })D(s)=H(s)=\sum _{n=1}^{\infty } h(n)n^{-s}. \end{aligned}$$

Since $L^{-1}(s, \pi \times \pi ^{\prime })$ converges absolutely for $\Re s>1$, we can write

$$\begin{aligned} D(s)=L^{-1}(s, \pi \times \pi ^{\prime }) H(s). \end{aligned}$$

(5.3)

Then it follows that

$$\begin{aligned} \mu (n) \lambda _{\pi \times \pi ^{\prime }}(n)=\sum _{d \mid n} \mu _{\pi \times \pi ^{\prime }}(d) h\left( \frac{n}{d}\right) . \end{aligned}$$

(5.4)

It follows from (3.5) that

$$\begin{aligned} L^{-1}(s, \pi \times \pi ^{\prime })=\prod _p\left( 1+\frac{\mu _{\pi \times \pi ^{\prime }}(p)}{p^s}+\frac{\mu _{\pi \times \pi ^{\prime }}\left( p^2\right) }{p^{2 s}}+\cdots +\frac{\mu _{\pi \times \pi ^{\prime }}\left( p^{mm^{\prime }}\right) }{p^{mm^{\prime } s}}\right) .\nonumber \\ \end{aligned}$$

(5.5)

Combining (5.1), (5.3), (5.5), (5.2) with (3.5), we obtian

$$\begin{aligned} H(s)=\prod _p\left( 1+O\left( \frac{\left| \alpha _{\pi \times \pi ^{\prime },1}(p)\right| ^2+\cdots +\left| \alpha _{\pi \times \pi ^{\prime }, mm^{\prime }}(p)\right| ^2}{p^{2 \sigma }}\right) \right) :=\sum _{n=1}^{\infty } \frac{h(n)}{n^s}. \end{aligned}$$

By (1.10), we know that H(s) converges absolutely in $\sigma > 1/2+\gamma$. Thus it follows from (5.4), the absolute inequality and (4.13) that

$$\begin{aligned} \begin{aligned} \sum _{n \leqslant x} \mu (n) \lambda _{\pi \times \pi ^{\prime }}(n) e\left( n \alpha \right)&=\sum _{m \leqslant x^{1-\varepsilon }} h(m) \sum _{d \leqslant x / m} \mu _{\pi \times \pi ^{\prime }}(d) e\left( d m \alpha \right) +O(x^{1-\varepsilon })\\&\ll _{\pi ,\pi ^{\prime }} \frac{x}{(\log x)^{1-\varepsilon }}, \end{aligned} \end{aligned}$$

uniformly in $\alpha$ $\in \mathbb {R}$.