1 Introduction

With the popularity of smart phones, mobile data service gets more and more attention. The increasing demand for higher capacity has brought great interest in multiple-input and multiple-output (MIMO) system. It is well known that the spectral efficiency of MIMO channel is much higher than that of the conventional single-antenna channels. In the last twenty years, MIMO has become a key technique to improve the data rate and link reliability. For 5G systems, an advanced MIMO technique which is called multi-user massive MIMO technique has been looked as an enabling technique.

As we know, MIMO system can provide two types of gains: diversity gain and multiplexing gain. Diversity gain is obtained when the same signals are sent at all transmitting antennas through different paths, thus the reliability of the system is improved. Multiplexing gain is obtained when different signals are sent through different antennas, therefore the data rate can be increased. Zheng and Tse [1, 2] found the fundamental trade-off between diversity gain and multiplexing gain for point-to-point MIMO channel and multiple access MIMO channels, respectively. Mehana and Nosratinia [3] studied the diversity performance of MIMO systems with MMSE receiver. However, most of the works only consider the MIMO systems with perfect channel state information at the receivers.

In multi-user MIMO (MU-MIMO) system, especially in massive MIMO system, due to the insufficient pilot resource, the reusing of pilot sequences will introduce severe pilot contamination which becomes the bottle neck of system performance. With emerging of massive MIMO techniques, the fundamental effect of pilot contamination should be studied for the system design. In [4], the asymptomatic sum-rate of MU-MIMO system with pilot contamination was studied. The authors in [5] have studied the asymptotic bit error rate (BER) of MU-MIMO with pilot contamination. However, to the best of the authors’ knowledge, there is no study on the performance analysis over the massive MIMO system for any given value of the SNR instead of the asymptotic regime.

For MU-MIMO system with pilot contamination, the outage and diversity of MMSE receiver remain unknown in this literature, which motivates us to research this problem. The objective in this paper is to derive the diversity performance of uplink MU-MIMO system with MMSE receiver. We focus on how the inter-cell interference and pilot contamination affect the diversity performance of the system.

The reminder of this paper is organized as follows. Section 2 introduces the system model and the equivalent model under pilot contamination. Section 3 obtains the channel capacity of the n-th user’s sum rate in target cell with pilot contamination. Section 4 aims to get the diversity performance of the system by deriving the lower bound and upper bound of the outage probability in high SNR regime. Section 5 focuses on similar topic to Sect. 4 but in the finite SNR regime. Numerical results and discussions are given in Sect. 6. Finally, Sect. 7 gives the main conclusions.

2 System Model of Multi-cell MU-MIMO

We consider uplink transmission of a multi-cell multi-user MIMO systems. There are L hexagonal cells, and each cell has one base station with M antennas. We assume that in each cell there are N users, and the n-th user is with \(K_{n}\) antennas. The total number of antennas is denoted as \(K = \sum _{n=1}^NK_{n}.\) Cell 1 is the target cell (the base-station in the target cell is denoted as BS 1), and other cells are interfering cells.

2.1 System Model

The received signal at BS 1 is given by

$${\varvec{y}}_1 = {\varvec{G}}_1{\varvec{x}}_1 + \sum _{l=2}^L{\varvec{G}}_{l}{\varvec{x}}_l + {\varvec{n}}_1$$
(1)

where \({{\varvec{y}}_1} = {\left[ y_1 \ldots y_M\right] }^\mathrm{T}\) is the received vector at BS 1. \({{\varvec{x}}_l} = {\left[ x_{l,1} \ldots x_{l,K}\right] }^\mathrm{T}\) is the vector of all K transmitting antennas, the entries of \({{\varvec{x}}_l}\) are i.i.d Zero Mean Circularly Symmetric Complex Gaussian (ZMCSCG) random variables with unit variance and zero mean. \({{\varvec{G}}_l} = {\left[ {\varvec{g}}_{l,1} \ldots {\varvec{g}}_{l,K}\right] }\) is the uplink channel matrix from cell l to BS 1. The k-th vector of \({{\varvec{G}}_l}\) namely the channel from the k-th transmitting antenna to BS 1 is \({\varvec{g}}_{l,k} = {\left[ g_{l,k,1} \cdots g_{{l,k,M}}\right] }^\mathrm{T}\). \({\varvec{n}}_1\) is the zero mean complex additive Gaussian noise with variance \(1/{\rho }\), where \(\rho\) is the uplink SNR.

In this paper, we consider the Wyner cell model [6, 7] which has been widely used in the studying of multi-cell multi-user systems. Assuming the interference factor from cell \(l\left( l \ne 1\right)\) is \(\alpha\), and \({\varvec{g}}_{l,k}\) is modelled as

$${\varvec{g}}_{l,k} = {\left\{ \begin{array}{ll} {\varvec{h}}_{l,k} &{} l = 1 \\ \sqrt{\alpha }{\varvec{h}}_{l,k} &{} l = 2, \ldots , L \end{array}\right. }$$
(2)

where \({\varvec{h}}_{l,k} = {\left[ h_{l,k,1} \cdots h_{l,k,M}\right] }^\mathrm{T}\) represents the small scale fast fading vector between the users in cell l and BS 1, and the entries are i.i.d ZMCSCG random variable with zero mean and unit variance.

2.2 Channel Estimation

We consider the imperfect CSI at the BS 1. In order to get the CSI, pilot sequences should be used. Due to the limited pilot resources, we consider the worst case pilot reuse [8], that is , all users in each cell transmit K orthogonal pilot sequences, and different cells share the same pilot signal sequences. To simplify analysis, we assume the time-orthogonal pilot sequences, that is the pilot sequences of the users in each cell can be denoted as the identity matrix \({\varvec{I}}_{K}\). The received signal at BS 1 is denoted as [9]

$${\varvec{Y}}_\mathrm{P} = {\varvec{G}}_{1} + \sum _{l=2}^{L}{\varvec{G}}_{l} + {\varvec{N}}_\mathrm{P}$$
(3)

where \({\varvec{Y}}_\mathrm{P}\) is the \(M \times K\) received matrix at BS 1, \({\varvec{N}}_\mathrm{P}\) is an \(M \times K\) noise matrix and each element is an i.i.d ZMCSCG random variable with variance \(\beta \rho ^{-1}\), where \(\beta\) denote the pilot-to-data power ratio.

From (2), we can see that

$${\mathcal {E}}({\varvec{g}}_{l,n}{\varvec{g}}_{j,m}^\mathrm{H}) = \mathrm{0} \ \left( j \ne l \; \mathrm{or} \ n \ne m\right) .$$

Then, to estimate \({\varvec{G}}_{l}\), \({\varvec{g}}_{l,k}\) can be estimated individually. From (3), the k-th column of \({\varvec{Y}}_\mathrm{P}\) can be expressed as

$${\varvec{y}}_{\mathrm{P},k} = \sum _{l=1}^{L}{\varvec{g}}_{l,k}+{\varvec{n}}_{\mathrm{P},k}$$
(4)

To get \({\varvec{g}}_{l,k}\), the MMSE estimator is adopted. From [10], the estimation of \({\varvec{g}}_{1,k}\) can be obtained as

$$\hat{{\varvec{g}}}_{1,k} = \left[ 1+\alpha \left( L-1\right) +\beta \rho ^{-1}\right] ^{-1}{\varvec{y}}_{\mathrm{P},k}.$$
(5)

The estimation of \({\varvec{g}}_{l,k}\) for \(l= 2, \ldots , L\) can be given by

$$\hat{{\varvec{g}}}_{l,k} = \alpha \hat{{\varvec{g}}}_{1,k}$$
(6)

Define

$$\begin{aligned} \hat{{\varvec{h}}}_{k}=\left[ 1+\alpha \left( L-1\right) +\beta \rho ^{-1}\right] ^{-1/2}{\varvec{y}}_{\mathrm{P},k}. \end{aligned}$$
(7)

Since \({\varvec{y}}_{\mathrm{P},k}\) is a complex Gaussian vector, we have

$$\begin{aligned} \hat{{\varvec{h}}}_{k}\thicksim {\mathcal {C}}{\mathcal {N}}(\varvec{0},{{\varvec{I}}_{M}}). \end{aligned}$$

With the notions in (5), (6) and (7), the estimated channel matrix \(\hat{{\varvec{G}}}_{1}\) can be characterized as

$$\begin{aligned} \hat{{\varvec{G}}}_{1} = \left[ 1+\alpha \left( L-1\right) +\beta \rho ^{-1}\right] ^{-1/2}\hat{{\varvec{H}}}, \end{aligned}$$
(8)

where \(\hat{\varvec{H}} = {\left[ \hat{\varvec{h}}_{1} \ldots \hat{\varvec{h}}_{K}\right] }\). Similarly, we can get the estimation of \({\varvec{G}}_{l}\) for \(l \ne 1\) which is given by

$$\begin{aligned} \hat{\varvec{G}}_{l} = \alpha \hat{\varvec{G}}_{1}, \end{aligned}$$
(9)

For descriptional convenience, we define estimation error vector \(\tilde{\varvec{g}}_{l,k}={\varvec{g}}_{l,k}-\hat{\varvec{g}}_{l,k}\). From [10], the covariance matrix of \(\tilde{\varvec{g}}_{l,k}\) is

$$\begin{aligned} {\mathcal {E}}(\tilde{\varvec{g}}_{l,k},\tilde{\varvec{g}}_{l,k}) = {\left\{ \begin{array}{ll} 1-\left[ 1+\alpha \left( L-1\right) +\beta \rho ^{-1}\right] ^{-1} &{} l = 1\\ \alpha -\alpha ^{2}\left[ 1+\alpha \left( L-1\right) +\beta \rho ^{-1}\right] ^{-1} &{} l\ne 1 \end{array}\right. } \end{aligned}$$

If the receiver of the cell 1 knows the estimated channel matrices, \(\hat{\varvec{G}}_{l}\), the system model in (1) can be re-expressed as

$$\begin{aligned} {\varvec{y}}_{1} = \hat{\varvec{G}}_1{\varvec{x}}_1 + \sum _{l=2}^L\hat{\varvec{G}}_{l}{\varvec{x}}_l + \underbrace{\sum _{l=1}^L\tilde{\varvec{G}}_{l}{\varvec{x}}_l+{\varvec{n}}_1}_\mathrm{equivalent \ noise} \end{aligned}$$
(10)

In (10), the noise introduced by channel estimation error combined with white Gaussian noise can be treated as the following equivalent noise as

$$\begin{aligned} \tilde{\varvec{n}}_{1} \triangleq \sum _{l=1}^L \tilde{\varvec{G}}_{l}{\varvec{x}}_{l}+{\varvec{n}}_{1}. \end{aligned}$$

We have

$$\begin{aligned} {\mathcal {E}}{\left( \tilde{\varvec{n}}_{1}\tilde{\varvec{n}}_{1}^\mathrm{H}\right) } = \tau {\varvec{I}}_M \end{aligned}$$
(11)

where \(\tau\) is the variance of equivalent noise given by

$$\begin{aligned} \tau = K\left[ {\alpha }L-{\alpha }+1-\frac{{\alpha }^{2}L-{\alpha }^{2}+1}{1+\alpha \left( L-1\right) +\beta \rho ^{-1}}\right] +{\rho }^{-1}. \end{aligned}$$
(12)

3 Channel Capacity of Multi-cell MU-MIMO with Pilot Contamination

After channel estimation, at BS 1, the linear MMSE receiver is used to estimate transmitted vector \(\hat{\varvec{x}}_{1}\). We focus on the symbol that transmitted by the \(k_n\)-th antenna of user n, where \(k_n \in \left\{ K_n (n-1) + 1, \ldots , n K_n\right\}\). For the linear model (10), with the estimated channel matrix \(\hat{\varvec{G}}_1\) at BS 1, according the the multi-user detection theory in [11], the signal-to-interference-plus-noise (SINR) \(\gamma _{k_n}\) can be denoted as

$$\begin{aligned} \gamma _{k_n} = \hat{\varvec{g}}_{1,k_n}^\mathrm{H} \left( {\sum _{j \ne k_n}^{K} \hat{\varvec{g}}_{1,j}\hat{\varvec{g}}_{1,j}^\mathrm{H}+ \sum _{l=2}^{L} \hat{\varvec{G}}_{l} \hat{\varvec{G}}_{l} ^ \mathrm{H} + \tau {\varvec{I}}_M}\right) ^{-1} \hat{\varvec{g}}_{1,k_n}. \end{aligned}$$
(13)

We should note that the equivalent noise is not Gaussian. However, as shown in [4], we can get the lower bound of the achievable sum-rate of linear MMSE for the n-th user in the cell 1

$$\begin{aligned} C_{n}= \sum _{k_n}\log _2 (1+\gamma _{k_n}). \end{aligned}$$
(14)

A standard calculation [11] yields the SINRs of the resulting set of virtual parallel channels in the form

$$\begin{aligned} \gamma _{k_n} = \frac{1}{\omega _{{k_n}}} - 1 , \end{aligned}$$

where

$$\begin{aligned} {\omega _{{k_n}}}={{\left[ {\left( {\varvec{I}}_{K}+\hat{\varvec{G}}_1^\mathrm{H}{\left( \sum \limits _{l = 2}^L\hat{\varvec{G}}_l\hat{\varvec{G}}_l^\mathrm{H}+\tau {\varvec{I}}_{M}\right) }^{-1}\hat{\varvec{G}}_1\right) }^{-1}\right] }_{k_n,k_n}} \end{aligned}$$
(15)

Correspondingly, the sum rate in (14) can be given by

$$\begin{aligned} C_{n}=-\sum _{k_n}\log _2{\omega _{{k_n}}}. \end{aligned}$$
(16)

Substituting (8) and (9) into (15), we can further write (15) as

$$\begin{aligned} {\omega _{{k_n}}}={\left[ {\left( {\varvec{I}}_{K}+\mu _1\hat{\varvec{H}}^\mathrm{H} {\left( {\varvec{I}}_{M}+\mu _2\hat{\varvec{H}}\hat{{\varvec{H}}}^\mathrm{H}\right) }^{-1}\hat{{\varvec{H}}}\right) }^{-1}\right] }_{k_n,k_n} \end{aligned}$$
(17)

where

$$\begin{aligned} \mu _1= & {} \frac{\rho ^2}{\kappa _1\rho ^2+\kappa _2\rho +\beta },\\ \kappa _1=\, & {} K\left[ \alpha ^2\left( L^2-3L+2\right) +2\alpha \left( L-1\right) \right] ,\\ \kappa _2=\, & {} \left( \alpha L - \alpha + 1\right) \left( K\beta + 1\right) ,\\ \mu _2=\, & {} \alpha ^2\left( L-1\right) \mu _1. \end{aligned}$$

Eventually, from the matrix inversion lemma [12]

$$\begin{aligned} {\varvec{CD}}\left( {\varvec{A}}+{\varvec{BCD}}\right) ^{-1}=\left( {\varvec{C}}^{-1}+{\varvec{DA}}^{-1}{\varvec{B}}\right) ^{-1}{\varvec{D}}{\varvec{A}}^{-1}, \end{aligned}$$

we can rewrite (17) as

$$\begin{aligned} {\omega _{{k_n}}}={\left[ {\left( {\varvec{I}}_{K}+\mu _1 {\left( {\varvec{I}}_{K}+\mu _2\hat{{\varvec{H}}}^\mathrm{H}\hat{{\varvec{H}}}\right) }^{-1}\hat{{\varvec{H}}}^\mathrm{H}\hat{{\varvec{H}}}\right) }^{-1}\right] }_{k_n,k_n} \end{aligned}$$
(18)

As we can see, (16) and (18) are useful and simple form of sum rate of n-th user in cell 1, which are the basis of following analysis.

4 Outage Analysis with Pilot Contamination in High SNR Regime

In this section, we are aiming to characterise the diversity gain d, as a function of the spectral efficiency \(R\left( b/s/Hz\right)\), the number of cells L, the interference between cells \(\alpha\) and the pilot-to-data power ratio \(\beta\). This requires a PEP analysis which is not easy to analyse. But, in the high SNR regime, PEP behaves exponentially like the outage probability [2], so the following analysis is based on the outage probability. Considering a simple case in [2], all \(K_n\) antennas are pulled up together, equivalent to a point-to-point case where there is one user with \(K_n\) antennas.

Outage occurs if the channel fails to support the target rate R, so the outage probability of n-th user in cell 1 is denoted as

$$\begin{aligned} P_\mathrm{out}^{n} = {\mathcal {P}}\left( C_{n} < R\right) \end{aligned}$$
(19)

Then the diversity of n-th user in cell 1 is defined as

$$\begin{aligned} d\left( R, L, M, K_{n}, \alpha , \beta \right) = {-}\lim _{\rho \rightarrow \infty }\frac{\log _2 P_\mathrm{out}^n}{\log _2\rho } \end{aligned}$$
(20)

However, the \(C_{n}\) in (16) makes futher analysis intractable. Instead, we derive the upper bound and lower bound of on the outage probability \({P}_\mathrm{out}^{n}\) and then we derive the aiming diversity function d.

Theorem 1

In a multi-cell MU-MIMO system with MMSE receiver, considering the pilot contamination, as to the diversity gain \(d\left( R, L, M, K_{n}, \alpha , \beta \right)\) for the n-th user in cell 1 consisting of \(K_{n}\) transmitting antennas and M receiving antennas under i.i.d Rayleigh fading channel and high SNR regime, we have

$$\begin{aligned} d\left( R, L, M, K_{n}, \alpha , \beta \right) = 0 \end{aligned}$$
(21)

Proof

The theorem is proved by developing the following upper bound and lower bound on \(P_\mathrm{out}^n\) for the MMSE reciver in (16).

4.1 Outage Upper Bound

We begin to derive the upper bound of the outage probability in (16). Since the function \(-\log _2\) function is convex, using Jensen’s inequality, we have

$$\begin{aligned} C_n \ge -K_n \log _2 \left( \frac{1}{K_n} \sum _{k_n} {\omega _{{k_n}}} \right) \end{aligned}$$
(22)

It clearly shows that the items in the summation of (22) are some of the \(K \times K\) matrix’s diagonal entries and obviously the sum is smaller than the sum of all the diagonal entries which is the trace of the \(K \times K\) matrix. So we have

$$\begin{aligned} C_n \ge -K_n \cdot \log _2 \Biggl \{ \frac{1}{K_n} \mathrm{Tr} \biggl [{\varvec{I}}_{K}+ \mu _1 {\left( {\varvec{I}}_{K}+\mu _2\hat{{\varvec{H}}}^\mathrm{H}\hat{{\varvec{H}}}\right) }^{-1}\hat{{\varvec{H}}}^\mathrm{H}\hat{{\varvec{H}}}\biggr ]^{-1} \Biggr \} \end{aligned}$$
(23)

Note that \(\hat{{\varvec{H}}}^\mathrm{H}\hat{{\varvec{H}}}\) is a complex central Wishart matrix [13]. Assume its ordered eigenvalues are \(0 \le \lambda _1 \le \cdots \le \lambda _K\). Then, the trace in (23) is the sum of eigenvalues. Therefore, (23) can be re-expressed as

$$\begin{aligned} C_n \ge -K_n \log _2 \left( \frac{1}{K_n} \sum _{j = 1}^K\frac{1}{1+\frac{\mu _1\lambda _j}{1+\mu _2\lambda _j}}\right) \end{aligned}$$
(24)

Substituting the expressions of \(\mu _1\) and \(\mu _2\) into (24), we can further get the lower bound of \(C_n\) as

$$\begin{aligned} C_n \ge -K_n \log _2 \left( \frac{1}{K_n} \sum _{j = 1}^K\frac{1}{1+\frac{\rho ^2\lambda _j}{\kappa _1\rho ^2+\kappa _2\rho +\beta +\alpha ^2\left( L-1\right) \rho ^2\lambda _j}}\right) \end{aligned}$$
(25)

Define [3]

$$\begin{aligned} \nu _j = 1 - \frac{\log _2\lambda _j}{\log _2\rho }, \end{aligned}$$
(26)
$$\begin{aligned} {\varvec{\nu }} = \left[ \nu _1 \cdots \nu _K \right] . \end{aligned}$$

From (26), we can derive \(\lambda _j = \rho ^{1-\nu _j}\). Replace \(\lambda _j\) with \(\nu _j\) in (25), we can have

$$\begin{aligned} C_n \ge -K_n \log _2 \left( \frac{1}{K_n} \sum _{j = 1}^K\frac{1}{1+\frac{\rho ^{3-\nu _j}}{\kappa _1\rho ^2+\kappa _2\rho +\beta +\alpha ^2\left( L-1\right) \rho ^{3-\nu _j}}}\right) \end{aligned}$$
(27)

Substitute \(C_n\) into (19), after some simplification, we can have the upper bound of the outage probability \(P_\mathrm{out}^{n}\) as

$$\begin{aligned} P_\mathrm{out}^{n} \le {\mathcal {P}} \left( \sum _{j = 1}^K\frac{1}{1+\frac{\rho ^{3-\nu _j}}{\kappa _1\rho ^2+\kappa _2\rho +\beta +\alpha ^2\left( L-1\right) \rho ^{3-\nu _j}}} \ge K_n2^{-\frac{R}{K_n}}\right) \end{aligned}$$
(28)

Using the joint PDF (Probability Density Function) of the ordered random variables \(\lambda _j\) is given by [13], we can derive the joint PDF of \({\varvec{\nu }}\) as

$$\begin{aligned} {\mathcal {P}} \left( {\varvec{\nu }} \right)&= T_{M,K}^{-1}\left( \log _2\rho \right) ^K \rho ^{\left( M-K+1\right) K } \prod _{i=1}^K \rho ^{-\left( M-K+1\right) \nu _i} \nonumber \\&\quad \times \prod _{i < j}^K \left( \rho ^{1-\nu _i} - \rho ^{1-\nu _j}\right) ^2 e^{-\sum _i \rho ^{1-\nu _i}} \end{aligned}$$
(29)

where \(T_{M,K}^{-1}\) is a normalizing constant.

Then the outage probability \(P_\mathrm{out}^{n}\) is the integral of \({\mathcal {P}} \left( {\varvec{\nu }} \right)\) with the limit \({\mathcal {A}}\) given by

$$\begin{aligned} {\mathcal {A}} = \left\{ {\varvec{\nu }}:\sum _{j = 1}^K\frac{1}{1+\frac{\rho ^{3-\nu _j}}{\kappa _1\rho ^2+\kappa _2\rho +\beta +\alpha ^2\left( L-1\right) \rho ^{3-\nu _j}}} \ge K_n2^{-\frac{R}{K_n}} \right\} . \end{aligned}$$
(30)

The following simplification follows from [1].

Firstly, \(T_{M,K}^{-1}\) is a constant which has no effect on the SNR exponent.

Secondly, in the high SNR regime, the term \(e^{-\sum _i \rho ^{1-\nu _i}}\) will decay to 0 if \(\nu _i < 1\). Then, we can replace the integral limit \({\mathcal {A}}\) with \({\mathcal {A}}^{'} = {\mathcal {A}} \ \cap \ {\mathcal {R}}_1^{K+}\), where \({\mathcal {R}}_1^{K+}\) is the set of K vectors with all entries bigger than 1.

Thirdly, the term \(e^{-\sum _i \rho ^{1-\nu _i}}\) approaches 1 for \(\nu _i > 1\) and e for \(\nu _i = 1\) and thus can be neglected. Furthermore, \(\nu _i > \nu _j\) if \(i < j\), so \(\left( \rho ^{1-\nu _i} - \rho ^{1-\nu _j}\right) ^2\) is determined by \(\rho ^{1-\nu _j}\).

Finally, the upper bound of \(P_\mathrm{out}^{n}\) is written as

$$\begin{aligned} P_\mathrm{out}^{n} \mathop {\le }\limits ^{\cdot } \rho ^{\left( M-K+1\right) K}\int _{\mathcal {A^{'}}} \prod _{i=1}^{K}\rho ^{-\left( M-K+1\right) \nu _i+2\left( i-1\right) \left( 1-\nu _i\right) }d({\varvec{\nu }}) \end{aligned}$$
(31)

where \(P_{out}^{n} \mathop {\le }\limits ^{\cdot } f(\rho )\) means exponentially less than \(f(\rho )\).

As for \({\mathcal {A}}\), the LHS (left hand side) of the equality in (30) is \({\mathcal {M}}\left( \nu _j\right)\) when \(\nu _j>1\), where \({\mathcal {M}}\left( \nu _j\right)\) is the number of \(\nu _j\) which satisfies \(\nu _j > 1\). So the limit of the integral finally can be denoted as

$$\begin{aligned} {\mathcal {A^{'}}} = \left\{ {\varvec{\nu }}: {\mathcal {M}}\left( \nu _j\right) \ge K_n2^{-\frac{R}{K_n}} \right\} \end{aligned}$$
(32)

Moreover, \({\mathcal {M}}\left( \nu _j\right) \le K\). Therefore, the integration over multiple variables in (31) can be separated as

$$\begin{aligned} P_\mathrm{out}^{n} \mathop {\le }\limits ^{\cdot } \rho ^{\left( M-K+1\right) K} \cdot \rho ^{-\left( M-K+1\right) K} = \rho ^0 \end{aligned}$$
(33)

It should be noticed that (33) shows that \(P_\mathrm{out}^{n}\) is exponentially smaller than \(\rho ^0\). Then we can get the lower bound of the diversity gain \(d\left( R,L,\alpha \right)\), which proves part of the Theorem 1.

4.2 Outage Lower Bound

In this section, we shall derive the upper bound of diversity gain \(d\left( R, L, M, K_{n}, \alpha , \beta \right)\) using the lower bound of the outage probability \(P_\mathrm{out}^{n}\). From Jensen’s inequality, we have

$$\begin{aligned} C_n \le K_n \log _2 \left( \frac{1}{K_n} \sum _{k_n} \frac{1}{\omega _{{k_n}}}\right) \end{aligned}$$
(34)

Replace the \(\hat{{\varvec{H}}}^\mathrm{H}\hat{{\varvec{H}}}\) with its SVD (Singular Value Decomposition) decomposition \(\hat{{\varvec{H}}}^\mathrm{H}\hat{{\varvec{H}}} = {\varvec{U}}^\mathrm{H} \varvec{\Xi } {\varvec{U}}\), where \(\varvec{\Xi }\) is a diagonal matrix with the eigenvalues of \(\hat{{\varvec{H}}}^\mathrm{H}\hat{{\varvec{H}}}\) being its entries. Then, \({\omega _{{k_n}}}\) can be further written as

$$\begin{aligned} {\omega _{{k_n}}} = \sum _{j=1}^K\frac{{\vert U_{j,k_n} \vert }^2}{1+\frac{\mu _1\lambda _j}{1+\mu _2\lambda _j}} \end{aligned}$$
(35)

where \(U_{j,k_n}\) is the \(\left( j,k_n\right)\)-th element of \({\varvec{U}}\). Let

$$\begin{aligned} \breve{k} = \mathrm{argmin}_{k_n}\left( \omega _{k_n}\right) . \end{aligned}$$

We can further amplify the \(C_n\) in (34) as

$$\begin{aligned} C_n \le K_n \log _2 \left( \frac{1}{\omega _{\breve{k}}} \right) \end{aligned}$$
(36)

Then the lower bound of the outage probability \(P_\mathrm{out}^n\) as

$$\begin{aligned} P_\mathrm{out}^n \ge {\mathcal {P}} \left( \omega _{\breve{k}} \ge 2^{-\frac{R}{K_n}}\right) \end{aligned}$$
(37)

Assume the event \({\mathcal {B}} = \lbrace {\vert U_{lk}\vert }^2 \ge K_n\rbrace\), then (37) equals

$$\begin{aligned} P_\mathrm{out}^n \ge {\mathcal {P}} \left( \frac{1}{K_n} \sum _{l=1}^{K} \frac{1}{1+\frac{\mu _1\lambda _l}{1+\mu _2\lambda _l}} \ge 2^{-\frac{R}{K_n}} \vert {\mathcal {B}}\right) {\mathcal {P}} \left( {\mathcal {B}} \right) \end{aligned}$$
(38)

From the properties of conditional probabilities \({\mathcal {P}} \left( {\mathcal {A}}{\mathcal {B}} \right) \ge {\mathcal {P}} \left( {\mathcal {A}} \right) {\mathcal {P}} \left( {\mathcal {B}} \right)\), and from [14], \({\mathcal {P}} \left( {\mathcal {B}} \right)\) is a \({\mathcal {O}}\left( 1\right)\) nonzero term, thus we have

$$\begin{aligned} P_\mathrm{out}^n \ge {\mathcal {P}} \left( \frac{1}{K_n} \sum _{l=1}^{K} \frac{1}{1+\frac{\mu _1\lambda _l}{1+\mu _2\lambda _l}} \ge 2^{-\frac{R}{K_n}}\right) \end{aligned}$$
(39)

It is obvious that (39) is asymptotically equivalent to (28), Thus, we find that the upper bound on the outage probability exponent coincides with the previously lower bound. Therefore, the proof of Theorem 1 is completed by considering the lower bound and upper bound.

5 Outage Analysis with Pilot Contamination in Finite SNR Regime

From previous analysis results, it is obvious that in high SNR regime, the system barely has no diversity gain due to pilot contamination. In this section, we aim to get the outage analysis with pilot contamination in finite SNR regime and we start from (24). Since eigenvalues are ordered, replace all the eigenvalues with \(\lambda _1\), and (24) can be written as follows

$$\begin{aligned} C_n \ge -K_n \log _2 \left( \frac{K}{K_n} \frac{1}{1+\frac{\mu _1\lambda _1}{1+\alpha ^2\left( L-1\right) \mu _1\lambda _1}}\right) \end{aligned}$$
(40)

Substitute (40) and (16) into (19), after some simplifications we get the upper bound of the outage probability as

$$\begin{aligned} P_\mathrm{out}^n \le {\mathcal {P}} \left( \lambda _1 \le \xi \right) \end{aligned}$$
(41)

where

$$\begin{aligned} \xi = \frac{\left( K2^{\frac{R}{K_n}}-K_n\right) \left( k_1+k_2\rho ^{-1}+\kappa \rho ^{-2}\right) }{K_n-\alpha ^2\left( L-1\right) \left( K2^{\frac{R}{K_n}}-K_n\right) } \end{aligned}$$
(42)

According to [15], the CDF(Cumulative Distribution Function) of the minimal ordered eigenvalues of Central-Noncorrelated Wishart matrix is

$$\begin{aligned} {F}_{\lambda _1}\left( x\right) =1-K_\mathrm{uc}{\mathrm {det}}\left[ {\varvec{S}}\left( x\right) \right] \end{aligned}$$
(43)

where

$$\begin{aligned} \begin{aligned} K_\mathrm{uc} = {\left[ \prod _{i=1}^K\left( M-i\right) !\prod _{j=1}^K\left( K-j\right) !\right] }^{-1}\\ {\varvec{S}}_{i,j}\left( x\right) ={\mathrm {\Gamma }}\left( M-K+i+j-1,x\right) \end{aligned} \end{aligned}$$
(44)

where \({\mathrm {\Gamma }}\left( k,x\right)\) is the incomplete Gamma function, which can be denoted as

$$\begin{aligned} {\mathrm {\Gamma }}\left( k,x\right) =\int _{x}^{\infty }\mu ^{k-1}e^{-\mu }d\mu , \end{aligned}$$

and \(1 \le i,j\le K\). So (41) is equivalent to \(P_{out}^n \le {F}_{\lambda _1}\left( \xi \right)\). Substitute it into (20), we can get the lower bound of diversity gain as

$$\begin{aligned} d\left( R,L,\alpha \right) \ge \frac{\rho K_{uc}}{1-K_{uc}{\mathrm {det}}\left[ {\varvec{S}}\left( \xi \right) \right] }\frac{\partial {\mathrm {det}}\left[ {\varvec{S}}\left( \xi \right) \right] }{\partial \rho } \end{aligned}$$
(45)

Using the derivatives of matrix theorem in [12], we can get the result of derivatives in (45), namely

$$\begin{aligned} \frac{\partial {\mathrm {det}}\left[ {\varvec{S}}\left( \xi \right) \right] }{\partial \rho }={\mathrm {det}}{\left[ {\varvec{S}}\left( \xi \right) \right] }\mathrm{Tr}\left[ {\varvec{S}}^{-1}\left( \xi \right) \frac{\partial {\varvec{S}}\left( \xi \right) }{\partial \rho }\right] \end{aligned}$$
(46)

where \(\frac{\partial {\varvec{S}}\left( \xi \right) }{\partial \rho }\) is also an matrix whose i-th row and j-th column element is

$$\begin{aligned} {\varvec{J}}_{i,j}=-\xi ^{M-K+i+j-2}e^{-\xi }\frac{\partial \xi }{\partial \rho } \end{aligned}$$
(47)

and

$$\begin{aligned} \frac{\partial \xi }{\partial \rho } = \frac{\left( K_n - K2^{\frac{R}{K_n}}\right) \left( k_2\rho ^{-2}+2\kappa \rho ^{-3}\right) }{K_n-\alpha ^2\left( L-1\right) \left( K2^{\frac{R}{K_n}}-K_n\right) } \end{aligned}$$
(48)

By substituting the derivatives from (42) and (46)–(48) to (45), we get the lower bound of diversity gain in finite SNR regime. And following similar steps in Sect. 4, we can get the upper bound of diversity gain in finite SNR regime.

Fig. 1
figure 1

The outage probability versus SNR at different values of \(\alpha\)

Full size image
Fig. 2
figure 2

The outage probability versus SNR for different values of L

Full size image
Fig. 3
figure 3

The diversity performance in finite SNR regime

Full size image
Fig. 4
figure 4

The diversity in finite SNR regime vary \(\kappa\)

Full size image

6 Discussion and Numerical Results

In this section, a set of Monte-Carlo simulations are generated for the sum rate in (16). We adopt a multi-cell system which is explained before. There are two users consisting of 1 and 2 antennas respectively and one base with \(M = 32\) antennas. The pilot sequence’s power is \(\kappa = 3\) times of data’s. There’re \(L = 2\) cells in all figures except Fig. 2. It shows that the outage probability when the interference \(\alpha = 0.1\) is much smaller than that of when \(\alpha = 0.2\) and the more antennas the user have, the smaller outage probability. But in the high SNR regime, all the outage curves decrease exponentially as \(\rho ^0\). Besides what has been discussed in Fig. 1, the Fig. 2 demonstrates that the more cells the system has, the lager outage probability will occur.

Figures 3 and 4 demonstrates the diversity performance in finite SNR. Figure 3 shows that, when \(\rho \in [0, 20]\), the the system in our article achieves diversity gain and diversity gain reaches max when \(\rho = 10(\hbox {dB})\). This figure explains the decreasing range of the outage probability in Figs. 1 and 2. Figure 4 demonstrates the interferences of different pilot sequences’ power factor \(\kappa\) in diversity performance. It shows the diversity increases with \(\kappa\) at the beginning then decrease. We can get the best \(\kappa\) is \(\kappa = 3\).

These results can be interpreted as follows: due to pilot contamination, the equivalent noise in the system increases exponentially as the sum rate increases. So the outage probability will remain stable in the high SNR regime. But in finite SNR, the sum rate is exponentially bigger than the equivalent noise, so the system achieves diversity gain. And at beginning, the bigger \(\kappa\) is, the smaller pilot sequences’ power is, thus the smaller pilot contamination. But when \(\kappa\) continues to increases, the channel estimation error increases, so the diversity gain decreases.

7 Conclusion

It is discovered from this paper that the diversity gain over the multi-cell with multiple users still exists when the pilot is contaminated by the inter-cell interferences. This gain is proved theoretically for any regime of finite SNR. However, this gain decreases substantially with the increase of SNR. For the regime of finite SNR, the optimal power is obtained for the pilot to maximize the diversity gain. Therefore, it is concluded that the diversity gain is still available even if the channel state information is imperfect due to the pilot contamination. However, this diversity gain does not exist when the SNR approaches to the positive infinite.