1 Introduction

1.1 Overview

In a wireless communication system, mobile transceiver is a small physical device and has limited transmit/receive power. One can place multiple independent antennas in a Base Station (BS) as there is no restriction in its physical size and power. Therefore, in many practical situations, a very low complexity system with multiple transmit antennas is desirable. A space–time block code (STBC) can be considered as the modulation scheme for multiple transmit antennas that provide full diversity and low decoding complexity.

The general STBCs having orthogonal designs are used to extend Alamouti’s diversity scheme to more than two antennas at the transmitter end. For complex signal constellations, it is not possible to find full-rate, full-diversity (FRFD) STBCs using (square) orthogonal designs. For real constellations, such a design is possible if the number of transmit antennas is two, four, or eight. If the square orthogonal design condition is relaxed, it is possible to find rectangular FRFD code designs. On the other hand, for complex signal constellations (e.g., QAM, 8 PSK), full-rate designs exist if and only if N t  = 2, i.e., the Alamouti code is the only FRFD STBC when the signal constellation is complex. The diversity order depends on the fading distribution, modulation format, number of diversity branches, whereas the array gain in multiple input multiple output (MIMO) systems depend on the fading distribution, modulation format, number of diversity branches, and the average per-hop signal to noise ratios (SNRs).

1.2 Literature Review

The performance of generalized selection combining (GSC) in a Rayleigh fading channel is discussed for the case of more than two receive antennas. The two transmit-two receive antenna Alamouti transmission technique, first investigated in [1], is generalized to more than two receive antennas, and is the only existing FRFD code for complex signal constellations.

In [2], exact closed-form expressions for (a) cumulative distribution function of SNR with transmit antenna selection (TAS)/GSC, (b) outage probability and symbol error rate, and (c) ergodic capacity are derived for cognitive relay networks with TAS/GSC in Nakagami-m fading channels. Further, high SNR approximation of ergodic capacity for the cases of (1) proportional interference power constraint, (2) fixed interference power constraint are derived and plotted.

Two new receiver selection schemes, generalized space–time sum of squares (GSTSoS) selection diversity and generalized sum of magnitudes (GSTSoM) selection diversity schemes are proposed in [3] where the first one provides the same performance as the conventional GSC, and the second one provides slightly poorer performance, but neither requires channel state information and both have much simpler implementations. GSC selects Ls receive SNR branches out of L diversity branches and combines the Ls selected signals using maximal ratio combining (MRC). Since MRC is generally affected by channel estimation errors and with GSC, the weak signals which are prone to these errors are excluded in combining. GSC (selecting L s Rx branches out of L available ones) is proved to outperform MRC with diversity order Ls.

Proper modeling is essential in order to evaluate the behavior of transmission and reception techniques in contemporary and future wireless systems [4]. In particular, frequency and time selectivity should be properly modeled, channel state information must be based on low velocities of the mobile receiver with adaptive control of transmission power/rate parameters being considered, and coded block error probabilities or mutual information being used to gauge performance [4]. In [5], performance analysis of MIMO systems with arbitrary number of transmit antenna selection and orthogonal space–time block coding in Rayleigh fading channels for imperfect CSI is presented. For the performance analysis, the moment generating function of the system effective SNR as well as its upper and lower bounds are derived. Using the approximate BER expressions and imperfect CSI, an adaptive antenna selection scheme is developed for minimizing the BER.

The BER performance of multilevel QAM with pilot-symbol-assisted modulation channel estimation in static and Rayleigh fading channels was evaluated in [6]. The impact of noise and estimator de-correlation on the BER was examined. The influence of pilot-symbol interpolation filter was also considered. In [7], on account of analyzing the performance of joint transmit and receive antenna selection with orthogonal space–time coding, characteristic functions of the joint output SNRs are used to analyze the performance of MIMO systems with transmit and receive antenna selection.

The authors in [8] used an approach based on Moment Generating Function to study the performance of MPSK with GSC and Equal Gain Combining (EGC) in various fading channels with independent and identically distributed (i.i.d) branches. Analytical closed-form expressions for the Bit Error Rate (BER), outage probability, and channel capacity of MPSK schemes were derived. The case of imperfect channel estimation was also considered and system performance was evaluated.

In [9], the performance of OSTBCs with TAS in a Ricean fading channel is analyzed. Out of a total of \( \text{L}_{t} \) transmit antennas, the receiver selects N antennas that maximize the received SNR. A low-rate feedback channel from the receiver to the transmitter is available to convey the indices of the selected transmit antennas.

Closed-form expressions were derived in [10] for single-user adaptive capacity of GSC, taking into account the effect of imperfect channel estimation at the receiver. Besides the various adaptation policies discussed in this paper, the impact of channel estimation on capacity statistics and symbol error rate of GSC systems are also analyzed. It can be deduced that the performance of generalized selection combining scheme is superior compared to that of conventional selection combining, and almost similar to that of MRC.

An asymptotic framework is applied to derive new closed-form expressions for outage probability and symbol error rate (SER) of amplify and forward (AF) relaying in MIMO networks with two distinct protocols in [11]: (a) transmit antenna selection with receiver maximal ratio combining (TAS/MRC), and (b) transmit antenna selection with receiver selection combining (TAS/SC). The novelty in [12] lies in choosing correlation between various Rayleigh fading MIMO channels in the context of MIMO systems employing transmit antenna selection with maximal ratio combining. Closed-form expressions for BER performance of orthogonal space–time block codes (OSTBC) MIMO and infinite series expansions for BER of transmit antenna selection (TAS)/MRC are made.

Errors in the feedback channel heavily degrade the performance of closed-loop MIMO systems. Examining the effect of imperfect TAS caused by feedback link errors on the performance of TAS/OSTBC with receive antenna selection and TAS/OSTBC over slow and frequency-flat Nakagami-m fading channels is the main objective in [13]. An upper bound is also derived on the capacity which was compared to the results from Monte Carlo simulations. The BER of BPSK in Rayleigh Fading using Alamouti transmission scheme and receiver selection diversity in the presence of channel estimation error was illustrated in [14]. Two new selection schemes, space–time Sum-of-Squares selection diversity and space–time Sum-of-Magnitudes selection diversity were proposed and proven to provide similar performance as SNR selection, but with much simpler implementations. The basic concepts of STBC and Code Matrix for various OSTBC systems were discussed in [15, 16].

The contributions of our research work includes deducing closed-form expressions for ergodic capacity for MIMO-OSTBC systems with generalized selection combining. Further, comparisons in BER performance of full rate full diversity, rate ¾ full diversity, and half rate, full diversity systems are also made. The ideal modulation scheme to be chosen for a particular range of SNRs are also found out. Closed-form expressions for ergodic capacity for the cases of high SNRs and low SNRs, and upper bounds of ergodic capacity are also derived.

1.3 Organization of the Paper

This paper is organized as follows: In Sect. 2, the system model is described and GSC scheme is introduced. The model used to estimate channel gain is also described. In Sect. 3, a SER analysis for GSC reception of MQAM in Rayleigh fading channels for various OSTBCs is illustrated. The impact of channel estimation error on system performance is considered and bounds on SER performance are also derived. In Sect. 4, approximate closed-form expressions for ergodic capacity are derived. Bounds on ergodic capacity and its approximations in different SNR regions are also investigated and plotted. In Sect. 5, numerical results are discussed. Finally, conclusions and future work are presented in Sect. 6.

2 System Model

Consider a system where OSTBC is applied to NT transmit and L receive antennas. Incoming data from the data source is first encoded by the OSTBC encoder. The outputs of OSTBC encoder consists of NT parallel streams which are then transmitted from NT transmit antennas. Modulation and demodulation blocks have not been discussed in the system model due to their irrelevance in performance analysis. The diversity order is equal to NTL. For complex signal constellations, modulation techniques such as MPSK and MQAM are used and for real signal constellations, MPAM is used.

2.1 System Model and Description

At the receiver, the received signal from each receiver antenna is first processed by a Space–Time (ST) combiner. ST combiner computes the receiver decision variables. Then, the SNR of the output signals is measured. Here, L s signals having the largest SNRs are selected out of L signals and combined using a MRC combiner as shown in Fig. 1. The combined signal is used to make the final decision on the transmitted symbols, and the signal estimate is based on the phase of the MRC combiner output. Then, coherent detection of MQAM for the case of independent Rayleigh fading with equal SNR averaged over fading is considered. Furthermore, the case of imperfect channel estimation and effects of channel estimation errors on system performance are considered. The GSC scheme is applied to an OSTBC-MIMO system and SER expressions are derived for arbitrary L and L s .

Fig. 1
figure 1

A MIMO NT × L system using generalized selection combining with OSTBC encoder

2.2 Channel Estimation

The channel estimate, \( \hat{g}_{j,i} \), can be obtained by transmitting pilot symbols. The channel estimate is assumed to be a zero-mean complex Gaussian random variable correlated with the true channel gain, g j,i . The channel gain and its estimates are related as \( g_{j,i} = k\hat{g}_{j,i} + d_{j,i} \) where \( k = \frac{{R_{c} }}{{\sigma_{{\hat{g}}}^{2} }} \) and d j,i  = (x l,i  +jy l,i ) [14]. The variance of real (or imaginary) component of d j,i is \( \sigma_{d}^{2} = (1 - \rho )\sigma_{g}^{2} \) [6], where channel estimation error, ρ is the squared amplitude of the cross-correlation coefficient of channel fading, and its estimate is given by \( \rho = \frac{{E^{2} \left[ {g\hat{g}} \right]}}{{E\left[ {\left| g \right|^{2} } \right]E\left[ {\left| {\hat{g}} \right|^{2} } \right]}} = \frac{{R_{c}^{2} }}{{\sigma_{g}^{2} \sigma_{{\hat{g}}}^{2} }} = \frac{{\sigma_{{\hat{g}}}^{2} }}{{\sigma_{g}^{2} }}k^{2} \) [1]. The values of ρ range from 0 to 1, and are used to specify the channel estimation error. With perfect channel estimation, ρ = 1. When channel estimation deteriorates, ρ tends to 0.

3 SER of GSC for Various MIMO OSTBC Systems

The SER for GSC in a multipath fading environment is obtained by averaging the conditional SER over the PDF of SNR of the MRC combiner output, \( \gamma_{GSC} \), and is given as

$$ \begin{aligned} P_{e,GSC} = E_{{\gamma_{GSC} }} \left[ {\Pr (e|\gamma_{GSC} )} \right] = \int\limits_{0}^{\infty } {\Pr (e|\gamma )} f_{{\gamma_{GSC} }} \left( \gamma \right)d\gamma , \hfill \\ \hfill \\ \end{aligned} $$
(1)

where \( \Pr \left( {e|\gamma_{GSC} } \right) \) is the conditional SER, \( f_{{\gamma_{GSC} }} (\gamma ) \) is the PDF of \( \gamma_{GSC} \), and γ is a particular value of random variable, \( \gamma_{GSC} . \)

3.1 SER of GSC for 8 × L Half-Rate Full-Diversity OSTBC System

The performance of GSC OSTBC system can be improved by increasing the number of transmit antennas to eight. This will increase the diversity order to 8Nr. Consider a system where OSTBC is applied to eight transmit antennas and L receive antennas. The code matrix for 8 × L HRFD OSTBC system with complex signal constellations is expressed as [16]

$$ X = \left[ {\begin{array}{*{20}c} {s_{1} } & {s_{2} } & {s_{3} } & {s_{4} } & {s_{5} } & {s_{6} } & {s_{7} } & {s_{8} } \\ { - s_{2} } & {s_{1} } & {s_{4} } & { - s_{3} } & {s_{6} } & { - s_{5} } & { - s_{8} } & {s_{7} } \\ { - s_{3} } & { - s_{4} } & {s_{1} } & {s_{2} } & {s_{7} } & {s_{8} } & { - s_{5} } & { - s_{6} } \\ { - s_{4} } & {s_{3} } & { - s_{2} } & {s_{1} } & {s_{8} } & { - s_{7} } & {s_{6} } & { - s_{5} } \\ { - s_{5} } & { - s_{6} } & { - s_{7} } & { - s_{8} } & {s_{1} } & {s_{2} } & {s_{3} } & {s_{4} } \\ { - s_{6} } & {s_{5} } & { - s_{8} } & {s_{7} } & { - s_{2} } & {s_{1} } & { - s_{4} } & {s_{3} } \\ { - s_{7} } & {s_{8} } & {s_{5} } & { - s_{6} } & { - s_{3} } & {s_{4} } & {s_{1} } & { - s_{2} } \\ { - s_{8} } & { - s_{7} } & {s_{6} } & {s_{5} } & { - s_{4} } & { - s_{3} } & {s_{2} } & {s_{1} } \\ {s_{1}^{*} } & {s_{2}^{*} } & {s_{3}^{*} } & {s_{4}^{*} } & {s_{5}^{*} } & {s_{6}^{*} } & {s_{7}^{*} } & {s_{8}^{*} } \\ { - s_{2}^{*} } & {s_{1}^{*} } & {s_{4}^{*} } & { - s_{3}^{*} } & {s_{6}^{*} } & { - s_{5}^{*} } & { - s_{8}^{*} } & {s_{7}^{*} } \\ { - s_{3}^{*} } & { - s_{4}^{*} } & {s_{1}^{*} } & {s_{2}^{*} } & {s_{7}^{*} } & {s_{8}^{*} } & { - s_{5}^{*} } & { - s_{6}^{*} } \\ { - s_{4}^{*} } & {s_{3}^{*} } & { - s_{2}^{*} } & {s_{1}^{*} } & {s_{8}^{*} } & { - s_{7}^{*} } & {s_{6}^{*} } & { - s_{5}^{*} } \\ { - s_{5}^{*} } & { - s_{6}^{*} } & { - s_{7}^{*} } & { - s_{8}^{*} } & {s_{1}^{*} } & {s_{2}^{*} } & {s_{3}^{*} } & {s_{4}^{*} } \\ { - s_{6}^{*} } & {s_{5}^{*} } & { - s_{8}^{*} } & {s_{7}^{*} } & { - s_{2}^{*} } & {s_{1}^{*} } & { - s_{4}^{*} } & {s_{3}^{*} } \\ { - s_{7}^{*} } & {s_{8}^{*} } & {s_{5}^{*} } & { - s_{6}^{*} } & { - s_{3}^{*} } & {s_{4}^{*} } & {s_{1}^{*} } & { - s_{2}^{*} } \\ { - s_{8}^{*} } & { - s_{7}^{*} } & {s_{6}^{*} } & {s_{5}^{*} } & { - s_{4}^{*} } & { - s_{3}^{*} } & {s_{2}^{*} } & {s_{1}^{*} } \\ \end{array} } \right]_{16\times8} . $$

This code matrix represents a half-rate code as eight symbols are transmitted in sixteen time slots. It is a full diversity code as the difference matrix of two transmitted codewords has rank one. Here, complex signal constellation modulation techniques (MPSK, MQAM) are used. At the receiver, Maximum Likelihood (ML) detection is used. The decoding complexity is minimum due to the orthogonality of the STBC used.

The received signal is processed by a Space–Time (ST) combiner which computes the receiver decision variable as

$$ \begin{aligned} y_{1,i} & = r_{1,i} \hat{g}_{1,i}^{*} + r_{2,i} \hat{g}_{2,i}^{*} + r_{3,i} \hat{g}_{3,i}^{*} + r_{4,i} \hat{g}_{4,i}^{*} + r_{5,i} \hat{g}_{5,i}^{*} + r_{6,i} \hat{g}_{6,i}^{*} + r_{7,i} \hat{g}_{7,i}^{*} + r_{8,i} \hat{g}_{8,i}^{*} \\ & \quad + r_{9,i} \hat{g}_{1,i}^{*} + r_{10,i} \hat{g}_{2,i}^{*} + r_{11,i} \hat{g}_{3,i}^{*} + r_{12,i} \hat{g}_{4,i}^{*} + r_{13,i} \hat{g}_{5,i}^{*} + r_{14,i} \hat{g}_{6,i}^{*} + r_{15,i} \hat{g}_{7,i}^{*} + r_{16,i} \hat{g}_{8,i}^{*} . \\ \end{aligned} $$
(2)

Substituting received signal and channel estimate values into (2), and normalizing the expression by dividing both sides of (2) by \( 8k\sigma_{{\hat{g}}}^{2} \), the obtained expression is termed as. Here,

$$ \begin{array}{l} {\text{Mean }}\left( {y_{1,i}^{{\prime }} } \right) = E\left[ {y'_{1,i} } \right] = \frac{{\left| {\hat{g}_{1,i} } \right|^{2} + \left| {\hat{g}_{2,i} } \right|^{2} + \left| {\hat{g}_{3,i} } \right|^{2} + \left| {\hat{g}_{4,i} } \right|^{2} + \left| {\hat{g}_{5,i} } \right|^{2} + \left| {\hat{g}_{6,i} } \right|^{2} + \left| {\hat{g}_{7,i} } \right|^{2} + \left| {\hat{g}_{8,i} } \right|^{2} }}{{4\sigma_{{\hat{g}}}^{2} }} = a_{i} , \hfill \\ {\text{Variance}}\left( {y_{1,i}^{{\prime }} } \right) = \sigma^{2} = E\left[ {y_{1,i}^{'2} } \right] - E^{2} \left[ {y_{1,i}^{'} } \right] = \frac{{\left( {16\sigma_{d}^{2} + 2\sigma_{n}^{2} } \right)}}{{16\rho \sigma_{{g^{2} }}^{2} }}a_{i} . \hfill \\ \end{array} $$

The variances of the real (or imaginary) components of g j,i and n j,i are denoted as \( \sigma_{g}^{2} \) and \( \sigma_{n}^{2} \), respectively. The average SNR per symbol of the received signal is defined as \( \bar{\gamma } = 8\sigma_{g}^{2} /\sigma_{n}^{2} \). The variance may be simplified to \( \frac{{\left( {1 - \rho } \right)\bar{\gamma }' + 1}}{{\rho \bar{\gamma }'}}a_{i} . \) Defining the effective SNR as \( \bar{\gamma }_{c} = \frac{{\rho \bar{\gamma }}}{{\left( {1 - \rho } \right)\bar{\gamma } + 1}} \). Then, the SNR of \( y_{1,i}^{{\prime }} \) is given by \( \frac{{a_{i}^{2} }}{{2\left( {\sigma^{ 2} } \right)}} \), where \( \sigma^{2} \) is the variance of the real (or imaginary) components of \( y_{1,i}^{{\prime }} \). Substituting this variance in the SNR expression, the SNR of \( y_{1,i}^{{\prime }} \) reduces to \( \frac{{\bar{\gamma }_{c} a_{i} }}{2}. \) At the MRC combiner, the selected L s ST combiner outputs are combined and \( \gamma_{GSC} \) is defined as sum of the SNR of \( y_{1,i}^{{\prime }} \), i.e.,

$$ \gamma_{GSC} = \frac{{\bar{\gamma }_{C} }}{2}\sum\limits_{i = 1}^{{L_{s} }} {a_{i} } . $$
(3)

As \( \hat{g}_{1,i} \) and \( \hat{g}_{2,i} \) are independent, zero-mean complex Gaussian random variables, a i has a Chi square distribution (central) with four degrees of freedom and its PDF is given by [3]

$$ f\left( {a_{i} } \right) = a_{i} \exp ( - a_{i} ),\quad \forall a_{i} > 0. $$
(4)

The joint PDF of the ordered set {a 1, a 2, …, a L } in GSC is expressed using order statistic as [17]

$$ f_{{a_{\left[ L \right]} }} = \left\{ {\begin{array}{ll} {L!f(a_{1} )f\left( {a_{2} } \right) \cdots f(a_{L} ),} & \quad {a_{1} > a_{2} \cdots > a_{L} } \\ {0,} & \quad {{\text{otherwise}} .} \\ \end{array} } \right. $$
(5)

Substituting (4) into (5), and combining the product of exponentials in the form of exponential of their sum, we have

$$ f_{{a_{\left[ L \right]} }} = \left\{ {\begin{array}{ll} {L!\prod\nolimits_{i = 1}^{L} {a_{i} \exp \left( { - \sum\nolimits_{i = 1}^{L} {a_{i} } } \right)} ,} & \quad {a_{1} > a_{2} \cdots > a_{L} } \\ {0,} & \quad {{\text{otherwise}} .} \\ \end{array} } \right. $$
(6)

Using the joint PDF of \( a_{[L]} , \) the SER expression is modified as

$$ P_{e,GSC} = \int_{0}^{\infty } {\int_{0}^{{a_{1} }} {\int_{0}^{{a_{2} }} { \cdots \int_{0}^{{a_{L - 1} }} {\Pr \left[ {e|\gamma_{GSC} } \right]} } } } f_{{a_{[L]} }} (a_{1} ,a_{1} , \ldots ,a_{L} )da_{L} a_{L - 1} \ldots a_{1} . $$
(7)

Let \( a_{\left[ L \right]} \) be an ensemble with \( a_{1} > a_{2} \cdots > a_{L} , \) which is a column vector represented as [17]

$$ a_{\left[ L \right]} = \left[ {\begin{array}{*{20}c} {a_{1} } & {a_{2} } & \cdots & {a_{L} } \\ \end{array} } \right]^{T} . $$

Using virtual branch relation, \( a_{[L]} = T_{VB} v_{[L]} . \) Here, \( T_{VB} \) is the upper triangular matrix represented as [18]

$$ T_{VB} = \left[ {\begin{array}{*{20}c} 1 & {\frac{1}{2}} & \cdots & {\frac{1}{L}} \\ 0 & {\frac{1}{2}} & \cdots & {\frac{1}{L}} \\ 0 & 0 & \ddots & \vdots \\ 0 & 0 & 0 & {\frac{1}{L}} \\ \end{array} } \right]. $$

Transforming the expression in (7), the SER expression can be modified as

$$ P_{e,GSC} = \int_{0}^{\infty } {\int_{0}^{\infty } {\int_{0}^{\infty } \cdots } } \int_{0}^{\infty } {\Pr [e|\gamma_{GSC} ]} f_{{v_{\left[ L \right]} }} \left( {v_{1} ,v_{2} , \ldots ,v_{L} } \right)dv_{L} dv_{L - 1} \ldots dv_{1} , $$
(8)

where v [L] = {v 1, v 2, …, v L } is a new set of independent variables. Using the distribution theory for transformation of random vectors,

$$ f_{{v_{\left[ L \right]} }} \left( {v_{1} ,v_{2} , \ldots ,v_{L} } \right) = Jf_{{a_{\left[ L \right]} }} \left( {a_{1} ,a_{2} , \ldots ,a_{L} } \right)|_{{a_{{\left[ L \right] = T_{VB} v_{\left[ L \right]} }} }} , $$

where J is the Jacobian of the virtual branch transformation defined as \( J = \left| {T_{VB} } \right| = \prod\nolimits_{n = 1}^{L} {\frac{1}{n}} = \frac{1}{L!}. \)

Here, the transformation matrix, \( T_{VB} \), is a upper triangular matrix. Hence, its determinant is equivalent to the product of the main diagonal elements. Substituting the value of J in the joint PDF of v [L], we have

$$ f_{{v_{\left[ L \right]} }} \left( {v_{1} ,v_{2} , \ldots ,v_{L} } \right) = \prod\limits_{i = 1}^{L} {a_{i} \exp \left( { - \sum\limits_{i = 1}^{L} {a_{i} } } \right)} . $$
(9)

For coherent detection of MQAM modulation, conditional SER is expressed as [18]

$$ \Pr \left( {e_{MQAM} |\gamma_{GSC} } \right) = \frac{q}{\pi }\int_{0}^{{{\pi \mathord{\left/ {\vphantom {\pi 2}} \right. \kern-0pt} 2}}} {\exp \left( { - \frac{{c_{MQAM} }}{{\sin^{2} \theta }}\gamma_{GSC} } \right)d\phi } - \frac{{q^{2} }}{4\pi }\int_{0}^{{{\pi \mathord{\left/ {\vphantom {\pi 4}} \right. \kern-0pt} 4}}} {\exp \left( { - \frac{{c_{MQAM} }}{{\sin^{2} \theta }}\gamma_{GSC} } \right)d\phi } , $$
(10)

where \( q = 4\left( {1 - \left( {\frac{1}{\sqrt M }} \right)} \right) \), and \( c_{MQAM} = \frac{3}{2(M - 1)} \) Substituting (9) and (10) in (8), SER expression for L = 3, Ls = 2 case with QAM modulation is given as

$$ P_{e,GSC} = \frac{0.1781}{\pi }\left[ {\int\limits_{0}^{{{\pi \mathord{\left/ {\vphantom {\pi 2}} \right. \kern-0pt} 2}}} {\frac{2}{{\left( {\frac{{0.25\bar{\gamma }'}}{{\sin^{2} \phi }} + 1} \right)^{2} \left( {\frac{{0.5\bar{\gamma }'}}{{3\sin^{2} \phi }} + 1} \right)}}} d\phi - \int\limits_{0}^{{{\pi \mathord{\left/ {\vphantom {\pi 4}} \right. \kern-0pt} 4}}} {\frac{1}{{\left( {\frac{{0.25\bar{\gamma }'}}{{\sin^{2} \phi }} + 1} \right)^{2} \left( {\frac{{0.5\bar{\gamma }'}}{{3\sin^{2} \phi }} + 1} \right)}}} d\phi } \right]. $$

Similarly, SER expression can be derived for higher order modulation schemes (M = 8, 16, 32) and multiple receive antennas (L = 4, 5, 6).

3.2 SER of GSC for 4 × L Full-Rate Full-Diversity (FRFD) OSTBC system

The code matrix for a 4 × L FRFD OSTBC system with real signal constellations is given as [16]

$$ S = \left[ {\begin{array}{*{20}c} {s_{1} } & {s_{2} } & {s_{3} } & {s_{4} } \\ { - s_{2} } & {s_{1} } & { - s_{4} } & {s_{3} } \\ { - s_{3} } & {s_{4} } & {s_{1} } & { - s_{2} } \\ { - s_{4} } & { - s_{3} } & {s_{2} } & {s_{1} } \\ \end{array} } \right]_{4\times4} . $$

Signals s1, s2, s3, and s4 corresponding to four information symbols are transmitted simultaneously during four consecutive symbol intervals. The corresponding received signals in these four intervals in the ith branch can be expressed as

$$ \left. {\begin{array}{*{20}c} {r_{1,i} = h_{1,i} s_{1} + h_{2,i} s_{2} + h_{3,i} s_{3} + h_{4,i} s_{4} + n_{1,i} } \\ {r_{2,i} = - h_{1,i} s_{2} + h_{2,i} s_{1} - h_{3,i} s_{4} + h_{4,i} s_{3} + n_{2,i} } \\ {r_{3,i} = - h_{1,i} s_{3} + h_{2,i} s_{4} + h_{3,i} s_{1} - h_{4,i} s_{2} + n_{3,i} } \\ {r_{4,i} = - h_{1,i} s_{4} - h_{2,i} s_{3} + h_{3,i} s_{2} + h_{4,i} s_{1} + n_{4,i} } \\ \end{array} } \right\}, $$
(11)

where \( h_{j,i} \), j = 1, 2, 3, 4 and i = 1, 2, … , L since there are 4 transmit and L receive antennas. The average SNR per symbol of received signal is defined as \( {\bar{\gamma }}_{ 0} = 4\upsigma_{\text{h}}^{ 2} /\upsigma_{\text{n}}^{ 2} \), where variances of hj,i and nj,i are denoted as \( \sigma_{h}^{2} \) and \( \sigma_{n}^{2} \) respectively. In matrix form, (11) can be written as

$$ \left[ {\begin{array}{*{20}c} {r_{1,i} } \\ {r_{2,i} } \\ {r_{3,i} } \\ {r_{4,i} } \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {h_{1,i} } & {h_{2,i} } & {h_{3,i} } & {h_{4,i} } \\ {h_{2,i} } & { - h_{1,i} } & {h_{4,i} } & { - h_{3,i} } \\ {h_{3,i} } & { - h_{4,i} } & { - h_{1,i} } & {h_{2,i} } \\ {h_{4,i} } & {h_{3,i} } & { - h_{2,i} } & { - h_{1,i} } \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {s_{1} } \\ {s_{2} } \\ {s_{3} } \\ {s_{4} } \\ \end{array} } \right] + \left[ {\begin{array}{*{20}c} {n_{1,i} } \\ {n_{2,i} } \\ {n_{3,i} } \\ {n_{4,i} } \\ \end{array} } \right]. $$

Let us assume H′ = XH H, where \( {\mathbf{X}} = \left( {\sum\nolimits_{j = 1}^{4} {\left| {h_{j,i} } \right|^{2} } } \right)\left( {{\mathbf{H}}^{{\mathbf{H}}} {\mathbf{H}}} \right)^{ - 1} . \) Since H matrix is known, H H H is represented as

Substituting H H H in X, X can be computed as a 4 × 4 Identity matrix. For decoding of transmitted symbols, s′ = H′r, and is given by

$$ \text{s}{^\prime} = \left( {\sum\limits_{j = 1}^{4} {\left| {h_{j,i} } \right|^{2} } } \right)\left[ {\begin{array}{*{20}c} {s_{1} } \\ {s_{2} } \\ {s_{3} } \\ {s_{4} } \\ \end{array} } \right] + \left[ {\begin{array}{*{20}c} {h_{1,i} n_{1,i} + h_{2,i} n_{2,i} + h_{3,i} n_{3,i} + h_{4,i} n_{4,i} } \\ {h_{2,i} n_{1,i} - h_{1,i} n_{2,i} - h_{4,i} n_{3,i} + h_{3,i} n_{4,i} } \\ {h_{3,i} n_{1,i} + h_{4,i} n_{2,i} - h_{1,i} n_{3,i} + h_{2,i} n_{4,i} } \\ {h_{4,i} n_{1,i} - h_{3,i} n_{2,i} + h_{2,i} n_{3,i} - h_{1,i} n_{4,i} } \\ \end{array} } \right]. $$

Variance of \( s' \) can be computed as, var(\( s' \)) \( = \sigma^{2} \left( {\left| {h_{1,i} } \right|^{2} + \left| {h_{2,i} } \right|^{2} + \left| {h_{3,i} } \right|^{2} + \left| {h_{4,i} } \right|^{2} } \right), \) where σ2 is the AWGN variance. The expression of Average SNR is given by \( {\bar{\gamma }}_{\text{i}} = {\bar{\gamma }}_{ 0} a_{i} , \) where \( a_{i} \) is defined as \( a_{i} = \left( {\left| {h_{1,i} } \right|^{2} + \left| {h_{2,i} } \right|^{2} + \left| {h_{3,i} } \right|^{2} + \left| {h_{4,i} } \right|^{2} } \right) \). At the MRC combiner, the selected Ls ST combiner outputs are combined and the sum of the SNR of selected branches is represented as

$$ \upgamma_{\text{GSC}} = \sum\limits_{i = 1}^{{L_{s} }} {{\bar{\gamma }}_{i} } = {\bar{\gamma }}_{0} \sum\limits_{i = 1}^{{L_{s} }} {a_{i} } . $$
(12)

The PDF of ai has a Chi square distribution. The PDF of SNR can be written in terms of the PDF of ai as \( f\left( {\gamma_{i} } \right) = \frac{{f\left( {a_{i} } \right)}}{{\bar{\gamma }_{0} }}. \) Using virtual branch relation \( \gamma_{[L]} = T_{VB} v_{[L]} \text{,} \) the ordered \( \gamma_{\left[ L \right]} \) are transformed into new set of independent variables \( v_{\left[ L \right]} \). The joint PDF of all SNR branches in GSC can be expressed as

$$ f_{{\gamma_{\left[ L \right]} }} = \frac{L!}{{\left( {\bar{\gamma }_{0} } \right)^{L} }}\prod\nolimits_{i = 1}^{L} {a_{i} \exp \left( { - \sum\nolimits_{i = 1}^{L} {a_{i} } } \right)} . $$
(13)

For coherent detection of MPAM, conditional SER is expressed as [18]

$$ \Pr (e|\gamma_{GSC} ) = R\int\limits_{0}^{\pi /2} {\exp ( - K(\phi )\gamma_{GSC} )d\phi ,} $$
(14)

where \( K\left( \varphi \right) = \frac{3}{{\left( {M^{2} - 1} \right)\sin^{2} \phi }} \) and \( R = 2\left( {M - 1} \right)/M\pi \). Substituting (13) and (14) into (8), the SER expression for GSC with MPAM becomes

$$ P_{e,GSC} = R\int\limits_{0}^{\infty } {\int\limits_{0}^{\pi /2} {\exp \left[ { - K(\phi )\bar{\gamma }_{0} \sum\limits_{i = 1}^{{L_{s} }} {a_{i} } } \right]} } f_{{v_{[L]} }} (v_{[L]} )d\phi dv_{[L]} . $$

The joint PDF of independent random variable v[L] for L = 2 is given as

$$ P_{e,GSC} = \frac{1.5}{{\pi \left( {{\bar{\gamma }}_{ 0} } \right)^{2} }}\int\limits_{0}^{\infty } {\int\limits_{0}^{\infty } {\int\limits_{0}^{{{{3\pi } \mathord{\left/ {\vphantom {{3\pi } 4}} \right. \kern-0pt} 4}}} {\exp \left[ { - \frac{{\left( {v_{1} + v_{2} } \right)}}{{5\sin^{2} \phi }}} \right]} } } \left( {v_{1} + \frac{{v_{2} }}{2}} \right)\left( {\frac{{v_{2} }}{2}} \right)\exp \left( {\frac{{ - v{}_{1} - v{}_{2}}}{{{\bar{\gamma }}_{ 0} }}} \right)d\phi dv_{2} dv_{1} . $$

Substituting the joint PDF in the SER expression, the SER for L = 2, Ls = 2 case with QAM modulation is given by

$$ f_{{v_{[L]} }} (v_{1} ,v_{1} , \cdots ,v_{L} ) = \left( {v_{1} + \frac{{v_{2} }}{2}} \right)\left( {\frac{{v_{2} }}{2}} \right)\exp \left( { - {{\left( {v_{1} + v_{2} } \right)} \mathord{\left/ {\vphantom {{\left( {v_{1} + v_{2} } \right)} {\bar{\gamma }_{0} }}} \right. \kern-0pt} {\bar{\gamma }_{0} }}} \right). $$
(15)

The integral in (15) can be further simplified as

$$ P_{e,GSC} = \frac{7}{{24\left( {{\bar{\gamma }}_{ 0} } \right)^{2} \pi }}\int\limits_{0}^{{{\pi \mathord{\left/ {\vphantom {\pi 2}} \right. \kern-0pt} 2}}} {\frac{1}{{\left( {\frac{1}{{5\sin^{2} \phi }} + \frac{1}{{{\bar{\gamma }}_{ 0} }}} \right)^{2} }}} d\phi . $$

Similarly, SER expression can be derived for higher order modulation schemes (M = 8, 16, 32) and multiple receive antennas (L = 3, 4, 5).

3.3 SER of GSC for 8 × L Full-Rate Full-Diversity (FRFD) OSTBC System

For real signal constellations and Nt = 8, the code matrix is expressed as [16]

$$ X = \left[ {\begin{array}{*{20}c} {x_{1} } & {x_{2} } & {x_{3} } & {x_{4} } & {x_{5} } & {x_{6} } & {x_{7} } & {x_{8} } \\ { - x_{2} } & {x_{1} } & { - x_{4} } & {x_{3} } & { - x_{6} } & {x_{5} } & {x_{8} } & { - x_{7} } \\ { - x_{3} } & {x_{4} } & {x_{1} } & { - x_{2} } & {x_{7} } & {x_{8} } & { - x_{5} } & { - x_{6} } \\ { - x_{4} } & { - x_{3} } & {x_{2} } & {x_{1} } & {x_{8} } & { - x_{7} } & {x_{6} } & { - x_{5} } \\ { - x_{5} } & {x_{6} } & { - x_{7} } & { - x_{8} } & {x_{1} } & { - x_{2} } & {x_{3} } & {x_{4} } \\ { - x_{6} } & { - x_{5} } & { - x_{8} } & {x_{7} } & {x_{2} } & {x_{1} } & { - x_{4} } & {x_{3} } \\ { - x_{7} } & { - x_{8} } & {x_{5} } & { - x_{6} } & { - x_{3} } & {x_{4} } & {x_{1} } & {x_{2} } \\ { - x_{8} } & {x_{7} } & {x_{6} } & {x_{5} } & { - x_{4} } & { - x_{3} } & { - x_{2} } & {x_{1} } \\ \end{array} } \right]_{8\times8} . $$

Here, the (i, j)th element of matrix represents transmission from antenna j during the ith time slot. The corresponding received signals in the ith branch can be expressed in matrix form as:

$$ \left[ {\begin{array}{*{20}c} {r_{1,i} } \\ {r_{2,i} } \\ {r_{3,i} } \\ {r_{4,i} } \\ {r_{5,i} } \\ {r_{6,i} } \\ {r_{7,i} } \\ {r_{8,i} } \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {h_{1,i} } & {h_{2,i} } & {h_{3,i} } & {h_{4,i} } & {h_{5,i} } & {h_{6,i} } & {h_{7,i} } & {h_{8,i} } \\ {h_{2,i} } & { - h_{1,i} } & {h_{4,i} } & { - h_{3,i} } & {h_{6,i} } & { - h_{5,i} } & { - h_{8,i} } & {h_{7,i} } \\ {h_{3,i} } & { - h_{4,i} } & { - h_{1,i} } & {h_{2,i} } & { - h_{7,i} } & { - h_{8,i} } & {h_{5,i} } & {h_{6,i} } \\ {h_{4,i} } & {h_{3,i} } & { - h_{2,i} } & { - h_{1,i} } & { - h_{8,i} } & {h_{7,i} } & { - h_{6,i} } & {h_{5,i} } \\ {h_{5,i} } & { - h_{6,i} } & {h_{7,i} } & {h_{8,i} } & { - h_{1,i} } & {h_{2,i} } & { - h_{3,i} } & { - h_{4,i} } \\ {h_{6,i} } & {h_{5,i} } & {h_{8,i} } & { - h_{7,i} } & { - h_{2,i} } & { - h_{1,i} } & {h_{4,i} } & { - h_{3,i} } \\ {h_{7,i} } & {h_{8,i} } & { - h_{5,i} } & {h_{6,i} } & {h_{3,i} } & { - h_{4,i} } & { - h_{1,i} } & { - h_{2,i} } \\ {h_{8,i} } & { - h_{7,i} } & { - h_{6,i} } & { - h_{5,i} } & {h_{4,i} } & {h_{3,i} } & {h_{2,i} } & { - h_{1,i} } \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {s_{1} } \\ {s_{2} } \\ {s_{3} } \\ {s_{4} } \\ {s_{5} } \\ {s_{6} } \\ {s_{7} } \\ {s_{8} } \\ \end{array} } \right] + \left[ {\begin{array}{*{20}c} {n_{1,i} } \\ {n_{2,i} } \\ {n_{3,i} } \\ {n_{4,i} } \\ {n_{5,i} } \\ {n_{6,i} } \\ {n_{7,i} } \\ {n_{8,i} } \\ \end{array} } \right], $$

where \( g_{j,i} \), j = 1, 2, … , 8, i = 1, 2, … , L, as there are 8 transmit and L receive antennas. The average SNR per symbol of received signal is defined as \( \bar{\gamma }_{0} ' = 8\sigma_{h}^{2} /\sigma_{n}^{2} , \) where the variances of hj,i and nj,i are denoted as \( \sigma_{g}^{2} \) and \( \sigma_{n}^{2} \) respectively. Let us assume H′ = XH H, where \( {\mathbf{X}} = \left( {\sum\nolimits_{j = 1}^{4} {\left| {h_{j,i} } \right|^{2} } } \right)\left( {{\mathbf{H}}^{{\mathbf{H}}} {\mathbf{H}}} \right)^{ - 1} . \) Now, H H H is represented as

For decoding, we can compute s′ = H′r. Now, Variance (s′) = \( \sigma^{2} \left( {\left| {h_{1,j} } \right|^{2} + \left| {h_{2,j} } \right|^{2} + \left| {h_{3,j} } \right|^{2} + \left| {h_{4,j} } \right|^{2} + \left| {h_{5,j} } \right|^{2} + \left| {h_{6,j} } \right|^{2} + \left| {h_{7,j} } \right|^{2} + \left| {h_{8,j} } \right|^{2} } \right), \) where σ2 is the AWGN variance. The expression of Average SNR is \( {\bar{\gamma }}_{\text{i}} = {\bar{\gamma }}_{ 0}^{ '} a_{i} , \) where \( a_{i} = \sigma^{2} \left( {\left| {h_{1,j} } \right|^{2} + \left| {h_{2,j} } \right|^{2} + \left| {h_{3,j} } \right|^{2} + \left| {h_{4,j} } \right|^{2} + \left| {h_{5,j} } \right|^{2} + \left| {h_{6,j} } \right|^{2} + \left| {h_{7,j} } \right|^{2} + \left| {h_{8,j} } \right|^{2} } \right). \) At the MRC combiner, the selected Ls ST combiner outputs are combined, and \( \upgamma_{\text{GSC}} \) is the sum of SNRs of the selected branches, i.e.,

$$ \upgamma_{\text{GSC}} = \sum\limits_{i = 1}^{{L_{s} }} {{\bar{\gamma }}_{\text{i}} } = {\bar{\gamma }}_{0}^{{\prime }} \sum\limits_{i = 1}^{{L_{s} }} {a_{i} } . $$

Using similar analysis as the last case, SER expression for L = 2, Ls = 1 case with QAM modulation is obtained as

$$ P_{e,GSC} = \frac{7}{{24\left( {{\bar{\gamma }}_{ 0}^{ '} } \right)^{2} \pi }}\int\limits_{0}^{{{\pi \mathord{\left/ {\vphantom {\pi 2}} \right. \kern-0pt} 2}}} {\frac{1}{{\left( {\frac{1}{{5\sin^{2} \phi }} + \frac{1}{{{\bar{\gamma }}_{ 0}^{ '} }}} \right)\left( {\frac{1}{{10\sin^{2} \phi }} + \frac{1}{{{\bar{\gamma }}_{ 0}^{ '} }}} \right)}}} d\phi . $$
(16)

3.4 SER of GSC for 3 × L 3/4 Rate Full-Diversity OSTBC System

For complex signal constellation and Nt = 3, the code matrix is expressed as [16]

$$ X = \left[ {\begin{array}{*{20}c} {x_{1} } & {x_{2} } & {x_{3} } \\ { - x_{2}^{*} } & {x_{1}^{*} } & 0 \\ {x_{3}^{*} } & 0 & { - x_{1}^{*} } \\ 0 & {x_{3}^{*} } & { - x_{2}^{*} } \\ \end{array} } \right]. $$

The corresponding received signals in the ith branch can be expressed as

$$ \begin{aligned} r_{1,i} = h_{1,i} x_{1} + h_{2,i} x_{2} + h_{3,i} x_{3} + n_{1,i} , \hfill \\ r_{2,i} = - h_{1,i} x_{2}^{*} + h_{2,i} x_{1}^{*} + n_{2,i} , \hfill \\ r_{3,i} = h_{1,i} x_{3}^{*} - h_{3,i} x_{1}^{*} + n_{3,i} , \hfill \\ r_{4,i} = h_{2,i} x_{3}^{*} - h_{3,i} x_{2}^{*} + n_{4,i} . \hfill \\ \end{aligned} $$

The corresponding received signals in the ith branch can be expressed in matrix form as

$$ \left[ {\begin{array}{*{20}c} {r_{1,i} } \\ {r_{2,i}^{*} } \\ {r_{3,i}^{*} } \\ {r_{4,i}^{*} } \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {h_{1,i} } & {h_{2,i} } & {h_{3,i} } \\ {h_{2,i}^{*} } & { - h_{2,i}^{*} } & 0 \\ { - h_{3,i}^{*} } & 0 & {h_{1,i}^{*} } \\ 0 & { - h_{3,i}^{*} } & {h_{2,i}^{*} } \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {x_{1} } \\ {x_{2} } \\ {x_{3} } \\ \end{array} } \right] + \left[ {\begin{array}{*{20}c} {n_{1,i} } \\ {n_{2,i}^{*} } \\ {n_{3,i}^{*} } \\ {n_{4,i}^{*} } \\ \end{array} } \right], $$

where h i,j , j = 1, 2, 3 and i = 1, 2, … , L as there are 8 transmit and L receive antennas. Average SNR per symbol of the received signal is defined as \( \bar{\gamma }_{0}^{''} = {{3\sigma_{h}^{2} } \mathord{\left/ {\vphantom {{3\sigma_{h}^{2} } {\sigma_{n}^{2} }}} \right. \kern-0pt} {\sigma_{n}^{2} }} \), where variances of hj,i and nj,i are denoted \( \sigma_{h}^{2} \) as \( \sigma_{n}^{2} \) and respectively. Let us assume H′ = XH H, where \( {\mathbf{X}} = \left( {\sum\limits_{j = 1}^{4} {\left| {h_{j,i} } \right|^{2} } } \right)\left( {{\mathbf{H}}^{{\mathbf{H}}} {\mathbf{H}}} \right)^{ - 1} . \) Now, H H H is represented as

$$ {\mathbf{H}}^{{\mathbf{H}}} {\mathbf{H}} = \left[ {\begin{array}{*{20}c} {\sum\nolimits_{j = 1}^{3} {\left| {h_{j,i}^{2} } \right|} } & 0 & 0 \\ 0 & {\sum\limits_{j = 1}^{3} {\left| {h_{j,i}^{2} } \right|} } & 0 \\ 0 & 0 & {\sum\limits_{j = 1}^{3} {\left| {h_{j,i}^{2} } \right|} } \\ \end{array} } \right]. $$

For decoding, we can compute s′ = H′r. The variance (s′) \( = \sigma^{2} \left( {\left| {h_{1,i} } \right|^{2} + \left| {h_{2,i} } \right|^{2} + \left| {h_{3,i} } \right|^{2} } \right) \). The expression of Average SNR is \( {\bar{\gamma }}_{i} = {\bar{\gamma }}_{ 0}^{ '} a_{i} , \) where \( a_{i} = \left( {\left| {h_{1,i} } \right|^{2} + \left| {h_{2,i} } \right|^{2} + \left| {h_{3,i} } \right|^{2} } \right). \) At the MRC combiner, the selected Ls ST combiner outputs are combined, and \( \upgamma_{\text{GSC}} \) is the sum of SNRs of the selected branches, i.e.,

$$ \gamma_{\text{GSC}} = \sum\limits_{i = 1}^{{L_{s} }} {{\bar{\gamma }}_{i} } = {\bar{\gamma }}_{0}^{{\prime \prime }} \sum\limits_{i = 1}^{{L_{s} }} {a_{i} } . $$

Using similar analysis, SER expression for L = 2, Ls = 1 case with QPSK modulation is given by

$$ P_{e,GSC} = \frac{7}{{24\left( {{\bar{\gamma }}_{ 0}^{{\prime \prime }} } \right)^{2} \pi }}\int\limits_{0}^{{{{3\pi } \mathord{\left/ {\vphantom {{3\pi } 4}} \right. \kern-0pt} 4}}} {\frac{1}{{\left( {\frac{0.5}{{\sin^{2} \phi }} + \frac{1}{{{\bar{\gamma }}_{ 0}^{{\prime \prime }} }}} \right)\left( {\frac{0.5}{{2\sin^{2} \phi }} + \frac{1}{{{\bar{\gamma }}_{ 0}^{{\prime \prime }} }}} \right)}}} d\phi . $$
(17)

3.5 Derivation of Bounds on BER Performance

In this section, upper and lower bounds on BER system performance are derived. Specifically, one lower bound and three upper bounds are derived using different approaches. The BER expression for Nt = 2, L = 2, Ls = 2 case with BPSK modulation is expressed as

$$ P_{e,GSC} = \int\limits_{0}^{\infty } {\int\limits_{0}^{\infty } {\left( {\frac{1}{\pi }\int\limits_{0}^{{{\pi \mathord{\left/ {\vphantom {\pi 2}} \right. \kern-0pt} 2}}} {\exp \left[ { - \frac{{\gamma_{GSC} }}{{\sin^{2} \phi }}} \right]} d\phi } \right)} } \left( {v_{1} + \frac{{v_{2} }}{2}} \right)\left( {\frac{{v_{2} }}{2}} \right)\exp \left( { - v{}_{1} - v{}_{2}} \right)dv_{2} dv_{1} . $$
(18)

The Q-function is represented as \( Q\left( x \right) = \frac{1}{\pi }\int\limits_{0}^{\pi /2} {\exp \left( { - \frac{{x^{2} }}{{2\sin^{2} \phi }}} \right)d\phi } . \) Therefore, (18) can be equivalently written as

$$ P_{e,GSC} = \int\limits_{0}^{\infty } {\int\limits_{0}^{\infty } {Q\left( {\sqrt {2\gamma_{GSC} } } \right)} } \left( {\frac{{v_{1} v_{2} }}{2} + \frac{{v_{2}^{2} }}{4}} \right)\exp \left( { - v{}_{1} - v{}_{2}} \right)dv_{2} dv_{1} . $$
(19)

Upper Bound 1:

Using identity \( Q\left( x \right) \le \frac{1}{2}\exp \left( { - \frac{{x^{2} }}{2}} \right),\quad \forall \,x \ge 0 \), the upper bound is derived as

$$ \left( {P_{e,GSC} } \right)_{{UB_{1} }} \le \frac{1}{2}\int\limits_{0}^{\infty } {\int\limits_{0}^{\infty } {\exp \left( { - \gamma_{GSC} } \right)} } \times \left( {\frac{{v_{1} v_{2} }}{2} + \frac{{v_{2}^{2} }}{4}} \right)\exp \left( { - v{}_{1} - v{}_{2}} \right)dv_{2} dv_{1} . $$
(20)

The integral in (20) can be further simplified to [19]

$$ \left( {P_{e,GSC} } \right)_{{UB_{1} }} \le \frac{7}{{48\left( {\frac{{\bar{\gamma }}}{2} + 1} \right)^{2} }}. $$

Upper Bound 2:

Using identity \( Q\left( x \right) < \frac{1}{{x\sqrt {2\pi } }}\exp \left( { - \frac{{x^{2} }}{2}} \right),\quad \forall \,{\text{x > 0}} \) [22], the upper bound is derived as

$$ \left( {P_{e,GSC} } \right)_{{UB_{2} }} < \frac{1}{{\sqrt {2\pi } }}\int\limits_{0}^{\infty } {\int\limits_{0}^{\infty } {\frac{1}{{\sqrt {2\gamma_{GSC} } }}\exp \left( { - \gamma_{GSC} } \right)} } \left( {\frac{{v_{1} v_{2} }}{2} + \frac{{v_{2}^{2} }}{4}} \right)\exp \left( { - v{}_{1} - v{}_{2}} \right)dv_{2} dv_{1} . $$

Upper Bound 3:

A loose upper bound on BER can be calculated by integrating (20) over the whole space

$$ \left( {P_{e,GSC} } \right)_{{UB_{3} }} \le \int\limits_{0}^{\infty } {\int\limits_{0}^{\infty } {\left( {\frac{1}{\pi }\int\limits_{0}^{{{\pi \mathord{\left/ {\vphantom {\pi 2}} \right. \kern-0pt} 2}}} {\exp \left[ { - \frac{{0.5\bar{\gamma }}}{{\sin^{2} \phi }}\left( {a_{1} + a_{2} } \right)} \right]} d\phi } \right)} } 2!a_{1} a_{2} \exp \left( { - a{}_{1} - a{}_{2}} \right)da_{2} da_{1} . $$
(21)

The integral in (21) can be further simplified to [19]

$$ \left( {P_{e,GSC} } \right)_{{UB_{1} }} \le \frac{1}{2\pi }\int\limits_{0}^{{{\raise0.7ex\hbox{$\pi $} \!\mathord{\left/ {\vphantom {\pi 2}}\right.\kern-0pt} \!\lower0.7ex\hbox{$2$}}}} {\frac{1}{{\left( {\frac{{0.5\bar{\gamma }}}{{\sin^{2} \phi }} + 1} \right)^{2} }}} d\phi . $$

Lower Bound:

Using identity \( Q\left( x \right) > \frac{1}{{x\sqrt {2\pi } }}\left( {1 - \frac{1}{{x^{2} }}} \right)\exp \left( { - \frac{{x^{2} }}{2}} \right),\quad \forall \,{\text{x > 1}} \) [22], the lower bound is derived as

$$ \left( {P_{e,GSC} } \right)_{LB} > \frac{1}{{\sqrt {2\pi } }}\int\limits_{0}^{\infty } {\int\limits_{0}^{\infty } {\frac{1}{{\sqrt {2\gamma_{GSC} } }}\left( {1 - \frac{1}{{2\gamma_{GSC} }}} \right)\exp \left( { - \gamma_{GSC} } \right)} } \left( {\frac{{v_{1} v_{2} }}{2} + \frac{{v_{2}^{2} }}{4}} \right)\exp \left( { - v{}_{1} - v{}_{2}} \right)dv_{2} dv_{1} . $$

Similarly, bounds on SER can be derived for different values of L and Ls.

4 Ergodic Capacity of GSC for MIMO-OSTBC Systems

Channel capacity is the maximum rate at which information can be transmitted with low probability of error. For fading channels, the channel coefficients change over time, and it is possible to average over their statistics by coding over large blocks of data.

4.1 Calculation of Ergodic Capacity for 2 × L, 4 × L and 8 × L OSTBC Systems

The ergodic capacity of a MIMO fading channel using STBC is obtained by averaging capacity over PDF of SNR of the MRC combiner output, \( \gamma_{GSC} \), and is given as

$$ C = \frac{K}{T}E_{{\gamma_{GSC} }} \left[ {\log \left( {1 + \frac{{\gamma_{GSC} }}{{N_{t} }}} \right)} \right], $$
(22)

where \( \gamma_{GSC} \) is the SNR of the MRC combiner output. Here, K is the number of symbols transmitted, and T is the number of time slots. So, \( \frac{K}{T} \) denotes the STBC rate. The capacity in (22) can be equivalently expressed as

$$ C = \frac{K}{T}\left[ {\int\limits_{0}^{\infty } {\log \left( {1 + \frac{{\gamma_{GSC} }}{{N_{t} }}} \right)f_{{\gamma_{GSC} }} \left( \gamma \right)d\gamma } } \right]. $$
(23)

Substituting the PDF of SNR of MRC combiner output in (23), the capacity expression becomes

$$ C = \frac{K}{T}\int\limits_{0}^{\infty } {\int\limits_{0}^{{a_{1} }} {\int\limits_{0}^{{a_{2} }} \cdots } } \int\limits_{0}^{{a_{L - 1} }} {\log \left( {1 + \frac{{\gamma_{GSC} }}{{N_{t} }}} \right)} f_{{a_{\left[ L \right]} }} \left( {a_{1} ,a_{2} , \ldots ,a_{L} } \right)da_{L} da_{L - 1} \cdots da_{1} . $$
(24)

Transforming the ordered random variables into independent random variables, the integral in (24) can be modified as

$$ C = \frac{K}{T}\int\limits_{0}^{\infty } {\int\limits_{0}^{\infty } {\int\limits_{0}^{\infty } \cdots } } \int\limits_{0}^{\infty } {\log \left( {1 + \frac{{\gamma_{GSC} }}{{N_{t} }}} \right)} f_{{v_{\left[ L \right]} }} \left( {v_{1} ,v_{2} , \cdots ,v_{L} } \right)dv_{L} dv_{L - 1} \cdots dv_{1} . $$
(25)

Substituting PDF of independent random variables in (25), the capacity expression for L = 2, Ls = 2 case is given as

$$ C = \frac{K}{T}\int\limits_{0}^{\infty } {\int\limits_{0}^{\infty } {\log_{2} } } \left( {1 + \frac{{\gamma_{GSC} }}{2}} \right)\left( {v_{1} + \frac{{v_{2} }}{2}} \right)\left( {\frac{{v_{2} }}{2}} \right)\exp \left( { - v{}_{1} - v{}_{2}} \right)dv_{2} dv_{1} , $$
(26)

where \( \gamma_{GSC} = \frac{{\bar{\gamma }}}{2}\sum\nolimits_{i = 1}^{{L_{s} }} {v_{i} .} \) Now,

  • for a 2 × L antenna system, \( \bar{\gamma }_{0}^{''} = {{2\sigma_{g}^{2} } \mathord{\left/ {\vphantom {{2\sigma_{g}^{2} } {\sigma_{n}^{2} }}} \right. \kern-0pt} {\sigma_{n}^{2} }}, \)

  • for a 4 × L antenna system, \( \bar{\gamma }_{0}^{''} = {{4\sigma_{g}^{2} } \mathord{\left/ {\vphantom {{4\sigma_{g}^{2} } {\sigma_{n}^{2} }}} \right. \kern-0pt} {\sigma_{n}^{2} }}, \)

  • for a 8 × L antenna system, \( \bar{\gamma }_{0}^{''} = {{8\sigma_{g}^{2} } \mathord{\left/ {\vphantom {{8\sigma_{g}^{2} } {\sigma_{n}^{2} }}} \right. \kern-0pt} {\sigma_{n}^{2} }}. \)

4.2 Approximations of Ergodic Capacity in Different SNR Regions

The ergodic capacity of MIMO OSTBC system can be approximated to different values in different SNR regions. Some approximations are based on properties of logarithmic series, which are used in capacity calculations. The expression of SNR can be approximated for high, low, and intermediate SNR values.

High SNR Approximation:

For, \( \frac{{\gamma_{GSC} }}{{N_{t} }} \gg 1 \)

$$ \ln \left( {1 + \frac{{\gamma_{GSC} }}{{N_{t} }}} \right) \cong \ln \left( {\frac{{\gamma_{GSC} }}{{N_{t} }}} \right). $$
(27)

Using (27), at high SNR values, the capacity expression in (26) can be approximated as

$$ C = \frac{K}{T}\frac{1}{\ln \left( 2 \right)}\int\limits_{0}^{\infty } {\int\limits_{0}^{\infty } {\ln } } \left( {\frac{{\gamma_{GSC} }}{2}} \right)\left( {v_{1} + \frac{{v_{2} }}{2}} \right)\left( {\frac{{v_{2} }}{2}} \right)\exp \left( { - v{}_{1} - v{}_{2}} \right)dv_{2} dv_{1} . $$

Low SNR Approximation 1:

For, \( \frac{{\gamma_{GSC} }}{{N_{t} }}\ll 1 \)

$$ \ln \left( {1 + \frac{{\gamma_{GSC} }}{{N_{t} }}} \right) \cong \left( {\frac{{\gamma_{GSC} }}{{N_{t} }}} \right) - \frac{1}{2}\left( {\frac{{\gamma_{GSC} }}{{N_{t} }}} \right)^{2} . $$
(28)

Using (28), at low SNR values, the capacity expression in (26) can be approximated as

$$ C = \frac{K}{T}\frac{1}{\ln \left( 2 \right)}\int\limits_{0}^{\infty } {\int\limits_{0}^{\infty } {\left[ {\left( {\frac{{\gamma_{GSC} }}{2}} \right) - \frac{1}{2}\left( {\frac{{\gamma_{GSC} }}{2}} \right)^{2} } \right]\left( {v_{1} + \frac{{v_{2} }}{2}} \right)\left( {\frac{{v_{2} }}{2}} \right)\exp \left( { - v{}_{1} - v{}_{2}} \right)dv_{2} dv_{1} } } . $$

Low SNR Approximation 2:

For \( \frac{{\gamma_{GSC} }}{{N_{t} }}\ll 1 \),

$$ \ln \left( {1 + \frac{{\gamma_{GSC} }}{{N_{t} }}} \right) \cong \sqrt {\left( {\frac{{\gamma_{GSC} }}{{N_{t} }}} \right)} . $$
(29)

Using (29), at low SNR values, the capacity expression in (26) can be approximated as

$$ C = \frac{K}{T}\frac{1}{\ln \left( 2 \right)}\int\limits_{0}^{\infty } {\int\limits_{0}^{\infty } {\sqrt {\left( {\frac{{\gamma_{GSC} }}{2}} \right)} } } \left( {v_{1} + \frac{{v_{2} }}{2}} \right)\left( {\frac{{v_{2} }}{2}} \right)\exp \left( { - v{}_{1} - v{}_{2}} \right)dv_{2} dv_{1} . $$

4.3 Derivation of Tighter Upper Bound on Ergodic Capacity

The Upper Bound on ergodic capacity of MIMO STBC system is derived using the following identity:

$$ E\left[ {\ln \left( {1 + \frac{{\gamma_{GSC} }}{{N_{t} }}} \right)} \right] \le 1 + E\left[ {\ln \left( {\frac{{\gamma_{GSC} }}{{N_{t} }}} \right)} \right]. $$
(30)

Using (30), the capacity expression in (26) can be upper bounded as

$$ C \le \frac{K}{T}\frac{1}{\ln \left( 2 \right)}\left[ {1 + \int\limits_{0}^{\infty } {\int\limits_{0}^{\infty } {\ln } } \left( {\frac{{\gamma_{GSC} }}{2}} \right)\left( {v_{1} + \frac{{v_{2} }}{2}} \right)\left( {\frac{{v_{2} }}{2}} \right)\exp \left( { - v{}_{1} - v{}_{2}} \right)dv_{2} dv_{1} } \right]. $$

5 Numerical Results

In this section, numerical SER results for GSC are shown for OSTBC with MQAM modulation in Rayleigh fading channels, and performance comparison is made with MPSK modulation scheme. The expressions for ergodic capacity of 2 × L, 4 × L and 8 × L MIMO OSTBC systems are derived and compared. Performance comparison of HRFD and FRFD OSTBC systems are made. Bounds for SER performance and ergodic capacity are derived and plotted. Furthermore, approximations of ergodic capacity in different SNR regions are derived and plotted. The impact of channel estimation error on SER performance for GSC scheme is discussed, and the SER is computed in the presence of channel estimation errors.

Figure 2 shows average SNR as a function of SNR per symbol for a 4 × 4 HRFD OSTBC system with imperfect channel estimation (ρ = 0.9). The performance degrades as compared to the perfect channel estimation case. The fading estimation error parameter, ρ, is assumed to be 0.9 for the imperfect channel estimation case. The parameter ρ may take any value between 0 and 1. At 10−2 SER, Ls = 2 provides 6.5 dB gain in performance compared to the Ls = 1 case. At 10−2 SER, Ls = 3 gives 1.5 dB gain compared to the Ls = 2 case. This shows that selection of more number of antennas result in smaller gain in performance.

Fig. 2
figure 2

SER of QPSK versus SNR for 4 × 4 Half Rate-Full Diversity OSTBC system with imperfect channel estimation

Figure 3 shows performance comparison of MQAM and MPSK for 4 × 3 HRFD OSTBC system with perfect channel estimation. At 10−2 SER, 16-QAM provides 5.75 dB gain in performance than 16-PSK, but QAM provides only an extra 0.5 dB gain in performance than QPSK. At 10−2 SER, 8-PSK provides 7 dB gain in performance than 16-PSK. SER system performance degrades for higher order modulation schemes for both MQAM and MPSK. As expected, SER performance for various systems improves with increase in average SNR.

Fig. 3
figure 3

Performance comparison of MQAM and MPSK for 4 Tx and 3 Rx system with perfect channel estimation

A comparison of SER for imperfect and perfect CSI systems versus SNR per symbol for Nt = 4, 8 and Nr = 4 HRFD OSTBC system antenna systems is plotted in Fig. 4. The fading estimation error parameter, ρ, is assumed to be 1 for the perfect channel estimation case. For the imperfect CSI case, ρ is assumed to be 0.9. At 10−2 SER, Nt = 4, perfect CSI case requires 2 dB lower SNR than Nt = 4, imperfect CSI case. At 10−4 SER, Nt = 8, perfect CSI case has 1.5 dB gain in SNR than Nt = 4, perfect CSI case. System performance degrades with imperfection in channel estimation and improves with increase in the number of transmit antennas.

Fig. 4
figure 4

Comparison of SER of imperfect and perfect CSI system versus SNR per symbol for Nt = 4, 8 and Nr = 4 half rate-full diversity MIMO OSTBC systems

Figure 5 shows SER as a function of Average SNR of QPSK for different antenna configurations with perfect channel estimation for HRFD MIMO-OSTBC systems. The fading estimation error parameter, ρ, is assumed to be 1, for the perfect channel estimation case. At 10−4 SER, 8 × 2 antenna system provides 6 dB gain in performance than the 2 × 2 system. At 10−4 SER, 8 × 4 antenna system provides 6 dB gain in performance compared to the 8 × 2 system.

Fig. 5
figure 5

SER versus average SNR of QPSK for different antenna configurations for HRFD OSTBC systems

Figure 6 shows performance bounds on BER of BPSK for 2 × 2 antenna systems with perfect channel estimation for which fading estimation error parameter, ρ, is assumed to be 1. For SER, three upper bounds and one lower bound are derived and plotted. In all cases, the number of selected receive antennas, Ls, is considered as two. At lower values of SNR, Upper Bound 1 is a tighter upper bound, while at high SNR, Upper Bound 2 is a tighter upper bound on SER. Upper Bound 3 is a loose upper bound computed by performing integration over the whole space. Lower bound is tighter for lower SNR values as compared to higher SNR values. Theoretical result lies between upper and lower bounds for all SNRs.

Fig. 6
figure 6

Performance bounds on BER of BPSK for 2 × 2 antenna system with perfect channel estimation

Figure 7 shows approximations of ergodic capacity in different SNR regions for 2 × 1 space–time diversity system with perfect channel estimation. One high SNR approximation and two low SNR approximations are derived and plotted. For lower SNR values in the range of −5 to 0 dB, theoretical capacity can be approximated as Low SNR Approximation 2. For intermediate SNR values in the range 0 to 7 dB, theoretical capacity can be approximated to Low SNR Approximation 1. For SNR values higher than 7 dB, theoretical capacity can be approximated as High SNR approximation.

Fig. 7
figure 7

Approximations of ergodic capacity in different SNR regions for 2 × 1 space–time diversity system with ρ = 1

An upper bound on ergodic capacity for different antenna systems with perfect channel estimation is derived and plotted in Fig. 8. Here, the number of transmit antennas is taken as two, and fading estimation error parameter, ρ, is assumed to be 1. As the number of receive antennas is increased, the upper bound is further tightened. Furthermore, upper bound is tightened more with increase in the value of Average SNR. At 30 dB SNR, L = 4 system has 14 bits/s/Hz gain in capacity than the L = 1 system.

Fig. 8
figure 8

Upper bound on ergodic capacity for different antenna systems with ρ = 1

Figure 9 shows SER of QPSK as a function of SNR per symbol for different code rates system with perfect channel estimation for L = 2, Ls = 2. Here, ρ is assumed to be 1, as it is a perfect channel estimation case of transmit antennas. Performance of STBC system improves with code rate. For 8 transmit antennas, full rate system gives 12 dB gain over half rate systems. For the same code rate, Nt = 8 has 3 dB gain over Nt = 4 system. For similar values of code rate, system performance improves with increase in the number of transmit antennas. This shows the effect of transmit antenna diversity on system performance.

Fig. 9
figure 9

SER of QPSK versus SNR per symbol for different code rates system with perfect channel estimation for L = 2, Ls = 2

Figure 10 shows SER of MPAM as a function of SNR per symbol FRFD systems with imperfect channel estimation. FRFD systems are not possible for complex signal constellations for more than two transmit antennas. So, MPAM modulation scheme is used. System performance is almost the same for both 4 and 8 transmit antenna case up to ρ = 0.9. After this value, performance improves with increase in the number of transmit antennas. At ρ = 0.5, the SER for L = 3 case is 0.0185 lower than its value for L = 2 case, whereas the SER for L = 4 case is 0.0034 lower than its value for L = 3 case.

Fig. 10
figure 10

SER of MPAM versus SNR per symbol for FRFD system with imperfect channel estimation

6 Conclusions and Future Work

In this paper, analytic SER and ergodic capacity results were derived for MIMO OSTBC systems with a generalized selection combining (GSC) scheme. Approximate closed form expressions for SER and ergodic capacity of GSC for Half Rate–Full Diversity, Full Rate–Full Diversity and 3/4 Rate-Full Diversity OSTBC were derived. The results of the analysis showed that system performance improves with increase in diversity gain. It was observed that performance of STBC depends on the code rate. Thus, performance of full rate system is found to be better than 3/4 rate, which in turn is better than the half rate OSTBC. It was observed that the performance of MQAM scheme is better than the MPSK modulation scheme. Results demonstrated that system performance degrades with higher order modulation. It was analyzed that SER performance for generalized selection combining degrades with increase in channel estimation error.

This paper can be further extended to include Space–Time Codes for Multi-User MIMO systems. For complex signal constellations, non-orthogonal STBCs with minimum decoding complexity can be analyzed for signal encoding in order to achieve full diversity and full rate, simultaneously. In future, the SER of new receive selection schemes, such as Generalized Space–Time SoS selection diversity and Generalized Space–Time SoM selection diversity can be measured and compared to the conventional GSC for more than two antennas.