1 Introduction

The new generation of wireless communication systems is providing multimedia services that require very high data rates. The high spectral efficiency can be achieved by using multiple antennas at both the transmitter and receiver, so multiple-input–multiple-output (MIMO) systems have gained popularity due to their capability in delivering high spectral efficiency and their robust performance against fading. Orthogonal frequency division multiplexing (OFDM) system is an effective system to combat multipath channel by inserting a cyclic prefix (CP). It converts the multipath fading of time-domain into flat fading of frequency-domain (FD) by the Fast Fourier Transform (FFT). Therefore, the receiver design is very simple. MIMO-OFDM-based transmission systems can thus provide very high data rates with a relatively simple receiver design and are now adopted in the recent cellular wireless communication standards such as long term evolution (LTE) and LTE-Advanced [1].

In multipath channel, the received signal in a MIMO receiver is corrupted by the inter-symbol interference (ISI), spatial interference, and co-antenna interference (CAI). Thus, for the adaptive decision feedback equalizer, the probability of making an error in the detection of subsequent symbols will be increased. It can result in error propagation, and its mitigation has to be considered when designing the receiver. In this regard, turbo equalizer [2, 3] is a joint equalization and decoding scheme used in communication systems, when considering coded data transmission over time dispersive channels. By iteratively exchanging soft extrinsic information between a soft-input soft-output (SISO) equalizer and a SISO decoder, the turbo principle can achieve large performance gains over separated equalizer and decoder structure. It has been shown to be an effective method to combat the ISI caused by frequency-selective channels. With such a receiver, in this paper, we compare the decoding complexity and performance of a turbo-coded iterative receiver with that of a LDPC-coded iterative receiver.

Soft-feedback equalizers (SFE) have been proposed in [48]. One important difference is that the SFE uses a structure similar to a DFE and combines the equalizer outputs and a priori information for ISI cancellation. The SFE coefficients are computed so as to minimize the mean-squared error (MSE) between the transmitted symbol and the equalizer output. The resulting equalizer coefficients depend on the quality of the equalizer output and the a priori information. However, SFE is actually an ISI cancelation device with both anti-causal and causal filters.

In this paper, we investigate an MMSE-based MIMO-OFDM soft-decision feedback equalization (SDFE) scheme for iterative turbo/LDPC coded systems. The equalization is performed in the frequency domain. We include offset compensation to make the reliability of both soft a priori information and soft decisions of the data symbols. When extrinsic information computed from the SDFE outputs are fed to the channel decoder as a priori information, the a posteriori information provided by the channel decoder minus the a priori information is used now to compute the feed forward and feedback filters coefficients for every sub-carrier. In later iterations, the SDFE coefficients are updated based on the a priori information and soft decisions of the data symbols. In addition, we provide approximate solutions for multilevel modulation schemes and show by means of computer simulations that our proposed algorithm outperforms the conventional solutions for linear MMSE equalization.

The rest of this paper is organized as follows. In Sect. 2, we describe the system model of the communications system. In Sect. 3, we describe the soft-decision feedback equalizer (SDFE) for MIMO-OFDM systems. In Sect. 4, we provide some simulation examples and discuss the simulation results to demonstrate the performance of the proposed algorithms. Finally, in Sect. 5, we give some conclusions.

Notations we use uppercase boldface letters to indicate matrices, and lowercase boldface letters are used to indicate vectors. \( \left( \cdot \right)^{H} \) denotes Hermitian, \( \left( \cdot \right)^{T} \) denotes transpose, \( \left( \cdot \right)^{*} \) denotes conjugate, I N is the identity matrix of size N × N, \( diag\left( {{\mathbf{V}}_{x}^{{}} } \right) \) is the diagonal matrix for the vector \( {\mathbf{V}}_{x}^{{}} \). Expectation is denoted by E{} and probability is denoted by P{}. The covariance operator is given by \( Cov\left( {x,y} \right) = E\left[ {xy_{{}}^{H} } \right] - E\left[ x \right]E\left[ {y_{{}}^{H} } \right] \).

2 System Model

In this section, we describe a soft-decision feedback equalizer (SDFE) for MIMO-OFDM systems. We first discuss the MIMO-OFDM system, and then we introduce the transmitter and SDFE model [8].

2.1 MIMO-OFDM Systems

The system model of the SDFE transmitter is shown in Fig. 1. We consider a MIMO-OFDM communication system with N subcarriers, n T transmit antennas, and \( n_{R} \) receive antennas that employs digital modulation with constellation size \( 2_{{}}^{m} \). Here, the input information sequence \( a_{j}^{t} \in \left\{ {0,1} \right\},\quad for \, \;j = 0,1, \ldots ,J - 1 \) representing the t-th channel encoder bits, are first encoded using a recursive systematic convolutional (RSC) or LDPC code with code rate γ. The RSC or LDPC encoder produces the coded bits \( b_{j}^{t} \in \left\{ {0,1} \right\},\quad \, j = 0,1, \ldots ,(J/\gamma ) - 1 \), J is the block length, and is followed by an interleaver and a constellation mapper. The interleaver is used to protect the data from burst error and for the purpose of turbo equalization that requires uncorrelated extrinsic information. However, interleaver is not necessary for LDPC codes because of their inherent random structure. The output of the t-th interleaved signal can be represented as

$$ {\mathbf{c}}^{t} = \left( {{\mathbf{c}}^{t} [1]\;{\mathbf{c}}^{t} [2]\; \ldots \;{\mathbf{c}}^{t} [k_{c} ]} \right) $$
(1)

where the vector c t[k] can be described as

$$ {\mathbf{c}}^{t} [k] = \left( {{\mathbf{c}}_{1}^{t} [k]\;{\mathbf{c}}_{2}^{t} [k]\; \ldots \;{\mathbf{c}}_{m}^{t} [k]} \right) $$
(2)

\( {\mathbf{c}}^{t} [k] \) denotes the m-bit coded bits of the k-th block to be transmitted by the t-th antenna, with bits \( {\mathbf{c}}_{{^{j} }}^{t} [k] \in \left\{ {0,1} \right\} \). The output of the block mapper is \( S^{t} \left( k \right) \). The transmitted symbols are selected from an independent and identically distributed (i.i.d.) sample and transmitted with equal power. The total transmit power is assumed to be unit, thus the symbol alphabet is \( 2_{{}}^{m}\text{-}ary \) constellation \( {\mathbf{Q}} = [\alpha_{1} \;\alpha_{2} \; \ldots \alpha_{{2^{m} }} ] \), where \( \alpha_{i}^{{}} \) corresponds to the bit pattern \( {\mathbf{q}}_{i} = [q_{i,1} \;q_{i,m} \; \ldots \;q_{i,m} ] \) with \( q_{{^{i,m} }}^{{}} \in 0,1 \). We require that the alphabet have zero mean \( \sum\nolimits_{i = 1}^{{2^{m} }} {\alpha_{i} } = 0 \) and unit energy \( \frac{1}{{2^{m} }}\sum\nolimits_{i = 1}^{{2^{m} }} {\left| {\alpha_{i} } \right|^{2} } = 1 \). To illustrate the algorithms, we consider the alphabets described as in Table 1.

The base-band description of the transmitted symbol \( x^{t} (n) \) at the t-th transmit antenna at time n can be described as follows,

$$ x_{{}}^{t} (n) = \frac{1}{\sqrt N }\sum\limits_{k = 0}^{N - 1} {S_{{}}^{t} (k)e_{{}}^{j2\pi nk/N} } , \, $$
(3)
Fig. 1
figure 1

MIMO-OFDM SDFE transmitter

Table 1 Symbol alphabet

We assume that the signal between each transmit and receive antenna propagates through L different paths. At the receiver, the received samples on the r-th receive antenna can be represented as

$$ y_{{}}^{r} (n) = \sum\limits_{t = 1}^{{n_{T} }} {\sum\limits_{l = 0}^{L - 1} {h_{{}}^{rt} \left( l \right)x_{{}}^{t} \left( {n - l} \right)} } + w_{r}^{{}} \left( n \right) $$
(4)

where \( h_{{}}^{rt} \left( l \right) \) is the l-th tap of the channel between the t-th transmitter and the r-th receiver, \( w_{r}^{{}} \left( n \right) \) is additive white Gaussian noise (AWGN) sample on the r-th receive antenna with zero mean and variance \( \sigma_{w}^{2} \). The received sample \( y_{{}}^{r} (n) \) is serially received using a serial-to-parallel converter. At antenna r, applying fast Fourier transform (FFT) on the time-domain received samples after removing the CP yields a FD signal vector that can be written in matrix form as follows,

$$ {\mathbf{z}}^{r} = \sum\limits_{t = 1}^{{n_{T} }} {{\mathbf{FH}}^{rt} {\mathbf{F}}^{H} {\mathbf{s}}^{t} } + {\mathbf{w}}^{r} \quad r = 1, \ldots ,n_{R} $$
(5)

where \( {\mathbf{F}} \) stands for the unitary fast Fourier transform (FFT) matrix, \( {\mathbf{H}}_{{}}^{rt} \) is the cyclic channel convolution matrix between the t-th transmitter and the r-th receiver, \( {\mathbf{s}}_{{}}^{t} \) is transmitted symbol, \( {\mathbf{w}}_{{}}^{r} \) is the FD noise sample and \( {\mathbf{z}}_{{}}^{r} \) is the received symbol that can be represented as follows,

$$ {\mathbf{z}}^{r} = [{\mathbf{z}}^{r} (0)\;{\mathbf{z}}^{r} (1)\; \ldots \;{\mathbf{z}}^{r} (N - 1)]^{T} $$
(6)
$$ {\mathbf{w}}^{r} = [w^{r} (0)\;w^{r} (1)\; \ldots \;w^{r} (N - 1)]^{T} $$
(7)

2.2 MIMO-OFDM SDFE Receiver

The iterative receiver structure of the MIMO-OFDM SDFE is shown in Fig. 2. The interleaver and deinterleaver are bypassed when LDPC coder is employed. A low-complexity iterative receiver can be designed by collecting all the samples received by \( n_{R}^{{}} \) antennas at frequency \( k \) [9]. Therefore, the received symbols can be represented as follows,

$$ {\mathbf{Z}}(k) = {\mathbf{H}}(k)\;{\mathbf{S}}(k) + {\mathbf{W}}(k),\quad \, k = 0,1, \ldots ,N - 1 $$
(8)

where the defined matrices \( {\mathbf{Z}}(k) \in C_{{}}^{{Nn_{R} \times 1}} \), \( {\mathbf{H}}(k) \in C_{{}}^{{Nn_{R} \times Nn_{T} }} \), \( {\mathbf{S}}(k) \in C_{{}}^{{Nn_{T} \times 1}} \), \( {\mathbf{W}}(k) \in C_{{}}^{{Nn_{R} \times 1}} \) and can be described as follows,

$$ {\mathbf{Z}}(k) = \left[ {{\mathbf{Z}}^{1} (k)\;{\mathbf{Z}}^{2} (k)\; \ldots \;{\mathbf{Z}}^{{n_{R} }} (k)} \right]^{T} $$
(9)
$$ {\mathbf{H}}(k) = \left[ {\begin{array}{*{20}c} {H^{11} (k)} & {H^{11} (k)} & \ldots & {H^{{1n_{T} }} (k)} \\ {H^{21} (k)} & {H^{22} (k)} & \ldots & {H^{{2n_{T} }} (k)} \\ \vdots & \vdots & \ddots & \vdots \\ {H^{{n_{R} 1}} (k)} & {H^{{n_{R} 2}} (k)} & \ldots & {H^{{n_{R} n_{T} }} (k)} \\ \end{array} } \right]^{T} $$
(10)
$$ {\mathbf{S}}(k) = \left[ {{\mathbf{s}}^{1} (k)\;{\mathbf{s}}^{2} (k)\; \ldots \;{\mathbf{s}}^{{n_{T} }} (k)} \right]^{T} $$
(11)
$$ {\mathbf{W}}(k) = \left[ {{\mathbf{w}}^{1} (k)\;{\mathbf{w}}^{2} (k)\; \ldots \;{\mathbf{w}}^{{n_{T} }} (k)} \right]^{T} $$
(12)
Fig. 2
figure 2

MIMO-OFDM SDFE receiver

In Fig. 3, the proposed SDFE turbo equalization receiver consists of a feed forward filter and a feedback filter. The feed forward filter and feedback filter are designed for every subcarrier. An estimate \( {\hat{\mathbf{s}}}(k) \) of the transmitted symbol on the k-th subcarrier based on MMSE estimation is given by

$$ {\hat{\mathbf{s}}}(k) = {\mathbf{f}}_{k}^{{}} {\mathbf{Z}}(k)^{T} + {\mathbf{b}}_{k}^{{}} {\tilde{\mathbf{S}}}(k) + {\mathbf{d}}_{k}^{{}} $$
(13)
$$ {\tilde{\mathbf{S}}}(K) = \left[ {{\tilde{\mathbf{s}}}^{1} (k)\;{\tilde{\mathbf{s}}}^{2} (k)\; \ldots\; {\tilde{\mathbf{s}}}^{{n_{T} }} (k)} \right]^{T} $$
(14)

where the feed forward filter matrices \( {\mathbf{f}}_{k}^{{}} \in C_{{}}^{{Nn_{T} \times Nn_{R} }} \), feedback filter matrices \( {\mathbf{b}}_{k}^{{}} \in C_{{}}^{{Nn_{T} \times Nn_{T} }} \), offset \( {\mathbf{d}}_{k}^{{}} \in C_{{}}^{{Nn_{T} \times 1}} \) and the \( \widetilde{{\mathbf{S}}}(K) \) is the past decided estimate of the N subcarriers by the t-th antenna, which is the function of soft a priori information and soft extrinsic information. The offset \( {\mathbf{d}}_{k}^{{}} \) compensation is to make the reliability of both soft a priori information and soft decisions of the data symbols. To design the equalizer [10, 11], the coefficients \( {\mathbf{f}}_{k}^{{}} \), \( {\mathbf{b}}_{k}^{{}} \) and \( {\mathbf{d}}_{k}^{{}} \) can be obtained by minimizing mean square error (MMSE) as follows,

$$ \frac{{\partial E\left\{ {\left| {{\hat{\mathbf{s}}}(k) - {\mathbf{s}}(k)} \right|^{2} } \right\}}}{{\partial {\mathbf{d}}_{k}^{*} }} = - 2E\left\{ {{\mathbf{s}}(k) - {\mathbf{f}}_{k}^{{}} {\mathbf{Z}}(k) - {\mathbf{b}}_{k}^{{}} {\tilde{\mathbf{S}}}(k) - {\mathbf{d}}_{k}^{{}} } \right\} = 0 $$
(15)
Fig. 3
figure 3

Block diagram of the SDFE receiver

We then obtain

$$ {\mathbf{d}}_{k}^{{}} = E\left\{ {{\mathbf{s}}(k)} \right\} - {\mathbf{f}}_{k}^{{}} {\mathbf{H}}E\left\{ {{\mathbf{S}}(k)} \right\} - {\mathbf{b}}_{k}^{{}} E\left\{ {{\tilde{\mathbf{S}}}(k)} \right\} $$
(16)

Similarity, the feed forward and feedback coefficients are obtained by minimizing the MSE,

$$ \frac{{\partial E\left\{ {\left| {{\hat{\mathbf{s}}}(k) - {\mathbf{s}}(k)} \right|^{2} } \right\}}}{{\partial {\mathbf{f}}_{k}^{*} }} = - 2E\left\{ {\left( {{\mathbf{s}}(k) - {\mathbf{f}}_{k}^{{}} {\mathbf{Z}}(k) - {\mathbf{b}}_{k}^{{}} {\tilde{\mathbf{S}}}(k) - {\mathbf{d}}_{k}^{{}} } \right){\mathbf{Z}}_{{}}^{H} \left( k \right)} \right\} = 0 $$
(17)
$$ \frac{{\partial E\left\{ {\left| {{\hat{\mathbf{s}}}(k) - {\mathbf{s}}(k)} \right|^{2} } \right\}}}{{\partial {\mathbf{b}}_{k}^{H} }} = - 2E\left\{ {\left( {{\mathbf{s}}(k) - {\mathbf{f}}_{k}^{{}} {\mathbf{Z}}(k) - {\mathbf{b}}_{k}^{{}} {\tilde{\mathbf{S}}}(k) - {\mathbf{d}}_{k}^{{}} } \right){\tilde{\mathbf{S}}}_{{}}^{H} \left( k \right)} \right\} = 0{^\prime} $$
(18)

So the Hermitian of \( {\mathbf{f}}_{k}^{{}} \) and \( {\mathbf{b}}_{k}^{{}} \) can be obtained as follows,

$$ {\mathbf{f}}_{k}^{H} = \left[ {\sigma_{w}^{2} {\mathbf{I}}_{{Nn_{R} }}^{{}} + {\mathbf{H}}\left( {{\mathbf{R}}_{k}^{ff} - {\mathbf{R}}_{k}^{fb} \left( {{\mathbf{R}}_{k}^{fb} } \right)_{{}}^{ - 1} {\mathbf{R}}_{k}^{fbH} } \right){\mathbf{H}}_{{}}^{H} } \right]_{{}}^{ - 1} {\mathbf{e}}_{k}^{{}} $$
(19)
$$ {\mathbf{b}}_{k}^{H} = - \left( {{\mathbf{R}}_{k}^{bb} } \right)_{{}}^{ - 1} \left( {{\mathbf{HR}}_{k}^{fb} } \right)_{{}}^{H} {\mathbf{f}}_{k}^{H} $$
(20)

where we defined covariance matrices \( {\mathbf{R}}_{k}^{ff} \in C_{{}}^{{Nn_{T} \times Nn_{T} }} \), \( {\mathbf{R}}_{k}^{fb} \in C_{{}}^{{Nn_{T} \times Nn_{T} }} \), \( {\mathbf{R}}_{k}^{bb} \in C_{{}}^{{Nn_{T} \times Nn_{T} }} \) and \( {\mathbf{e}}_{k}^{{}} \in C_{{}}^{{Nn_{R} \times n_{T} }} \) as follows,

$$ {\mathbf{R}}_{k}^{ff} = E\left\{ {{\mathbf{S}}(k){\mathbf{S}}(k)_{{}}^{H} } \right\} - E\left\{ {{\mathbf{S}}(k)} \right\}E\left\{ {{\mathbf{S}}(k)_{{}}^{H} } \right\} $$
(21)
$$ {\mathbf{R}}_{k}^{fb} = E\left\{ {{\mathbf{S}}(k){\tilde{\mathbf{S}}}(k)_{{}}^{H} } \right\} - E\left\{ {{\mathbf{S}}(k)} \right\}E\left\{ {{\tilde{\mathbf{S}}}(k)_{{}}^{H} } \right\} $$
(22)
$$ {\mathbf{R}}_{k}^{bb} = E\left\{ {{\tilde{\mathbf{S}}}(k){\tilde{\mathbf{S}}}(k)_{{}}^{H} } \right\} - E\left\{ {{\tilde{\mathbf{S}}}(k)} \right\}E\left\{ {{\tilde{\mathbf{S}}}(k)_{{}}^{H} } \right\} $$
(23)
$$ {\mathbf{e}}_{k}^{{}} = {\mathbf{H}}\left( {E\left\{ {{\mathbf{S}}(k){\mathbf{s}}(k)_{{}}^{H} } \right\} - E\left\{ {{\mathbf{S}}(k)} \right\}E\left\{ {{\mathbf{s}}(k)_{{}}^{H} } \right\}} \right) $$
(24)

and \( {\mathbf{e}}_{k}^{{}} \) denotes a matrix of all zero, expect for the k-th subcarrier. At the output of the equalizer, we assume that the estimate \( {\hat{\mathbf{s}}}(k) \in Q \) as an output of an equivalent AWGN channel, so we can employ the AWGN channel model to denote the extrinsic log likelihood ratios (LLR) \( \lambda_{k}^{t} \). Then, we can compute from another equivalent AWGN channel to get a priori LLR \( \left( {\lambda_{k}^{t} } \right)_{{}}^{p} \). Here, superscript \( ()^{p} \) denotes the quantities obtained from the previous iteration. We use the extrinsic information \( \lambda_{k}^{t} \) plus a priori information \( \left( {\lambda_{k}^{t} } \right)_{{}}^{p} \) to denote a full LRR \( L_{k}^{t} \) that can be used to compute \( \tilde{s}_{{}}^{t} \left( k \right) \) and covariance matrices.

3 Soft Decision Feedback Equalization

According to the previous section, the k-th subcarrier estimate \( \hat{s}\left( k \right) \) can be rewritten as

$$ {\mathbf{s}}\left( k \right) = {\mathbf{f}}_{k}^{{}} \left( {{\mathbf{Z}}\left( k \right) - {\mathbf{H}}E\left\{ {{\mathbf{S}}\left( k \right)} \right\}} \right) + {\mathbf{b}}_{k}^{{}} \left( {{\tilde{\mathbf{S}}}\left( k \right) - E\left\{ {{\tilde{\mathbf{S}}}\left( k \right)} \right\}} \right) + E\left\{ {{\mathbf{s}}\left( k \right)} \right\} $$
(25)

The block diagram of the SDFE receiver is shown in Fig. 3. In order to perform turbo equalization, the a priori information needs to be updated. In the first iteration, the input symbols are initialized to be no a priori information. So \( {\mathbf{H}}E\left\{ {{\mathbf{S}}\left( k \right)} \right\} = 0 \), \( E\left\{ {{\mathbf{s}}\left( k \right)} \right\} = 0 \) and \( E\left\{ {{\tilde{\mathbf{S}}}\left( k \right)} \right\} = 0 \), the employed SDFE becomes the traditional decision feedback equalizer. The feed forward filter coefficient can be represented as follows,

$$ {\mathbf{f}}_{k}^{H(1)} = \left[ {\sigma_{w}^{2} {\mathbf{I}}_{{Nn_{R} }}^{{}} + {\mathbf{HH}}_{{}}^{H} } \right]_{{}}^{ - 1} {\mathbf{e}}_{k}^{{}} $$
(26)

In order to compute \( {\mathbf{f}}_{k}^{{}} \), \( {\mathbf{b}}_{k}^{{}} \) and \( {\mathbf{d}}_{k}^{{}} \), covariance matrices \( {\mathbf{R}}_{k}^{ff} \), \( {\mathbf{R}}_{k}^{fb} \) and \( {\mathbf{R}}_{k}^{bb} \) have to be decided first. The covariance matrices \( {\mathbf{R}}_{k}^{ff} \), \( {\mathbf{R}}_{k}^{fb} \) and \( {\mathbf{R}}_{k}^{bb} \) is a key step for finding estimate \( \widehat{{\mathbf{s}}}(k) \). The \( {\mathbf{R}}_{k}^{ff} \) can be computed using the a priori information from decoder [10]. For calculating \( {\mathbf{R}}_{k}^{fb} \) and \( {\mathbf{R}}_{k}^{bb} \), we make the assumption that the transmitted symbols are independent and identically distributed (iid), so we have \( E\left\{ {{\mathbf{s}}_{{}}^{p} \left( n \right){\tilde{\mathbf{s}}}_{{}}^{q} \left( m \right)_{{}}^{*} } \right\} = E\left\{ {{\tilde{\mathbf{s}}}_{{}}^{p} \left( n \right){\tilde{\mathbf{s}}}_{{}}^{q} \left( m \right)_{{}}^{*} } \right\} = 0 \), except when \( p = n \) and \( m = n \). However, we can simplify the calculation to be individual expectation values \( \varepsilon_{{}}^{t} \), \( \chi_{{}}^{t} \) and \( \zeta_{{}}^{t} \), that can be described as follows,

$$ \varepsilon_{{}}^{t} = E\left\{ {\tilde{s}_{{}}^{t} \left( k \right)} \right\} $$
(27)
$$ \chi_{{}}^{t} = E\left\{ {s_{{}}^{t} \left( k \right)\tilde{s}_{{}}^{t} \left( k \right)_{{}}^{*} } \right\} $$
(28)
$$ \xi_{{}}^{t} = E\left\{ {\tilde{s}_{{}}^{t} \left( k \right)\tilde{s}_{{}}^{t} \left( k \right)_{{}}^{*} } \right\} $$
(29)

However, these individual expectation values \( \varepsilon_{{}}^{t} \), \( \chi_{{}}^{t} \) and \( \zeta_{{}}^{t} \) have to be determined according to the quality of both the soft decisions of data symbols and the soft a priori information. Since the individual expectation values \( \varphi_{{}}^{t} \), \( \chi_{{}}^{t} \) and \( \zeta_{{}}^{t} \) are functions of \( {\tilde{\mathbf{s}}}\left( k \right) \), and \( {\tilde{\mathbf{s}}}\left( k \right) \) is a function of extrinsic LLR \( \lambda_{k,j}^{t} \) and a priori information \( \left( {\lambda_{k}^{t} } \right)_{{}}^{p} \). As a result, knowledge of the extrinsic information \( \lambda_{k}^{t} \) and a priori information \( \left( {\lambda_{k}^{t} } \right)_{{}}^{p} \) is needed to find the equalization matrix.

The received signal \( {\mathbf{Z}}\left( k \right) \) goes through the feed forward filter \( {\mathbf{f}}_{k}^{H(1)} \). At the first iteration, using Eq. (26), because the input symbols are initialized to be no a priori information, we can obtaine the estimate symbols \( \hat{s}_{{}}^{t} \left( k \right) \). At the output of the equalizer, we assume the output signal is of an equivalent AWGN channel that can be used to calculate the extrinsic LLR \( \lambda_{k,j}^{t} \) [12, 13]. We compute from another equivalent AWGN channel to get a priori LLR \( \left( {\lambda_{k}^{t} } \right)_{{}}^{p} \). The full LLR \( L_{k}^{t} \) can be used to calculate the probability values \( P\left( {\widetilde{s}^{t} (k) = \alpha_{i} } \right) \). Then using the probability values \( P\left( {\widetilde{s}^{t} (k) = \alpha_{i} } \right) \) can calculate the past decided estimate \( {\tilde{\mathbf{s}}}\left( k \right) \). Since the individual expectation values \( \varepsilon_{{}}^{t} \), \( \chi_{{}}^{t} \) and \( \zeta_{{}}^{t} \) are functions of \( {\tilde{\mathbf{s}}}\left( k \right) \), these individual expectation values \( \varepsilon_{{}}^{t} \), \( \chi_{{}}^{t} \) and \( \zeta_{{}}^{t} \) can be computed by past decided estimate \( {\tilde{\mathbf{s}}}\left( k \right) \). Finally, using these individual expectation values \( \varepsilon_{{}}^{t} \), \( \chi_{{}}^{t} \) and \( \zeta_{{}}^{t} \) can obtain the covariance matrices \( {\mathbf{R}}_{k}^{fb} \) and \( {\mathbf{R}}_{k}^{bb} \).

3.1 SDFE for Multilevel Modulation

In order to demodulate the symbols, at the output of the equalizer [11, 1416], we assume that the estimate symbol \( {\tilde{\mathbf{s}}}\left( k \right) \) is output signal of an equivalent AWGN channel having transmitted symbols \( {\mathbf{s}}\left( k \right) \) as its input signal, that is

$$ \hat{s}_{{}}^{t} \left( k \right) = A_{{}}^{t} s_{{}}^{t} \left( k \right) + v_{{}}^{t} \left( k \right) $$
(30)

where \( A_{{}}^{t} \) is the equivalent amplitude of the signal at the output of the t-th antenna and can be represented as follows,

$$ A_{{}}^{t} = E\left\{ {\hat{s}_{{}}^{t} \left( k \right)s_{{}}^{t} \left( k \right)} \right\} = \left[ {\mathbf{A}} \right]_{t}^{{}} $$
(31)

where \( \left[ {\mathbf{A}} \right]_{t}^{{}} \) is the t-th diagonal element of matrix \( {\mathbf{A}} \in {\mathbf{C}}_{{}}^{t \times t} \) and \( {\mathbf{A}} = {\mathbf{f}}_{k}^{{}} {\mathbf{e}}_{k}^{{}} \), \( v_{{}}^{t} \left( k \right) \) is a complex white Gaussian noise with zero mean and variance \( \sigma_{v}^{2} = A_{{}}^{t} \left( {1 - A_{{}}^{t} } \right) \). The estimates \( \hat{s}_{{}}^{t} \left( k \right) \) are complex Gaussian distributed, \( \hat{s}^{t} \left( k \right) \sim N\left( {A^{t} s^{t} \left( k \right),\sigma_{v}^{2} } \right) \). In order to calculate the bit extrinsic probabilities [17], at first, we have to approximate the symbol extrinsic probabilities. We use the equivalent AWGN channel assumption given in (30), then the a posterior probability can be written as follows,

$$ P\left( {s^{t} \left( k \right)\left| {\hat{s}^{t} \left( k \right)} \right.} \right) = \frac{{P\left( {\hat{s}^{t} \left( k \right)\left| {s^{t} \left( k \right)} \right.} \right)P\left( {s^{t} \left( k \right)} \right)}}{{P\left( {\hat{s}^{t} \left( k \right)} \right)}} $$
(32)

The symbol extrinsic probability can be described as follows,

$$ P\left( {\hat{s}_{{}}^{t} \left( k \right)\left| {s_{{}}^{t} \left( k \right) = \alpha_{i}^{{}} } \right.} \right) = \frac{1}{{\sigma_{v}^{2} \pi }}\exp \left( { - \frac{{\left| {\hat{s}_{{}}^{t} \left( k \right) - A_{{}}^{t} \alpha_{i}^{{}} } \right|_{{}}^{2} }}{{\sigma_{v}^{2} }}} \right) $$
(33)

where \( \alpha_{i}^{{}} \) is defined in Table 1. So the a posteriori LLR can be obtained as

$$ \begin{aligned} L\left( {c_{k,j}^{t} \left| {\hat{s}^{t} \left( k \right)} \right.} \right) & = \log \frac{{P\left( {c_{k,j}^{t} = 0\left| {\hat{s}^{t} \left( k \right)} \right.} \right)}}{{P\left( {c_{k,j}^{t} = 1\left| {\hat{s}^{t} \left( k \right)} \right.} \right)}} \\ & = \log \frac{{\sum\nolimits_{{\forall \alpha_{j}^{{}} :c_{k,j}^{t} = 0}} {P\left( {\hat{s}^{t} \left( k \right)\left| {s^{t} \left( k \right) = \alpha_{i}^{{}} } \right.} \right)P\left( {s^{t} \left( k \right) = \alpha_{i}^{{}} } \right)} }}{{\sum\nolimits_{{\forall \alpha_{j}^{{}} :c_{k,j}^{t} = 1}} {P\left( {\hat{s}^{t} \left( k \right)\left| {s^{t} \left( k \right)} \right. = \alpha_{i}^{{}} } \right)P\left( {s^{t} \left( k \right) = \alpha_{i}^{{}} } \right)} }} \\ & = \underbrace {{\log \frac{{\sum\nolimits_{{\forall \alpha_{j}^{{}} :c_{k,j}^{t} = 0}} {P\left( {\hat{s}^{t} \left( k \right)\left| {s^{t} \left( k \right) = \alpha_{i}^{{}} } \right.} \right)\prod\nolimits_{{\forall j_{{}}^{'} :j \ne j}} {P\left( {c_{k,j'}^{t} } \right)} } }}{{\sum\nolimits_{{\forall \alpha_{j}^{{}} :c_{k,j}^{t} = 1}} {P\left( {\hat{s}^{t} \left( k \right)\left| {s^{t} \left( k \right)} \right. = \alpha_{i}^{{}} } \right)\prod\nolimits_{{\forall j_{{}}^{'} :j \ne j}} {P\left( {c_{k,j'}^{t} } \right)} } }}}}_{{\lambda_{k,j}^{t} }} + \underbrace {{\log \frac{{P\left( {c_{k,j'}^{t} = 0} \right)}}{{P\left( {c_{k,j'}^{t} = 1} \right)}}}}_{{\left( {\lambda_{k,j}^{t} } \right)^{p} }} \\ \end{aligned} $$
(34)

The a priori LLR \( \left( {\lambda_{k,j}^{t} } \right)_{{}}^{p} \) represents priori information on the occurrence probability of \( c_{k,j}^{t} \), and it is obtained by interleaving the soft output of the decoder from the previous iteration. For the first iteration, no a priori information is available and hence we have \( \left( {\lambda_{k,j}^{t} } \right)_{{}}^{p} = 0 \). The extrinsic LLR \( \lambda_{k,j}^{t} \) of coded bit \( c_{k,j}^{t} \) is given by

$$ \lambda_{k,j}^{t} = \log \frac{{\sum\nolimits_{{\forall \alpha_{j}^{{}} :c_{k,j}^{t} = 0}} {P\left( {\hat{s}_{{}}^{t} \left( k \right)\left| {s_{{}}^{t} \left( k \right) = \alpha_{i}^{{}} } \right.} \right)\prod\nolimits_{{\forall j_{{}}^{'} :j \ne j}} {P\left( {c_{k,j'}^{t} } \right)} } }}{{\sum\nolimits_{{\forall \alpha_{j}^{{}} :c_{k,j}^{t} = 1}} {P\left( {\hat{s}_{{}}^{t} \left( k \right)\left| {s_{{}}^{t} \left( k \right)} \right. = \alpha_{i}^{{}} } \right)\prod\nolimits_{{\forall j_{{}}^{'} :j \ne j}} {P\left( {c_{k,j'}^{t} } \right)} } }} = \log \frac{{\sum\nolimits_{{\forall \alpha_{j}^{{}} :c_{k,j}^{t} = 0}} {\exp \left( { - \rho_{k,i}^{t} + \frac{{\sum\nolimits_{\forall j':j \ne j} {\tilde{q}_{i,j}^{{}} L\left( {c_{n,j'}^{{}} } \right)} }}{2}} \right)} }}{{\sum\nolimits_{{\forall \alpha_{j}^{{}} :c_{k,j}^{t} = 1}} {\exp \left( { - \rho_{k,i}^{t} + \frac{{\sum\nolimits_{\forall j':j \ne j} {\tilde{q}_{i,j}^{{}} L\left( {c_{n,j'}^{{}} } \right)} }}{2}} \right)} }} $$
(35)

where the notation \( \forall \alpha_{i}^{{}} :c_{k,j}^{t} = 1 \) represents the subset of the k-th subcarrier symbol \( s_{{}}^{t} \left( k \right) \) in Q with a given value of \( c_{k,j}^{t} = 1 \) and \( \tilde{q}_{i,j}^{{}} \) can be represented as follows,

$$ \tilde{q}_{i,j}^{{}} \approx \left\{ {\begin{array}{*{20}c} { + 1,} & {q_{i,j}^{{}} = 0} \\ { - 1,} & {q_{i,j}^{{}} = 1} \\ \end{array} } \right. $$
(36)

The approximation of the LLR \( \lambda_{k,j}^{t} \) can be given in Table 2 [18]. Let the a priori LLR \( (\lambda_{k,j}^{t} )^{p} \) of \( c_{k,j}^{t} \) be computed from another equivalent AWGN channel with output as follows,

$$ \Theta_{n}^{t} = s_{{}}^{t} \left( k \right) + \upsilon_{{}}^{t} \left( k \right) $$
(37)

where AWGN noise \( \upsilon_{{}}^{t} \left( k \right) \) is with zero mean and variance \( \sigma_{v}^{2} \). So we obtain

$$ \left( {\lambda_{k,j}^{t} } \right)_{{}}^{p} = \log \frac{{P\left( {c_{k,j}^{t} = 0} \right)}}{{P\left( {c_{k,j}^{t} = 1} \right)}} = \Theta_{n}^{t} \phi_{p}^{t} $$
(38)

where

$$ E\left\{ {\left| {\left( {\lambda_{k,j}^{t} } \right)_{{}}^{p} } \right|_{{}}^{2} } \right\} = \left( {\phi_{p}^{t} } \right)_{{}}^{2} + 2\phi_{p}^{t} $$
(39)
Table 2 Approximation of LLR \( \lambda_{k,j}^{t} \) for multilevel modulations

So \( \phi_{p}^{t} \) can be estimated from the a priori information. The log likelihood function of \( \lambda_{k,j}^{t} \) can then be described as follows,

$$ \ln f\left( {\left( {\lambda_{1,j}^{t} } \right)^{p} , \ldots \left( {\lambda_{N,j}^{t} } \right)^{p} \left| {\phi_{p}^{t} } \right.} \right) = \frac{ - N}{2}\ln \left( {4\pi } \right) - \frac{ - N}{2}\ln \left( {\phi_{p}^{t} } \right) - \frac{{\sum\nolimits_{k = 1}^{N} {\sum\nolimits_{j = 1}^{Q} {\left( {\left( {\lambda_{k,j}^{t} } \right)^{p} - \phi_{p}^{t} } \right)^{2} } } }}{{4\phi_{p}^{t} }} $$
(40)

Taking partial differentiation with respect to \( \phi_{p}^{t} \), we obtain

$$ \frac{{\partial \left( {\left( {\lambda_{1,j}^{t} } \right)^{p} , \ldots \left( {\lambda_{N,j}^{t} } \right)^{p} \left| {\phi_{p}^{t} } \right.} \right)}}{{\partial \phi_{p}^{t} }} = \frac{ - N}{{2\phi_{p}^{t} }} + \frac{{2\sum\nolimits_{k = 1}^{N} {\sum\nolimits_{j = 1}^{Q} {\left( {\left( {\lambda_{k,j}^{t} } \right)^{p} - \phi_{p}^{t} } \right)\phi_{p}^{t} } } }}{{4\left( {\phi_{p}^{t} } \right)^{2} }} + \frac{{\sum\nolimits_{k = 1}^{N} {\sum\nolimits_{j = 1}^{Q} {\left( {\left( {\lambda_{k,j}^{t} } \right)^{p} - \phi_{p}^{t} } \right)^{2} } } }}{{4\phi_{p}^{t} }} = 0 $$
(41)

Manipulating the above equation, we can get

$$ \begin{aligned} & \frac{ - N}{{2\phi_{p}^{t} }} + \frac{{\sum\nolimits_{k = 1}^{N} {\sum\nolimits_{j = 1}^{Q} {\left( {\lambda_{k,j}^{t} } \right)^{p} - K_{c} \phi_{p}^{t} } } }}{{2\phi_{p}^{t} }} + \frac{{\sum\nolimits_{k = 1}^{N} {\sum\nolimits_{j = 1}^{Q} {\left( {\left( {\lambda_{k,j}^{t} } \right)^{p} } \right)^{2} } } - 2\phi_{p}^{t} \sum\nolimits_{k = 1}^{N} {\sum\nolimits_{j = 1}^{Q} {\left( {\lambda_{k,j}^{t} } \right)^{p} + K_{c}^{{}} \left( {\left( {\lambda_{k,j}^{t} } \right)^{p} } \right)^{2} } } }}{{4\phi_{p}^{t} }} = 0 \\ & \quad \Rightarrow \sum\limits_{k = 1}^{N} {\sum\limits_{j = 1}^{Q} {\left( {\left( {\lambda_{k,j}^{t} } \right)^{p} } \right)^{2} } } - 2\phi_{p}^{t} N - N\left( {\phi_{p}^{t} } \right)^{2} = 0 \\ \end{aligned} $$
(42)

Final, we obtain

$$ \left( {\phi_{p}^{t} } \right)_{{}}^{ML} = \sqrt {1 + \frac{1}{N \times Q}\sum\limits_{k = 1}^{N} {\sum\limits_{j = 1}^{Q} {\left( {\left( {\lambda_{k,j}^{t} } \right)_{{}}^{p} } \right)^{2} } } } - 1 $$
(43)

After obtaining the extrinsic LLR \( \lambda_{k,j}^{t} \) and the a priori LLR \( \left( {\lambda_{k,j}^{t} } \right)_{{}}^{p} \), we can further compute the full LLR \( L_{k,j}^{t} \) of coded bit \( c_{k,j}^{t} \) and can be described as follows,

$$ L_{k,j}^{t} = \lambda_{k,j}^{t} + \left( {\lambda_{k,j}^{t} } \right)_{{}}^{p} $$
(44)

where \( \left( {\lambda_{k,j}^{t} } \right)_{{}}^{p} = \log \frac{{P\left( {c_{k,j}^{t} = 0} \right)}}{{P\left( {c_{k,j}^{t} = 1} \right)}} \). The probability \( P\left( {\tilde{s}_{{}}^{t} \left( k \right) = \alpha_{i}^{{}} } \right) \) is given by [1825]

$$ \begin{aligned} P\left( {\tilde{s}_{{}}^{t} \left( k \right) = \alpha_{i}^{{}} } \right) & = \prod\limits_{j = 1}^{m} {\frac{{\exp \left( {\tilde{q}_{i,j}^{{}} \cdot L_{k,j}^{t} } \right)}}{{1 + \exp \left( {\tilde{q}_{i,j}^{{}} \cdot L_{k,j}^{t} } \right)}}} \\ & = \prod\limits_{j = 1}^{m} {\frac{{\exp \left( {\tilde{q}_{i,j}^{{}} \cdot L_{i,j}^{t} /2} \right)}}{{\exp \left( { - \tilde{q}_{i,j}^{{}} \cdot L_{k,j}^{t} /2} \right) + \exp \left( {\tilde{q}_{i,j}^{{}} \cdot L_{k,j}^{t} /2} \right)}}} \\ & = \prod\limits_{j = 1}^{m} {\frac{{\cosh \left( {L_{k,j}^{t} /2} \right)\left[ {1 + \tilde{q}_{i,j}^{{}} \cdot \tanh \left( {\frac{{L_{k,j}^{t} }}{2}} \right)} \right]}}{{2\cosh \left( {L_{k,j}^{t} /2} \right)}}} \\ & = \prod\limits_{j = 1}^{m} {\frac{1}{2}\left( {1 + \tilde{q}_{i,j}^{{}} \cdot \tanh \left( {\frac{{L_{k,j}^{t} }}{2}} \right)} \right)} \\ \end{aligned} $$
(45)

Using Eq. (45), the computation of \( \tilde{s}_{{}}^{t} \left( k \right) \) is given by

$$ \tilde{s}_{{}}^{t} \left( k \right) = \sum\limits_{\alpha \in Q} {\alpha_{i}^{{}} P} \left( {\tilde{s}_{{}}^{t} \left( k \right) = \alpha_{i}^{{}} } \right) $$
(46)

By using equal probability and symmetries of \( \alpha_{i} \in Q \), from Eq. (46), the expectation values \( \varepsilon_{{}}^{t} \), \( \chi_{{}}^{t} \) and \( \zeta_{{}}^{t} \) can be computed as follows,

$$ \varepsilon_{{}}^{t} = E\left\{ {\tilde{s}_{{}}^{t} \left( k \right)} \right\} = 0 $$
(47)
$$ \chi_{{}}^{t} = E\left\{ {s_{{}}^{t} \left( k \right)\tilde{s}_{{}}^{t} \left( k \right)_{{}}^{*} } \right\} = \alpha_{i}^{{}} E\left\{ {\tilde{s}_{{}}^{t} \left( k \right)_{{}}^{*} \left| {s_{{}}^{t} \left( k \right) = \alpha_{i}^{{}} } \right.} \right\} $$
(48)
$$ \zeta_{{}}^{t} = E\left\{ {\tilde{s}_{{}}^{t} \left( k \right)\tilde{s}_{{}}^{t} \left( k \right)_{{}}^{*} } \right\} = E\left\{ {\tilde{s}_{{}}^{t} \left( k \right)\tilde{s}_{{}}^{t} \left( k \right)_{{}}^{*} \left| {s_{{}}^{t} \left( k \right) = \alpha_{i}^{{}} } \right.} \right\} $$
(49)

Finally, using Eqs. (48) and (49), we can obtain the covariance matrices \( {\mathbf{R}}_{k}^{fb} \) and \( {\mathbf{R}}_{k}^{bb} \). Based on the assumptions \( E\left\{ {s_{{}}^{p} \left( n \right)\tilde{s}_{{}}^{q} \left( m \right)_{{}}^{*} } \right\} = E\left\{ {\tilde{s}_{{}}^{p} \left( n \right)\tilde{s}_{{}}^{q} \left( m \right)_{{}}^{*} } \right\} = 0 \), we obtain

$$ {\mathbf{R}}_{k}^{fb} = {\mathbf{B}}I_{N} , \quad {\mathbf{B}} = diag\left( {\chi^{1} ,\chi^{2} , \ldots ,\chi^{t} } \right) $$
(50)
$$ {\mathbf{R}}_{k}^{bb} = {\mathbf{Z}}I_{N} ,\quad {\mathbf{Z}} = diag\left( {\zeta^{1} ,\zeta^{2} , \ldots ,\zeta^{t} } \right) $$
(51)

Figure 4 is the algorithm flow diagram.

Fig. 4
figure 4

Algorithm flow diagram

4 Simulation Results

In this section, we provide several simulation examples to demonstrate the performance of the proposed receivers. The number of transmitted antennas and the number of the received antennas are two. We present simulation results for the Turbo and LDPC coded MIMO-OFDM transmission with iterative soft decision feedback equalization (SDFE) over multipath fading channels. Table 3 shows the simulation parameters.

Table 3 Simulation parameters

The turbo coded transmitted binary bits are encoded by a rate \( \gamma = 1/2 \) recursive systematic convolutional (RSC) code with generator matrix in octal notation G = (7, 5)8. The encoder is the recursive convolutional (2,1,2) codes where the last entry denotes the constrained length. The employed decoding MAP algorithm in the convolutional decoder is the Log-MAP algorithm. The encoded bits are interleaved using a random interleaver of size \( J/\gamma \) (J is the number of uncoded bits in one block). The used (w r , w c ) LDPC code is characterized by an m × n binary matrix which is called the parity check matrix H, where w c is the column weight and w r is the row weight. In this paper, we employ the irregular quasi-cyclic (QC) code. The matrix size is 3 × 6, the matrix includes a sub-matrix of 87 × 87 identity matrix. The code rate is \( \gamma = 1/2 \). The LDPC decoding algorithm is the message-passing or belief-propagation (BP) algorithm. For all simulations, the modulation scheme is BPSK, QPSK or 16 QAM, the channel order is five, the number of sub-carriers is N = 64 or 32, the CP length is 16 or 8, the number of iteration is 1–5 and E s  = 1 is the average transmitted symbol energy.

Figure 5 shows the bit error rate performance comparisons with BPSK modulation over Proakis ISI channel [19]. The FFT size is 32 and the CP length is 8. The iteration number is 1 or 4. Figure 5 shows that the performance of the receiver with SDFE structure is better than that with linear equalizer. The performance of SDFE structure with LDPC code is better than that with turbo code. The gain with SDFE structure over linear equalizer is about 2 dB. Figure 6 shows the bit error rate performance comparisons with BPSK modulation over Proakis ISI channel. The FFT size is 64 and the CP length is 16. From the simulation, we find that the performance of the system with larger FFT size and CP length is better than that with smaller ones. This result makes sense since the SDFE combines both soft a priori information and soft decisions of the data symbols for equalization. Figures 7 and 8 show the bit error rate performance comparisons with QPSK modulation over Proakis ISI channel. Figure 7 is with the same FFT size and CP length as those of Fig. 5. Figure 8 is with the same FFT size and CP length as those of Fig. 6. From the results of Figs. 7, 8, we find that the proposed iterative SDFE receiver also works well with complex signal. Figure 9 shows the bit error rate performance comparisons with 16QAM modulation. The proposed iterative SDFE also works with high-order modulation. Figures 10, 11 and 12 show the bit error rate performances of the proposed iterative SDFE with LDPC/turbo coding schemes and different modulation and different iteration number. From these figures, we find that with increasing iteration number, the performance of the proposed receivers can be improved dramatically.

Fig. 5
figure 5

Bit error rate performance comparisons with BPSK modulation, FFT size 32 and CP length 8

Fig. 6
figure 6

Bit error rate performance comparisons with BPSK modulation, FFT size 64 and CP length 16

Fig. 7
figure 7

Bit error rate performance comparisons with QPSK modulation, FFT size 32 and CP length 8

Fig. 8
figure 8

Bit error rate performance comparisons with QPSK modulation, FFT size 64 and CP length 16

Fig. 9
figure 9

Bit error rate performance comparisons with 16QAM modulation, FFT size 64 and CP length 16

Fig. 10
figure 10

Bit error rate performance comparisons with different iterations: Turbo code, BPSK modulation, FFT size 64 and CP length 16

Fig. 11
figure 11

Bit error rate performance comparisons with different iterations: LDPC code, QPSK modulation, FFT size 64 and CP length 16

Fig. 12
figure 12

Bit error rate performance comparisons with different iterations: LDPC code, 16QAM modulation, FFT size 64 and CP length 16

5 Conclusion

In this paper, we propose a FD iterative SDFE scheme for MIMO OFDM communication systems in multi-path fading channels. The proposed iterative SDFE not only combats error propagation but also be suitable for multilevel modulation systems. Under a Gaussian approximation of the a priori information and the SDFE output, the output of an equivalent AWGN channel and expressions for the a posteriori LLR and the extrinsic LLR are derived for multilevel modulation. In addition, we also investigate the iterative receivers by employing irregular LDPC and turbo codes. The interleaver and deinterleaver are not necessary for LDPC code because of their inherent random structure. For the same modulation scheme, we show that the proposed receiver with LDPC code is with performance better than that with turbo code in the MIMO-OFDM systems.