1 Introduction

The emergence of new wireless devices and the ever-escalating demand for voice and data traffic indicates the need to evolve and meet the high capacity and coverage requirements of the wireless network. Cisco Visual Networking Index (VNI) estimates a sevenfold growth in the worldwide data traffic from 2017 to 2020 [1]. This necessitates the 5G wireless communication networks to introduce new technologies to address the high data rate, seamless coverage, high capacity demand with low power consumption, and maintaining QoS. The massive MIMO technology and dense heterogeneous networks (HetNet) have been identified as an inevitable solution to address the unprecedented capacity demand with better energy efficiency [2]. Massive MIMO is a technology where the macro base station is furnished with massive antennas to serve the clients concurrently at the same time and frequency resource. These massive antennas make sure to provide better signal energy focus on the anticipated users which results in high spectral efficiency (SE) and energy efficiency (EE) under realistic scenarios [3,4,5]. Further, the massive MIMO utilizes degrees of freedom and offers services to its clients at the same time and frequency band thus enhancing throughput of the system. The propagation channels corresponding to various users in the massive MIMO network decorrelate when the number of the antenna array is increased [6]. Massive MIMO can smartly utilize the 5G spectrum and it is capable of extending the coverage through high gain beamforming strategy and higher order spatial multiplexing [7]. Massive MIMO operates in time division duplex (TDD) mode to acquire channel state information for the utilization of channel reciprocity [8]. Time division duplex (TDD) mode of operation enables the users to utilize the same time and frequency resource thereby minimizing the feedback overhead and channel training in the downlink [9]. In a dense HetNet with small cell access points (SCAs), are cost-effective low power base stations that can provide capacity and coverage enhancements ensuring QoS. Along with the typical macro base station, enormous small cell access points (SCAs) are deployed to relieve the traffic from the macro base station. Moreover, it could lead to the problem of routing the traffic that arises from SCAs to the backhaul network. Enabling wired backhauling to enormous SCAs is the main drawback to practice in reality [10, 11]. The advancement in massive MIMO technology can bridge the gap in wireless backhauling of SCAs and it is considered as a viable and cost-effective solution [12]. Small cell access points (SCAs) can localize the data traffic thereby minimizing the distance between the clients and transmitters resulting in less propagation loss and enhanced energy efficiency. Besides providing the enhancement in the SE and EE, massive MIMO and network densification are envisioned as prominent technologies of the 5G network and B5G [13]. With the recent trends such as massive MIMO and the network densification in B5G creates a natural concern to reduce the power consumption to attain thousand-fold energy efficiency [14]. Additionally, full-duplex (FD) communication paved its way with the recent developments in self interference cancellation capability via beamforming strategies and shared antenna methodology [15,16,17,18,19]. In Inband FD communication, the nodes are enabled to perform concurrent transmission and reception of information in the same frequency resources. Thus, FD communication attains remarkable enhancement in the SE of the system. Henceforth, the FD technology at the SCA and enabling massive MIMO at the macro base station (MBS) imparts an eminent architecture combining the benefits of aforesaid technologies.

1.1 Related work

The above-mentioned works highlight the effectiveness of massive MIMO systems, HetNet, and FD technology. Therefore, the work improvement achieved by utilizing the aforementioned technologies is discussed. The combination of Massive MIMO and small cell have grabbed significant research interest. In [20], distributed power allocation strategy in HetNet is analyzed and EE assuring a reduced transmission rate is validated. In [21], rate maximization and transmit power minimization subjected to power constraints are jointly addressed in HetNet. It is proved that the EE is improved. However, the above works focused on power optimization in HetNets with small cells but it did not consider the impact of massive MIMO technology on HetNet. In [22], spectral and energy efficiency is maximized by utilizing power allocation and antenna selection strategy in massive MIMO systems. In [23], a scalable power control strategy is analyzed in massive MIMO systems to maximize the SE and to provide maximum fairness among users in the network. In [24], Power optimization in massive MIMO systems with the zero-forcing (ZF) beamforming strategy is validated. The zero-forcing (ZF) beamforming strategy reduces the self interference thereby enhancing the downlink (DL) rate. In [25], the power allocation method considering QoS constraints for massive MIMO systems and the impact of the number of clients and the antennas on EE is also discussed. The above-mentioned works concentrated on power optimization in massive MIMO systems and not on HetNet with small cells. In [26, 27], conceptualize massive MIMO based HetNet scenario is discussed. Here, the massive MIMO utilizes degrees of freedom and offers services to its clients at the same time and frequency band thus enhancing throughput of the system. Further in [28], the network power consumption is greatly enhanced by employing massive MIMO enabled HetNet with small cells. This analysis is done by considering dynamic and static power consumption. Here, MZF beamforming is utilized by the transmitters to serve the users. The downlink (DL) transmit power minimization subjected to QoS constraints at the users is achieved utilizing the MZF beamforming in [28]. In [29], the authors have discussed the eigen beamforming strategy for computing the channel capacity under specific power constraint considering all the antennas. The water filling algorithm is utilized to allocate power in various eigenmodes. It lacks accuracy due to the output restraint of the power amplifier. In [30], the analysis related to complexity has been analyzed and it is observed that the optimal system design can achieve energy and spectral efficiency with reduced complexity. Authors in [8, 19] examined the gain of massive MIMO and small cells together. Here, the power allocation and backhaul problem is not addressed. The FD technology remains as a backhaul solution in massive MIMO enabled HetNet with small cells as stated in [31, 32]. Authors in [18] have developed a self-backhaul strategy where SCAs are enabled with FD technology. As a consequence, SCAs permits the reception of information from the MBS while concurrently communicates the information to its clients in the DL. Similarly, SCAs permit the reception of information from its clients in the uplink (UL) while simultaneously communicates the information to the MBS. It has also manifested the efficacy of the FD self backhaul strategy without considering the interference issue in HetNet. Authors in [33, 34], discussed sum rate maximization and transmit power minimization guaranteeing QoS by incorporating power allocation strategies in FD enabled massive MIMO systems. Here, the backhaul transmit power is not considered in the power allocation strategy. The summary of the related works is presented in Table 1. From the literature, it is observed that extensive research is focused on power optimization in massive MIMO technology and HetNet with small cells individually. Very few of the research work focused on power optimization in massive MIMO enabled HetNet with small cells. Parameters such as sum rates, EE, SE, and fairness are maximized by utilizing a power allocation strategy in the network. Backhauling the massive MIMO enabled HetNet with small cells is an important research issue. Even though a massive MIMO enabled HetNet with small cells could furnish a high data rate between the clients and MBS, the transmissions occurring between the MBS and SCAs results as the bottleneck of the network. Therefore, a reliable backhaul solution is required to enhance the data rate of clients in the network. Thus, enabling FD technology at the SCA is considered as a backhaul solution. FD backhauling allows the transmission and reception of information using the same spectrum. However, power allocation becomes challenging when jointly considering massive MIMO enabled HetNet with FD small cells.

Table 1 Summary of the related works
Full size table

1.2 Contribution of the paper

We propose a HetNet architecture comprising of MBS enabled with massive MIMO overlaying multiple SCAs with FD capability serving macro user equipments (MUEs) and small cell user equipments (SUEs). Here, simultaneous transmissions occur from the MBS to its user (access link) and from the MBS to SCAs (backhaul link) at the same frequency band. The power allocation problem of the MBS and SCAs with FD capability is formulated as a non-convex optimization problem. Then it is reformulated as a convex problem utilizing semi-definite relaxation. Scaled beamforming design and power optimization algorithm is coherently addressed to maximize the system sum rate with reduced power consumption guaranteeing QoS at the clients. The complexity analysis has been performed to compute the optimal system performance.

1.3 Organization of the paper

The remaining sections of the article are organized as follows: Section 2 discusses the system model. Section 3 discusses the joint power optimization and scaled beamforming algorithm. Section 4 illustrates the simulation result analysis and finally, the conclusion is discussed in section 5.

2 System model

The massive MIMO enabled HetNet with a small cell network comprising of one macro base station (MBS) equipped with a number of M antennas and randomly deployed N number of SCAs with a single antenna as depicted in Fig. 1. It is considered that the MBS is capable of providing information to K single-antenna MUEs and every SCA serves one single-antenna SUEs. Here K + N « M, which denotes massive MIMO is used and M can be a large value ranging from 8 to 100 or even more [28]. We assume that all the base stations and user terminals are synchronized and operated on TDD protocol. The variables or representation utilized for deriving the equations are presented as in Table 2.

Fig. 1
figure 1

Massive MIMO enabled HetNet with FD small cells

Full size image
Table 2 Variable/Representation used in the system model
Full size table

2.1 Channel model

In channel modeling, the fading channel matrix is represented as \( \mathrm{D}=\left[{\mathrm{d}}_{k,0}{\mathrm{d}}_{k,0}^H,.\dots, {\mathrm{d}}_{k,n}{d}_{k,n}^H\right] \).Rayleigh flat fading random variable is represented as \( {\mathrm{d}}_{\mathrm{k},0}^{\mathrm{H}}\in {\mathrm{C}}^{\mathrm{M}\times 1} \) and \( {\mathrm{d}}_{\mathrm{k},\mathrm{n}}^{\mathrm{H}}\in {\mathrm{C}}^{\mathrm{N}\times 1} \)

In addition, MBS employs MZF beamforming technique to perfectly eradicate the multiuser interference. The \( {\mathrm{x}}_{\mathrm{k},0}^{\mathrm{m}} \) and \( {\mathrm{x}}_{\mathrm{k},\mathrm{n}}^{\mathrm{s}} \) denotes the data signals at the MBS and the nth SCA serving k user equipments (UE). It is obtained from the Gaussian codebook and it is represented as \( {\mathrm{x}}_{\mathrm{k},\mathrm{n}}^{\mathrm{s}}\sim \mathrm{CN}\left(0,1\right) \) for n = 0,……, N. Then, nk is a complex guassian noise variable and variance is \( {\sigma}_k^2 \) . The beamforming vectors \( {\mathrm{z}}_{\mathrm{k},0}^{\mathrm{m}}\in {\mathbb{C}}^{\mathrm{M}\times 1} \), \( {\mathrm{z}}_{\mathrm{k},\mathrm{n}}^{\mathrm{s}}\in {\mathbb{C}}^{\mathrm{N}\times 1} \) are multiplied with the data signals to acquire the transmitted signals.

$$ {\mathrm{x}}_{\mathrm{n}}=\sum \limits_{\mathrm{k}=1}^{\mathrm{K}}{\mathrm{z}}_{\mathrm{k},\mathrm{n}}^{\mathrm{s}}{\mathrm{x}}_{\mathrm{k},\mathrm{n}}^{\mathrm{s}},\kern1.75em \mathrm{n}=0,.\dots, \mathrm{N} $$
(1)

Here \( {\mathrm{z}}_{\mathrm{k},\mathrm{n}}^{\mathrm{s}}\ne 0, \) and this optimization variable is used to find the solution for ‘n’ transmitters that can provide services to the kth UE. The received signal at the kth MUE which experience the inter-tier interference from SCAs is written as

$$ {\mathrm{y}}_{\mathrm{k}}^{\mathrm{m}}=\sqrt{{\mathrm{P}}_{\mathrm{k}}^{\mathrm{m}}}{\mathrm{d}}_{\mathrm{k}}^{{\mathrm{m}}^{\mathrm{H}}}{\mathrm{z}}_{\mathrm{k}}^{\mathrm{m}}{\mathrm{x}}_{\mathrm{k}}^{\mathrm{m}}+\sum \limits_{\mathrm{n}=1}^{\mathrm{N}}\sqrt{{\mathrm{P}}_{\mathrm{n}}^{\mathrm{s}}}{\mathrm{d}}_{\mathrm{n}\mathrm{k}}^{{\mathrm{s}}^{\mathrm{H}}}{\mathrm{x}}_{\mathrm{n}}^{\mathrm{s}}+{\mathrm{n}}_{\mathrm{k}}^{\mathrm{m}} $$
(2)

The first term in Eq. (2) represents the intended signal received by MUEs from the MBS and the subsequent term indicates the cross-tier interference from SCAs. The received signal-to-interference-plus-noise ratio (SINR) of kth MUE is derived using the Eq. (2) and it is as follows:

$$ \mathrm{SIN}{\mathrm{R}}_{\mathrm{k}}^{\mathrm{m}}=\frac{{\mathrm{P}}_{\mathrm{k}}^{\mathrm{m}}{\left\Vert {\mathrm{d}}_{\mathrm{k}}^{{\mathrm{m}}^{\mathrm{H}}}{\mathrm{z}}_{\mathrm{k}}^{\mathrm{m}}\right\Vert}^2}{\sum_{\mathrm{n}=1}^{\mathrm{N}}{\mathrm{P}}_{\mathrm{n}}^{\mathrm{s}}{{\mathrm{d}}_{\mathrm{n}\mathrm{k}}^{{\mathrm{s}}^{\mathrm{H}}}}^2+{\upsigma}_{\mathrm{k}}^2} $$
(3)

SCAs acquire information from the MBS and concurrently transmit the information to SUEs utilizing the same frequency resource. Self-interference from itself and co-tier interference from other SCAs are the commonly occurring interference at the SCAs. Thus, the received signal at the nth SCA is written as

$$ {\mathrm{y}}_{\mathrm{n}}^{\mathrm{b}}=\sqrt{{\mathrm{P}}_{\mathrm{n}}^{\mathrm{b}}}\ {\mathrm{d}}_{\mathrm{n}}^{{\mathrm{b}}^{\mathrm{H}}}{\mathrm{z}}_{\mathrm{n}}^{\mathrm{b}}{\mathrm{x}}_{\mathrm{n}}^{\mathrm{b}}+\sum \limits_{{\mathrm{n}}^{\prime }=1,\ne \mathrm{n}}^{\mathrm{N}}\sqrt{{\mathrm{P}}_{{\mathrm{n}}^{\prime}}^{\mathrm{s}}}\ {\mathrm{d}}_{{\mathrm{n}}^{\prime}\mathrm{n}}^{{\mathrm{b}}^{\mathrm{H}}}{\mathrm{x}}_{{\mathrm{n}}^{\prime}}^{\mathrm{s}}+\sqrt{\upgamma {\mathrm{P}}_{\mathrm{n}}^{\mathrm{s}}}\ {\mathrm{x}}_{\mathrm{n}}^{\mathrm{s}}+{\mathrm{n}}_{\mathrm{n}}^{\mathrm{b}} $$
(4)

The first term in Eq. (4) represents the intended signal, the subsequent term indicates the co-tier interference and the third term indicates the self-interference and self-interference cancellation methods are utilized to evaluate the value of γ. The investigation carried out in this work is a common case. Hence, the received SINR at the nth SCA is represented as

$$ \mathrm{SIN}{\mathrm{R}}_{\mathrm{n}}^{\mathrm{b}}=\frac{{\mathrm{P}}_{\mathrm{n}}^{\mathrm{b}}{\left\Vert {\mathrm{d}}_{\mathrm{n}}^{{\mathrm{b}}^{\mathrm{H}}}{\mathrm{z}}_{\mathrm{n}}^{\mathrm{b}}\right\Vert}^2}{\sum_{{\mathrm{n}}^{\prime }=1,\ne \mathrm{n}}^{\mathrm{N}}{\mathrm{P}}_{{\mathrm{n}}^{\prime}}^{\mathrm{s}}{{\mathrm{d}}_{{\mathrm{n}}^{\prime}\mathrm{n}}^{{\mathrm{b}}^{\mathrm{H}}}}^2+\upgamma {\mathrm{P}}_{\mathrm{n}}^{\mathrm{s}}+{\upsigma}_{\mathrm{k}}^2} $$
(5)

The SUEs suffer from co-tier interference due to MBS and cross-tier interference due to SCAs. Thus, the received signal at the nth SCA user can be written as

$$ {\displaystyle \begin{array}{c}{\mathrm{y}}_{\mathrm{n}}^{\mathrm{s}}=\sqrt{{\mathrm{P}}_{\mathrm{n}}^{\mathrm{s}}}{\mathrm{d}}_{\mathrm{n}}^{{\mathrm{s}}^{\mathrm{H}}}{\mathrm{x}}_{\mathrm{n}}^{\mathrm{s}}+\sum \limits_{{\mathrm{n}}^{\prime }=1,\ne \mathrm{n}}^{\mathrm{N}}\sqrt{{\mathrm{P}}_{{\mathrm{n}}^{\prime}}^{\mathrm{s}}}\ {\mathrm{d}}_{{\mathrm{n}}^{\prime}\mathrm{n}}^{{\mathrm{s}}^{\mathrm{H}}}{\mathrm{x}}_{{\mathrm{n}}^{\prime}}^{\mathrm{s}}+\sum \limits_{\mathrm{k}=1}^{\mathrm{K}}\sqrt{{\mathrm{P}}_{\mathrm{k}}^{\mathrm{m}}}\ {\mathrm{d}}_{\mathrm{n}}^{{\mathrm{m}}^{\mathrm{H}}}{\mathrm{z}}_{\mathrm{k}}^{\mathrm{m}}{\mathrm{x}}_{\mathrm{k}}^{\mathrm{m}},\\ {}\kern3.75em +\sum \limits_{{\mathrm{n}}^{\prime }=1}^{\mathrm{N}}\sqrt{{\mathrm{P}}_{{\mathrm{n}}^{\prime}}^{\mathrm{b}}}{\mathrm{d}}_{\mathrm{n}}^{{\mathrm{m}}^{\mathrm{H}}}{\mathrm{z}}_{{\mathrm{n}}^{\prime}}^{\mathrm{b}}{\mathrm{x}}_{{\mathrm{n}}^{\prime}}^{\mathrm{b}}+{\mathrm{n}}_{\mathrm{n}}^{\mathrm{s}}\end{array}} $$
(6)

The first term in the above Eq. (6) denotes the intended signal, the subsequent term indicates the co-tier interference from SCAs, the third and fourth terms indicates the cross-tier interference from the MBS. Therefore, the received SINR at the nth SCA user is expressed as

$$ \mathrm{SIN}{\mathrm{R}}_{\mathrm{n}}^{\mathrm{s}}=\frac{{\mathrm{P}}_{\mathrm{n}}^{\mathrm{s}}{{\mathrm{d}}_{\mathrm{n}}^{{\mathrm{s}}^{\mathrm{H}}}}^2}{\sum \limits_{\mathrm{k}=1}^{\mathrm{K}}{\mathrm{P}}_{\mathrm{k}}^{\mathrm{m}}{\left\Vert {\mathrm{d}}_{\mathrm{n}}^{{\mathrm{m}}^{\mathrm{H}}}{\mathrm{z}}_{\mathrm{k}}^{\mathrm{m}}\right\Vert}^2+\sum \limits_{{\mathrm{n}}^{\prime }=1}^{\mathrm{N}}{\mathrm{P}}_{{\mathrm{n}}^{\prime}}^{\mathrm{b}}{\left\Vert {\mathrm{d}}_{\mathrm{n}}^{{\mathrm{m}}^{\mathrm{H}}}{\mathrm{z}}_{{\mathrm{n}}^{\prime}}^{\mathrm{b}}\right\Vert}^2+{\sum}_{{\mathrm{n}}^{\prime }=1,\ne \mathrm{n}}^{\mathrm{N}}{\mathrm{P}}_{{\mathrm{n}}^{\prime}}^{\mathrm{s}}{{\mathrm{d}}_{{\mathrm{n}}^{\prime}\mathrm{n}}^{{\mathrm{s}}^{\mathrm{H}}}}^2+{\upsigma}_{\mathrm{k}}^2} $$
(7)

3 Joint power optimization and scaled beamforming algorithm

In this section, problem realization is done and the beamforming algorithm with less complexity is discussed. The main objective is to attain power optimization while sustaining QoS. The QoS constraints denote the data rate [bits/s/Hz] that every UE must attain in parallel. It is denoted as log2(1 + SINRk) ≥ γk, where γk is known as the fixed QoS target. The aggregate SINR of the kth UE assuming the perfect channel estimation can be represented as

$$ \mathrm{SIN}{\mathrm{R}}_{\mathrm{k}}=\frac{{\left|{\mathrm{d}}_{\mathrm{k},0}^{\mathrm{H}}{\mathrm{z}}_{\mathrm{k},0}\right|}^2+{\sum}_{\mathrm{n}=1}^{\mathrm{N}}{\left|{\mathrm{d}}_{\mathrm{k},\mathrm{n}}^{\mathrm{H}}{\mathrm{z}}_{\mathrm{k},\mathrm{n}}\right|}^2}{\sum \limits_{\mathrm{i}=1;\mathrm{i}\ne \mathrm{k}}^{\mathrm{K}}\left({\left|{\mathrm{d}}_{\mathrm{k},0}^{\mathrm{H}}{\mathrm{z}}_{\mathrm{i},0}\right|}^2+{\sum}_{\mathrm{n}=1}^{\mathrm{N}}{\left|{\mathrm{d}}_{\mathrm{k},\mathrm{n}}^{\mathrm{H}}{\mathrm{z}}_{\mathrm{i},\mathrm{n}}\right|}^2\right)+{\upsigma}_{\mathrm{k}}^2} $$
(8)

The total power consumed per subcarrier is denoted as Pd + Ps where Pd, Ps are dynamic and static power [28].

$$ {\mathrm{P}}_{\mathrm{d}}={\uprho}_{\mathrm{o}}{\sum}_{\mathrm{k}=1}^{\mathrm{K}}{\left\Vert {\mathrm{z}}_{\mathrm{k},0}\right\Vert}^2+{\sum}_{\mathrm{n}=1}^{\mathrm{N}}{\uprho}_{\mathrm{n}}{\sum}_{\mathrm{k}=1}^{\mathrm{K}}{\left\Vert {\mathrm{z}}_{\mathrm{k},\mathrm{n}}\right\Vert}^2 $$
(9)
$$ {\mathrm{P}}_{\mathrm{s}}=\frac{\upeta_{\mathrm{o}}}{\mathrm{C}}\mathrm{M}+{\sum}_{\mathrm{n}=1}^{\mathrm{N}}\frac{\upeta_{\mathrm{n}}}{\mathrm{C}}\mathrm{N} $$
(10)

Here, C denotes the number of subcarriers, ρo, ρn signifies the efficiency of power amplifiers at the MBS and SCA, while ηo, ηn denotes the circuit power per antenna at the MBS and SCA. Therefore, the power optimization while sustaining QoS is obtained by

$$ {\displaystyle \begin{array}{l}\underset{{\mathrm{z}}_{\mathrm{k},\mathrm{n}}\forall \mathrm{k},\mathrm{n}}{\operatorname{minimize}}{\mathrm{P}}_{\mathrm{d}}+{\mathrm{P}}_{\mathrm{s}}\\ {}\mathrm{subject}\ \mathrm{to}\ {\log}_2\left(1+\mathrm{SIN}{\mathrm{R}}_{\mathrm{k}}\right)\ge {\upgamma}_{\mathrm{k}}\forall \mathrm{k}\\ {}{\sum}_{\mathrm{k}=1}^{\mathrm{K}}{\mathrm{z}}_{\mathrm{k},\mathrm{n}}^{\mathrm{H}}{\mathrm{z}}_{\mathrm{k},\mathrm{n}}\le {\mathrm{q}}_{\mathrm{n}}\forall \mathrm{n}\end{array}} $$
(11)

The QoS constraints in the above-mentioned Eq. (11) make it a non-convex problem and it can be reformulated as a convex problem utilizing semi-definite relaxation. To achieve a convex semi-definite problem formulation from Eq. (11), the matrix is expected to be positive semi-definite and it is represented as \( {\mathbb{Z}}_{\mathrm{k},\mathrm{n}}={\mathrm{z}}_{\mathrm{k},\mathrm{n}}^{\mathrm{H}}{\mathrm{z}}_{\mathrm{k},\mathrm{n}}\forall \mathrm{n} \) and the matrix rank is less than or equal to 1 i.e., rank \( \left({\mathbb{Z}}_{\mathrm{k},\mathrm{n}}\right)\le 1 \). Rank could be zero which implicates \( {\mathbb{Z}}_{\mathrm{k},\mathrm{n}}=0 \). Hence, by including MBS and SCAs, the expression is given as

$$ {\displaystyle \begin{array}{l}\underset{{\mathrm{z}}_{\mathrm{k},\mathrm{n}}\ge 0\forall \mathrm{k},\mathrm{n}}{\operatorname{minimize}}\sum \limits_{\mathrm{n}=0}^{\mathrm{N}}{\uprho}_{\mathrm{n}}\sum \limits_{\mathrm{k}=1}^{\mathrm{K}}\mathrm{tr}\left({\mathbb{Z}}_{\mathrm{k},\mathrm{n}}\right)+{\mathrm{P}}_{\mathrm{s}}\\ {}\mathrm{subject}\ \mathrm{to}\kern1.5em \operatorname{rank}\left({\mathbb{Z}}_{\mathrm{k},\mathrm{n}}\right)\le 1\forall \mathrm{k},\mathrm{n}\\ {}\sum \limits_{\mathrm{n}=0}^{\mathrm{N}}\left({\mathrm{d}}_{\mathrm{k},\mathrm{n}}^{\mathrm{H}}\right)\left(\left(1+\frac{1}{{\overset{\sim }{\upgamma}}_{\mathrm{k}}}\right){\mathbb{Z}}_{\mathrm{k},\mathrm{n}}-\sum \limits_{\mathrm{k}=1}^{\mathrm{K}}{\mathbb{Z}}_{\mathrm{i},\mathrm{n}}\right){\mathrm{d}}_{\mathrm{k},\mathrm{n}}\ge {\upsigma}_{\mathrm{k}}^2\forall \mathrm{k},\\ {}{\sum}_{\mathrm{k}=1}^{\mathrm{K}}\mathrm{tr}\left({\mathbb{Z}}_{\mathrm{k},\mathrm{n}}\right)\le {\mathrm{q}}_{\mathrm{n}}\forall \mathrm{n}\end{array}} $$
(12)

Here, qn and \( {\sigma}_k^2 \) denotes per-antenna constraints at the transmission node and variance. Then, the QoS target is transformed into SINR targets which are as follows.

The QoS target is given as log2(1 + SINRk) ≥ γk ∀ k ,

After applying antilog on both sides, we get \( \mathrm{SIN}{\mathrm{R}}_{\mathrm{k}}\ge {2}^{\upgamma_{\mathrm{k}}}-1\forall \mathrm{k} \).

Substituting the SINRk value from Eq. (8) in the above-mentioned equation as

$$ \frac{{\left|{\mathrm{d}}_{\mathrm{k},0}^{\mathrm{H}}{\mathrm{z}}_{\mathrm{k},0}\right|}^2+{\sum}_{\mathrm{n}=1}^{\mathrm{N}}{\left|{\mathrm{d}}_{\mathrm{k},\mathrm{n}}^{\mathrm{H}}{\mathrm{z}}_{\mathrm{k},\mathrm{n}}\right|}^2}{\sum \limits_{\mathrm{i}=1;\mathrm{i}\ne \mathrm{k}}^{\mathrm{K}}\left({\left|{\mathrm{d}}_{\mathrm{k},0}^{\mathrm{H}}{\mathrm{z}}_{\mathrm{i},0}\right|}^2+{\sum}_{\mathrm{n}=1}^{\mathrm{N}}{\left|{\mathrm{d}}_{\mathrm{k},\mathrm{n}}^{\mathrm{H}}{\mathrm{z}}_{\mathrm{i},\mathrm{n}}\right|}^2\right)+{\upsigma}_{\mathrm{k}}^2}\ge {2}^{\upgamma_{\mathrm{k}}}-1 $$
(13a)

The above equation can be rewritten considering MBS and SCAs

$$ \frac{\sum_{\mathrm{n}=0}^{\mathrm{N}}{\left|{\mathrm{d}}_{\mathrm{k},\mathrm{n}}^{\mathrm{H}}{\mathrm{z}}_{\mathrm{k},\mathrm{n}}\right|}^2}{\sum \limits_{\mathrm{i}=1;\mathrm{i}\ne \mathrm{k}}^{\mathrm{K}}\left({\left|{\mathrm{d}}_{\mathrm{k},0}^{\mathrm{H}}{\mathrm{z}}_{\mathrm{i},0}\right|}^2+{\sum}_{\mathrm{n}=1}^{\mathrm{N}}{\left|{\mathrm{d}}_{\mathrm{k},\mathrm{n}}^{\mathrm{H}}{\mathrm{z}}_{\mathrm{i},\mathrm{n}}\right|}^2\right)+{\upsigma}_{\mathrm{k}}^2}\ge {2}^{\upgamma_{\mathrm{k}}}-1 $$
(13b)
$$ \frac{\sum_{\mathrm{n}=0}^{\mathrm{N}}{\left|{\mathrm{d}}_{\mathrm{k},\mathrm{n}}^{\mathrm{H}}{\mathrm{z}}_{\mathrm{k},\mathrm{n}}\right|}^2}{2^{\upgamma_{\mathrm{k}}}-1}\ge \sum \limits_{\mathrm{i}=1;\mathrm{i}\ne \mathrm{k}}^{\mathrm{K}}\left({\left|{\mathrm{d}}_{\mathrm{k},0}^{\mathrm{H}}{\mathrm{z}}_{\mathrm{i},0}\right|}^2+{\sum}_{\mathrm{n}=1}^{\mathrm{N}}{\left|{\mathrm{d}}_{\mathrm{k},\mathrm{n}}^{\mathrm{H}}{\mathrm{z}}_{\mathrm{i},\mathrm{n}}\right|}^2\right)+{\upsigma}_{\mathrm{k}}^2 $$
(13c)
$$ \frac{\sum_{\mathrm{n}=0}^{\mathrm{N}}{\left|{\mathrm{d}}_{\mathrm{k},\mathrm{n}}^{\mathrm{H}}{\mathrm{z}}_{\mathrm{k},\mathrm{n}}\right|}^2}{2^{\upgamma_{\mathrm{k}}}-1}\ge {\sum}_{\mathrm{n}=0}^{\mathrm{N}}\left({\left|{\mathrm{d}}_{\mathrm{k},\mathrm{n}}^{\mathrm{H}}{\mathrm{z}}_{\mathrm{k},\mathrm{n}}\right|}^2\right)+{\sum}_{\mathrm{i}=1}^{\mathrm{K}}\left({\sum}_{\mathrm{n}=0}^{\mathrm{N}}{\left|{\mathrm{d}}_{\mathrm{k},\mathrm{n}}^{\mathrm{H}}{\mathrm{z}}_{\mathrm{i},\mathrm{n}}\right|}^2\right)+{\upsigma}_{\mathrm{k}}^2 $$
(13d)
$$ \frac{\sum_{\mathrm{n}=0}^{\mathrm{N}}{\left|{\mathrm{d}}_{\mathrm{k},\mathrm{n}}^{\mathrm{H}}{\mathrm{z}}_{\mathrm{k},\mathrm{n}}\right|}^2}{2^{\upgamma_{\mathrm{k}}}-1}-{\sum}_{\mathrm{n}=0}^{\mathrm{N}}\left({\left|{\mathrm{d}}_{\mathrm{k},\mathrm{n}}^{\mathrm{H}}{\mathrm{z}}_{\mathrm{k},\mathrm{n}}\right|}^2\right)-{\sum}_{\mathrm{i}=1}^{\mathrm{K}}\left({\sum}_{\mathrm{n}=0}^{\mathrm{N}}{\left|{\mathrm{d}}_{\mathrm{k},\mathrm{n}}^{\mathrm{H}}{\mathrm{z}}_{\mathrm{i},\mathrm{n}}\right|}^2\right)\ge {\upsigma}_{\mathrm{k}}^2 $$
(13e)
$$ {\sum}_{\mathrm{n}=0}^{\mathrm{N}}{\mathrm{d}}_{\mathrm{k},\mathrm{n}}^{\mathrm{H}}\left(\left(1+\frac{1}{2^{\upgamma_{\mathrm{k}}}-1}\right){\mathbb{Z}}_{\mathrm{k},\mathrm{n}}-\sum \limits_{\mathrm{i}=1}^{\mathrm{K}}{\mathbb{Z}}_{\mathrm{i},\mathrm{n}}\right)\ {\mathrm{d}}_{\mathrm{k},\mathrm{n}}\ge {\upsigma}_{\mathrm{k}}^2 $$
(13f)

where QoS target is transformed to SINR target which is given by,\( {\tilde{\gamma}}_k={2}^{\gamma_k}-1\forall k \).

Equation (13a) to (13f) illustrate the procedure to convert non-convex optimization problem to convex optimization problem (i.e) to convert Eqs. (11) to (12). Now, Eq. (12) indicates the convex problem excluding the rank constraints. Yet optimality could be attained by relaxing the constraints. As in [28], \( \left\{{\mathbb{Z}}_{\mathrm{k},\mathrm{n}}^{\ast}\forall \mathrm{k},\mathrm{n}\right\} \)is considered to be the optimal solution to Eq. (12) and every UE has three choices to attain optimality: \( {\mathbb{Z}}_{k,n}^{\ast }=0,1\le n\le N; \)(when the MBS only serves the UE), \( {\mathbb{Z}}_{\mathrm{k},0}^{\ast }=0\ \mathrm{and}\ {\mathbb{Z}}_{\mathrm{k},\mathrm{n}}^{\ast }=0\ \mathrm{for}\ \mathrm{i}\ne \mathrm{n} \)(when the UE is served only by the nth SCA point), \( {\sum}_{\mathrm{k}=1}^{\mathrm{K}}\mathrm{tr}\left({\mathbb{Z}}_{\mathrm{k},\mathrm{n}}^{\ast}\right)={\mathrm{q}}_{\mathrm{n}} \)(When the UE is served by a combination of MBS and SCAs). The convex optimization problem could provide an optimal solution in polynomial time using standard algorithms. To obtain optimal results, every UE is assigned only one transmitter even though UEs could be aided by multi-flow transmission. UEs which are very near to SCAs are merely served by it and the remaining UEs are served by the MBS. The transition areas surrounding SCA could not able to satisfy the QoS targets, so it leads to a change in the transition areas which has a multi-flow transmission. Thus, it minimizes transmission complexity.

3.1 Scaled beamforming approach

The optimal beamforming related to spatial soft-cell has moderate complexity but this becomes impracticable during the real time implementation with a large number of M and N. The centralized algorithm can be acquired utilizing primal or dual decomposition methods. It becomes non-viable to practice in reality since it requires iterative signaling of the parameters. Multi-flow ZF beamforming is less complex and non-iterative. When n = 0…N at the transmitter, the following equation is calculated.

$$ {\mathrm{u}}_{\mathrm{k},\mathrm{n}}=\frac{{\left({\sum}_{\mathrm{i}=1}^{\mathrm{K}}\frac{1}{\upsigma_{\mathrm{i}}^2}\ {\mathrm{d}}_{\mathrm{i},\mathrm{n}}{\mathrm{d}}_{\mathrm{i},\mathrm{n}}^{\mathrm{H}}+\frac{\mathrm{K}}{{\overset{\sim }{\upgamma}}_{\mathrm{k}}{\mathrm{q}}_{\mathrm{n}}}\mathrm{I}\right)}^{-1}{\mathrm{d}}_{\mathrm{k},\mathrm{n}}}{\left\Vert {\left({\sum}_{\mathrm{i}=1}^{\mathrm{K}}\frac{1}{\upsigma_{\mathrm{i}}^2}{\mathrm{d}}_{\mathrm{i},\mathrm{n}}{\mathrm{d}}_{\mathrm{i},\mathrm{n}}^{\mathrm{H}}\kern0.5em +\frac{\mathrm{K}}{{\overset{\sim }{\upgamma}}_{\mathrm{k}}{\mathrm{q}}_{\mathrm{n}}}\mathrm{I}\right)}^{-1},{\mathrm{d}}_{\mathrm{k},\mathrm{n}}\right\Vert}\forall \mathrm{k}, $$
(14)

Here the MBS resolves the convex optimization problem.

$$ {\displaystyle \begin{array}{c}\begin{array}{c}\operatorname{minimize}\\ {}{\mathrm{P}}_{\mathrm{k},\mathrm{n}}\ge 0\forall \mathrm{k},\mathrm{n}\end{array}{\sum}_{\mathrm{n}=0}^{\mathrm{N}}{\uprho}_{\mathrm{n}}{\sum}_{\mathrm{k}=1}^{\mathrm{K}}\left({\mathrm{P}}_{\mathrm{k},\mathrm{n}}\right)+{\mathrm{P}}_{\mathrm{s}}\\ {}\mathrm{subject}\ \mathrm{to}\kern2em \sum \limits_{\mathrm{k}=1}^{\mathrm{K}}{\mathrm{P}}_{\mathrm{k},\mathrm{n}}\le {\mathrm{q}}_{\mathrm{k},\mathrm{n}}\forall \mathrm{n},\\ {}\sum \limits_{\mathrm{n}=0}^{\mathrm{N}}{\mathrm{P}}_{\mathrm{k},\mathrm{n}}\left(1+\frac{1}{{\overset{\sim }{\upgamma}}_{\mathrm{k}}}\right)-\sum \limits_{\mathrm{i}=1}^{\mathrm{K}}{\mathrm{P}}_{\mathrm{k},\mathrm{n}}\ge {\upsigma}_{\mathrm{k}}^2\forall \mathrm{k}\end{array}} $$
(15)

The allotment of power (Pk, n)∗ ∀ k provides a solution for Eq. (15) at the nth SCA and the calculation is as follows

$$ {\mathrm{z}}_{\mathrm{k},\mathrm{n}}={\left(\sqrt{{\mathrm{P}}_{\mathrm{k},\mathrm{n}}}\right)}^{\ast }{\mathrm{u}}_{\mathrm{k},\mathrm{n}}\forall \mathrm{k} $$
(16)

When ZF beamforming is applied in the above algorithm, the problem could be formulated as a power allocation issue which is represented in Eq. (15). It is less complex regardless of the massive antennas present in the base station. Since this algorithm does not work iteratively, only a few scalar parameters are varied among MBS and SCA to sustain the coordination. The UEs who are in the proximity of SCA are affected.

The conventional ZF technique nullifies the cross-tier interference of the users who are present in the cell edges. The system complexity is increased to nullify the cross-tier interference. Even though the cross-tier interference disappears, thermal noise is present. Therefore, it is limited and comparable in some aspects with that of thermal noise. Thus the complexity level is reduced by relaxing the interference constraints, and the number of antennas can be increased for providing a larger rate when compared to the ZF method.

A less complex optimal beamforming algorithm is achieved through scaling the noise by a factor of ψi, n while sustaining the desired signal power. The non-iterative and less complex ZF beamforming serves as a source for the scaled beamforming algorithm. Here power optimization is accomplished with less complexity by scaling the interference noise power which is in the unit norm as in Eq. (17) and it is written as

$$ {\overset{\sim }{\mathrm{u}}}_{\mathrm{k},\mathrm{n}}=\frac{{\left({\sum}_{\mathrm{i}=1}^{\mathrm{K}}\frac{1}{\upsigma_{\mathrm{i}}^2}{\mathrm{d}}_{\mathrm{i},\mathrm{n}}{\mathrm{d}}_{\mathrm{i},\mathrm{n}}^{\mathrm{H}}+\left({\uppsi}_{\mathrm{i},\mathrm{n}}\right)\frac{\mathrm{K}}{{\overset{\sim }{\upgamma}}_{\mathrm{k}}{\mathrm{q}}_{\mathrm{n}}}\mathrm{I}\right)}^{-1}{\mathrm{d}}_{\mathrm{k},\mathrm{n}}}{\left\Vert {\left({\sum}_{\mathrm{i}=1}^{\mathrm{K}}\frac{1}{\upsigma_{\mathrm{i}}^2}{\mathrm{d}}_{\mathrm{i},\mathrm{n}}{\mathrm{d}}_{\mathrm{i},\mathrm{n}}^{\mathrm{H}}+\left({\uppsi}_{\mathrm{i},\mathrm{n}}\right)\frac{\mathrm{K}}{{\overset{\sim }{\upgamma}}_{\mathrm{k}}{\mathrm{q}}_{\mathrm{n}}}\mathrm{I}\right)}^{-1}{\mathrm{d}}_{\mathrm{k},\mathrm{n}}\right\Vert}\forall \mathrm{k}, $$
(17)

Where \( {\uppsi}_{\mathrm{i},\mathrm{n}}=\left(\mathrm{M}-1\right)+\left(\mathrm{i}-\mathrm{n}\right),\kern0.5em \mathsf{i}=\mathsf{1},.\dots, \mathsf{K}\kern0.5em \mathsf{and}\ \mathsf{n}=\mathsf{1},.\dots, \mathsf{N} \)

The allotment of power (Pk, n)∗ ∀ k provides a solution for Eq. (15) at the nth SCA and the calculation is as follows

$$ {\mathrm{z}}_{\mathrm{k},\mathrm{n}}={\left(\sqrt{{\mathrm{P}}_{\mathrm{k},\mathrm{n}}}\right)}^{\ast }{\overset{\sim }{\mathrm{u}}}_{\mathrm{k},\mathrm{n}}\forall \mathrm{k} $$
(18)

3.2 Power optimization algorithm

In massive MIMO enabled HetNet with FD small cell network interference between the network access and backhaul links is considered to be limiting factor which can hinder the system performance. Initially, beamforming is designed at the base stations to optimize the power consumption under QoS constraints. Then the power optimization is carried out through the power allocation algorithm. The power allocation (PA) algorithm is formulated for the access and backhaul part of the network to maximize the sum rates of users. Furthermore, we have considered the backhaul capacity to ensure the efficient allocation of power resources. The main aim is to calculate the power required for transmission by MBS and SCAs assuring peak power constraints.

In the DL, SCAs permits the reception of information from the MBS while concurrently communicates the information to its clients at the same time and frequency band. Similarly, SCAs permit the reception of information from its clients in the UL while simultaneously communicates the information to the MBS. Here, in this study, SCAs are considered as special MUEs. In the DL, the MBS transmits information to MUEs and SCAs, and SCAs transfer the information to their clients concurrently in the same frequency band. In the UL, the SCAs receive information from their clients and transmit them to the MBS, and the MBS receives information from MUEs and SCAs simultaneously in the same frequency band. In massive MIMO enabled HetNet with FD small cells, the transmission or reception of the access and backhaul of SCAs is possible in the same frequency and time similar to the transmission or reception of information to the SUEs and MUEs.The main aim is to maximize the data rate of access part of the network while considering the backhaul capacity limits. The power optimisation problem is formulated as the maximization of the sum of user data rates.

$$ \mathrm{Data}\ \mathrm{Rate}\ \mathrm{of}\ \mathrm{MBS}:\kern2.00em {\mathrm{R}}_{\mathrm{k}}^{\mathrm{m}}=\mathrm{B}{\log}_2\left(1+\mathrm{SIN}{\mathrm{R}}_{\mathrm{k}}^{\mathrm{m}}\right) $$
(19)
$$ \mathrm{Date}\ \mathrm{Rate}\ \mathrm{of}\ \mathrm{Backhaul}:\kern2.00em {\mathrm{R}}_{\mathrm{n}}^{\mathrm{b}}=\mathrm{Blo}{\mathrm{g}}_2\left(1+\mathrm{SIN}{\mathrm{R}}_{\mathrm{n}}^{\mathrm{b}}\right) $$
(20)
$$ \mathrm{Data}\ \mathrm{rate}\ \mathrm{of}\ \mathrm{SCAs}:\kern2.00em {\mathrm{R}}_{\mathrm{n}}^{\mathrm{s}}=\mathrm{Blo}{\mathrm{g}}_2\left(1+\mathrm{SIN}{\mathrm{R}}_{\mathrm{n}}^{\mathrm{s}}\right) $$
(21)

The total data rate R is derived from SCAs, MUEs, SUEs represented by \( {\mathrm{R}}_{\mathrm{n}}^{\mathrm{b}},{\mathrm{R}}_{\mathrm{k}}^{\mathrm{m}},{\mathrm{R}}_{\mathrm{n}}^{\mathrm{s}} \) is formulated as

$$ \mathrm{R}\left(\mathrm{P}\right)=\sum \limits_{\mathrm{k}=1}^{\mathrm{K}}{\mathrm{R}}_{\mathrm{k}}^{\mathrm{m}}+\sum \limits_{\mathrm{n}=1}^{\mathrm{N}}{\mathrm{R}}_{\mathrm{n}}^{\mathrm{s}}+\sum \limits_{\mathrm{n}=1}^{\mathrm{N}}{\mathrm{R}}_{\mathrm{n}}^{\mathrm{b}} $$
(22)

The power allocation strategy of the MBS and SCAs is written as \( \mathrm{P}=\left[{\mathrm{P}}_1^{\mathrm{m}},.\dots, {\mathrm{P}}_{\mathrm{K}}^{\mathrm{m}},{\mathrm{P}}_1^{\mathrm{b}},.\dots, {\mathrm{P}}_{\mathrm{N}}^{\mathrm{b}},{\mathrm{P}}_1^{\mathrm{s}},.\dots, {\mathrm{P}}_{\mathrm{N}}^{\mathrm{s}}\right]. \)The Power allocation (PA) strategy adopted by the MBS and SCA to enhance the sum rate can be achieved by using

$$ {\displaystyle \begin{array}{c}\begin{array}{c}\max \\ {}\mathrm{P}\end{array}\ \mathrm{R}\left(\mathrm{P}\right)\\ {}\begin{array}{c}\mathrm{s}.\mathrm{t}.\kern1em {\mathrm{S}}_1:{\mathrm{R}}_{\mathrm{n}}^{\mathrm{s}}\le {\mathrm{R}}_{\mathrm{n}}^{\mathrm{b}},\forall \mathrm{n}\\ {}{\mathrm{S}}_2:{\sum}_{\mathrm{k}=1}^{\mathrm{K}}{\mathrm{P}}_{\mathrm{k}}^{\mathrm{m}}\le {\mathrm{P}}_{\mathrm{m}\mathrm{ax}}^{\mathrm{m}}\\ {}{\mathrm{S}}_3:{\mathrm{P}}_{\mathrm{n}}^{\mathrm{s}}\le {\mathrm{P}}_{\mathrm{m}\mathrm{ax}}^{\mathrm{s}},\forall \mathrm{n}\\ {}{\mathrm{S}}_4:{\mathrm{P}}_{\mathrm{n}}^{\mathrm{b}}\le {\mathrm{P}}_{\mathrm{m}\mathrm{ax}}^{\mathrm{b}},\forall \mathrm{n}\\ {}{\mathrm{S}}_5:{\mathrm{R}}_{\mathrm{k}}^{\mathrm{m}}\ge {\mathrm{R}}_{\mathrm{m}\mathrm{in}},\forall \mathrm{k}\\ {}{\mathrm{S}}_6:{\mathrm{R}}_{\mathrm{n}}^{\mathrm{s}}\ge {\mathrm{R}}_{\mathrm{m}\mathrm{in}},\forall \mathrm{n}\end{array}\end{array}} $$
(23)

Where S1 denotes that the SCAs backhaul downlink rate should not be less than SCAs access downlink rate to assure QoS. Here, S2, S3 and S4 specifies the transmit power constraint of the MBS, SCA, and backhaul link. The \( {\mathrm{P}}_{\mathrm{m}\mathrm{ax}}^{\mathrm{m}} \) and \( {\mathrm{P}}_{\mathrm{max}}^{\mathrm{s}} \) represents the maximum transmit power of the MBS and the SCAs respectively. S5 and S6 denotes the minimum data rate requirement or QoS requisite of MUEs and SUEs.

The Eq. (23) denotes non-linear coupling among the optimization parameters. It is computationally intractable to find the optimal solution. So, a low complexity algorithm can be developed to compute the optimal solution. The coupled problem is solved by dividing it into small subproblems utilising eigenvalue decomposition method. In this power allocation algorithm, maximum transmit power is assigned in the MBS and the backhaul station i.e., SCA. The transmit power is reduced until the data rate difference becomes minimal. Pkis computed for all users in the network using the Eq. (24). The optimal power allocation according to the water filling algorithm is utilized to enhance the data rates in the access links by assuming the interference that occurs at backhaul as noise.

$$ {\mathrm{P}}_{\mathrm{k}}=\Big\{{\displaystyle \begin{array}{c}{\upnu}_{\mathrm{k}}\mathrm{B}-\frac{\mathrm{K}}{{\mathrm{A}}_{\mathrm{k}}\left(\mathrm{N}-\mathrm{K}\right)},\kern2em \mathrm{if}\kern0.5em {\mathrm{A}}_{\mathrm{k}}>\frac{1}{\upnu_{\mathrm{k}}}\\ {}0\kern0.5em ;\kern4em \mathrm{otherwise}\end{array}} $$
(24)

Where B represents bandwidth, νk denotes the threshold for power allocation according to user priority. νk ∈ [0, 1] denotes each user’s priority. In case of equal priority, \( {\upnu}_{\mathrm{k}}=\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$\mathrm{K}$}\right. \)for all k. This corresponds to water filling distribution with variable threshold levels that can be varied by the user priorities. Then, \( {\mathrm{A}}_{\mathrm{k}}=\frac{\upsigma_{\mathrm{k}}^2+{\mathrm{P}}_{\mathrm{k}}^{\mathrm{b}}{\mathrm{d}}_{\mathrm{nk}}}{\upgamma_{\mathrm{k}}} \) is the kth MUEs link condition. Here \( {\sigma}_k^2 \) denotes the self-interference variance, \( {P}_k^b \) denotes the backhaul power, dnk denotes the channel gain, and γk represents the interfering channel gain with respect to backhaul. The channel gains dnk and γk follows a uniform distribution.

After determining the access power, both access and backhaul rates can be determined. If the backhaul rate \( {R}_k^b \)is greater than the access rate \( {R}_k^{\mathrm{m}} \), the backhaul power \( {P}_k^b \) is minimized by a factor of δmin. If \( {R}_k^b \)is lesser than \( {R}_k^{\mathrm{m}}, \) the MBS should minimize its transmission power \( {P}_k^m \) by a factor of δmin. In this algorithm, the MBS and the SCA transmission powers are optimized along with the backhaul transmission power by considering backhaul capacity limits.

Power Allocation (PA) Algorithm:

figure c

4 Simulation results and analysis

This section evaluates the performance of the joint power optimization and scaled beamforming algorithm in the massive MIMO enabled HetNet with FD small cell networks. The convex optimization problem has been efficiently carried out utilizing algorithmic toolbox SeDuMi which is accessible in the modeling language CVX [35]. Table 3 list the parameters utilized for simulation.

Table 3 Simulation parameters
Full size table

The total power consumption versus QoS target per user is illustrated in Fig. 2 with M = 50 and N = 2. The PA and scaled Beamforming scheme are compared with PA and ZF Beamforming, MZF beamforming, and MBS schemes. It is observed that total power per subcarrier of PA and scaled beamforming scheme, PA and ZF beamforming scheme, MZF beamforming, and MBS scheme increases with an increase in QoS target per user.

Fig. 2
figure 2

Total power consumption versus QoS constraints

Full size image

Table 4 shows comparisons of total power consumption and QoS target per user between MBS, MZF beamforming, PA, and ZF beamforming and PA and scaled beamforming. From Table 4, it observed that total power per subcarrier is reduced marginally in PA and ZF beamforming when compared to the MZF beamforming technique. This is due to the suitable power allocation method is employed in the network. Further, the total power per subcarrier is reduced in PA and scaled beamforming method when compared to MZF beamforming, because appropriate joint power allocation scheme with scaled beamforming method is implemented in the network. So, PA and scaled beamforming method outperformed when compared to MZF beamforming, PA and ZF beamforming and MBS schemes.

Table 4 Comparison of Total power consumption and QoS target per user
Full size table

Figures 3 and 4 shows the DL sum rate and UL sum-rate against σ2in dB. DL and UL sum rate is assessed for FD mode without PA and beamforming, MZF beamforming, FD with PA and ZF beamforming, FD with PA, and scaled beamforming. When the value σ2is large, the sum rates obtained during UL and DL transmissions are decreased. It is because the larger the value of σ2, it indicates the significant interference resulting in the increased backhaul power consumption. Therefore, the available power for MUEs is low and on the other hand, MBS assigns a smaller amount of power to SCAs backhaul link which can affect the achievable sum rate of MUEs and SUEs during the DL and UL transmissions.

Fig. 3
figure 3

Downlink (DL) sum rate

Full size image
Fig. 4
figure 4

Uplink (UL) sum rate

Full size image

Tables 5 and 6 show the comparisons of the DL sum rate and UL sum-rate respectively. From Tables 5 and 6, it is noticed that the sum-rate is marginally increased in FD with PA and ZF beamforming when compared to MZF beamforming. This is due to the appropriate power allocation method is employed in the network. Further, the sum rate is increased in FD with PA and scaled beamforming when compared to MZF beamforming. Conclusively, the FD with PA and scaled beamforming outperforms when compared with other approaches because the PA algorithm assigns the power to the MUEs, SUEs and the scaled beamforming algorithm reduces the interference that occurs during transmissions.

Table 5 Comparison of Downlink sum rate
Full size table
Table 6 Comparison of Uplink sum rate
Full size table

4.1 Complexity analysis

The complexity analysis has been carried out by varying the number of transmission antennas at the MBS, SCAs, and the number of UEs. Figure 5 clearly shows the total power consumption of the network based on the different number of UEs. It is inferred that when the UEs in the network increases, eventually it increases the power consumption. Figure 6 shows the total power consumption of the network based on the different number of antennas at the MBS. If the number of transmission antennas at the MBS increases, it creates additional complexity resulting in increased power consumption. Tables 7 and 8 illustrate the comparison of total power per subcarrier and the number of UEs, and total power per subcarrier and the number of MBS respectively. It is evident from Tables 7 and 8, is that the FD with PA with scaled beamforming algorithm provides better results when compared to other approaches. Complexity analysis carried out relying on the different number of UEs is computed utilizing the relation

$$ \mathrm{Complexity}=\mathrm{Number}\ \mathrm{of}\ \mathrm{UEs}\ \mathrm{lo}{\mathrm{g}}_{10}\;\left({\mathrm{P}}_{\mathrm{Average}}\right) $$
(25)
$$ \mathrm{Complexity}=\mathrm{Antennas}\ \mathrm{at}\ \mathrm{the}\ \mathrm{MBS}\ \mathrm{lo}{\mathrm{g}}_{10}\;\left({\mathrm{P}}_{\mathrm{Average}}\right) $$
(26)
Fig. 5
figure 5

Total power per subcarrier versus Number of UEs with M = 50, N = 2

Full size image
Fig. 6
figure 6

Total power per subcarrier versus Antennas at the MBS

Full size image
Table 7 Comparison of Total power per subcarrier and Number of UEs
Full size table
Table 8 Comparison of Total power per subcarrier and antennas at MBS
Full size table

Where (PAverage) denotes the average of the total power consumed while differing the number of transmission antennas at the MBS and UEs.

From Figs. 7 and 8, it is observed that the system becomes more complicated with the inclusion of the total number of UEs and the number of antennas at the MBS with the increased total power consumption. Thus, it can be concluded that the complexity possesses a direct relation with the increment in the different numbers of UEs and antennas at the MBS, which have a great impact on the average total power consumption. Shannon channel capacity theorem is directly proportional to the bandwidth. The Eqs. (25) and (26) are assessed with the existing Shannon channel capacity theorem. Thus, the FD with PA and scaled beamforming algorithm optimizes the power with reduced complexity and provides better performance when compared to FD with PA and ZF beamforming, MZF beamforming, and FD methods.

Fig. 7
figure 7

Complexity analysis on differing the number of UEs when M = 50, N = 2

Full size image
Fig. 8
figure 8

Complexity analysis on differing the number of antennas

Full size image

4.2 Computational complexity analysis

The number of additions and multiplications required to compute the total power per subcarrier, downlink, and uplink sum-rate of the proposed algorithm is illustrated in Table 9. The number of SCA’s are represented as N and number of user’s are represented as K.

Table 9 Computational complexity of the proposed algorithm
Full size table

5 Conclusion

This paper has evaluated the performance of the joint power optimization and scaled beamforming algorithm in a massive MIMO enabled HetNet with small cells. The FD transmission mode enables to utilize the access and backhaul links concurrently in the same frequency band. Moreover, we express the power allocation problem as an optimization problem considering co-tier and cross tier interference. Utilizing the concepts of convex optimization, tractable, and computable analytical modeling is done to enhance the sum rates and minimize the power consumption while operating in FD mode. Furthermore, along with the power optimization, the scaled beamforming technique is used to sustain the low complexity and it is compared with the conventional ZF beamforming and the optimal beamforming technique considering only MBS. The PA algorithm assigns the power to the MUEs and SUEs to enhance the sum rates and the scaled beamforming technique reduces the interference issue. From the numerical results, it is evident that the joint power optimization and scaled beamforming technique escalates the network capacity and outperforms the PA along with conventional ZF beamforming and the optimal case without considering PA and beamforming technique. Conclusively, the complexity analysis has been done and the results clearly show that optimizing the number of transmission antennas as well as users remains a crucial design parameter to obtain the optimal system performance.