1 Introduction

With an explosive demand of the wireless broadband services, the future wireless system will witness a rapid growth of high data rate applications with very diverse quality-of-services (QoS) requirements. To support such applications under the limited resources and harsh wireless channel conditions, the dynamic resource allocation which achieves both the higher system spectral efficiency and better QoS has been identified as one of the most promising techniques. The wireless networks have been characterized by the fixed spectrum allocation policy, where the governmental agencies assign spectrum to the license holders on a long term basis for large geographical regions [1]. However, according to the recent measurements made by the Federal Communications Commission (FCC), a large portion of the assigned spectrum is used sporadically leading the fact that fixed spectrum allocation policy has resulted in severe underutilization of several bands both in temporal and spatial manner [2], that motivates for investigation of the cognitive radio network, which proposes secondary access to the already-licensed spectrum as a means to mitigate spectrum scarcity. It is a novel concept in the wireless communications communities, which aims to have more adaptable and aware communication device that can make better use of the available natural resources [3] and offers a solution to the spectral crowding problem by introducing the opportunistic usage of the radio frequency spectrum that are not heavily occupied by the licensed users [4]. It improves the spectrum usage by facilitating coordination between the licensees and secondary/cognitive users in the secondary spectrum markets by allowing license exempt use of the licensed spectrum with times or locations when it is unused or lightly used.

In general, for wireless communication systems, the channel capacity is used as a basic performance measurement tool for the analysis and design of new and more efficient techniques to improve the spectral efficiency. The adaptive power transmission scheme that achieves the Shannon capacity under the fading environment is considered in [5] and average transmit power constraint along with the availability of channel state information (CSI) at the cognitive transmitter was initially considered in [6]. Later, the power optimization problem with peak and average transmit power constraints have been investigated [7]. However, in the spectrum-sharing systems, CSI is used at the secondary transmitter to adaptively adjust the transmission resources [8, 9]. In [9], the knowledge of the secondary link CSI and information about the channel between the secondary transmitter (ST) and primary receiver (PR) at the ST have been used to obtain the optimal power transmission policy of the secondary user (SU) under constraints on the peak and average received-power at the primary receiver. Ghasem and Sousa [10] have demonstrated that the secondary user may take advantage in the fading environment between the primary and secondary user by opportunistically transmitting with high power when its signal, as received by the licensed receiver, is deeply faded. One of the most efficient ways to determine the spectrum occupancy is to sense the activity of primary users operating in the secondary user’s range of communication [11]. Practically, it is difficult for a secondary user to have direct access to the CSI pertaining to the primary user link. Recent works on the spectrum-sharing systems concentrated on sensing the primary transmitter’s activity is based on the local processing at the secondary user side [12]. In this context, the sensing ability is provided by a sensing detector mounted on the secondary user’s equipment, which scans the spectrum for specific time [13]. The activity statistics of the primary user’s signal in the shared spectrum is computed and, based on the sensing information [14], the cognitive user has capability to determine the local presence of the primary transmitter in a specific spectrum band. For instance, the received signals at energy-based detector [15, 16] are used to detect the presence of unknown primary transmitters. However, using this sensing information obtained from the spectrum sensor and considering that the secondary transmitter does not have information about the state of its corresponding channel, the power adaptation strategy that maximizes the channel capacity of the secondary user’s link is investigated in [17]. Rezki and Alouini [18] have considered the limited/imperfect CSI at the secondary transmitter and computed the Ergodic channel capacity. Further, in [19] the power allocation for erroneous estimated channel gain between the secondary user and primary base station is performed through the geometric programming problem which is solved by Lagrange dual decomposition. However, only underlay spectrum sharing model is considered in [19]. Parsaeefard and Sharafat [20] have considered the cognitive nodes as relay nodes and illustrated the power and channel allocation strategy to the cognitive users in the Rayleigh fading environment. In [21], the rate loss constraint (RLC) is considered instead of conventional interference power constraint in order to protect the primary user, and the channel capacity of cognitive user which utilizes primary users OFDM (orthogonal frequency division multiplexing) subcarriers, is maximized by RLC and cognitive user transmit power constraint. However, in [1821] the authors have computed the channel capacity of the cognitive user without considering the channel sensing information available at secondary transmitter.

In this paper, we have emphasized on the cognitive radio wireless communication system with maximum achievable Ergodic channel capacity, considering single cognitive user. In a collaborative communication framework, either extra relay terminals assist the communication between some dedicated sources and their corresponding destinations and/or allow the users in a network to help each other to achieve higher communication system capacity than the single point-to-point communication between source and destination [22, 23]. However, in this paper we have considered point-to-point communication between the cognitive users without any kind of cooperation/collaboration among them therefore, if more than one cognitive user’s are competing to access the primary user’s same spectrum hole, then due to probable inter cognitive user’s interference the maximum achievable channel capacity is upper bounded by only single cognitive user’s case. The proposed spectrum-sharing system has a pair of primary transmitter (PT) and primary receiver (PR) as well as a pair of secondary transmitter (ST) and secondary receiver (SR) as shown in Fig. 1. The small-scale fading effects over the transmit power of secondary transmitter in the proposed system has been explored. However, in [24] such type of system model is considered without the fading in the link channel between the secondary transmitter and primary receiver. We have investigated the Ergodic channel capacity for the Nakagami-m fading channel in the secondary and primary links and the power of secondary transmitter is controlled based on:

Fig. 1
figure 1

The proposed spectrum-sharing system model

  1. 1.

    Sensing information about the primary user’s activity, and

  2. 2.

    CSI of the secondary and primary link.

Further, the constraint on average interference at the primary radio receiver is considered for the channel capacity. Since the cognitive user is able to adapt any modulation strategy, therefore it can change its modulation strategy according to the fading environment and hence both adaptation policies in the rate and power can established [25], which is referred as the variable rate and power transmission scheme. In this context, we have also considered the variable rate and power M-QAM transmission strategy in the cognitive radio communication system where the rate and power of the secondary transmitter is adaptively controlled based on the availability of secondary user’s link CSI and the sensing information about the primary user’s activity. The remainder of the paper is organized as follows. Section 2 is concern with the spectrum sharing system model. Section 3 discusses about the power and rate adaptation policy for the Nakagami fading channels and in Sect. 4 the numerical simulation results of the proposed spectrum sharing model is presented. Finally, Sect. 5 concludes the work.

2 Spectrum sharing system

2.1 System model

This proposed spectrum-sharing system consist of a pair of primary transmitter (PT) and primary receiver (PR) as well as a pair of secondary transmitter (ST) and receiver (SR) as shown in Fig. 1. In this scenario, the secondary user is allowed to use the spectrum band assigned to the primary user as long as the interference power imposed by secondary transmitter on the primary receiver is less than a predefined threshold value that is the interference temperature limit. We have considered that the primary user link that is the channel between the primary transmitter and primary receiver is a stationary block-fading channel. According to the definition of block-fading, the channel gain remains constant over some block length T and after that time, the channel gain changes to a new independent value based on its distribution [24].

The average transmit power of the primary transmitter is assumed to be P t and its average ON/active time is α or average OFF/inactive time is \( \bar{\alpha } = 1 - \alpha \) [17]. In addition to this, we have considered a discrete-time flat-fading channel with perfect CSI at the receiver and transmitter of the secondary user. As shown in Fig. 1, the secondary receiver generates and estimates the channel power gain (\( \hat{\gamma }_{s} \)) between the secondary transmitter and secondary receiver. We assume that the channel power gain is fed back to the secondary transmitter error-free and without delay. Further, the channel gain between the transmitter and receiver of the secondary user, secondary transmitter and primary receiver as well as between the primary transmitter and secondary transmitter is given by \( \sqrt {\gamma_{s} } \), \( \sqrt {\gamma_{p} } \), and \( \sqrt {\gamma_{m} } \), respectively. However, the channel power gains \( \gamma_{s} \), \( \gamma_{p} \), and \( \gamma_{m} \) are independent of each other. We have obtained the cognitive radio communication system Ergodic channel capacity by considering the distribution of \( \gamma_{s} \) and \( \gamma_{p} \) as the Nakagami-m distribution. \( d_{m} \), \( d_{s} \) and \( d_{p} \) are the distances between secondary transmitter to primary transmitter, secondary transmitter to secondary receiver, and secondary transmitter to primary receiver, respectively. Moreover, the channel between the primary transmitter and secondary receiver is considered additive white Gaussian noise channel (AWGN), denoted as n and can be modeled as zero-mean Gaussian random variable with variance \( N_{0} B \), where N 0 and B denote the noise power spectral density and the signal bandwidth, respectively. x is the data transmitted from secondary transmitter and \( \hat{x} \) is the estimated transmitted data at secondary receiver as shown in Fig. 1.

2.2 Spectrum sensing module

As shown in Fig. 1, the secondary transmitter is equipped with a spectrum sensing detector whose function is to sense the frequency band of primary user for secondary user’s transmissions. However, based on the received signals, the detector computes a single sensing metric denoted by ξ, [16]. The sensing metric is the total primary signal power in the number of independent signal samples [17]. We consider the statistics of ξ conditioned on the primary user being active or idle, are known prior to the secondary transmitter. Using the energy detection method for sensing information on the primary user being active or idle, the sensing parameter ξ is modeled according to Chi square probability distribution functions (pdfs) with ν degrees of freedom as discussed in [15], where ν is related to the number of samples used in the sensing period, N We define the pdf of ξ given that the primary transmitter is active or idle by, \( f_{1} \left( \xi \right) \) and \( f_{0} \left( \xi \right) \), respectively that is the \( f_{1} \left( \xi \right) \) and \( f_{0} \left( \xi \right) \) are conditional probability. According to [26, p. 941], for a large number of ν (e.g., ≥30), one can approximate the Chi square distribution with a Gaussian pdf. Since the number of observation samples can be large enough for the approximation to be valid. We choose \( f_{1} \left( \xi \right)\sim {\mathcal{N}}(\mu_{1,} \delta_{1}^{2} ) \) and \( f_{0} \left( \xi \right)\sim {\mathcal{N}}(\mu_{0,} \delta_{0}^{2} ) \) where (\( \mu_{1,} \delta_{1}^{2} \)) and \( (\mu_{0,} \delta_{0}^{2} ) \) are given by [12]. The probability distribution of \( \xi \) depends on [17]:

$$ \left. {\begin{array}{*{20}l} {\left. \begin{aligned} {\upmu}_{1} = {\text{N}}\left( {\frac{{{\text{P}}_{\text{t}} }}{{{\text{d}}_{\text{m}}^{2} }} + 1} \right) \hfill \\ {\updelta}_{1}^{2} = 2{\text{N}}\left( {\frac{{{\text{P}}_{\text{t}} }}{{{\text{d}}_{\text{m}}^{2} }} + 1} \right)^{2} ,\,{\text{and}} \hfill \\ \end{aligned} \right\}} \hfill & {\text{when PT is active}} \hfill \\ {\begin{array}{*{20}l} \, \hfill \\ { \left. \begin{aligned} {\upmu}_{0} = {\text{N}} \hfill \\ {\updelta }_{0}^{2} = 2{\text{N}} \hfill \\ \end{aligned} \right\}} \hfill \\ \end{array} } \hfill & {\text{when PT is idle}} \hfill \\ \end{array} } \right\} $$
(1a)

and the probability distribution of \( \xi \) is given as [12]:

$$ \left. \begin{aligned} f_{0} \left( \xi \right) = \frac{1}{{\sqrt {2\pi \delta_{0}^{2} } }}exp \left( {\frac{{ - \left( {\xi - \mu_{0} } \right)^{2} }}{{2\delta_{0}^{2} }}} \right) \hfill \\ f_{1} \left( \xi \right) = \frac{1}{{\sqrt {2\pi \delta_{1}^{2} } }}exp \left( {\frac{{ - \left( {\xi - \mu_{1} } \right)^{2} }}{{2\delta_{1}^{2} }}} \right) \hfill \\ \end{aligned} \right\} $$
(1b)

In this paper, we have used the energy detector for spectrum sensing due to its easy implementation and low computational complexity as discussed in [15]. The other sensing detectors can also be used for spectrum sensing because, the authors main motive is to compute the sensing metric \( \upxi \), which can represent the total signal power observed or the correlation between the observed signal and a known signal pattern [17]. However, the main difference lie in the number of samples required for the same performance in different detectors and that depends on the required signal-to-noise ratio [15]. In addition to this, the secondary user transmission should be limited so that it does not cause harmful interference to the primary. Therefore a limit or constraint is set at PR called average interference power constraint or simply interference constraint. When PU is active, ST cannot transmit power which crosses the average interference power constraint at the primary receiver, which is given as [24]:

$$ E_{{\gamma_{s,} \xi ,\gamma_{p} }} \left[ {P(\gamma_{s,} \gamma_{p, } \xi )\gamma_{p} |PU is ON} \right] \le Q_{\text{int}} ;\forall \gamma_{s} ,\gamma_{p, } \xi $$
(2)

where the transmit power of SU is \( P(\gamma_{s,} \gamma_{p, } \xi ) \) and expectation over the joint pdf of random variables \( {\upgamma}_{\text{s}} ,{\upgamma}_{\text{p }} \) and \( \xi \) is denoted by \( {\text{E}}_{{{\upgamma}_{{{\text{s}},}} {\upxi},{\upgamma}_{\text{p}} }} \left[ \cdot\right] \). \( Q_{int} \) is the interference limit set at PR that is the maximum interference power, which it can tolerate without degrading its own performance. The constraint defined in (2) is used to compute the Ergodic channel capacity. However, the average interference power constraint is considered only because we have considered that the licensed user performance is measured by the average signal-to-noise ratio (SNR) and not by instantaneous SNR. Moreover, the Ergodic channel capacity under the average received power constraint is, in general, higher than that of the peak received power constraint due to the more restrictive nature of the peak as opposed to the average interference power constraint.

3 Rate and power adaptation policy for M-QAM

The data rate and power is a potential transmission strategy, which adjust the transmit power and data rate of cognitive radio systems to improve the spectrum efficiency for utilizing the shared spectrum [24, 2729]. Further, the data rate adaptation is a spectral efficient technique and its adaptation can be achieved either through the variation of the symbol time duration [30] or by varying the constellation size [31]. However, the former method is spectral inefficient and requires variable-bandwidth system design as discussed in [32]. The variable data rate adaptation policy using varying constellation size is fixed bandwidth and spectral efficient method [32]. The Ergodic channel capacity under adaptation policy of the variable data rate and power transmission strategy in M-QAM signal constellation is considered with the knowledge of CSI and spectrum-sensing information at the secondary transmitter side, which satisfy the predefined bit-error-rate (BER) requirements and adhering to the constraints on the average interference power at the primary user. In this case, the cognitive radio adapt the transmit power according to the:

  1. 1.

    primary and secondary channel power gain \( \gamma_{p} \) and \( \gamma_{s} \), respectively,

  2. 2.

    primary user’s activity states \( {\upxi} \), subjected to the average interference, and,

  3. 3.

    instantaneous bit-error-rate constraint \( P_{b} \left( {\gamma_{s} ,\xi } \right) = P_{b} \).

The \( P_{b} \) bound for each value of \( \gamma_{s} \) and \( {\upxi} \), which is given as [24]:

$$ P_{b} \left( {\gamma_{s} ,\xi } \right) \le 0.2\exp \left( {\frac{ - 1.5}{M - 1}\frac{{P(\gamma_{s,} \gamma_{p, } \xi )\gamma_{s} }}{{N_{0} B}}} \right) $$
(3)

where M is the constellation size or the number of symbols in the particular modulation format. \( P(\gamma_{s,} \gamma_{p, } \xi ) \) is the transmit power of secondary transmitter. To satisfy the conditions as discussed in (3), we can adjust the values of M and \( P(\gamma_{s,} \gamma_{p, } \xi ) \). However, Eq. (3) holds for \( M \ge 4 \) [24]. We can also write (3) by the following mathematical expression:

$$ P_{b} \left( {\gamma_{s} ,\xi } \right) \le 0.2exp\left( {\frac{ - 1.5}{M - 1}SNR_{ss} } \right) $$
(3a)

where \( SNR_{ss} \) is the signal-to-noise power ratio of the secondary transmitter to secondary receiver. For both adaptive data rate and adaptive power transmission policy, Eq. (3) should be satisfied for the following constraint on average interference power:

$$ \frac{{P(\gamma_{s,} \gamma_{p, } \xi )\gamma_{p} }}{{N_{0} B}} \le Q_{int} $$
(3b)

or

$$ SNR_{sp} \le Q_{int} $$

where \( SNR_{sp} \) is the signal-to-noise power ratio of secondary transmitter to primary receiver. After some mathematical manipulation of Eq. (3), we yields the following maximum constellation size for a given \( P_{b} \left( {\gamma_{s} ,\xi } \right) \):

$$ M\left( {\gamma_{s} ,\xi } \right) = 1 + K\left( {\frac{{P(\gamma_{s,} \gamma_{p, } \xi )\gamma_{s} }}{{N_{0} B}}} \right) $$
(3c)

Moreover, we can achieve the constellation size that is the value of M in M-QAM modulation format for an arbitrary chosen bit-error-rate, the average interference power and the ratio of \( \frac{{\gamma_{s} }}{{\gamma_{p} }} \) and is given by the following expression:

$$ M = 1 + K\left( {\frac{{\gamma_{s} }}{{\gamma_{p} }}} \right)Q_{int} $$
$$ M = 2^{n} = 2^{{log_{2} \left( {1 + K\left( {\frac{{\gamma_{s} }}{{\gamma_{p} }}} \right)Q_{int} } \right)}} $$
(4)

where

$$ K = \frac{ - 1.5}{{In\left( {5P_{b} } \right)}} < 1 $$
(5)

and n is the number of bits per symbol. However, for \( M < 4 \), suppose for BPSK the error rate is given in [32]. Therefore, the Ergodic channel capacity under average interference power constraint and given \( P_{b} \) is:

$$ \frac{{C_{er} }}{B} = \mathop {\hbox{max} }\limits_{{P(\gamma_{s} ,\gamma_{p} ,\xi )}} \mathop {\iint }\nolimits log_{2} \left( {1 + \frac{{K\gamma_{s} P(\gamma_{s,} \gamma_{p, } \xi )}}{{N_{0} B}}} \right)f_{s} \left( {\gamma_{s} } \right)f_{p} (\gamma_{p} )(\alpha f_{1} \left( \xi \right) + \bar{\alpha }f_{0} \left( \xi \right))d\gamma_{s} d\gamma_{p} $$
(6)

With the constraint:

$$ \mathop {\iint }\nolimits \gamma_{p} P(\gamma_{s,} \gamma_{p, } \xi )f_{s} \left( {\gamma_{s} } \right)f_{p} (\gamma_{p} )f_{1} \left( \xi \right)d\gamma_{s} d\gamma_{p} \le Q_{int} $$
(7)

The transmitter power \( P\left( {\gamma_{s,} \gamma_{p, } \xi } \right) \) of cognitive transmitter is the joint function of secondary channel gain, primary channel gain and sensing metric. Asghari and Aissa [24] have provided a mathematical expression for the channel capacity of the secondary user’s link for power adaptation policies under interference and peak power constraint with the sensing pdf’s. However, the primary user’s link channel power gain \( \gamma_{p} \), which is presented in (6) was not considered in [24]. Now, we have to maximize the Ergodic capacity of the system as given by (6) and satisfy the constraint given in (7). Therefore, to yield the optimal power allocation \( P\left( {\gamma_{s,} \gamma_{p, } \xi } \right) \), we form the Lagrangian multiplier, \( \lambda \) [33] and construct the following Lagrangian function:

$$ \begin{aligned} L\left( {P\left( {\gamma_{s,} \gamma_{p, } \xi } \right),\lambda } \right)\, & = \mathop {\iint }\nolimits log_{2} \left( {1 + \frac{{K\gamma_{s} P(\gamma_{s,} \gamma_{p, } \xi )}}{{N_{0} B}}} \right)f_{s} \left( {\gamma_{s} } \right)f_{p} (\gamma_{p} )(\alpha f_{1} \left( \xi \right) + \bar{\alpha }f_{0} \left( \xi \right))d\gamma_{s} d\gamma_{p} \\ &\quad - {\uplambda }\left( {\mathop {\iint }\nolimits \gamma_{p} P(\gamma_{s,} \gamma_{p, } \xi )f_{s} \left( {\gamma_{s} } \right)f_{p} (\gamma_{p} )f_{1} \left( \xi \right)d\gamma_{s} d\gamma_{p} - Q_{int} } \right) \\ \end{aligned} $$
(8)

\( L\left( {P\left( {\gamma_{s,} \gamma_{p, } \xi } \right),\lambda } \right) \) is the concave function of \( P(\gamma_{s,} \gamma_{p, } \xi ) \) and interference constraint defined in (7) is convex, therefore the 1st order condition that is the derivative of \( L\left( {P\left( {\gamma_{s,} \gamma_{p, } \xi } \right),\lambda } \right) \) w.r.t \( P\left( {\gamma_{s,} \gamma_{p, } \xi } \right) \) is sufficient KKT condition for the optimality [34] and the sufficient condition allows us to obtain the solution. Now, the optimization problem being convex (i.e. this problem is a maximization problem with a concave cost function and a convex set of constraints), there is a unique solution. Hence, the solution given by the sufficient condition is the only solution and is given by:

$$ \frac{{\partial L\left( {P,\lambda } \right)}}{\partial P} = \frac{1}{{1 + \frac{{K\gamma_{s} P\left( {\gamma_{s} ,\gamma_{p} ,\xi } \right)}}{{N_{0} B}}}}\frac{{K\gamma_{s} }}{{N_{0} B}}\left( {\alpha f_{1} \left( \xi \right) + \bar{\alpha }f_{0} \left( \xi \right)} \right)f_{s}\left(\gamma_{s}\right)f_{p} \left( {\gamma_{p} } \right) - \lambda \gamma_{p} f_{1} \left( \xi \right)f_{s} \left( {\gamma_{s} } \right)f_{p} \left( {\gamma_{p} } \right) = 0 $$

or

$$ \frac{\partial L(P,\lambda )}{\partial P} = \frac{{K\gamma_{s} }}{{N_{0} B + K\gamma_{s} P\left( {\gamma_{s,} \gamma_{p, } \xi } \right)}}\left( {\alpha f_{1} \left( \xi \right) + \bar{\alpha }f_{0} \left( \xi \right)} \right) - {\uplambda }\gamma_{p} f_{1} \left( \xi \right) = 0 $$
(9)

and

$$ P\left( {\gamma_{s,} \gamma_{p, } \xi } \right) = \frac{{\gamma_{\mu } \left( \xi \right)}}{{{\uplambda}\gamma_{p} }} - \frac{{N_{0} B}}{{\gamma_{s} K}} $$
(10a)

If we assume \( P\left( {\gamma_{s,} \gamma_{p, } \xi } \right) = 0 \) for some values of \( \gamma_{s,} \) \( \gamma_{p, } \) and \( \xi \), which take placed in the condition defined below and after putting \( P\left( {\gamma_{s,} \gamma_{p, } \xi } \right) = 0 \) in (10a), we get:

$$ \frac{{\gamma_{p} }}{{\gamma_{s} }} > \frac{{\gamma_{\mu } \left( \xi \right)K}}{{{\uplambda}N_{0} B}} $$
(10b)

Therefore, from (10a) and (10b), the power \( P\left( {\gamma_{s,} \gamma_{p, } \xi } \right) \) is adapted to maximize the Ergodic channel capacity as defined in (6), which is given as:

$$ P\left( {\gamma_{s,} \gamma_{p, } \xi } \right) = \left\{ {\begin{array}{*{20}c} {\frac{{\gamma_{\mu } \left( \xi \right)}}{{{\uplambda}\gamma_{p} }} - \frac{{N_{0} B}}{{\gamma_{s} K}},\frac{{\gamma_{p} }}{{\gamma_{s} }} \le \frac{{\gamma_{\mu } \left( \xi \right)K}}{{{\uplambda}N_{0} B}}} \\ {0, \frac{{\gamma_{p} }}{{\gamma_{s} }} > \frac{{\gamma_{\mu } \left( \xi \right)K}}{{{\uplambda}N_{0} B}}} \\ \end{array} } \right. $$
(10c)

where

$$ \gamma_{\mu } \left( \xi \right) = \alpha + \bar{\alpha }\frac{{f_{0} \left( \xi \right)}}{{f_{1} \left( \xi \right)}}. $$
(11)

The optimal power allocation obtained by (10a) represents the more transmission power, which can be used when \( \gamma_{s} \) increases and \( \gamma_{p} \) decreases and the average interference constraint at primary receiver is satisfied. This is due to the primary user’s fading channel advantage which has enhanced the cognitive user’s capacity. The sensing decision is considered in Eq. (11) and it is observed that when the conditional probability that the PU is idle (\( f_{0} \left( \xi \right) \)) gets higher than that of being active (\( f_{1} \left( \xi \right) \)), then the value of \( {\upgamma}_{{\upmu}} \left( {\upxi} \right) \) has an ascending behavior and \( {\upgamma}_{{\upmu}} \left( {\upxi} \right) > 1 \) otherwise, \( {\upgamma}_{{\upmu}} \left( {\upxi} \right) < 1 \). Therefore, as the conditional probability distribution of the primary user being idle gets higher than being active, \( {\upgamma}_{{\upmu}} \left( {\upxi} \right) \) increases and, consequently, we can increase the secondary user’s transmission power without causing harmful interference to the primary receiver. Note that when \( {\upgamma}_{{\upmu}} \left( {{\upxi}} \right) = 1 \), the secondary transmitter has no information about the primary user activity. Accordingly, it considers that the primary user is always active \( \left( {\frac{{{\text{f}}_{0} \left( {\upxi} \right)}}{{{\text{f}}_{1} \left( {\upxi} \right)}} = 1} \right) \) and continuously transmits with the same power level with which it is already transmitting. For \( {\upgamma}_{{\upmu}} \left( {{\upxi}} \right) \), the values of \( f_{0} \left( \xi \right) \) and \( f_{1} \left( \xi \right) \) should be taken at that value of \( \xi \) which is computed by the sensing detector for a given detection and false alarm probabilities. The higher value of \( {\upxi} \) as compared to threshold that is the energy computed in a particular time interval over a spectrum, indicates the presence of PU signal and vice versa [17]. However, if we modify probability of false alarm, value of \( {\upxi} \) is also modified. By substituting Eq. (10a) in (7), we get:

$$ \mathop {\iint }\nolimits_{0}^{{\frac{{K\gamma_{\mu } \left( \xi \right)}}{{{\uplambda}_{0} N_{0} B}}}} \left( {\frac{{\gamma_{\mu } \left( \xi \right)}}{{{\uplambda}_{0} }} - \frac{{N_{0} B\gamma_{p} }}{{\gamma_{s} K}}} \right)f_{s} \left( {\gamma_{s} } \right)f_{p} (\gamma_{p} )f_{1} \left( \xi \right)d\gamma_{s} d\gamma_{p} = Q_{int} $$

where \( {\uplambda}_{0} \) is determined in such a way that average interference power constraint in (7) is equal to \( Q_{int} \).

$$ \mathop {\iint }\nolimits_{0}^{{\frac{{K\gamma_{\mu } \left( \xi \right)}}{{{\uplambda}_{0} N_{0} B}}}} \left( {\frac{{\gamma_{\mu } \left( \xi \right)}}{{{\uplambda}_{0} N_{0} B}} - \frac{{\gamma_{p} }}{{\gamma_{s} K}}} \right)f_{s} \left( {\gamma_{s} } \right)f_{p} (\gamma_{p} )f_{1} \left( \xi \right)d\gamma_{s} d\gamma_{p} = \frac{{Q_{int} }}{{N_{0} B}} = \Phi $$

or

$$ \mathop {\iint }\nolimits_{0}^{{K\gamma_{\mu } \left( \xi \right)\gamma_{0} }} \left( {\gamma_{\mu } \left( \xi \right)\gamma_{0} - \frac{{\gamma_{p} }}{{\gamma_{s} K}}} \right)f_{s} \left( {\gamma_{s} } \right)f_{p} (\gamma_{p} )f_{1} \left( \xi \right)d\gamma_{s} d\gamma_{p} = \Phi $$
(12)

where \( \gamma_{0} = \frac{1}{{\lambda_{0} N_{0} B}} \), and \( \Phi = \frac{{Q_{\text{int}} }}{{N_{0} B}} \) is the average SNR [8]. By substituting (10a) in (6), gives the following Ergodic channel capacity expression:

$$ \frac{{C_{er} }}{B} = \mathop \smallint \nolimits_{{\frac{{\gamma_{s} }}{{\gamma_{p} }} \ge \frac{{N_{0} B {\uplambda}_{0} }}{{K\gamma_{\mu } \left( \xi \right)}} = \frac{1}{{K\gamma_{0} \gamma_{\mu } \left( \xi \right)}}}}^{\infty } log_{2} \left( {1 + \frac{{K\gamma_{s} }}{{N_{0} B}}\left[ {\frac{{\gamma_{\mu } \left( \xi \right)}}{{{\uplambda}_{0} \gamma_{p} }} - \frac{{N_{0} B}}{{\gamma_{s} K}}} \right]} \right)f_{s} \left( {\gamma_{s} } \right)f_{p} (\gamma_{p} )(\alpha f_{1} \left( \xi \right) + \bar{\alpha }f_{0} \left( \xi \right))d\gamma_{s} d\gamma_{p} $$

or

$$ \frac{{C_{er} }}{B} = \mathop \smallint \nolimits_{{\frac{{\gamma_{s} }}{{\gamma_{p} }} \ge \frac{{N_{0} B {\uplambda}_{0} }}{{K\gamma_{\mu } \left( \xi \right)}} = \frac{1}{{K\gamma_{0} \gamma_{\mu } \left( \xi \right)}}}}^{\infty } log_{2} \left( {\frac{{K\gamma_{s} \gamma_{\mu } \left( \xi \right)}}{{N_{0} B {\uplambda}_{0} \gamma_{p} }}} \right)f_{s} \left( {\gamma_{s} } \right)f_{p} (\gamma_{p} )(\alpha f_{1} \left( \xi \right) + \bar{\alpha }f_{0} \left( \xi \right))d\gamma_{s} d\gamma_{p} $$

or

$$ \frac{{C_{er} }}{B} = \mathop \smallint \nolimits_{{\frac{{\gamma_{s} }}{{\gamma_{p} }} \ge \frac{{N_{0} B {\uplambda}_{0} }}{{K\gamma_{\mu } \left( \xi \right)}} = \frac{1}{{K\gamma_{0} \gamma_{\mu } \left( \xi \right)}}}}^{\infty } log_{2} \left( {\frac{{K\gamma_{s} \gamma_{\mu } \left( \xi \right)\gamma_{0} }}{{\gamma_{p} }}} \right)f_{s} \left( {\gamma_{s} } \right)f_{p} (\gamma_{p} )(\alpha f_{1} \left( \xi \right) + \bar{\alpha }f_{0} \left( \xi \right))d\gamma_{s} d\gamma_{p} $$
(13)

or

$$ \frac{{C_{er} }}{B} = {}_{{\frac{{\gamma_{s} }}{{\gamma_{p} }} \ge \frac{{N_{0} B {\uplambda}_{0} }}{{K\gamma_{\mu } \left( \xi \right)}}}}^{{E_{{\gamma_{s} ,\gamma_{p} ,\xi }} }} \left[ { log_{2} \left( {\frac{{K\gamma_{u} \left( \xi \right)\gamma_{s} }}{{\lambda_{0} N_{0} B\gamma_{p} }}} \right)} \right] $$
(14)

where \( C_{er} \) denotes the Ergodic capacity and Ε[·] denotes the expectation operator. Equation (14) is similar to that presented in [24, Equation (30)] except the term \( \gamma_{p} \), which is due to the consideration of the primary channel gain in the cognitive user’s system capacity. However, when only the power adaptation policy is adapted instead of power and rate adaptation policy, then the additional constraint of (5) is not needed and the Ergodic channel capacity of adaptive power transmission policy is given by the following mathematical expression by substituting K = 1 in Eq. (14):

$$ \frac{{C_{er} }}{B} = {}_{{\frac{{\gamma_{s} }}{{\gamma_{p} }} \ge \frac{{N_{0} B {\uplambda}_{0} }}{{\gamma_{\mu } \left( \xi \right)}}}}^{{E_{{\gamma_{s} ,\gamma_{p} ,\xi }} }} \left[ { log_{2} \left( {\frac{{\gamma_{u} \left( \xi \right)\gamma_{s} }}{{\lambda_{0} N_{0} B\gamma_{p} }}} \right)} \right] $$
(15)

Comparing the Ergodic capacity of power adaptation policy in (15) and rate and adaptation policy for M-QAM modulation format in (14), the Eq. (14) reveals that there is an effective power loss of K for adaptive M-QAM as compared to (15). However, for the adaptive power transmission policy the probability of error is significantly more and fixed, which is 0.0446 in comparison to the adaptive rate and power transmission policy where the probability of bit error can be varied according to the quality-of-service requirement.

4 Effect of channel conditions

In this section, we have explored the fading channel effect on the cognitive radio communication systems performance and numerically computed the Ergodic channel capacity in different fading environment.

  • Nakagami-m fading

The Nakagami-m distribution often provides the best fit to the urban [35] and indoor [36] multipath propagation and gives AWGN, Rayleigh and Rician fading channel model by adjusting the fading parameter m, which is the ratio of line-of-sight (LOS) signal power to the multipath signal power. The channel fading model based on Nakagami distribution, both \( \gamma_{s} \) and \( \gamma_{p} \) would be distributed according to the following Gamma distribution [10]:

$$ f\left( \gamma \right) = \frac{{m^{m} z^{m - 1} }}{\Gamma (m)}e^{ - m\gamma } $$

and the pdf \( f_{s} \left( {\gamma_{s} } \right)f_{p} (\gamma_{p} ) \) is given as:

$$ f_{s} \left( {\gamma_{s} } \right)f_{p} \left( {\gamma_{p} } \right) = \left( {\frac{{m_{0} }}{{m_{1} }}} \right)^{{m_{0} }} \frac{{z^{{m_{1} - 1}} }}{{\beta \left( {m_{0} ,m_{1} } \right)\left( {x + \frac{{m_{0} }}{{m_{1} }}} \right)^{{m_{0} + m_{1} }} }} $$
(16)

where \( m_{0} \) and \( m_{1} \) are \( m \) parameters [10] for \( \gamma_{p} \) and \( \gamma_{s} \), respectively. \( \frac{{\gamma_{p} }}{{\gamma_{s} }} = z \), where z is a random variable. \( \beta (\cdot) \) is the beta function. When \( m_{0} = m_{1} = m \), the Eq. (16) becomes:

$$ f_{s} \left( {\gamma_{s} } \right)f_{p} \left( {\gamma_{p} } \right) = \frac{{z^{m - 1} }}{{\beta \left( {m,m} \right)\left( {z + 1} \right)^{2m} }} $$
(17)

By substituting (17) in (12), we yield the following value of secondary transmit power, which satisfy the average interference constraint for the Nakagami-m fading channel:

$$ \mathop {\iint }\nolimits_{0}^{{K\gamma_{\mu } \left( \xi \right)\gamma_{0} }} \left( {\gamma_{\mu } \left( \xi \right)\gamma_{0} - \frac{{\gamma_{p} }}{{\gamma_{s} K}}} \right)\frac{{z^{m - 1} }}{{\beta \left( {m,m} \right)\left( {z + 1} \right)^{2m} }}f_{1} \left( \xi \right)d\gamma_{s} d\gamma_{p} = \frac{{Q_{int} }}{{N_{0} B}} $$
(18)

and the Ergodic channel capacity from (13), for the Nakagami-m fading environment is given by:

$$ \frac{{C_{er} }}{B} = \mathop \smallint \nolimits_{{\frac{{\gamma_{s} }}{{\gamma_{p} }} \ge \frac{{N_{0} B {\uplambda}_{0} }}{{K\gamma_{\mu } \left( \xi \right)}} = \frac{1}{{K\gamma_{0} \gamma_{\mu } \left( \xi \right)}}}}^{\infty } log_{2} \left( {\frac{{K\gamma_{s} \gamma_{\mu } \left( \xi \right)\gamma_{0} }}{{\gamma_{p} }}} \right)\frac{{z^{m - 1} }}{{\beta \left( {m,m} \right)\left( {z + 1} \right)^{2m} }}(\alpha f_{1} \left( \xi \right) + \bar{\alpha }f_{0} \left( \xi \right))d\gamma_{s} d\gamma_{p} $$
(19)

4.1 Rayleigh fading

Since the Nakagami-m distribution with fading parameter equal to 1 represent the Rayleigh fading channel, and the pdf \( f_{s} \left( {\gamma_{s} } \right)f_{p} (\gamma_{p} ) \) will have log-logistic distribution [10]. Therefore by substituting m = 1 in Eq. (18), we get:

$$ \mathop \smallint \nolimits_{0}^{{K\gamma_{\mu } \left( \xi \right)\gamma_{0} }} \left( {\gamma_{\mu } \left( \xi \right) \gamma_{0} - \frac{z}{K}} \right)\frac{1}{{\left( {1 + z} \right)^{2} }} f_{1} \left( \xi \right)dz = \frac{{Q_{int} }}{{N_{0} B}} $$

or

$$ f_{1} \left( \xi \right)\left( { - \frac{1}{K}\log_{2} \left( {1 + K\gamma_{\mu } \left( \xi \right)\gamma_{0} } \right) + \gamma_{\mu } \left( \xi \right) \gamma_{0} } \right) = \frac{{Q_{int} }}{{N_{0} B}} = \Phi $$
(20)

Therefore the capacity of the cognitive radio communication system in the Rayleigh fading environment is achieved by putting m = 1 in (19):

$$ \frac{{C_{er} }}{B} = \mathop \smallint \nolimits_{{\frac{1}{{\gamma_{0} \gamma_{\mu } \left( \xi \right)}}}}^{\infty } log_{2} \left( {K\gamma_{0} \gamma_{\mu } \left( \xi \right)z} \right)\frac{1}{{\left( {1 + z} \right)^{2} }}(\alpha f_{1} \left( \xi \right) + \bar{\alpha }f_{0} \left( \xi \right))dz $$

or

$$ \frac{{C_{er} }}{B} = \left( {\alpha f_{1} \left( \xi \right) + \bar{\alpha }f_{0} \left( \xi \right)} \right)log_{2} (1 + K\gamma_{\mu } \left( \xi \right)\gamma_{0} \left( \Phi \right)) $$
(21)

where \( \gamma_{0} \left( \Phi \right) \) is from the (20) for a given \( \Phi \). Equation (21) gives the Ergodic channel capacity of adaptive rate and power transmission policy under the Rayleigh fading environment. Further, the capacity of adaptive power transmission policy under the Rayleigh fading environment is as given below:

$$ \frac{{C_{er} }}{B} = \left( {\alpha f_{1} \left( \xi \right) + \bar{\alpha }f_{0} \left( \xi \right)} \right)log_{2} (1 + \gamma_{\mu } \left( \xi \right)\gamma_{0} \left( \alpha \right)) $$
(22)

4.2 Rician fading

The Nakagami-m distribution with the fading parameter greater than or equal to 2 represent the Rician fading channel. Now, by substituting m = 2 in (18), we get the following expression for Rician fading channel:

$$ \mathop \smallint \nolimits_{0}^{{K\gamma_{\mu } \left( \xi \right)\gamma_{0} }} \left( {\gamma_{\mu } \left( \xi \right) \gamma_{0} - \frac{z}{K}} \right)\frac{6z}{{\left( {1 + z} \right)^{4} }} f_{1} \left( \xi \right)dz = \frac{{Q_{int} }}{{N_{0} B}} $$

or

$$ f_{1} \left( \xi \right)\left( {\frac{{3K\gamma_{0} \gamma_{\mu } \left( \xi \right) + 2}}{{6K\left( {1 + K\gamma_{0} \gamma_{\mu } \left( \xi \right)} \right)^{2} }} + \frac{{\gamma_{0} \gamma_{\mu } \left( \xi \right)}}{6} - \frac{2}{6K}} \right) = \frac{{Q_{int} }}{{N_{0} B}} = \alpha $$
(23)

Therefore, for the spectrum-sharing system operating under predefined power constraints and a target BER value \( P_{b} \), the Rician fading channel capacity expression of the secondary user’s link, based on the adaptive rate and power M-QAM transmission policy, is obtained by putting m = 2 in (19):

$$ \frac{{C_{er} }}{B} = \mathop \smallint \nolimits_{{\frac{1}{{K\gamma_{0} \gamma_{\mu } \left( \xi \right)}}}}^{\infty } log_{2} \left( {K\gamma_{0} (\alpha )\gamma_{\mu } \left( \xi \right)z} \right)\frac{6z}{{\left( {1 + z} \right)^{4} }}(\alpha f_{1} \left( \xi \right) + \bar{\alpha }f_{0} \left( \xi \right))dz $$
(24)

where \( \gamma_{0} \left( \alpha \right) \) is from the (23) for a given α. Also, the Ergodic channel capacity of adaptive power transmission policy in the Rician fading environment is given by the following expression:

$$ \frac{{C_{er} }}{B} = \mathop \smallint \nolimits_{{\frac{1}{{\gamma_{0} \gamma_{\mu } \left( \xi \right)}}}}^{\infty } log_{2} \left( {\gamma_{0} \gamma_{\mu } \left( \xi \right)z} \right)\frac{6z}{{\left( {1 + z} \right)^{4} }}(\alpha f_{1} \left( \xi \right) + \bar{\alpha }f_{0} \left( \xi \right))dz $$
(25)

Similarly, we can compute the channel capacity for different fading parameters values, however it leads to cumbersome mathematical expressions.

5 Simulation results

In this section, we have numerically simulated the proposed spectrum sharing system model that operates under constraints on the average received-interference power in the Nakagami-m fading environment for the adaptation strategies such as variable power and variable rate and power as presented in the preceding Sects. 3 and 4.

The position of terminals as shown in Fig. 1 is assumed in such a way that \( d_{s} = d_{p} = 1 \) (unit) and \( {\text{d}}_{\text{m}} = 3 \) (unit). The channel gain \( \sqrt {\upgamma_{\text{s}} } \) and \( \sqrt {\upgamma_{\text{p}} } \) is distributed according to the Nakagami-m fading pdf. Furthermore, we assume \( {\text{N}}_{0} {\text{B }} = 1 \) and the sensing detector calculates the sensing-information metric in N = 30 observation samples. We suppose that the primary user remains active 50 % of the time (α = 0.5) and set the PU’s transmit power Pt = 1. Figure 2a illustrates the distribution of conditional probabilities \( f_{0} \left( \xi \right) \) and \( f_{1} \left( \xi \right) \) corresponding to the different values of detected energy by sensing detector in the particular number of samples. Moreover, these distributions are used for the computation of \( \gamma_{\mu } \left( \xi \right) \) for different detected energy values in a particular interval as shown in Fig. 2b and three regions have been recognized for the parameter \( \upgamma_{\upmu} \left(\upxi \right) \), namely, \( \upgamma_{\upmu} \left(\upxi \right) > 1 \), \( \upgamma_{\upmu} \left(\upxi \right) = 1 \) and \( \upgamma_{\upmu} \left(\upxi \right) < 1 \). In Fig. 2b, when \( \upgamma_{\upmu} \left(\upxi \right) > 1 \) represent that the probability of the PU to be idle is higher than that of being active otherwise, \( \upgamma_{\upmu} \left(\upxi \right) < 1 \). The power and rate is adapted according to the channel gains and the sensing information. Moreover, the higher power levels are used by secondary user’s when the probability of primary user being inactive is significantly more [higher values of \( \upgamma_{\upmu} \left(\upxi \right) \)] in comparison to the case for which \( \upgamma_{\upmu} \left(\upxi \right) \) is less. We have considered the bit-error-probability 10−2, 10−4 and 10−6 for the adaptive rate and power transmission policy for these two cases: (\( \upgamma_{\upmu} \left(\upxi \right) > 1 \) and \( \upgamma_{\upmu} \left(\upxi \right) < 1 \)). For the Rayleigh fading environment or Nakagami-m distribution with m = 1, Fig. 3 shows the variation of Ergodic channel capacity with Q int for adaptive power and adaptive rate and power transmission policy, while considering the sensing information metric available at the cognitive user. The simulation results in Fig. 3 are presented for the value of parameter \( \upgamma_{\upmu} \left(\upxi \right) < 1 \).

Fig. 2
figure 2

a Spectrum sensing probability density functions given that the primary user is idle \( {\text{f}}_{0} \left( {\upxi} \right) \) and active \( {\text{f}}_{1} \left( {\upxi} \right) \), b \( {\upgamma}_{{\upmu}} \left( {\upxi} \right) \) variation for N = 30, \( P_{t} = 1 \), α = 0.5 and \( {\text{d}}_{\text{m}} = 3 \) [24]

Fig. 3
figure 3

Ergodic channel capacity under the adaptive power and adaptive rate and power transmission in the Rayleigh fading channel for M-QAM modulation for \( {\upgamma}_{{\upmu}} \left( {\upxi} \right) = 0.8 \) with interference power constraint at the primary receiver

It is clear from Fig. 3, that as the interference tolerance (Q int ) at primary receiver increases, the capacity of secondary user increases due to the increase in transmit power of the secondary user. The Ergodic capacity of adaptive rate and power transmission policy is less in comparison to that of the adaptive power transmission policy since there is additional constraint on target BER. In addition to this, as the required BER decreases, the Ergodic capacity of the system is less as depicted from Fig. 3. For example, the capacity for P b of 10−6 is less than that for \( P_{b} = 10^{ - 2} \) due to the more strict constraint on the required error rate. In Fig. 4, we have considered the value of the parameter \( \upgamma_{\upmu} \left(\upxi \right) > 1 \) which shows that the probability of the primary user is being active is more than being inactive and leads to increase in the transmit power, consequently results the increase of capacity of the secondary user in comparison to the capacity that is shown in Fig. 3 where \( \upgamma_{\upmu} \left(\upxi \right) < 1 \). Further, without considering the sensing information available at the secondary user, the capacity variation with Q int have been validated with Fig. 5 of [10], which is the case when only the average interference power constraint is considered. The effects of average interference power Q int on the capacity in Nakagami-m fading environment with m = 2 that is the Rician fading channel for the adaptive power and adaptive rate and power transmission is shown in Fig. 6 for the case when \( \upgamma_{\upmu} \left(\upxi \right) < 1 \). However, for the adaptive power and adaptive rate and power transmission policy and the comparison of the capacity for three cases of BER that is 10−2, 10−4 and 10−6 is presented in Fig. 6.

Fig. 4
figure 4

Ergodic channel capacity, under adaptive power and adaptive rate and power transmission in Rayleigh fading channel for M-QAM modulation for \( {\upgamma}_{{\upmu}} \left( {\upxi} \right) = 1.2 \) with interference power constraint at primary receiver

Fig. 5
figure 5

The capacity under the average interference-power constraint as reported in [10]

Fig. 6
figure 6

Ergodic channel capacity, under adaptive power and adaptive rate and power transmission in Rician fading channel for M-QAM modulation and \( {\upgamma}_{{\upmu}} \left( {\upxi} \right) = 0.8 \) with interference power constraint at primary receiver which it can tolerate without degrading performance

Further, in Fig. 7 the capacity in Rician fading environment (Nakagami-m distribution with m = 2) for \( \upgamma_{\upmu} \left(\upxi \right) > 1 \) is presented. However, the comparison of Fig. 6 with Fig. 7 reveals that the significant enhancement in the capacity is due to the higher power adaptation of the secondary transmitter. Moreover, the capacity comparison between Rayleigh and Rician fading environment demonstrate that the capacity of cognitive radio network for later case is less than that of former for a given Q int . The reason lies in the fact that severe primary channel Rayleigh fading has given advantage to the secondary transmitter to increase its transmission power while keeping interference power constraint constant in comparison to the Rician fading channel with m = 2, which is less severe due to the presence of line-of-sight (LOS) component. Moreover, Fig. 8(a), (b) show the adaptation in the constellation size according to the channel gain ratio of the secondary-to-primary user and average interference power for different BER, respectively. It is also clear from Fig. 8(a), (b) that the number of bits per symbol or the constellation size of modulation technique increases as the channel gain ratio of the secondary transmitter to primary receiver increases, or the average interference power limit at primary receiver increases for the chosen BER. Thus significantly better channel conditions of the secondary link leads to the adaptation of higher modulation format.

Fig. 7
figure 7

Ergodic channel capacity under the adaptive power and adaptive rate and power transmission in Rician fading environment for M-QAM modulation and \( {\upgamma}_{{\upmu}} \left( {\upxi} \right) = 1.2 \) with interference power constraint at primary receiver

Fig. 8
figure 8

The constellation size adaptation a according to the signal-to-noise power ratios of secondary-to-primary user for \( Q_{int} = 0\,{\text{dB}} \), and b with the interference power constraint for the given signal-to-noise power ratios (10 dB) of secondary-to-primary user for adaptive power and rate transmission policy with \( P_{b} = 10^{ - 2} \), 10−4, and 10−6

6 Conclusion

In this paper, we have considered a spectrum-sharing concept in the cognitive radio system where the secondary user’s transmit power and rate can be adjusted based on the sensing information of the primary user and secondary user’s as well as secondary-to-primary user’s fading environment. In addition to this, the spectrum-sharing system will be operated under the average interference power constraints of the primary receiver. In this context, we have demonstrated the Ergodic capacity of the cognitive radio communication system with power and rate adaptation policy in different fading environment for chosen BER. Since the Nakagami-m distribution is fit for both the Rayleigh and Rician fading distribution by varying the fading parameter, therefore the Ergodic capacity for both these distributions have been presented. In addition to this, the numerically simulated results for the Ergodic capacity are presented for both the adaptive power and adaptive rate and power transmission policy, which reveal that the adaptive power transmission has more capacity than that of the adaptive rate and power transmission policy at the cost of BER. Moreover, we have demonstrated that the sensing parameter knowledge has provided an opportunity to control the secondary user’s transmission parameters such as rate and power according to different primary user’s activity levels observed by the sensing detector. However, the secondary transmitter can adapt different modulation by varying the value of M in M-QAM according to the channel conditions, BER and interference constraints. Further, it is also illustrated that the capacity in case of Rician fading environment is less as compare to that of Rayleigh fading because of LOS component present in the secondary transmitter to primary receiver has provided more prominent effect on the capacity of secondary user in comparison to that present in secondary transmitter to secondary receiver link.