Introduction

Abstract

With the dramatically increasing demand in mobile traffic, the dilemma between spectrum shortage and waste, which is mainly caused by the traditional static and pre-allocated spectrum access policy, has became a serious problem facing the wireless communication systems.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1.1 Interference Coordination in Dynamic Spectrum Access

1.1.1 Preliminaries

With the dramatically increasing demand in mobile traffic, the dilemma between spectrum shortage and waste, which is mainly caused by the traditional static and pre-allocated spectrum access policy, has became a serious problem facing the wireless communication systems. To solve this problem, dynamic spectrum access (DSA) is an efficient and promising approach. In recent years, based on the development of cognitive radio (CR) technology [1], in which the users have the ability to sense the environment, learn from history information, and then adjust their decisions in a smart and dynamic manner, DSA has drawn great attentions and can be used in several application scenarios. For example, in the primary-secondary spectrum access systems, the secondary users opportunistically access the channels which are unoccupied by the primary users; in heterogeneous networks, the small cells access the channels according to the spatial and temporary traffic distribution; in LTE-U systems, the users also dynamically access the licensed and unlicensed channels.

In DSA systems, the users perform spectrum access in a distributed and auto-nomous manner; furthermore, due to the feature of open transmission in wireless communications, interference becomes the primary concern [2]. Here, the considered interference metric is generalized. Specifically, it mainly includes: (i) The traditional PHY-layer interference: the interference signal received from other transmitters. (ii) The MAC-layer interference: in both time division multiple access (TDMA) and carrier sensing multiple access (CSMA) schemes, a user cannot transmit when its neighboring users are transmitting. In this scenario, the interference is different from the traditional PHY-layer interference as it only cares about the number of interfering users but not about the received interference signal. To capture this effect, the MAC-layer interference can be defined and analyzed. (iii) More generalized interference: we can generalize the interactions among the users when their objectives are conflicting. In this book, we will analyze the above-mentioned interference metrics in different scenarios.

1.1.2 Challenges and Problems

The key task in DSA systems is to choose the appropriate channels to mitigate mutual interference among multiple users. To summarize, there are some new challenges and problems:

• 1. Developing efficient distributed optimization approaches. In methodology, centralized optimization approaches have the following limitations: (i) they may cause heavy communication overhead as global information of all the users is required, and (ii) as the network scales up, the computational complexity becomes huge and unacceptable. Thus, it is desirable to develop efficient distributed optimization interference mitigation approaches.

• 2. Addressing the combinatorial nature of DSA problems. Generally, the decision variables in DSA are discrete, i.e., choosing a channel from the available channel set. For such combinatorial optimization problems, the well-known convex optimization approaches cannot be applied.

• 3. Coping with the incomplete and dynamic information constraints in wireless environment. Information is key to decision [3]. Unfortunately, incomplete information, e.g., a user only has its individual action-payoff information and partial environment information, and dynamic information constraints, e.g., the channel state is time-varying, cause challenges and new problems.

1.2 Game-Theoretic Solutions for Interference Coordination

Game theory [4] is a branch of applied mathematics, which provides efficient distributed approaches for solving problems involving multiple interactive decision-makers. Although it was originally studied in economics, it has been extensively applied into several application scenarios, e.g., biology [5], social activities [6], and engineering [7]. Since the pioneer work on applying game theory in power control [8, 9], it has been regarded as an important optimization tool for wireless networks [10, 11]. Naturally, it is very suitable for solving wireless optimization problems which are directly related to economic events and activities, e.g., spectrum auction [12–14] and incentive mechanism [15]. More importantly, it can be applied to solve any other involving multiple interactive users in wireless optimization problems, e.g., distributed power control [16], self-organizing networking [17], multiple access control [18], and heterogeneous network selection [19].

1.2.1 Motivation of Applying Game Models

In this book, we focus on developing game-theoretic interference coordination approaches for DSA systems [21]. The motivation of using game models are summarized as follows:

• 1. Game models provide a good and promising framework for distributed optimization, as the players in game take actions distributively and autonomously. Furthermore, the interactions among the players can be well analyzed and addressed using game theory.

• 2. The combinatorial nature of DSA problems can be easily addressed. In game models, the players choose actions from their available action set, which is always discrete. In this sense, it is believed that game models are very suitable for solving combinatorial optimization problems.

• 3. The dynamic and incomplete constraints can be solved, via careful design of repeated play in game models. Through repeated play, useful information can be obtained to guide the players to take actions. As a result, incomplete and dynamic information constraints can be addressed.

• 4. Smart and intelligent decision can be achieved. It is expected that learning is the core of future wireless communications [20]. On one hand, the outcome of the game is predicable and hence the performance can be improved. On the other hand, through repeated play, the players can learn from the past information and the feedback from the environment, adjust their behaviors, and finally achieve desirable and stable outcomes.

1.2.2 A General Framework of Game-Theoretic Solutions

1.2.2.1 Basic Game Models

Generally, a noncooperative game is denoted as ${G}=\{\mathbb {N},{A}_n, u_n\}$, where

• $\mathbb {N}$ is the set of players. The player set can be defined flexibly. For example, in the spectrum auction problems, players are the operators. In network management and planning, the base stations with their serving clients are the players. In wireless access problems, the mobile users are the players.

• ${A}_n$ is the available action set of player n. In most scenarios, the available action is a single decision variable, e.g., channel, time, power. However, it can also be defined as a combination of multiple decision variables, e.g., joint channel selection and power control, joint relay selection and power control.

• $u_n$ is the utility function of n. Denote $a_n \in {A}_n$ as the chosen action of player n, and $a_{-n}$ as the chosen action profile of all players except n. Then, the utility function is generally expressed as $u_n(a_n,a_{-n})$. In some scenarios, the utility of a user is only affected by the actions of neighboring users and hence the utility is then determined by $u_n(a_n,a_{J_{n}})$, where $J_n$ is the neighboring users and $a_{J_n}$ is their chosen actions.

In some application scenarios, the players may choose mixed strategies over their available action sets. Formally, the mixed strategy of player n is denoted as $\sigma _n(a_n), a_n \in A_n$, which corresponds to the probability of player n choosing action $a_n$. The mixed strategy action space of player n is then given by ${{\Sigma }_{n}}=\left\{ {{\sigma }_{n}}:\sum \nolimits _{{{a}_{n}}\in {{{A}}_{n}}}{{{\sigma }_{n}}}({{a}_{n}})=1,0\le {{\sigma }_{n}}({{a}_{n}})\le 1 \right\}$. Denote $\sigma _{-n}$ as the mixed strategy profile of all the players except n, then the expected achievable utility function of player n is determined by $u_n(\sigma _n,\sigma _{-n})=\sum \limits _{a \in {A}} \Big ( \prod \limits _{k\in \mathbb {N}} \sigma _k(a_k)\Big ) u_n(a)$.

Since all the players in non-cooperative games are selfish and rational, i.e., they all maximize their utilities, it is important to study the stable solutions of the game. In the following, some important definitions are presented.

Definition 1.1

An action profile $a^*=(a^*_1,\ldots ,a^*_{-n})$ is a pure strategy Nash equilibrium (NE) if and only if no player can improve its utility by deviating the current chosen action unilaterally, i.e.,

$$\begin{aligned} {u_n}({a^*_n},a^*_{-n}) \ge {u_n}(a_n,{a^*_{-n}}), \forall n \in \mathcal {N}, \forall a_n \in \mathcal {A}_n, a_n\ne a^*_n \end{aligned}$$

(1.1)

The concept of NE was first coined by John. Nash [22], who was awarded the 1994 Nobel Prize in Economics. It is the most important solution concept in game theory. Based on NE, some other useful concepts of equilibria, e.g., correlated equilibrium (CE) [23], evolutionary stable strategy (ESS) [24], and conjectural equilibrium [25], are also extensively studied and used.

Fig. 1.1

The general framework of game-theoretic solutions for optimization problems in wireless communications

1.2.2.2 A General Framework of Game-Theoretic Solutions

For developing game-theoretic solutions for optimization problems in wireless communications, a general framework is shown in Fig. 1.1. It is seen that there are two key steps [26]: (i) game design and formulation, and (ii) distributed learning.

• 1. Game design and formulation. One needs to first identify the player set and the corresponding available action set, and then define the utility functions of the players. Defining utility function is very important since it inherently determines the properties and achievable performance of the game-theoretic models. There are three featured principles for defining utility function in wireless communications: (i) making the stable states optimal or near-optimal, which is the ultimate purpose of optimization in wireless communications, (ii) addressing the inherent features of wireless communications, e.g., channel fading, time-varying traffic, and user mobility, and (iii) having clear physical meanings. That is, it should explicitly be related to the optimization metrics in wireless communications, e.g., achievable throughput, interference, delay, or energy-efficiency.

• 2. Design of distributed learning. In pure game theory, it is always assumed that the players can perfectly monitor the environment and the actions chosen by other players. As a result, some efficient algorithms, e.g., best response [27] and fictitious play [4], can be used to adjust their strategies toward stable solutions. However, in the presence of incomplete and dynamic information constraints in wireless communications, the task of achieving stable and desirable solutions is challenging. Thus, efforts should be given to: (i) developing efficient distributed learning algorithms which converge to stable solutions, e.g., NE, CE, and ESS, and (ii) achieving desirable stable solutions, e.g., maximizing the throughput or minimizing the interference.

Denote $a_n(k)$ as the action of player n in the kth iteration, and $a_{-n}(k)$ as the action profile of all other players except n. Due to the interactions (interference, congestion or competition) among the multiuser players, the received payoff $r_n(k)$ of each player is jointly determined by the action profile of all players, i.e., $r_n(k)={{g}_{n}}\left( {{a}_{n}}(k),{{a}_{-n}}(k) \right)$, where the payoff function $g_n(\cdot )$ may be determinate or random. Generally, the players perform the following learning procedure to update their actions:

$$\begin{aligned} {{a}_{n}}(k+1)=F\left( {{a}_{n}}(k),{{a}_{-n}}(k);{{r}_{n}}(k),{{r}_{-n}}(k) \right) , \end{aligned}$$

(1.2)

Since the action update of a player is based on the profiles of chosen action and received payoff in the last iteration, the system evolution can be described as $\left\{ {{a}_{n}}(k),{{a}_{-n}}(k) \right\} \rightarrow \left\{ {{r}_{n}}(k),{{r}_{-n}}(k) \right\} \rightarrow \left\{ {{a}_{n}}(k+1),{{a}_{-n}}(k+1) \right\}$. Thus, the optimization objective of the learning algorithm is to find a stable action profile to maximize the system utility.

As discussed before, the incomplete and dynamic information constraints may pose some challenges. Specifically, (i) a player may not know the information about all other players, (ii) the received payoff $r_n(k)$ may be random and time-varying. Thus, the update rule needs to be carefully designed to guarantee the convergence toward desirable solutions. In addition, the update rule given in (1.2) is coupled since it needs to know information about others. To reduce the information exchange among users, it is desirable to develop uncoupled learning algorithms, i.e.,

$$\begin{aligned} {{a}_{n}}(k+1)=F_u\left( {{a}_{n}}(k),{{r}_{n}}(k) \right) , \end{aligned}$$

(1.3)

where only the individual information of action and received payoff are needed. An illustrative diagram of distributed learning in games is shown in Fig. 1.2.

Fig. 1.2

The illustrative diagram of distributed learning algorithms in games

1.3 Organization and Summary

In the following, we present the definition of exact potential game [27], which admits promising properties and have been extensively used in wireless communication networks [28]. Potential game admits several promising properties and the most important two are [27]: (i) every potential game has at least one pure strategy NE, (ii) the global or local maxima of the potential function correspond to a pure strategy NE. Furthermore, there are several efficient distributed learning algorithms which converges to NE of potential games in the presence of incomplete and dynamic information constraints.

Definition 1.2

A game is a potential game (EPG) if there exists an exact potential function $\phi _e: {{A}_1} \times \cdots \times {{A}_N} \rightarrow R$ such that for all $n \in \mathbb {N}$, all $a_n \in {A}_n$, and $a'_n \in {A}_n$,

$$\begin{aligned} u_n(a_n,a_{-n})-u_n(a'_n,a_{-n}) = \phi _e(a_n,a_{-n})-\phi _e(a'_n,a_{-n}) \end{aligned}$$

(1.4)

That is, the change in the individual utility function caused by the unilateral action change of an arbitrary user is the same with that in the exact potential function. In this book, several potential game-based interference coordination approaches are presented and analyzed, the summary of game-theoretic interference coordination approaches in this work is shown Fig. 1.3. Specifically, the rest of this book is organized as follows.

• In Chap. 2, an interference mitigation game with time-varying channels is formulated and a stochastic learning automata-based algorithm is applied. The distinct feature is that the channel gains are randomly changing, which is common in practical wireless networks.

• In Chap. 3, an MAC-layer interference mitigation game with orthogonal channels is formulated and the binary log-linear learning algorithm is applied.

• In Chap. 4, an MAC-layer interference mitigation game with partially overlapping channels is formulated and the simultaneous log-linear learning algorithm is applied.

• In Chap. 5, a generalized interference mitigation game for sequential channel sensing and access is formulated and a modified stochastic learning automata-based algorithm is applied. The distinct feature is that the active user set is time-varying.

• Finally, future direction and research issues are presented in Chap. 6.

Fig. 1.3

Summary of game-theoretic interference coordination approaches in this book