SDN-based resource allocation for heterogeneous LTE and WLAN multi-radio networks

Abstract

To meet increasing bandwidth demand, cellular network operators have increasingly deployed wireless local area networks (WLANs) on top of their cellular networks [e.g., long term evolution (LTE)]. State-of-the-art smartphones can utilize such heterogeneous LTE and WLAN radio bandwidth by operating the multi-radio interfaces simultaneously. This paper investigates how such heterogeneous radio bandwidth should be controlled and allocated to multi-radio user devices. A software-defined networking (SDN)-based resource allocation framework is proposed that can properly orchestrate heterogeneous radio bandwidth in emerging LTE/WLAN multi-radio networks in a centralized and holistic manner. The proposed framework is implemented and evaluated in a computing environment that combines a real SDN controller with simulated network forwarding entities.

1 Introduction

As smart devices such as smartphones are prevalent in mobile cellular networks, mobile traffic demand has increased [1]. Such mobile traffic increase is one of the important technical challenges facing mobile network operators. In the fourth generation long term evolution (LTE) networks, operator-owned wireless local area network (WLAN) hotspots can be integrated into the cellular network cores with the packet data network gateway (P-GW) being an anchor [2]. Such WLAN hotspots can offload data traffic from cellular networks to alleviate congestion at LTE access networks; for example, internet protocol (IP) flow mobility [3–5]).

Though the recent trend of exploiting heterogeneous LTE/WLAN multi-radio technologies has emerged in the form of selecting the radio-in-use in commercial smartphones, multi-radio enabled smart devices will provide the simultaneous use of heterogeneous LTE and Wi-Fi radio interfaces in the future in order to utilize multiple radios in more efficient and sophisticated ways; namely, seamless flow mobility and bandwidth aggregation [6]. Furthermore, exploiting heterogeneous LTE and WLAN radio bandwidth from a holistic network resource allocation perspective needs investigation as exemplified by a recent work [7]. How such enhanced heterogeneous bandwidth can be centrally allocated to multi-radio user devices becomes important in terms of network utilization.

Software-defined networking (SDN) is drawing intense interest due to its disruptive network paradigm change. It has the potential for enabling various new services by separating the control plane from the data plane [8], including data center networks [9] and network virtualization [10].

Recently, attempts to employ SDN to LTE cellular networks have been made as movements toward the next-generation core network architecture. These efforts include focusing on eliminating tunnels in the existing evolved packet core (EPC) [11], selective traffic redirection to circumvent EPC [12], scaling EPC with a hierarchy of SDN controllers [13], gateway functions placement in EPC [14], placement of EPC functions based on SDN and network functions virtualization [15] and SDN-based services in LTE/WLAN dense networks [16]. Industry research and prospect includes Bell Lab’s cellular SDN architecture [17] and Juniper’s view on virtualization for telco mobile networks [18]. However, research on the SDN-based allocation problem of heterogeneous network resources in emerging LTE/WLAN networks needs investigation with considerations for multi-radio user devices.

This is the first study to apply the SDN paradigm to heterogeneous resource allocation in emerging LTE and WLAN integrated networks, and to specify the functional decomposition of the centralized solution into the network entities. Figure 1 illustrates a state-of-the-art heterogeneous LTE/WLAN multi-radio network and entities, in which LTE evolved node B (eNB) belonging to a single P-GW and WLAN access point (AP) located within the area of an LTE eNB are considered. The P-GW can be labeled an ingress/egress point for a collection of multiple eNBs and APs. Each user’s smartphone has an LTE interface and a Wi-Fi interface that can operate simultaneously. The LTE interface can operate in both uplink and downlink directions at the same time, assuming widely deployed frequency division duplex (FDD)-based LTE whereas the Wi-Fi interface can only operate in one direction (either uplink or downlink) with contention-based shared medium access.

Although the canonical distributed network utility maximization (NUM) framework has been widely studied for resource allocation in past literature [19–22], the present study proposes an SDN-based resource allocation framework that can respect heterogeneous resources in a centralized holistic manner based on the emerging SDN paradigm. The feasibility of the proposed framework through proof-of-concept implementation is demonstrated. Open research issues regarding future SDN-based mobile networks and resource allocation therein are also discussed.

Fig. 1

A heterogeneous LTE/WLAN multi-radio network with an SDN controller (P-GW packet data network gateway, S-GW serving gateway, ePDG evolved packet data gateway, eNB evolved node B, UE user equipment, AP access point)

2 Related work

2.1 SDN-based wireless mobile networks

Tailoring cellular networks, especially LTE networks, to the SDN paradigm has recently drawn the attention of research communities and industries. Some studies address the issues and solutions for an SDN-based core network (a wired part) [11–14, 23, 24], whereas others focus more on the issues and directions for an SDN-based radio access network (a wireless part) [25, 26].

Jin et al. propose a scalable and flexible LTE core network architecture that can program forwarding rules to middle boxes based on service policies [11]. Nagaraj et al. propose offloading flows from the EPC to other IP transport networks based on flow classification [12]. Moradi et al. look into the scalability issue of SDN-based cellular WANs and propose a three-level control of switch regions and a recursive topology discovery mechanism [13]. Basta et al. examine possible scenarios for placement of EPC functions with considerations for network functions virtualization (NFV) [14]. Heinonen et al. propose a mechanism for dynamically switching user plane tunnels in SDN-based cellular core networks [23]. Pentikousis et al. study applying SDN to mobile core networks and propose a flow-based forwarding architecture to benefit operators [24]. Compared with these works on SDN-based EPC, the present paper focuses on resource allocation in radio networks anchored at the same core network.

Gudipati et al. consider applying the SDN concept to LTE radio access networks, and propose abstracting multiple eNBs as a single big base station and centrally managing radio resources together for load balancing and utility optimization [25, 26]. The controller makes the abstraction of radio resources into the three dimension of frequency, time, and space, and uses an interference map to calculate radio resource allocation. It is more fine-grained than user rate-based resource allocation in the present paper. However, it deals only with user association between neighboring cells of single-radio LTE network and it does not propose particular algorithms.

There are works that seek joint coordination of radio access networks and transport core networks [27, 28]. Bernardos et al. consider accommodating various wireless mobile network in the SDN realm and propose a high-level architecture and required northbound and southbound interfaces [27]. Tan et al. propose wireless SDN architecture with a distributed user plane and a centralized function-oriented control plane, and demonstrate end-to-end quality of service (QoS) management and content-aware data broadcast in converged LTE and WLAN networks [28]. While these works target design and architecture [27, 28], the present paper focuses on a heterogeneous resource allocation problem particularly.

2.2 SDN-based control of heterogeneous wireless mobile networks

Researches relevant to the present study consider SDN-based control of heterogeneous wireless mobile networks, such as Mendonca et al.’s work [29], CROWD [16, 30], OpenRoads [31], and SoftMobile [32]. Mendonca et al. examine the possibility of applying the SDN approach in heterogeneous wired networks, infrastructure-based wireless networks, and infrastructure-less wireless networks; for example, mobile ad hoc networks [29]. They consider typical use cases like SDN-based gateway assignment and data offloading. Ali-Ahmad et al. deal with very dense networks of multiple radio technologies and propose a two-tier controller architecture for scalability in the control of dense networks [16, 30]. Yap et al. propose an OpenFlow-based wireless architecture for orchestrating and virtualizing heterogeneous wireless networks, and demonstrate a case study of WiMAX to/from Wi-Fi handover [31]. Chen et al. consider complex control plane issues in multi-radio access technology (RAT) heterogeneous mobile networks, and propose a holistic control framework for providing global network view (spectrum, connection, interference map, etc.) and abstraction along with coordination algorithms (e.g., joint scheduling and interference cancellation) [32]. Unlike these works, the present paper proposes a resource allocation framework for user-level fairness in heterogeneous wireless networks.

2.3 Software for SDN research and development

Many researches use the combination of a real SDN controller and simulated switches for the purpose of proof-of-concept implementation. Mininet [33, 34] is the first one for such combination and has been widely used for fast SDN prototyping. It is simple and easy to use as proof-of-concept prototyping. However, it does not include wireless radio modules and hence cannot be used for research targeted for wireless networks. OFOmnet [35] is OpenFlow extension for the OMNeT++ simulator. It supports wired Ethernet model but does not support wireless network models. Estinet [36] a commercial tool that provides combined OpenFlow simulation and emulation on the same machine. It supports a Wi-Fi model but does not support a LTE model. ns-3 [37] is most popular in academia for research purposes. It includes open source LTE model and WLAN model. The present paper takes the approach of exploiting well-defined wireless models (LTE, WLAN) in ns-3 and adding necessary OpenFlow modules to make forwarding entities compatible with OpenFlow controller.

Software for SDN implementation is typically open source, for example, OpenStack [38], Open vSwitch [39], and Indigo [40]. A promising testbed would be to configure such software on top of real controller machines and real physical forwarding entities for LTE and WLAN networks targeted by the present paper. However, the present paper takes a simulation-emulation approach for low-cost, proof-of-concept implementation of them and performance evaluation. It utilizes IRIS [41, 42], developed by Electronics and Telecommunications Research Institute (ETRI), as an open source controller and enables OpenFlow functions in simulated switches in ns-3 to make interoperable with IRIS.

3 SDN-based resource allocation framework

Fig. 2

The architecture of SDN-based heterogeneous resource allocation (both downlink and uplink)

3.1 Architecture

With the introduction of an SDN controller to the target network, all the cellular network entities with packet forwarding functionality are presumed to follow forwarding rules instructed by the central SDN controller. Figure 2 illustrates this study’s proposal that the SDN controller should centrally gather network topologies and make decisions on packet forwarding and heterogeneous resource allocation in the target LTE/WLAN multi-radio network, and then instruct forwarding rules and resource allocation rules (including each user’s LTE downlink rate share, LTE uplink rate share, and WLAN bidirectional rate share) to the network entities. Compared with separate control of heterogeneous resources, the centralized holistic architecture for heterogeneous resource allocation can well respect both total network utilization and user rate fairness as demonstrated in performance evaluation.

A link discovery module at the SDN controller uses OpenFlow protocol messages (e.g., Hello, Port Status messages) to locate switching entities in the network, and triggers the link discovery process by exploiting OpenFlow Packet Out message with Link Layer Discovery Protocol (LLDP) packet. Depending on the link discovery result when changes in each user u’s attachment and radio states occur, the SDN controller generates relevant forwarding rules for forwarding entities (P-GW, S-GW, and eNB for LTE, and ePDG and AP for WLAN) . These rules can be exchanged between the SDN controller and the switching entities using OpenFlow protocol [8] (e.g., OpenFlow Flow Modify messages).

All global information about the target network (e.g., user device’s attachment, user device’s radio states, or user device flows) is centrally available at the SDN controller, thereby making it the best place to allocate heterogeneous radio resources with little communication overhead. The resulting user resource allocation rules regarding each user u’s rate share information ($x^\mathrm{lte,dl}_{u}$ for LTE downlink, $x^\mathrm{lte,ul}_{u}$ for LTE uplink, and $x^\mathrm{wifi}_{u}$ for WLAN bidirectional link) are delivered to switching entities for downlink traffic regulation, and further to user devices for uplink traffic regulation. Since the WLAN link is bidirectional, an actual portion of $x^\mathrm{wifi}_{u}$ in the downlink direction and in the uplink direction, respectively, needs to be determined by a user device and notified to the involved switching entities. The resource allocation rules are exchanged by extending OpenFlow 1.0 Vendor messages. It is noted that OpenFlow 1.3 meter table gadget can now serve rate control purposes.

3.2 Design details

Figure 3 illustrates the more detailed software design of an SDN controller with proposed heterogeneous resource allocation capability and forwarding entities that it controls. This figure also highlights how the proposed framework is designed and implemented in an integration of a real SDN controller and switches in a packet-level simulator space.

IRIS is utilized as a real SDN controller, which is a carrier-grade SDN controller developed by Electronics and Telecommunications Research Institute (ETRI), South Korea [41]. Like Floodlight, IRIS is Java-based and portable on any operating systems. IRIS is adopted due to the each of programming new controller modules and better performance (e.g., supporting more flows). The basic building blocks of IRIS include a link discovery module (discovering links between neighbor switches), a topology management module (constructing an entire of topology of network switches), a device management module (keeping track of devices attached to switches), and a forwarding module (determining forwarding rules for flows). IRIS is also enhanced with a heterogeneous resource allocation module proposed in this study.

ns-3 is employed as a simulator containing forwarding entities in an LTE and WLAN multi-radio network. ns-3 is a reliable open source packet-level simulator widely used in academia. Furthermore, its LTE and WLAN forwarding entities (LTE eNB and P-GW, and WLAN AP and router) are properly implemented by built-in simulation models, which are not supported in other simulators like Mininet [33] and OMNet++ [35]. EstiNet [36] supports LTE models, but it is not open source-based. The ns-3 LTE and WLAN forwarding entities are enhanced with OpenFlow compliant modules by integrating OpenFlow Software Implementation Distribution (OFSID). In the present study, the forwarding entities in the simulation space are also enhanced to use ns-3 built-in emulation devices (class EmuNetDevice) to communicate with the real SDN controller throughout a real connected network. The LTE simulation model in ns-3.19 is employed, which includes spectrum channel model, Gaussian interference-based PHY model, and MIMO antenna model. LTE simulation parameters such as 10 MHz bandwidth, multi-antenna spatial multiplexing mode, and periodic wideband channel quality indicator (CQI) can be set. The WiFi simulation model in ns-3.19 is used, which includes YansWiFi channel. Employing these well-verified built-in modules can assure the reliability of performance evaluation of the proposed framework.

The joint use of the real SDN controller and separate forwarding entities can make the functional verification of the proposed framework more reliable in terms of OpenFlow compliance and proof-of-concept demonstration. Furthermore, the use of the scalable carrier-grade SDN controller which can support more than five million flows at a time [41] and configure a hierarchy of controllers [42] can increase the applicability and feasibility of the proposed framework to large-scale LTE/WLAN multi-radio networks in practice.

Fig. 3

The integration of a real SDN controller and OpenFlow-based LTE/WLAN forwarding entities in a packet-level simulation space

3.3 Heterogeneous resource allocation module

A heterogeneous resource allocation module, included in the SDN controller, computes LTE and WLAN rate share information for each active user that actually has data traffic to be communicated. Typically, only a few tens of users remain active to transmit and receive data in real cellular networks, so idle state users do not need to be considered in resource allocation. Any resource allocation algorithm can be adopted in the proposed framework, but this paper proposes two price-based fair heterogeneous resource allocation algorithms to the problem. It is natural that each user’s rate share will be reduced in some fair manner as the number of users in a cell increases.

3.3.1 Problem formulation: heterogeneous LTE/WLAN multi-radio resource allocation

The problem of resource allocation in LTE and WLAN multi-radio networks is formulated in Equation (1) by considering all available heterogeneous bandwidth of state-of-the-art smartphone users. The per-user utility function $U_{u}(\cdot )$ of the sum of all heterogeneous bandwidth allocated to a user is employed. The formulation can reflect heterogeneous bandwidth aggregation enabled in an LTE and WLAN multi-radio network [6]. As usual, it is assumed that the access link capacity is much lower than that of the corresponding core network so we only focus on access radio links in formulation . As in canonical NUM frameworks [19–21], this study’s formulation can reflect a network utilization aspect and a fairness aspect and can make tradeoffs between them by using various utility functions.

$$\begin{aligned} \text {max}&\sum _{u} U_{u} {(x_{u})}=\sum _{u} U_{u} {(x_{u}^{\mathrm{lte,dl}}+x_u^{\mathrm{lte,ul}}+x_u^\mathrm{wifi})} \nonumber \\ \text {subject to}&\sum _{u} \tfrac{x_{u}^{\mathrm{lte,dl},b}}{r_{u}^{\mathrm{lte,dl},b}} \le 1, \quad \forall b, \nonumber \\&\sum _{u} \tfrac{x_{u}^{\mathrm{lte,ul},b}}{r_{u}^{\mathrm{lte,ul},b}} \le 1,\quad \forall b, \nonumber \\&\sum _{u} \tfrac{x_{u}^{\mathrm{wifi},a}}{R^{\mathrm{wifi},a}} \le 1, \quad \forall a, {\mathbf {0}}\preceq {\mathbf {x}} \end{aligned}$$

(1)

We first describe the key radio resources in formulation. LTE radio resource is in two-dimensional grids of time and frequency [43]. Frequency is typically 5, 10, and 20 MHz in the field, and time unit is 1 ms subframe per resource scheduling period. LTE FDD, widely used in the field, is assumed so LTE downlink and uplink radio resources are separated in the frequency domain. The unit of resource allocation is resource block (RB) of 12 subcarriers and 7 symbols, which corresponds to time-frequency block of 0.5 ms and 180 kHz. For each RB, the eNB determines which user (that is UE) will use it at which modulation and coding scheme (MCS) level. Downlink and uplink MCS level is impacted by both the user’s wanted signal strength and inter-cell interference. In formulation, the allocated resource is abstracted into the user’s LTE downlink rate ($x^{\mathrm{lte,dl}}_u$) and uplink rate ($x^{\mathrm{lte,ul}}_u$) which are determined by the number of RBs and MCS level for the user. Meanwhile, WLAN radio resource is allocated in unit of frame airtime since the user receives the whole frequency bandwidth at once, for example, 20 MHz for IEEE 802.11n. Also, WLAN radio resource is either in uplink direction or in downlink direction in the same frequency. In formulation, the allocated resource is abstracted into the user’s WLAN rate $(x^{\mathrm{wifi}}_u)$ which is determined by the airtime for the user. Note that the proposed SDN-based framework considers all these radio resources (LTE downlink, LTE uplink, and WLAN) collectively.

In Eq. (1), $x_{u}$ is the aggregate rate allocated for user $u \in U$, including LTE downlink rate share i.e., link utilization $(x_{u}^{\mathrm{lte,dl}})$, LTE uplink rate share $(x_{u}^{\mathrm{lte,ul}})$ and WLAN bidirectional rate share $(x_{u}^{\mathrm{wifi}})$. U is a set of user devices in LTE eNBs. Note that $x_{u}^{\mathrm{lte,dl}} = x_{u}^{\mathrm{lte,dl}, {\mathfrak {q}}(u)}$ and $x_{u}^{\mathrm{lte,ul}} = x_{u}^{\mathrm{lte,ul}, {\mathfrak {q}}(u)}$ where ${\mathfrak {q}}(u)$ represents the LTE eNB where user u camps on. $r_{u}^{\mathrm{lte,dl},b}$ and $r_{u}^{\mathrm{lte,ul},b}$ represent each user u’s achievable downlink rate and uplink rate in its eNB b, respectively. Note also that $x_{u}^{\mathrm{wifi}} = x_{u}^{\mathrm{wifi}, {\mathfrak {p}}(u)}$ where ${\mathfrak {p}}(u)$ represents a WLAN AP, if any, associated with user u. If user u does not have any associated WLAN AP, $x_{u}^{\mathrm{wifi}}=0$.

The first constraint implies that the sum of each user’s ratio of its allocated LTE downlink rate to its achievable downlink rate must be below one (hundred percent airtime) in an LTE eNB $b \in B$ (a set of LTE eNBs). Likewise, the second constraint means that the sum of each user’s ratio of its actual LTE uplink rate to its achievable uplink rate must be below one. The third constraint involves each WLAN AP $a \in A$ (a set of WLAN APs) where all its associated users can only consume the WLAN link capacity collectively in both downlink and uplink $(R^{\mathrm{wifi},a})$. Here, for the brevity of the article, different per-user achievable rates are considered for LTE only, but not for WLAN, although multi-rate features can be similarly applied to WLAN in formulation. ${\mathbf {x}}$ is a user rate vector with $0\le x_{u},\forall u\in U$.

3.3.2 Proportional fair resource allocation algorithm

The Lagrangian can be obtained by specifying a set of Lagrange multipliers, ${\varvec{\lambda }}$, ${\varvec{\mu }}$, and ${\varvec{\nu }}$, individually associated with LTE downlinks, LTE uplinks, and WLAN links (both downlink and uplink), respectively, by the three constraints in Problem (1). $\lambda _b$ serves as the price information for downlink in an LTE eNB b whereas $\mu _b$ serves as the price information for uplink in b. $\nu _a$ serves as the price information for a WLAN AP a. As in proximal optimization algorithms, the quadratic term is subtracted from the original objective function to make a new objective function strictly concave with respect to $x_u^{\mathrm{lte,dl}}$, $x_u^{\mathrm{lte,ul}}$, and $x_u^{\mathrm{wifi}}$. c is a small constant.

$$\begin{aligned}&L({\mathbf {x}},{\varvec{\lambda }},{\varvec{\mu }},{\varvec{\nu }})= \sum _{u} U_{u} {\left( x_{u}^{\mathrm{lte,dl}}+x_u^{\mathrm{lte,ul}}+x_u^{\mathrm{wifi}}\right) } - c\sum _{u} \Biggl (\left( x_{u}^{\mathrm{lte,dl}}-y_{u}^{\mathrm{lte,dl}}\right) ^2\nonumber \\&\qquad +\left( x_u^{\mathrm{lte,ul}}-y_u^{\mathrm{lte,ul}}\right) ^2 +\left( x_u^{\mathrm{wifi}}-y_u^{\mathrm{wifi}}\right) ^2\Biggr )-\sum _{b=1}^B \lambda _{b}\left( \sum _{u} \tfrac{x_{u}^{\mathrm{lte,dl},b}}{r_{u}^{\mathrm{lte,dl},b}} - 1\right) \nonumber \\&\qquad -\sum _{b=1}^B \mu _{b} \left( \sum _{u} \tfrac{x_{u}^{\mathrm{lte,ul},b}}{r_{u}^{\mathrm{lte,ul},b}} \!-\! 1\right) \!-\!\sum _{a=1}^{A} \nu _{a} \left( \sum _{u} \tfrac{x_{u}^{\mathrm{wifi},a}}{R^{\mathrm{wifi},a}}-1 \right) \end{aligned}$$

(2)

The dual function for the Lagrangian is

$$\begin{aligned} D({\varvec{\lambda }},{\varvec{\mu }}, {\varvec{\nu }})= & {} \max _{{\mathbf {x}}\succeq {\mathbf {0}}}L({\mathbf {x}},{\varvec{\lambda }}, {\varvec{\mu }},{\varvec{\nu }}) \end{aligned}$$

(3)

and the dual problem to (2) is

$$\begin{aligned} \text {min}&D({\varvec{\lambda }},{\varvec{\mu }}, {\varvec{\nu }})\nonumber \\ \text {subject to}&{\varvec{\lambda }} \succeq 0, {\varvec{\mu }} \succeq 0, {\varvec{\nu }}\succeq {\mathbf {0}} \end{aligned}$$

(4)

The gradient projection method is applied to solve the dual problem (4). The Lagrange multiplier for the LTE downlinks ${\varvec{\lambda }}$, the LTE uplink ${\varvec{\mu }}$, and the WLAN links ${\varvec{\nu }}$ is updated at each iteration by subtracting the step size $\gamma$ multiplied by the partial derivative of the dual function with respect to $\lambda _b$ and $\mu _b$, and $\nu _a$, respectively. $[z]^{+}=\max \{z,0\}$ and $c=\frac{1}{2}$ are used.

$$\begin{aligned} \lambda _{b}(t+1)&=\left[ \lambda _{b}(t)-\gamma \frac{\partial D({\varvec{\lambda }}(t), {\varvec{\mu }}(t), {\varvec{\nu }}(t))}{\partial \lambda _{b}}\right] ^{+}\nonumber \\&=\Biggl [\lambda _{b}(t)-\gamma \Biggl ( 1-\sum _{u} \tfrac{x^{\mathrm{lte,dl},b}_{u}({\varvec{\lambda }}(t), {\varvec{\mu }}(t), {\varvec{\nu }}(t))}{r_{u}^{\mathrm{lte,dl},b}}\Biggr )\Biggr ]^{+} \nonumber \\ \mu _{b}(t+1)&=\left[ \mu _{b}(t)-\gamma \frac{\partial D({\varvec{\lambda }}(t), {\varvec{\mu }}(t), {\varvec{\nu }}(t))}{\partial \mu _{b}}\right] ^{+}\nonumber \\&=\Biggl [\mu _{b}(t)-\gamma \Biggl ( 1-\sum _{u} \tfrac{x^{\mathrm{lte,ul},b}_{u}({\varvec{\lambda }}(t), {\varvec{\mu }}(t), {\varvec{\nu }}(t))}{r_{u}^{\mathrm{lte,ul},b}}\Biggr )\Biggr ]^{+} \nonumber \\ \nu _{a}(t+1)&=\left[ \nu _a(t)-\gamma \frac{\partial D({\varvec{\lambda }}(t), {\varvec{\mu }}(t), {\varvec{\nu }}(t))}{\partial \nu _{a}}\right] ^{+}\nonumber \\&=\Biggl [\nu _{a}(t)-\gamma \Biggl ( 1-\sum _{u} \tfrac{x^{\mathrm{wifi},a}_{u}({\varvec{\lambda }}(t), {\varvec{\mu }}(t), {\varvec{\nu }}(t))}{R^{\mathrm{wifi},a}}\Biggr )\Biggr ]^{+} \end{aligned}$$

(5)

The SDN controller’s resource allocation module updates each user rate $x_u$ in response to the updates of Lagrange multipliers ${\varvec{\lambda }}$, ${\varvec{\mu }}$, and ${\varvec{\nu }}$.

$$\begin{aligned} x^{\mathrm{lte,dl}}_{u}(t+1)&= \Biggl [(1-\gamma )x^{\mathrm{lte,dl}}_{u}(t)+\gamma y^{\mathrm{lte,dl}}_{u}(t) + \alpha \Biggl ( 1 - \frac{\lambda _{{\mathfrak {q}}(u)}(t)}{r_u^{\mathrm{lte,dl},{{\mathfrak {q}}(u)}}U'_{u}(x_{u}(t))} \Biggr )\Biggr ]^{+}\nonumber \\ x^{\mathrm{lte,ul}}_{u}(t+1)&= \Biggl [(1-\gamma )x^{\mathrm{lte,ul}}_{u}(t)+\gamma y^{\mathrm{lte,ul}}_{u}(t) + \alpha \Biggl ( 1 - \frac{\mu _{{\mathfrak {q}}(u)}(t)}{r_u^{\mathrm{lte,ul},{{\mathfrak {q}}(u)}}U'_{u}(x_{u}(t))} \Biggr )\Biggr ]^{+}\nonumber \\ x^{\mathrm{wifi}}_{u}(t+1)&= \Biggl [(1-\gamma )x^{\mathrm{wifi}}_{u}(t)+\gamma y^{\mathrm{wifi}}_{u}(t) + \alpha \Biggl ( 1 - \frac{\nu _{{\mathfrak {p}}(u)}(t)}{R^{\mathrm{wifi},{{\mathfrak {p}}(u)}}U'_{u}(x_{u}(t))} \Biggr )\Biggr ]^{+}\nonumber \\ x_{u}(t+1)&=x_{u}^{\mathrm{lte,dl}}(t+1)+x_u^{\mathrm{lte,ul}}(t+1)+x_u^{\mathrm{wifi}}(t+1)\nonumber \\ y^{i}_{u}(t+1)&=(1-\beta )y^{i}_{u}(t)+\beta x^{i}_{u}(t) \end{aligned}$$

(6)

At each iteration for user u, link rate variables ($x^{\mathrm{lte,dl}}_{u}$, $x^{\mathrm{lte,ul}}_{u}$, and $x^{\mathrm{wifi}}_{u}$) and corresponding auxiliary variables ($y^{\mathrm{lte,dl}}_{u}$, $y^{\mathrm{lte,ul}}_{u}$, and $y^{\mathrm{wifi}}_{u}$) are updated. The superscript i can represent LTE downlink, LTE uplink or WLAN link. Eqs. 5,  6 form the convergence loop to reach a proportional fair resource state and are included as a module of the SDN controller. Whenever changes in user device attachment, user device radio states, or user flows occur, the convergence loop is continuously iterated from the previous point, rather than from the initial point, for faster convergence. The convergence of the proposed algorithm is guaranteed by the use of proximal optimization theory as introduced in Bertsekas et al. [44], applied to wired multipath routing in Lin et al. [45], and applied to wireless mesh multipath routing in Li et al. [46].

3.3.3 Max-min fair resource allocation algorithm

Max-min fairness has been widely studied in wireless network domains in terms of power control, link scheduling, topology control, routing, channel assignment, and rate allocation [47]. Max-min fairness should also be a natural and important fairness criteria for heterogeneous multi-radio networks. The second proposed resource allocation algorithm seeks to maximize the minimum user rate by solving the following formulation.

$$\begin{aligned} \text {max} \min _{u} {x_{u}}&=\min _{u} {(x_{u}^{\mathrm{lte,dl}}+x_u^{\mathrm{lte,ul}}+x_u^{\mathrm{wifi}})} \nonumber \\ \text {subject to} \sum _{u} \tfrac{x_{u}^{\mathrm{lte,dl},b}}{r_{u}^{\mathrm{lte,dl},b}}&\le 1, \quad \forall b, \nonumber \\ \sum _{u} \tfrac{x_{u}^{\mathrm{lte,ul},b}}{r_{u}^{\mathrm{lte,ul},b}}&\le 1, \quad \forall b, \nonumber \\ \sum _{u} \tfrac{x_{u}^{\mathrm{wifi},a}}{R^{\mathrm{wifi},a}}&\le 1, \quad \forall a, {\mathbf {0}}\preceq {\mathbf {x}} \end{aligned}$$

(7)

The price information and update rules are the same as in Eq. (5). The user rate is updated according to the following rules in Eq. (8).

$$\begin{aligned} x^{\mathrm{lte,dl}}_{u}(t+1)&= \Biggl [(1-\gamma )x^{\mathrm{lte,dl}}_{u}(t)+\gamma y^{\mathrm{lte,dl}}_{u}(t) + \alpha ( 1 - \lambda _{{\mathfrak {q}}(u)}(t)x_{u}(t))\Biggr ]^{+} \nonumber \\ x^{\mathrm{lte,ul}}_{u}(t+1)&= \Biggl [(1-\gamma )x^{\mathrm{lte,ul}}_{u}(t)+\gamma y^{\mathrm{lte,ul}}_{u}(t) + \alpha ( 1 - \mu _{{\mathfrak {q}}(u)}(t)x_{u}(t))\Biggr ]^{+}\nonumber \\ x^{\mathrm{wifi}}_{u}(t+1)&= \Biggl [(1-\gamma )x^{\mathrm{wifi}}_{u}(t)+\gamma y^{\mathrm{wifi}}_{u}(t) + \alpha ( 1 - \nu _{{\mathfrak {p}}(u)}(t)x_{u}(t))\Biggr ]^{+}\nonumber \\ x_{u}(t+1)&=x_{u}^{\mathrm{lte,dl}}(t+1)+x_u^{\mathrm{lte,ul}}(t+1)+x_u^{\mathrm{wifi}}(t+1)\nonumber \\ y^{i}_{u}(t+1)&=(1-\beta )y^{i}_{u}(t)+\beta x^{i}_{u}(t). \end{aligned}$$

(8)

4 Performance evaluation

The integration of the real SDN controller (IRIS) and forwarding entities in the well-established packet-level simulator space (ns-3) can serve as a versatile simulation-cum-emulation test environment for LTE/WLAN multi-radio network scenarios. First, the functional verification of the proposed framework can be assured. Second, the network utilization and fairness performance of the proposed framework in various scenarios can be characterized.

The parameters of the resource allocation algorithms are as follows: $\gamma =0.1$ and $\alpha =0.2$. For the purpose of fast convergence, $\beta =0.9$ is also used. Recall that $\sum _{u}U_{u}(x_{u})=\sum _{u}\log {(x_{u})}$ is maximized for proportional fair resource allocation [20] and $\min _{u}x_u$ is maximized for max-min fair resource allocation. Scenarios with an LTE cell with WLAN APs available within it and a neighbor LTE cell are used to reflect LTE inter-cell interference. In the first cell, there are three users, and users 1, 2, and 3 are placed at cell center, mid-cell, and cell edge locations, respectively. Each user can be in the coverage of an individual Wi-Fi AP depending on a scenario. There are eight scenarios where users 1, 2, and/or 3 can be placed either in Wi-Fi AP or not in Wi-Fi AP. The number of users is small so that important factors like locations and available AP can be easily analyzed. In Scenario 1, none have Wi-Fi AP available. In Scenario 2, only user 1 at a cell center has Wi-Fi AP. In Scenario 3, only user 2 at mid-cell has Wi-Fi AP. In Scenario 4, only user 3 at cell edge has Wi-Fi AP. In Scenario 5, users 1 and 2 have individual Wi-Fi APs. In Scenario 6, users 1 and 3 have individual Wi-Fi APs. In Scenario 7, users 2 and 3 have individual Wi-Fi APs. In Scenario 8, users 1, 2, and 3 all have individual Wi-Fi APs. In the second cell, there are three users, and user 4, 5, and 6 are placed at a cell edge, mid-cell, and cell center location, respectively. Due to inter-cell interference between the two cells, each user’s downlink MCS and uplink MCS level is different depending on its location. Initially, all six users use LTE only, and later at simulation time 5 the three users in the first cell also start to utilize 10 Mbps share of individually available, if any, Wi-Fi radio. This scenario adequately highlights the adaptation of resource allocation in response to heterogeneous bandwidth change. User datagram protocol (UDP) is used to provide only multiplexing and de-multiplexing functions of a transport layer. UDP does not perform transport-layer end-to-end congestion control (e.g., TCP’s congestion control) so user rates are not influenced by congestion control. This helps investigate the performance impact of the proposed resource allocation accurately. UDP lacks reliable delivery but it is expected that underlying medium access layer (MAC) layer provides reliability mechanism such as hybrid automatic repeat request (HARQ).

Fig. 4

The total network utility for eight different scenarios. a The topology and scenarios. b The total network utility

4.1 Total network utility

Figure 4 summarizes the total network utility for the eight aforementioned scenarios by using a utility function $\log (\cdot )$. For comparison purposes, the baseline case of resource allocation is included, which does not consider heterogeneous LTE and WLAN resources together, but instead separately considers each resource (indicated as “non-SDN” in the figure). In all scenarios, the SDN-based proportional allocation outperforms others, which demonstrates the strength of holistic consideration of heterogeneous resources. Interestingly, in some scenarios where mid-cell or cell edge users only have WLAN APs available (Scenarios 4, 5, 6, and 7), the baseline case shows lower network utility than the SDN-based max-min allocation, which seeks to equalize all user rates by sacrificing the total network utility. The reason for this is that the baseline case did not consider LTE and WLAN resources together, thereby demonstrating the need for allocating heterogeneous LTE and WLAN resources holistically.

Fig. 5

The convergence of the two SDN-based resource allocation algorithms (in Scenario 7)

4.2 Convergence of resource allocation algorithms

Figure 5 illustrates the convergence property of the proposed resource allocation algorithms in Scenario 7. Suppose that all user rates start from 2 Mbps initially, but in general the use of the gradient projection method provides convergence regardless of initial values [20]. In SDN-based proportional allocation, user rates converge to proportionally fair states after about 2000 iterations of the convergence loop. Note that the iteration number indicates the computation number in the convergence loop in the SDN controller’s resource allocation module, so it should not be confused as time. At simulation time 5 there is a change in heterogeneous bandwidth, i.e., additional use of Wi-Fi radio for users 2 and 3. From this point, the convergence loop is triggered and the convergence to new fair states takes about 3000 iterations (from iteration number 5000). Meanwhile, in SDN-based max-min allocation, user rates converge within 1000 iterations, both initially and in response to heterogeneous bandwidth change.

4.3 Trajectory of user rates

Figure 6 shows the change of user rates over time. From simulation time 1, the user’s application starts to send and receive packets according to the resource allocation rules from the SDN controller. There is some latency prior to reaching fair states which is related to the computational time in the convergence loop and exchange of OpenFlow Vendor messages. At simulation time 5, there is the addition of Wi-Fi radio at some users, and after some latency user rates show proportionally fair states. Note that fluctuation in user rates is due to the inherent nature of Wi-Fi radio’s medium access. There is some randomness in packet transmission and reception because of the presence of collision and collision avoidance mechanism. Meanwhile, LTE eNB’s deterministic RB scheduling enables a more stable nature of the rates achieved when using LTE only.

Fig. 6

The trajectory of user rates

Fig. 7

The tradeoff between total network throughput and user fairness

4.4 Tradeoff between network utilization and fairness

In general, there is a fundamental tradeoff between total network utilization and fairness in any network. Figure 7 illustrates how the proposed SDN-based resource allocation makes a tradeoff between heterogeneous resource utilization and multi-radio user fairness. In this figure, Jain’s fairness index is used and eight different scenarios are indicated. The SDN-based proportional allocation seeks to increase total network utilization by using both LTE and WLAN resources while maintaining the same fairness as when using LTE resources only (Scenario 1). The SDN-based proportional allocation of multiple heterogeneous radio resources respects the same rationale in proportionally fair allocation of a single resource in traditional cellular networks. It can be observed that the SDN-based max-min allocation seeks user equalization while utilizing heterogeneous resources. Note that the baseline case cannot provide a good tradeoff in half of the scenarios (Scenarios 3, 4, 6, and 7).

The tradeoff between network utilization (throughput) and fairness can be better understood based on $\alpha$-fairness [48, 49]. The $\alpha$-fairness framework uses the utility function for user u is given in Eq. 9 and attempts to maximize the total utility sum.

$$\begin{aligned} U_u(x_u)=\frac{(x_u)^{1-\alpha }}{1-\alpha } \end{aligned}$$

(9)

Setting $\alpha =0$ seeks to maximize the sum of user throughput by giving less resources to users who need the higher ratio of resource per unit throughput and thus sacrificing fairness. Setting $\alpha \rightarrow \infty$ means to achieve throughput equalization by giving more resources to such users and is known as max-min fairness. Between the two extreme cases, setting $\alpha \rightarrow 1$ (corresponding to $U_u(x_u)=\log (x_u)$) attempts to strike a balance between throughput and fairness and is known as proportional fairness. Although the tradeoff in SDN-based resource allocation [49] and that in SDN-based multi-radio resource allocation in the present paper are studied in simulation, examining the tradeoff mathematically deserves one of interesting further research topics [50].

Fig. 8

The queueing analysis of packet delay. a The queueing model. b The average packet delay

5 Discussion

5.1 Analytical model for packet delay

In general, packet delay is one of important performance metrics. The packet delay in the proposed SDN-based mobile network can be investigated by using the queueing theory-based approaches taken by some prior works [51, 52]. Figure 8a describes the queueing analysis model for the proposed SDN-based LTE/WLAN network. It is assumed that the queue at the P-GW and the SDN controller follows M/M/1 model as in the prior works. The queue at the eNB follows M/D/1 model as in [5] since the radio transmission time is deterministic. It is notable that in the proposed queueing model for the SDN-based mobile network, only the P-GW needs to consult the controller for determining paths for downlink packets to user nodes. Meanwhile, the eNB may just forward downlink packets over the LTE air without need to consult the controller. The arrival rate at the P-GW is denoted by $\lambda$. With the probability $q^{nf}$, there is no matching flow entry for a packet and thus the packet is forwarded to the controller. The controller determines a forwarding rule for the packet and forward the packet back to the P-GW. Suppose that the service rate at the P-GW and the controller are $\mu _s$ and $\mu _c$, respectively. Following the novel modified Jackson model for SDN in [52], the average time spent in the queues of the P-GW and the controller is obtained as $\frac{1+q^{nf}}{\mu _s-(1+q^{nf})\lambda } + \frac{q^{nf}}{\mu _c - q^{nf}\lambda }$. The packet departing the P-GW is assumed to arrive in the queue at an eNB with the equal arrival rate $\lambda _b$, which is $\lambda$ divided by the number eNBs attached to the P-GW. The service rate at the eNB is $\mu _b$. Based on the M/D/1 model, the packet waiting time and service time over the LTE air are obtained as $\frac{\lambda _b}{2\mu _b(\mu _b-\lambda _b)}$ and $\frac{1}{\mu _b}$, respectively. Putting all together, the average time in the whole system for a packet is obtained in Eq. 10.

$$\begin{aligned} E[T]= \frac{1+q^{nf}}{\mu _s-(1+q^{nf})\lambda } + \frac{q^{nf}}{\mu _c - q^{nf}\lambda } + \frac{\lambda _b}{2\mu _b(\mu _b-\lambda _b)} + \frac{1}{\mu _b} \end{aligned}$$

(10)

Figure 8b presents a brief illustration of the average packet delay for different arrival rate $\lambda$. The parameters $\mu _c=4175$ (responses per second), $\frac{1}{\mu _s}=9.8$ (us), and $q^{nf}\in \{0.04, 0.2, 0.5, 1\}$ are used in the same way as the prior works [51, 52]. The $\mu _b$ is 5000 packets per second assuming 10 MHz LTE bandwidth. The arrival rate $\lambda$ is varied up to $\mu _c$. For the lower value of $q^{nf}$, the delay is almost less changing since the probability of packet being forwarded to the controller is low, and thus, the controller queue has only to process light traffic. Meanwhile, for the higher value of $q^{nf}$, the controller queue suffers from heavy traffic and the queueing delay at the controller becomes a major factor. Although a simple analytical model for the SDN-based mobile network is examined here, we believe that more thorough study considering practical aspects deserves an interesting future work.

5.2 Consideration for priority traffic

The proposed resource allocation framework deals with only the aggregate traffic for each user but does not consider priority traffic or differentiation of traffic service. To support priority traffic, the current formulation should be modified in a similar way to a previous work [53]. For example, suppose that a user u requires priority traffic service class c with minimum guaranteed rate $Q_c$. Then an additional constraint, $x^{\mathrm{lte,dl}}_u + x^{\mathrm{lte,ul}}_u + x^{\mathrm{wifi}}_u \ge Q_c$, should be included in Eq. 1 and as a result, a corresponding price variable associated with the constraint should be included in Eq. 2. Similarly constraints for other users with priority traffic can be set. However, the issue of resource allocation of priority traffic in the SDN domain deserves in-depth study and it is left for possible future work.

6 Concluding remarks

This is the first study, to the best of the authors’ knowledge, to apply and extend the SDN paradigm to the problem of heterogeneous resource allocation in emerging LTE and WLAN multi-radio networks. An SDN-based centralized holistic framework was designed to allocate heterogeneous radio resources to multi-radio smartphone users with the intended objectives of network utilization and fairness. The proposed framework was implemented in an integration of a real SDN controller and LTE/WLAN forwarding entities based on well-established modules in a simulation space, which enables more reliable functional verification and performance evaluation. Price-based heterogeneous resource allocation algorithms in LTE/WLAN multi-radio networks were also proposed and evaluated the integrated test environment.

Though the focal point of the paper was to introduce the utility of SDN-based heterogeneous resource allocation in emerging LTE/WLAN multi-radio networks, some important practical considerations involved with future SDN-based large-scale cellular networks arose, which suggest other possible research directions. The first issue is related to the inherent dynamic behavior of wireless mobile networks. Each user’s perceived signal quality and correspondingly assigned MCS vary depending on location and time and user devices could change the point of attachment, i.e., handover, frequently depending on user movement. Though up to 80 % of mobile traffic comes from static situations where user MCS is rather stable [54], traffic in dynamic situations should also be recognized and a different allocation scheme of heterogeneous resources should be applied to it than other kinds of traffic. The authors think that this deserves another stream of research on the SDN-based evolved packet system mobility management (EMM) aspect of LTE/EPC which also involves the near-edge deployment design of multiple SDN controllers. The second issue is scalability. A single SDN controller cannot coordinate an ISP’s entire cellular network consisting of, for example, tens of millions of users and dozens of base stations in South Korea. A hierarchy of multiple SDN controllers is a promising method of enhancing the scalability of SDN-based cellular networks and heterogeneous resource allocation therein similar to recently proposed scalable architectures [13, 16]. Though IRIS adopted in this paper can also configure a hierarchy of controllers for large-scale networks by abstracting a controller’s regional network as a single switch [42], the method of deploying them to LTE/WLAN multi-radio networks, as well as placing and decomposing functions to them, deserves future research.