Performance issues and solutions in SDN-based data center: a survey

Abstract

Due to the development of the Internet and the smart end systems such as smartphones, portable laptop, and other smart mobile devices, as well as the emergence of concepts such as cloud computing, social networks, and Internet of Things, 4G and 5G have changed network requirements. A software-defined network (SDN) is a new architecture to support new network requirements. This architecture is composed of three layers, data, control, and application plane. A lot of papers have been published in well-known journals to improve network performance, so we classify the solutions for performance improvement based on three SDN layers and three SDN-based spheres which are (a) SDN-data center, (b) wireless networks, and (c) software-defined wide area network. This paper surveys the articles which have been published by Springer, IEEE, Elsevier, and ACM. We extract the performance improvement solutions and resources which are required for performance control and the metrics to evaluate the network performance. This article can help network enthusiasts to better understand, investigate, and improve the performance of SDN-based networks.

1 Introduction

The growth of the Internet is fast and has changed IT (information technology) requirements. The number of mobile end systems is increasing and is accelerating the use of the Internet by this equipment. Therefore, services diversity has created various network traffic requirements based on demand, which have caused needs for agility and flexibility in networks. To provide agility and flexibility in the network, the SDN architecture is needed. Some networks which need more agility and flexibility because of the high service demand rate are cellular network 4G and 5G, IoT, cloud computing, data center inter- and intra-network, and optical network. The traditional networks cannot support new network requirements, and they should be changed into architecture with innovation capability.

The scalability and the on-demand services in cloud computing, big data, security, and node mobility in computer networks are opening rapidly; the demand for supporting these concepts is on the rise. The traditional network architecture is gradually challenging problems that should be solved for the future networks. To prevail these shortages, there has been a lot of interest in both universities and industries in the future networks researching.

To consider security in new concepts like cloud computing, there are initiative solutions such as the way which has been proposed in [1] for the cloud storage system. Blockchain is the other solution that can be used in the networks as mentioned in [2] and [3] for the data storage mechanism in a wireless sensor network and a wireless body network, respectively.

Therefore, there are many research projects around the world that offer different architectures for the future networks [4]. More users, according to the research on the future networks, have realized that the lack of traditional networks development is the closed network. In general, closed networks, according to [5], face the following problem.

• (a) Large-scale networks managing complexity.

• (b) Network services provisioning complexity.

• (c) Network equipment control complexity.

• (d) Deploying new network protocols complexity.

SDN architecture is proposed as the main structure of the future networks [6]. SDN separates the control layer and data layer to change the traditional closed network architecture. This architecture provides a centralized controller and makes network programmable. SDN has changed network design and management. In general, three main layers named application, control, and data plane are the main layers of this architecture. These layers communicate with each other with the north and south APIs as shown in Fig. 1. The network equipment is bare forwarding elements which switch and transmit traffics without any decision ability; this element is called FE. This equipment sends packet-in for the first packet of flow or for a flow which has the rule in FE to send flow toward the controller otherwise transmits flow based on strategy exist in the controller and impose to FEs. Controllers are hardware and software servers which consist of network strategy, decision, and actions for ingress flow in each equipment. The application layer provides network services such as firewall, routing, and so on. The northbound API is defined for communication between the application layer and the controllers, and the southbound is designated for the controller to communicate with equipment. The southbound API has been designed to impose controller strategic decisions on the equipment, and the northbound provides network programmability with the application layer. Controllers can create the opportunity to design new protocols and new network services. The OpenFlow protocol is one of the major protocols in SDN [7, 8], which enables the control of the switch flow table. If network equipment vendors implement the OpenFlow protocol in their equipment, the controllers can add, remove, and edit the flows with the standard mechanism. Many users incorrectly believe that OpenFlow is equivalent to SDN, but OpenFlow is just one standard protocol for the southbound API.

Fig. 1

SDN architecture with two main APIs [9]

According to the performance importance in the networks, there is no survey article with network performance improvement concentration in SDN. So this paper focuses on three types of the network which are a) data center networks, b) wireless networks, and c) WAN. The contribution of this paper is an investigation of the performance improvement problems and solutions which have been proposed by the researchers in the mentioned type of the networks. In this paper, we inquire the papers which have been published by Springer, IEEE, Elsevier, and ACM between 2007 and 2019 to classify the performance improvement solutions in the SDN-based network containing DCN & cloud, Wireless, and WAN. Figure 2 shows their distribution based on the years 2007 to 2019 for Springer, IEEE, Elsevier, and ACM papers, meanwhile the other type of publications like RFC, patent, web site, and papers that are related to SDN performance improvement solutions are used to enrich this paper. The keywords which are searched simultaneously are SDN, Performance improvement, and efficiency.

Fig. 2

Distribution of papers in Springer, IEEE, Elsevier, and ACM

The bar chart in Fig. 2 can help the researchers to inquire the journals with performance improvement in the SDN topic among ACM, Elsevier, IEEE, and Springer accurately.

The rest of the paper is organized as follow. Section 2 expresses the history of SDN emergence and effective events. Section 3 shows the related works. We will discuss SDN performance improvement in two basic taxonomies which are SDN architecture and communication infrastructure types. These taxonomies will be expressed in Sects. 4 and 5, respectively. Section 6 discusses performance in SDN. Finally, Sect. 7 concludes the paper. Figure 3 shows the framework of this paper to guide ones to study the current paper.

Fig. 3

General view of this paper framework

2 History of SDN

This section expresses the history of SDN emergence and the momentous events which influence its maturity are discussed. The main idea of ​​the software-defined network is the project known as “Stanford Clean Slate Project” [10]. The main purpose of the project is to design an architecture that provides simple management and robust security in enterprise networks [11]. Therefore, Martin Casado who was Ph.D. student proposes a controller to enable network administrators to set their flow-based policies in the network. Nick McKeown et al. introduced the idea of the OpenFlow protocol in 2008, which initiated the programmability in networks and introduced SDN in 2009 [12] and [13]. The proposed architecture needed a foundation for development, standardization, and commercialization, so The Open Source Networks Foundation (ONF) has been established as a nonprofit institution to do those missions for SDN [14]. There existed a new requirement for research about software-defined networking; hence, the software-defined networking group was founded to examine SDN architecture from different points in 2012. This group has been established to provide solutions for SDN setup and operation in the near prospect [15]. SDN separates control and data traffic, and the main brain of the network has been changed to the controller, so the controller which is needed was launched as a project named OpenDaylight in 2013 by the guidance of the Linux Foundation. The project was supported by companies such as Cisco, Microsoft, HP, IBM, VMware, and Brocade [16]. This architecture is used in different network fields, and there are specialized researches about the future of this architecture. Telecommunication companies researching telecommunication markets like Infonetics have predicted that SDN markets will reach eleven billion by 2018 [17]. The other research shows that AT&T has been able to expand over 100 cities through demand-based network services in 2015 through SDN [18]. The combination of giant companies to prepare the world for the futuristic technologies has been seen at the ONUG 2016 conference that Yahoo and Verizon announced to be made ready for SDN and DevOps for the future. Yahoo was building a major change in its infrastructure, with the ultimate goal of reaching automated virtualization networks [19]. SDN requires security, so well-known companies like Juniper has begun its partnership with several leading security service providers in 2017 to create a safe infrastructure for SDN. Juniper’s allies have been Carbon Black, Netskope, CipherCloud, ForeScout, and Aruba. They tried to take into account all aspects of security in a way that others cannot attribute the weakness to SDN-based products on the company [20]. Ericson and Verizon have been joined to use manageable services in order to hasten IT changes in the company, so they launched “network as a service” for SD-WAN, virtual security service, and enterprise radio access network [21]. The history of SDN is shown in Fig. 4.

Fig. 4

History of SDN in a glance

3 Related works

Bruno et al. investigated programmable networks and then reviewed SDN architecture in [22]. They described SDN architectures in detail and presented the SDN implementation and test platform. They focused on SDN direction to information-centric networking (ICN) with heterogeneous nodes. Rowshanrad et al. in [23] have reviewed SDN architecture and then introduced the programmable networks and well-known controllers and simulators by which this architecture can be simulated. This survey can be helpful for the researchers to be informed the trend of SDN like software-defined ICN, virtualization, wireless and mobile networks, cloud and data centers, multimedia over SDN, and the works done on security of SDN. Fellow et al. have surveyed SDN comprehensively from network top to down and reviewed the architecture of these networks [24]. The benefits of SDN and network functions virtualizing have been explored by Jammal et al. and expressed the challenges of reliability, security, and scalability in SDN [25]. Various papers have addressed the challenges and security solutions in SDN [26], [27], and [28]. In [29], the authors discussed the current situation and the necessities for using SDN and this architecture deployment methods. In [30] and [31], the authors surveyed articles from the perspective of three layers of data, controller, and application. Farhady and et al. [32] revealed the various components of SDN and examined the related open-source and commercial products. In [33] and [34], the authors addressed various ways for the Quality of Service (QoS) betterment in SDN. Karakus et al. [35] reviewed the challenges and methods of scalability in SDN. In [36], the authors have provided an overview of the upgrade methods in SDN. In [37], the authors studied and categorized programming languages ​​in SDN. In [38], the authors have addressed the challenges and methods of the SDN that exist in wide area network (WAN). In [39], the authors have surveyed the challenges in the SDN-based network and provided solutions for scalability, availability, reliability, and performance. Jungmin son et al. have surveyed SDN utilizing for cloud computing which focuses on DCN power optimization, traffic engineering, network virtualization, and security [40].

All surveys which have been studied in this paper examined SDN from different perspectives that we can summarize their reviews. The researchers in these surveys have paid to SDN and its architecture. They introduce the SDN components and classified them into commercial and open source. They also present SDN simulators, controllers, and platforms in their reviews. They express the necessity and requirements of SDN for deployment. According to recent studied reviewed papers, SDN can be utilized in ICN, virtualization, wireless and mobile network, cloud and data center network. It plays a significant role in networking because this architecture makes networks programmable, flexible, and agile. The programming languages using in SDN are examined in the surveys for guiding the researchers. Non-functional challenges in SDN that have been surveyed are reliability, scalability, and security. Based on the reviews, power optimization, traffic engineering, and network virtualization are recent paper SDN problems. The problem which has not ever stated in the recent SDN reviews is performance improvement. Our survey discusses this problem and its metrics to evaluate and the solutions for performance improvement. Architecture in detail, but we can categorize their information into three main groups generally which are SDN introduction, challenges, solutions, and taxonomy. They have examined the SDN and introduced this architecture in detail. They have studied other researches and stated challenges and solutions related to SDN; finally, they propose their taxonomy to guide the researchers in this filed. According to the studied surveys which are about SDN, we found that major networks in which SDN is used are programmable networks, network virtualization, ICN, and cloud computing. SDN has three main non-functional challenges which are scalability, security, and performance. These surveys show that we can categorize these review topics into four momentous groups which are SDN architecture, SDN platforms and tools, SDN trends toward the future, and SDN challenges. To our best knowledge so far, the surveys have not paid to the performance improvements in SDN for metrics, solutions and network resources extraction. In this paper, after papers examination and categorization based on SDN architecture, the solution based on four infrastructure spheres will be scrutinized. Finally, issues, solutions, metrics, and network resources will be classified and discussed.

4 Performance improvement from SDN architecture perspective

This section belongs to the recent researches in SDN architecture in data, control, and application plane separately. This section focuses on performance improvement to show what methods and resources exist to ameliorate performance in the different layers of SDN which are data, control, and application.

4.1 Data plane in SDN

The recent investigations that have been done on the FEs and its interface toward the controller are focused in this paper. The southbound API specifies the rule of adding, removing, and modifying flows from FE and the flow tables are located in the FEs, whose type of memory and its function are important in the storing, deleting, and searching of flows.

4.1.1 Southbound API protocols

The open flow has been known as an open-source API for adding, removing, and controlling rules from the flow tables which have been placed in FEs, whose functions are discussed in [41]. This paper has added capability for relationship between control and data layers. Curtis et al. [42] have surveyed the OpenFlow protocol and proposed DevoFlow which has reduced the overhead of OpenFlow for high-performance networks. The relevant workgroup in the IETF has provided the RFC 5810, named ForCES, to access the goals that OpenFlow had traced [43]. The ForCES standard has provided a dynamic model in comparison to OpenFlow, but it has been less used because it was not available in open source [24] and [43]. To migrate toward an OpenFlow-based network, the network equipment should change, so it causes more expensive. Authors in [44] have present the closed flow method to take advantage of SDN without changing network elements and new investment. The other standards for the southbound API have been proposed by the IETF in RFC 4655 called PCE and in RFC 6830 named LISP [45] and [46]. Communication networks are similar to pipes to transfer data from one node to another node. These communications are becoming dominant with software-based infrastructures for data transmission and data inspecting, while data are moving between hosts. Wood has developed OpenNetVM and open-source framework for high-speed packet processing to expedite network middleware scrutiny [47].

4.1.2 TCAM space

Flows are stored in the FEs and need to reduce the memory reading and writing time to improve performance in the data plane. The network flow tables are stored in the high-speed and very expensive memory called ternary content addressable memory (TCAM), so control of table size is important for cost management. Paper [48] has presented a decision forest model making to store more data in memory. Kannan et al. have proposed compact TCAM in the paper [49], which stored more entities in the TCAM by decreasing the size of flows index. They have used shorter tags for identifying flows than the original number of bits used to store the flow entities. This approach has reduced the power and the cost of a flow with a decrease in the number of flows which are supported by FEs. In [50], a source routing approach has been proposed that amends the scalability, reduces the convergence time, causes fault tolerance, and decreases the number of ingress flows. In paper [51], Ms. Rifai and her colleagues have proposed and evaluated a compression algorithm called MINNIE in an essay to enhance the efficiency of TCAM in SDN. Moulierac et al., in the paper [52], used the integer linear programming (ILP) optimization method to ameliorate the input flows size space in SDN. Paper [53] has suggested an algorithm to promote the rules for limited-size TCAM. In [54], adaptive hard timeout method (AHTM) has been proposed to improve the flow table utilization. This method optimizes the timeouts of flow entries to make the flow table efficient. In [55], Elliott and et al. have presented the system with FPGA to save tens to hundreds of thousands of filter rules while supporting real-time updates of the ruleset through online reconfigurability and modern line rates of 10G or higher. The other CPU base approaches impose significant latency in comparison with FPGA-based method.

4.2 Control plane in SDN

In SDN, control plane is composed of software and hardware called the “controller” which is acting as a network OS. The controller links some FEs. Distributed network controller and management in the traditional network have been changed into a centralized network controller and management in SDN. Therefore, in this architecture, the network administrators can regulate traffic based on application demands with rule setup in controllers. A controller works like middleware in the network. Many controllers are provided for the SDN architecture, which includes commercial and open-source ones. For commercial instance, Cisco, Juniper, NEC, Nicira, and big switch networks presented their own products for controller, which typically implement their own southbound API protocol which is similar to the OpenFlow protocol, for example, APIC is provided by Cisco as a controller [56] and Big Network Controller by Big Switch Company [57]. Research centers and universities usually use open-source controller based on OpenFlow. These controllers are centralized but some of them support distribution. Roberto Bifulco et al. add ARP responder to improve the ARP packets and decrease the ARP broadcasting [58].

4.2.1 Open-source controller

These controllers are extensible and development is available. These are attractive to researchers and academics. These controllers also are used in the commercial controller.

4.2.1.1 Centralized controller

Some controllers work in a centralized manner, and there is no way to communicate with other controllers. Casado et al. presented the first controller in the Ethane project in the year 2007 [11]. NOX controller was implemented in C++ language and was presented as the first open-source controller in 2008 [59]. The NOX controller is installed on the server. This server also stores the necessary data for the network global view. Maestro controller was introduced in 2011 and was intended to improve the parallelization of controllers in the article [60]. This research has shown that the throughput (Thr) of Maestro can meet about linear scalability on an eight-core server machine. This paper has used the parallelized method and has implemented the controller on NOX and eventually offers Maestro as a controller. Floodlight controller is a Java-based and open-source software that uses the OpenFlow protocol as its southbound API and big switch network uses floodlight to implement its own controller [61]. Beacon has been introduced in 2010, aiming to improve the controller performance. This controller is Java based and open source and also supports the OpenFlow protocol as southbound API and generally; it is common for training and researching [62]. OpenDaylight has also been introduced by Linux Foundation [16]. A customized OpenFlow controller has been developed in Ruby and C with the use of Trema as a full-stack and easy-to-use framework [63]. Ryu is an SDN core based on components. It presents software modules and paves the way for developers to implement new network modules to control. This controller supports NMS protocols consist of OpenFlow, Netconf, and OF-config. It carries five versions of OpenFlow and Nicira Extensions. This controller is open source and frees under license of Apache 2.0 [64]. Due to the architecture of SDN controllers, the problem is the single point of failure, so controllers need to be scalable to overcome this problem.

4.2.1.2 Distributed controller

Some controllers can have decentralized architecture, and they have defined communication methods with other controllers, which paper [65] expressed taxonomy of controllers in SDN and stated distributed approaches in three categories, (a) distributed flat controller design, (b) hierarchical controllers design, and (c) hybrid controllers design. In distributed methods, controllers are still conceptually centralized in SDN but can be physically distributed as shown in Fig. 5.

Fig. 5

An overview distributed controllers [65]

Koponen et al. presented Onix which supports distribution as a distributed control. Onix provides a general API for implementation over the controller; Onix controller in 2010 was introduced [66]. Mr. Toutonchian and his colleague at the University of Toronto, with the development of the NOX controller, could provide a new controller with scalability named HyperFlow in 2010 [67]. DISCO controller was presented as a distributed controller in 2014, which differed from the past distributed controllers. This controller is used for heterogeneous networks. These controllers independently control different domains and can communicate with each other through the east–west API. This controller has used OpenFlow-based Floodlight and AMQP protocol to develop [68]. The controller called ONOS can be used for WAN. The first version of ONOS was developed to overcome the problem of scalability and the second version to better performance. This controller is suitable for large-scale networks [69]. A few numbers of researches have been done on the evaluation of controllers. Mr. Abdullah Shah et al. have examined the performance of the four controllers, NOX, Beacon, Maestro, and Floodlight in terms of thread scalability, switches scalability, latency, and throughput, meanwhile, these controllers are multi-threaded [70]. A controller with the support of OpenFlow basically has two paradigms: reactive and proactive. In [71], the authors proposed a methodology to evaluate OpenFlow controller performance. Both of the two paradigms from the point of performance were examined in identified controllers. The performance evaluation was done in a physical environment and emulation. The performance evaluation was done in a physical environment and emulation. It was examined different OpenFlow-based controller and using a varied number of OpenFlow devices. Finally, this paper presented some conclusions about controller scalability. Ms. Shiva Rowshanrad and her colleagues at Shiraz University have reviewed and compared the OpenDayLight with Floodlight. In this paper, packet loss and delay in different topologies with different loads have been examined [72]. Monaco et al. expressed the purpose and design of the Yans controller in [73] and also stated that controllers have the role of the operating system in the network. In [74], Xie et al. reviewed the existing controllers for performance, scalability, and security. All reviewed papers are summarized in Table 1.

Table 1 Centralized and decentralized controllers

4.2.2 Important commercial controller

In this section, several hardware vendors, network service providers, and network solutions consultants which designed and developed their own controllers, are introduced. Cisco introduced a controller named APIC which is a software-based controller. This controller automates and simplifies network configuration, provisioning, and management in comparison with a single x86 or virtualization-enabled computer [56]. HP has launched a controller called HP VAN SDN that provides a unified control in an OpenFlow-enabled network. This controller simplifies management, provisioning, orchestration and facilitates the delivery of a new generation of application-based network services [75]. NEC provided NECPF6800 control over a network that supports the OpenFlow protocol as a southbound API [76]. The VSC controller provides a controller as a service and multi-tenant to the controller [77]. VMware has introduced a controller called VMware NSX to manage virtual machines and tunnels [78]. Huawei as a hardware and service provider has developed a controller called Agile controller [79]. Microsoft has also provided a controller for controlling virtual and physical resources on its Windows Server 2016 [80].

4.3 Application plane in SDN

SDN adds programmability to networks so users and IT employee can solve network management, QoS, routing, and network performance problems in the application layer. In SDN, Applications can be used in different networks like cloud computing, data centers, cellular networks, and the Internet of Things (IoT) and can solve the network issues. To automate and optimization in SDN, new information analysis concept like big data can be used.

4.3.1 Application plane and NMS

The two tools named ENVI and SAGE have been developed to monitor and evaluate OpenFlow-based networks [81]. Open Networking Foundation (ONF) has defined rules to provide an extension into the SDN framework. Devlic et al. have proposed an initial architecture to integrate network management and SDN framework with ONF rule considering. Architectural considerations include the definition of the required entities and their interactions. Architecture on the access and aggregation was proposed as a network use-case, ultimately. It also plans procedures that can explain how network management extensions can expedite network-wide and node-specific management & configuration [82]. Kim et al. presented a network control framework called lithium that facilitates network management [83]. Sundaresan et al. have launched and developed a pilot platform called “BISmark” to collect information about network topology, availability, accessibility, etc. [84]. In [85], Kim et al. designed and implemented “Procera” to facilitate network operations and management of all aspects. This product combines top-level networking policies and low-level network configuration. In [86], a framework for managing things on the Internet of Things was presented. In [87], the vertical structure of the equipment management was presented for configuration, control, and monitoring of resources, and the management of heterogeneous equipment and communication between different networks. This structure was implemented using the “Yang” language. 5G technology needs cellular flexibility and more resource manageability that require the use of SDN architecture. Chang et al. expressed the merit of quick converging to the global optimal solution was used to design the fitness function of the network resource management. The ability of fast random search of particle swarm optimization was used to realize the update and optimization [88]. Das et al. provided a framework based on SDN with huge help resources for network management called FlowComb [89].

4.3.2 Application plane and QoS

Kim et al. implemented an API to program in the controllers to manage QoS in an SDN controller that provides automated control over the QoS in networks [90]. In [91], the authors proposed OpenQoS controller for end-to-end multimedia services. With the optimization of OpenFlow-based controllers, Egilmez and colleagues presented a method to optimize QoS for scalable video streaming. They implemented this optimization in a controller over an OpenFlow network. In this case, they solved two optimization problems to improve QoS in video streaming. They prioritize frames routing in a way to transfer the base layer as a flow without any losses in QoS, and then, the enhancement layers are routed as a flow with losses in QoS [92]. The A-SDN standard was implemented by the IETF to provide a framework for SDN that supports QoS and implement network services with deploying network function as a software application [93]. KO et al. have provided OpenQFlow as a branch of OpenFlow to support QoS in SDN, besides, this protocol is stateful and uses micro-flows [94]. Civanlarand et al. described an architecture for flow-based QoS in OpenFlow-based networks, which consisted of a central controller and a large number of FEs. They focused on analyzing the optimization method for routing and the requirements for providing QoS in scalable video streaming in [95].

4.3.3 Application plane and data center network

Google designed and implemented the B4 network to connect its data centers with the aim to maximize utilization of bandwidth (BW) between them control with easier method [96]. Tavakoli et al. [97] developed NOX controllers for data centers and addressed the needs of the data center. Thanh and colleagues develop an OpenFlow testbed to measure and analyze energy-aware data centers deeply, which consists of hardware infrastructure and simulation [98]. Macapuna et al. proposed a new data center architecture called “SiBF,” which changed data centers problem into a software challenge [99]. Fang et al. proposed an approach to detect congestion in data centers using SDN [100]. Martini et al. have proposed a coordinated approach based on the architecture of SDN for virtual data centers that make the trade-off between computational and network resources [101]. Wang et al. presented a dynamic controller with considering response time decrease in the controller [102].

4.3.4 Application plane and cloud computing

Cloud computing is provided at three levels: “Infrastructure as a Service (IaaS),” “Platform as a Service (PaaS)” and “Software as a Service (SaaS).” Providing the network as a service (NaaS) is difficult and complicated due to the limited network resources controlling. SDN provides a robust API for network management that enables the implementation of NaaS. Feng and colleagues have suggested an idea with a combination of cloud computing and networking concepts named network as a service (NaaS). This conception begets openness criteria for related protocols and designs to provision, delivery, and network resource consumption. They extend OpenFlow-based architecture as a cloud-based network [103]. Benson et al. have presented a novel cloud networking system called CloudNaaS. Users can leverage CloudNaaS to support the growth of applications with the help of an extensible, rich, and flexible network function chains. What makes CloudNaaS hugely efficient is cloud infrastructure and high-speed programmable network element [104]. Banikazemi et al. have proposed a Meridian-based SDN framework to support a network service model for cloud computing [105]. Raghavendra et al. presented a software architecture for network management operations; this scheme is including a shared graph library. The authors presented a mechanism to qualify practical computational times using dynamic all pair shortest path algorithm with dynamic graph query which is scalable lightweight [106]. Tidbits in [107] stated that SDN eliminates network infrastructure constraints and provides a suitable platform for creating cloud computing.

4.3.5 Application plane and others

Caraguay et al. described IoT and SDN with an inspection of the combination of these technologies [108]. Hoffmann in [109] stated that SDN and network function virtualization (NFV) have been considered in recent years; they focused on mobile networks to make it cost-efficient. They have used SDN and NFV concept with using standardized software and hardware for these networks. Bifulco et al. scrutinized the scalability of SDN-based networks and then proposed a distributed strategy based control plane functions which enable scale-out of the system. They proved the separation strategy with a simulation that results in less state engaged by individual OpenFlow controllers; in addition, it is also more effective with localization of network traffic [110]. Fichera et al. presented an experimental scheme for 5G with use of SDN-based edge networks, cloud computing, and IoT topics. In this architecture, there is an orchestrator to provision data delivery path based on reliability and robustness of 5G networks requirements [111]. Wang et al. used SDN to solve the mobility problem in the IP network and provided an architecture [112]. Cui et al. in the paper [113] presented the good ability of SDN to solve problems prevailing with big data application like data processing, data transferring and optimization in data centers. In addition, network data in SDN have big data criteria and can be analyzed by big data algorithm to control and manage networks too. They showed big data and SDN benefit from each other. In this section, the work done in the data layers, control, and application has been dealt with and a summary of them done in it has been examined, which is summarized in Table 2.

Table 2 Performance improvement issues categorization based on SDN architecture in different papers

5 Performance improvement in SDN-based networks

Based on our recent studies, we can classify the networks in three groups, generally which are (1) SDN-DC & cloud, (2) wireless, and (3) SD-WAN and then categorize the performance improvement solutions with SDN based on the three mentioned network types. The network types which need more agility, flexibility, and innovation are shown in Fig. 6. These types are more common in the recent papers, and we have clustered the solutions which are presented in Sects. 5.1, 5.2, and 5.3.

Fig. 6

Networks type which uses SDN to improve agility, flexibility, and innovation

5.1 Performance improvement in SDN-DC & cloud

In SDN-enabled optical networks, significant resource consumption reduction for heterogeneous bandwidth (vs. homogeneous bandwidth) is a result of research [114] done by Mandal et al. Iovanna et al. proposed a hierarchical routing approach; they have used a novel virtualization scheme for real network operation adaptively to configure heterogeneous packet-optical network domains dynamically. Finally, they validated their idea by simulation in the real network and showed that it is suitable for transport SDN [115]. Paolucci et al. focused on flexgrid optical networks and proposed a novel super-filter technique. They compacted spectrum-contiguous light path and bettered performance measures in spectral efficiency [116]. Zhou et al. have studied flexible load of fibers and verified different distances 10, 20, and 30 km fiber to support different optical power budget up to 44db and the capacity between 200 mbps to 9.6 Gbps. They have proposed a novel software-defined FlexPON to control bit loading adaptively. The proposed scheme is composed of three innovative features which are (a) DSP enabled flexible transceiver and flexible link, (b) OLT side access network resource virtualization, and (c) software-defined programmable network function and resource scheduling. This solution is to deliver high quality of service (QoS) to end users with highly diverse wired and wireless [117]. Zhao presented an optical as a service (OaaS) and designed to provide the optical network resources directly for the data centers with considering both data center and network resources, which can be implemented with software-defined networking (SDN). Under SDN enabled OaaS architecture, a new concept of network provisioning is proposed different from legacy connection (light path) provisioning, because the client’s requests may include not only one end-to-end connection but also several end-to-end connections, even a network [118]. Zhang et al. designed the optical network architecture called SD-DON to save energy. This solution is offered to incorporate an optical network unit (ONU) sleep/doze modes into dynamic bandwidth allocation (DBA) in order to decrease energy consumption [119]. Zeng et al. used SDN in rearrangement the multicast-trees to minimize spectrum usage and operation complexity. They have proposed the routing algorithm in inter-DC to select light path dynamically [120]. Liu et al. studied multipath routing problems for fat-tree networks scenario. They proved that the problem is NP-completeness and then proposed a practical solution based on SDN. Their algorithm has been implemented in an OpenFlow-based and centralized controller to collect the necessary information to select an optimized route. They used Mininet and Data Center Network (DCN) to prove that their algorithm has made throughput and end-to-end delay optimized [121].

Peng et al. proposed a new smoothed optical DCN to provide dynamic, programmable, and highly available connectivity services for DCN. They expressed LIGHTNESS which is low latency and high throughput dynamic network infrastructures for high-performance data center interconnects. This architecture tries to provide the requirements of new and emerging DC and cloud applications, dynamically [122]. Gu et al. combined SDN and network coding for intra-data center communication and proposed a flexible and efficient passive optical networks with focusing on edge and aggregation layer in DC. This scheme has provided a low delay, good fairness, increased efficiency, and network flexibility. They use dynamic bandwidth allocation, wavelength assignment, and scheduling to minimize the load of the core layer in DCN and provide a low delay, good fairness, increased efficiency, and network flexibility [123]. Ma et al. provided a platform for evaluation SDN-based network from the points of conformance and interoperability. Conformance testing is done to show that the equipment meets the needed conditions. Interoperability testing is performed to evaluate that the equipment from different vendors can be communicated with each other or not [124]. Wang et al. presented a latency-aware flow scheduling system to provide required effective bandwidth to make sure the network to meet targeted QoS and QoE with considering of existed changes in inter-VM traffic patterns and background application VM patterns [125]. In [126], Li et al. presented a solution to the problem of resources scheduling in cloud-based networks by using Hadoop and allocate bandwidth according to the network conditions. Lali and colleagues have evaluated the performance based on latency, packet delivery ratio, and processing overhead in SDN, by comparing the performance of traditional networks and SDN improvements [127]. Murali et al. proposed two heuristic algorithms forward local rerouting (FLR) and backward local rerouting (BLR) for solving the proactive fault-tolerance problem in SDN, in which they considered the optimal backup path in TCAM memory and bandwidth efficiency. FLR starts the path selection when a link is failed to recovery the end-to-end path. This algorithm selects the path with the least number of additional switches. BLR chooses the backup path based on TCAM-aware algorithm, since the backup baths need minimum TCAM, in the other hand, the minimum number of additional rules as well as the amount of bandwidth reserved for the backup paths. This study has shown that the number of needed rules decreased to 75% and network restoring time is up to 50 ms and has been improved in the bandwidth for backups [128]. In the cloud-based data centers, energy consumption is an important parameter. In SDN, the resource of the network can be allocated by a need to control energy consumption. As mentioned in this research, switches are the most energy consumer that should be controlled by this method. Subbiah et al. in [129] proposed an energy-efficient routing algorithm in open virtual switch to select the optimal path based on minimum power consumption. Chiang proposed a multipath transmission scheme to equally balance the traffic of flow over the disjoint paths designated by the SDN controller, which was able to learn the topology of the network and discover multiple transmission paths [130].

TING-YU MU et al. reduce traffic between the controller and switches and controller load with the machine learning technique in cloud computing servers. They use deep reinforcement learning to better controller overhead up to  %60 and 14% improvement in the table-hit ratio [131]. In [132], the authors proposed an algorithm to differentiate between heavy flows, elephant flows, and bulky flows and present efficient algorithms to detect flows of the different types based on multi-parameters. This algorithm improves the scalability of the network and traffic control. Qiang Gao et al. proposed a delay-constrained multicast routing algorithm which had been derived from anycast to avoid traffic congestion in multicast trees dynamically [133]. X.Qin et al. proposed a dynamic routing for VM migration to improve the performance in the SDN-based networks [134]. They offered an early notification (E-N) algorithm that can deal with bursty traffic and packet loss which exist in live VM migration, and then develop a dynamic routing algorithm to improve the network performance. Zhang et al. tried to solve the performance problem in data centers with big data, so they proposed an efficient flow detection mechanism, and a novel algorithm to differentiate scheduling algorithm based on the link performance situation to improve performance [135]. Lin et al. offered the greedy algorithm to improve the network performance based on user request scheduling to improve network performance. The network requests are considered from three points, (1) the amount of data, (2) time, and (3) the coupling relationship between requests. The amount of data is presented by bandwidth and time of requests. Time presents the length of requests. The coupling relationship shows the relation between request, for instance, the first request with a small amount of data used for negotiation and the second request with large amount data used for transmission [136].

5.2 Performance improvement in SD-wireless network

Costa-Requena et al. provided an idea for the integration of SDN and mobile networks named SDMN, which benefits from dynamic and efficient caching. They used SDN in LTE networks to optimize bandwidth and latency by providing content caching and then presented the benefits of this integration [137]. Yang et al. presented a congestion control method that reduces the response time in non-balanced conditions. The burst traffic is a problem in radio networks which causes congestion. The heterogeneous radio network integration is considered to reduce network congestion as a cost-effective scheme. In this paper, the authors use the coverage overlapping of varying wireless networks like LTE and Wifi to transfer the traffics. They also perform the SDN as an architecture to make this solution dynamic. The throughput degradation of adapting their mechanism is 4.15% on average [138]. Anderson et al. have controlled encryption in the boundary between the mobile packet network and data networks. They use SDN and NFV (network function virtualization) to code the traffics which are required in DPI, video optimization, content caching, etc. system dynamically. This increases the QoE by adaptive regulating and managing of traffics [139]. Narmanlioglu et al. stated the eNodeB assignment to each mobile operator (MO) with varying numbers and locations of associated user equipment (UE) and MOs’ demands. They use QoS-aware and QoS-unaware scheduling algorithms adaptively and show that the LTE network works with better performance terms which are fairness, data rate, and satisfied-MO-ratio. They perform the virtualized controller to develop this concept and evaluate with Monte Carlo simulations with time-varied numbers and location of user equipment [140]. Anil Kumar Rangisetti et al. worked on the load imbalance issue and have proposed software-defined LTE radio area network (SD-LTE-RAN) framework. They used a QoS-aware load balance (QALB) algorithm. They implemented a novel balancing algorithm which considers loads of neighbor cell, UEs QoS profile, and their throughput. This algorithm maintains 80% of the guaranteed rate for 70% of the cells, while the previous algorithms only hold 50% of the cells. This algorithm has reduced average network overload by 15% on the network and in a mobility scenario; it has been decreased by 10% [141]. In paper [142], Al-Rubaye et al. proposed a new platform based on SDN named IIoT. This platform has been implemented in a controller to gather network real-time information to manage demand, resources, and system reliability and protect the network against failure. WSNs have limitation in their performance which is sensor resource restrictions. The resources are power supply, memory, processing units, and communication capabilities. In [143], the authors consider both sensor resource restriction and random node density in a geographical area. They cluster the sensors into virtual zones (VZs) in order to balance the number of cluster heads (CHs) based on the node density of VZ. They have used the whale optimization algorithm (WOA) in SDN. Chakrabarty et al. have presented an SDN architecture for the secure IoT named BlackSDN to improve security and performance in IoT. They showed that the SDN can ameliorate the throughput of the network with the proposed method and SDN in the broadband networks [144]. El-Mougy et al. studied researches done to combine IoT and SDN. They analyzed SDN base IoT to manage different types of networks to provide sensing-as-a service (SaaS) as a momentous cloud IoT application like wireless sensor networks and mobile networks. The performance is throughput and latency which are measured in this research [145]. Tomovic et al. worked on fog computing which enables some data to be analyzed and managed at the network edge. They proposed a new model that is composed of IoT, fog computing, and SDN to provide the low and predictable latency of the applications in the network [146]. Jang et al. developed an architecture based on SDN for the Internet of Things that are used in smart home and have improved QoS parameters such as fairness, delay, service priority, and quality of experience by allocating proper bandwidth [147]. Bera studied different SDN-based networks to achieve an effective method for IoT in edge, core, and data center networking. This study is a survey about the SDN for IoT [148]. There is a language called P4 which is a top-down approach to describe a protocol-independent data path pipeline, independent from the network platform. Meanwhile, the open data plane (ODP) follows an open-source bottom-up method to write platform-independent data plane applications. Patra et al. offered the multi-architecture compiler system for abstract data planes (MACSAD) as an approach to converge P4 and ODP through a common compilation process delivering portability of data plane applications without compromising target performance improvements [149]. Jan Willem et al. worked to provide dynamic adaptive streaming over HTTP (DASH) for internet video streaming over HTTP in wireless testbed. This system provisions target bitrate signaling and dynamic traffic control to give proper bandwidth sharing among clients; in addition, it is privacy-friendly and encrypted video streaming fully support. The main performance metrics are bandwidth and delay [150]. In paper [151], authors proposed a method for mobility-induced packet losses are tackled smartly to avoid the reduction in sending rate and proposed SDN assisted TCP does not violate the end-to-end semantics and no modification is required in the TCP protocol running at the end hosts. Malak Sadik offered a system with a centralized controller to assist in the handover discovery, handover decision, and co-channel interference coordination. They try to support QoS, Quality of Experience (QoE), reduce delay, and interference in multi-tier LTE/Femto and D2D networks. This algorithm predicts the QoE in UE to increase network capacity and reduce interference. They use SDN-based fuzzy MADM handover scheme and reduce unnecessary handovers, blocking probability, and total handover delay with parallel throughput increase [152]. Lun Tang et al. worked on 5G to provide promising technology demanding services; they try to reach a compromise between network stability and network slice embedding (NSE) reliability [153]. This paper improves the reliability of NSE. The resource allocation while ensuring the stability of the queue is proposed in this work. Nir Gazit et al. worked on the mobile operator for the next generation networks and proposed the online placement and evaluated in the realistic scenario [154]. They decrease the reliance on the third party which is shown in %60 cost reduction. In [155], research on safety application was done which try to reduce delay and energy consumption in vehicular ad hoc networks with SDN/NFV technology. The authors proposed a collision detection algorithm for performance improvement. Schiller et al. have worked on mobile edge computing (MEC) platform and try to solve traffic engineering with SDN/NFV in 5G. They proposed a controller to manage traffics to better user Quality of Experience, e.g., latency, in comparison to direct communication [156]. Atzori proposed a solution for IoT paradigm and tried to control the workload and power consumption in parallel with extended lifetime with SDN architecture [157]. Coronado et al. have expressed a method to manage mobility in WLAN with multicast rate adaptation algorithm in SDN to better the performance metric like throughput and channel utilization [158]. Jindal has worked on VANET and expressed an approach to better delay, throughput, and network load with SDN and deep learning concept which the experimental test confirm this claim but its method has worse packet loss in comparison with only SDN-based VANET [159]. In [160], an approach based on SDN was proposed to improve performance with the sensing task, sensing a delay, and active sleep better control. Xi Zhang et al. work on 5G to improve QoS in this networks, so they proposed SDN architecture to manage wireless resources to improve performances with virtual networks without offloading, virtual networks with WiFi offloading, and virtual networks with device-to-device offloading [161]. Bellavista has suggested a model for collaboration with the mobile ad hoc networks (MANET) and fiber-wireless (FiWi) with a hierarchical SDN architecture to provide fairness and load balancing [162]. Mahmood worked on VANET which consists of heterogeneous wireless media like 4G LTE, Wi-Fi, WiMAX, DSRC, and terahertz communication to overcome the bottleneck and decrease the latency [163]. Chahal has proposed an SDN-based mechanism to avoid LTE from congestion and ameliorate the network performance [164]. A static stream control window can prompt issues, being too small or too large and requiring adjustment in an attempt to get to the current ideal value, so in this paper, they control TCP stack rate in eNodeB.

5.3 Performance improvement in SD-WAN

In [165], Bouet et al. worked on deep packet inspection deployment cost and proposed an optimization problem for its engine deployment cost. They used the genetic algorithm for optimization in different use cases with various traffic model and varied cost structure. They minimized the number of engines, network load, and unanalyzed flows. They showed that this optimization had reclined the cost of DPI engines deployment up to 58% with required bandwidth provisioning constraint. In [166], Li et al. discussed and proposed an optimization problem to maximize network utilization with practical bandwidth virtualization. Rodrigues discussed procedures to minimize losses on routes update layers across layers for monitoring of scalable traffics [167]. Hyunwoo et al. proposed a system to provide QoE for reliable video streaming with the help of real-time monitoring and select the appropriate path dynamically. They used MPLS for traffic engineering (TE) and off-the-shelf SDN platform [168]. Ji et al. used cross-layer architecture, hardware elements like adaptive EDFA-Raman amplifier, multi-degree supreme channel, and flexible network switching to optimize multi-flow switching [169]. Akyildiz et al. focused on four topics consisting of network management, fault tolerance, topology update, and traffic analysis with pattern extraction to characterize and then surveyed the state-of-the-art tools to engineer network traffic for SDNs. In addition, the authors discussed some existing traffic engineering tools from the points of industry and academy groups. Furthermore, open research issues for traffic engineering in SDN were discussed meticulously [170]. Huang et al. proposed a multiplexing design basis of rules so that all flows of a session move toward a destination but through distinct paths. The authors proposed a solution for an optimization problem to minimize space occupation in multiple unicast seasons with considering of QoS as constraints. They also formulated routing engineering, rule placement, and rule multiplexing schemes as an optimization problem [171]. Huang et al. proposed a multi-objective optimization problem to maximize statistical resource sharing with a service guarantee and minimize shared information [172]. Amiri et al. focused on cloud gaming data centers and suggested an optimization algorithm to select the optimal path based on linear programming (LP). They used a method for allocation of game server to game sessions’ base on the type of games, server loads, and path delay. Their result showed that this algorithm minimizes the average delay of all players by 9.6% [173]. Feng Tao proposed a scheme that per link has own price, bandwidth, and delay. They used SDN control application to meet a fair bandwidth allocation, min-cost, and minimum global delay. They integrated bandwidth allocation and flow table allocation in SDN and showed the difference between traditional network and SDN-based networks [174]. Tsung-Feng Yu et al. proposed an adaptive routing called ARVS for video streaming with considering of QoS based on SDN. They separated the behavior of network for the base layer and enhancement layer to introduce two different levels of QoS-flow. This method was evaluated with different network loads with a feasible path and shortest path routing algorithm. The result showed a 77.3% decrease in packet loss rate (PLR) for the base layer and a 51.4% increase in coverage [175]. In [176], Tomovic et al. investigated different routing algorithms to guarantee network performance in SDN, dynamically. Bandwidth rejection ratio and low computational time are used as a performance indicator in SDN. They used bandwidth and path delay for constraint in the optimization. Lara and colleagues proposed an additional virtual layer between the physical topology and end-users in the WAN. They proved this virtualization has significant advantages and endurable delays [177]. Saldana et al. have stated an optimization problem in OpenFlow-based SDN with bandwidth utilization objective function. They investigated the overhead of small packet-based flows and emerging services in a controller to reduce overhead. Their result showed that 68% of bandwidth in IP4 and 78% in IPv6 were saved [178]. Hongyu et al. developed a strategy to save on the access layer of SDN-based wireless networks and used a combination linear programming algorithm to optimize delivery time, QoS, and energy consumption [179]. In [180], a method has been provided for end-to-end bandwidth optimizing in SDN. Zhang et al. proposed a dynamic scheduling algorithm for bandwidth allocations based on bandwidth demand to provide high efficiency with an optimal scheduling algorithm for bandwidth allocation [181]. Zhu et al. defined a multi-objective optimization problem to meet the QoS demand and supported the largest parallel flow, using the MOPSO algorithm to find the optimized path [182]. Guo et al. used a type of genetic algorithm as a heuristic algorithm for a migration sequence of the routers to optimize traffic engineering. They evaluated by simulation and compared the greedy algorithm and static migration algorithm as proposed one [183]. Qing Li et al. proposed an update scheme which is fast and consistent in SDN-based network with the help of a new structure of relation graph. They focused on an update operation sequence to avoid inconsistencies. The result showed the update operation accelerated up to 40% on average [184]. Qiang Gao et al. presented an innovation in multicasting. They deployed multicast modules as a plug-in for the operating system in SDN. They used the routing algorithm to optimize and avoid congestion in multicast trees. The experimental result showed this framework is feasible and performs in a use case with high load [185]. Tolga Bagci et al. expressed an optimization problem for ISP revenue. They focused on cost per-flow for each service level as constraints. Their solution is based on users request for service and paying to optimize QoE in SDN controller [186]. Qin et al. proposed a model predictive control (MPC) for CDN to improve two performance parameters of response time and bandwidth. They used the intelligent algorithm to select server and routing path based on performance metrics [187]. Lan et al. focused on data center networks and stated an optimization problem and proposed a dynamic load balance path in SDN. Their algorithm changed the path toward the less congested path in SDN-based data center and decrease packet loss with updating of flow tables in each switch. Their solution is composed of two algorithms which are based on a single link and multilink. In the single link, they routed traffic to one link but, in multilink, controller directed traffic to multilink. Their result showed that throughput increased to 9.5 and18.5 percent per flow in comparison with LABERIO, respectively. They also tested their algorithm in a fat-tree topology and showed 8.3 and 24.8 percent improvement in throughput [188]. Bryskin et al. used GMPLS to move to SDN. The GMPLS protocol was introduced in 2000, and it was invested heavily [189]. Tomovic et al. used a logical centralized topology in distributed controller topology in SDN backbone and proposed a new routing based on bandwidth-delay. Their algorithm maximized utilization of network resources with efficient traffic engineering and classifies traffic flows in a finite number of categories based on the level of services to provide QoS [190]. Gao et al. formulated an optimization problem using mixed-integer linear programming and solved by an evolutionary algorithm ant colony and proved by simulation [191]. Al Hasrouty et al. used a multiparty VoIP system in SDN and proposed the solution to diffuse stream adaptively and dynamically to optimize the performance of the participant. They proposed a distributed algorithm and adaptive dynamic flow bandwidth balancing and latency [192]. In [193], the authors provided a method to collect a plurality of network information from the nodes, forward the network information based on a plurality of domain-specific policies, receive traffic optimization information based on the network information, and compute forwarding decisions for the nodes based on the traffic optimization information in SDN which is presented in the application layer and decides-based collected information. Gomes and colleagues used the combination of SDN and NFV and proposed virtual software-defined networks (VSDNs), and for assigning VNF, they used two algorithms, the BRAR algorithm to get the network into a state the elasticity and BEE-focus algorithm was proposed to select the appropriate route to improve bandwidth and energy efficiency [194]. Tuysuz in the survey article examined the papers from the point of energy efficiency and collated possible energy gain ratios of the existing approaches. This study discussed energy efficiency methods and future research [195]. Wibowo et al. focused on the OpenFlow protocol and its future challenges. They studied SDN and multi-domain SDN and presented a review of prevailing researches status [196]. Xu et al. focused on SDN-based data centers and general scenarios to evaluate. They optimized energy consumption and utilization of links and switches by flow scheduling. Their optimization minimized energy consumption and maximized link and switch utilization in a parallel way with deadline setup [197]. Valerrian Pasca et al. prioritized flows based on bandwidth and low latency in SDN. They classified traffics and demand using machine learning techniques (MLT). There are diverse applications and varied requirements in nature. Their algorithm routed flows based on application performance needs. Their proposed AMPS controller using MLT for flow prioritization and path allocation based on application requirements [198]. In [199], Lakiotakis focused on networked music performance (NMP) systems in SDN with considering the interaction between application requirements and network performance. Abdelhak Bentaleb and his coworkers worked on online reinforcement learning (ORL-SDN) to optimize the Quality of Experience (QoE) with presentation-quality stability, fairness, and bandwidth utilization. They formulate the adaptation process with partially observable Markov decision process and solve the per-cluster optimization problem and solve this optimization problem using ORL-SDN [200].

In [201], the authors proposed an intelligent algorithm for path controlling in the controller to save energy in hybrid SDN-based networks. They maximize control network control ability with a given SDN switches upgrading budget constraint and maximize energy saving in ISPs. Jingyu Hua et al. improve delay and bandwidth simultaneously to maximize the network utilization in SDN with hierarchical control plane [202]. Xiaohong Huang et al. proposed an algorithm to allocate the bandwidth to support better performance with the scalability of the networks [203]. Wai-Xi Liu et al. proposed an algorithm based on deep learning to predict the network performance for caching for ICN (information center network) [204]. The authors in [205] have offered methods for off-loading congested controllers and congested in-band control channels using control flow tables. They improved scalability and performance in the networks. Authors in [206] offered adaptive cache management (ACME) to adjust the sizes and shapes of buckets to make flow table efficient and better performance in SDN. Jian Yang et al. proposed an approach for video streaming to improve performance and efficiency, so they use SDN and then offered a variable-length interval cache strategy for RTP streaming, which can realize the self-adaptive adjustment of the size of cached video segments based on their access patterns [207]. Hayes et al. proposed the classification algorithm in SDN networks to improve performance with resource allocation approach [208].

We extract the performance metrics in each paper which are used to improve and cluster them based on the network type in Table 3. According to reviewed papers, the goal and the solutions for performance improvement which are proposed are presented. This table is one of our survey contributions.

Table 3 Performance improvement metrics, goals, and solutions in SDN based on three network types SDN-DC & cloud, wireless, and SD-WAN

6 Discussion

To improve the performance in SDN, it is needed to determine the metrics in the first step. In the second step, problems and solutions should be examined and categorized. In this paper, we have studied SDN performance improvement to pave the way for SDN researchers’ innovation. We have studied various issues in recent articles from well-known conferences and journals to find the most common performance improvement solutions in different spheres which are SDN-DC & cloud, wireless network, and SD-WAN. In this section, the problems, solutions, metrics, measures, and resources are discussed from different perspectives. SDN is composed of three layers and northbound, southbound, and east–west APIs which are summarized in Fig. 7.

Fig. 7

Overview of data, control, and application plane in SDN

6.1 Data layer performance discussion

Due to data plane in SDN and the proposed improvement methods, we can divide data plane solutions for improvement approaches into southbound API and TCAM memory space. There are various southbound APIs for communicating between the controller and the FEs. The OpenFlow protocol is commonly used as an open-source southbound API. To improve the SDN-based network performance, some changes in the functional methods have been implemented and presented, and other various standards have been proposed as southbound API whether commercial or open source. The memory space problem optimization is clustered in the data plane problem and tried to minimize memory space with the maximum number of stored flows in the flow table. Due to the high energy consumption in the TCAM, there is a direct relation between the amount of memory space and the energy consumption which causes improvement in energy saving. In this paper, improvement solutions for performance are considered, but for the future work, scrutiny can be narrowed to improvement in energy consumption.

6.2 Control layer performance discussion

According to our recent studied papers, we have classified controllers into two general categories based on open-source and commercial ones. There are some metrics which are considered in the commercial controller to select and develop in the open-source controllers. There are two main problems about the controller to improve the performance which are controller placement problem and controller performance problem. The controller placement problem (CPP) tries to optimize the number of controllers and their placement to improve the performance. The other issue which is investigated about the controller performance is the software techniques to parallelize the processes and threads. The controller performance can be summarized into three topics: (1) multi-core processor, (2) flow table partitioning, and (3) task batching.

• Multi-core processing approaches which have been implemented in the commercial and open-source controllers allocate the processes to multi-cores to parallelize the controller’s modules and make the controllers faster.

• Flow table partitioning which apportions the flow tables to reduce the search and decision time to optimize the performance of the network.

• Task batching is the other way to batch the modules of the controller to avoid the resource-wasting.

6.3 Application layer performance discussion

Different types of applications have been defined in the application plane. We have divided the application plane into five main categories: network management, quality of service, data center, and cloud computing and other applications based on SDN. The number of papers which have been presented at various well-known conferences shows the attention of researchers to network management and automatic orchestration. The core of cellular networks like 4G and 5G, as well as the Internet of Things, use SDN-based networks to increase flexibility in facing new service demands. In this plane, the QoS and the QoE are evaluated and we can categorize the solution into routing and resource allocation improvement and optimization. Figure 7 shows three layers of SDN architecture with south and north API to communicate between the layers. According to the number of papers published in journals, the most focus is on the application layer with northbound API to improve network performance.

6.4 Performance metrics discussion

The reviewed articles have used the measures to evaluate the performance which are joint with the traditional networks. These measures are:

• (a) BW: The capacity of the wired or wireless link to transfer the max amount of data from one point to another point in one second, and its unit is bit per second (bps).

• (b) Thr: The amount of data processed and transferred from one point to another point through networks in one second, and its unit is also bit per second (bps).

• (c) Delay: The time lasts to transfer data from one node to another node through a network that consists of propagation, transitions, queuing, and processing delay. Its unit is second.

• (d) Jitter: The variation of delay through two points in networks. Its unit is second.

• (e) BP (blocking probability): it indicates the probability of flows which are blocked. This metric’s unit is percent.

• (f) Bit error rate (BER): The number of bits has been altered through a channel in the result of noise, interference, distortion, and synchronization. Its unit is a percentage of altered bits to total transferred bits.

• (g) PLR: The percentage of packet loss with respect to a packet sent.

• (h) Flow rate (FR): The number of flows to be transferred through the network in per second.

• (i) Energy consumption: The percent of the energy which is saved.

Different metrics have been used to evaluate the performance in different SDN-based networks, so this metrics in different network types are compared in Table 4.

Table 4 Performance metrics comparison in SDN-based networks

To evaluate the performance in the network, we can use bandwidth, throughput, delay, jitter, BP, bit error rate, packet loss rate, flow rate, and energy. These measures can be classified into bandwidth and power which are two resources in the network generally. To improve network performance, resources have to be measurable and controllable to improve network performance. The bandwidth resource which shows the capability of data transferring in per time unit is the main resource of networks. It has a direct effect on performance metrics. The power as the other resource includes energy consumption which can be measured and controlled.

6.5 Performance problems discussion

Performance improvement has been stated as the problem to control and develop resources in SDN. Problems can be divided into two groups which are network resource scaling and network resource controlling. The problems can be expressed in (a) SDN layers category or (b) TCP/IP layers one. The SDN layers’ problems are:

• 1. Data layer: The problem statement in the data layer is organized into southbound API and TCAM space control. The solutions should be proposed to solve the southbound API overhead decrease and quick flow registering. The other problem is TCAM space control to prevent congestion control and lessen TCAM space. TCAM space reduction can influence the cost and read/write speed on FEs.

• 2. Control layer: The problem which should be solved is increasing flow rate, and make controllers distributed.

• 3. Application layer: The problem can be support many network applications like NMS, QoS, Data centers, cloud computing, and others. This layer can follow performance with implementation of different network applications.

The TCP/IP layers’ problems are:

• 1. Physical layer: bandwidth expanding with microwave, optical spectrum resource is a problem. The other is power management to decrease energy consumption.

• 2. Datalink layer: congestion control problem with cache space management.

6.6 Network resources discussion

The networks have their own specified resources which can be classified as:

• (a) BW: it is the capacity of the channel to communicate.

• (b) Optical resources: it is the resources which exist in the optical channel like wavelength, frequency.

• (c) Cache: this resource points the caching resources like TCAM, queue, flow table. This resource also indicates the caching methods in the WAN.

• (d) Power: this is the power which is used to transfer data.

• (e) Microwave resource: it is the resources like frequency in wireless media.

• (f) Cost: this is the cost of the network to transfer data.

In summary, the papers are classified based on the network resources which are used in the researches in Table 5.

Table 5 Performance improvement resources in recent review papers

6.7 Solutions discussion

These solutions have been generalized into three terms which have been extracted from the reviewed papers.

• (a) SDN architecture benefits: the benefits of the global view of the controller, programmability, and flexibility have been used with the researchers.

• (b) Resource allocation with awareness: Fairness and awareness in resource allocation make better the performance of the networks.

• (c) Distributed control plane: Multi-core processors, flow table partitioning, flow batching, and task batching in the controller with the different architecture of the controllers’ aid the innovation for performance improvement.

7 Conclusion

In this survey, we have examined the performance improvement in SDN and generalize the solutions, metrics, and resources which have been proposed in the recent papers published in ACM, IEEE, Elsevier, and Springer. We have divided the performance improvement issues into two momentous contributions which are SDN architecture and network types. According to the reviewed papers, we categorize the performance issues in data, control, and application plane. In the data plane, the network utilization is proposed with southbound API changes to reduce the number of messages, and TCAM space-saving to flow table minimization, energy consumption decrease, and cost reduction. The controller issues are CPP and controller performance. Application layer concentrates the adaptability solutions based on network condition to improve network performance. The application layer is the most interesting plane for performance improvement. The network performance is evaluated with metrics which are BW, Thr, delay, jitter, BP, BER, PLR, FR, and energy consumption. The resources which are controlled to ameliorate the network performance are BW, optical resources, cache, power, microwave resource, and cost. The solutions based on the network types which are (a) SDN-DC & cloud, (b) wireless network, and (c) SD-WAN were classified. The complete model of performance in SDN includes controllers and network with this point that the controller should be able to handle requests parallelized with performance measures improvement in the network. This article is useful for researchers working on the performance of SDN with usage considerations and leads them to evaluate, improve, and optimize SDN-based networks performance with respect to more network types. The survey with a focus on the security improvement which can classify threats and solutions in SDN can be the future work.