1 Introduction

Enterprise Information Systems (EIS) has developed rapidly in recent years, it helps companies improve the functionality of their business processes by integrating large amounts of information, and the Internet of Things (IoT) are the basic of EIS [1]. IoT connects enormously wide range of low-cost sensors into networks, and enables seamless information exchange and data processing in enterprise computing [2, 3]. EIS integrates a series of new technologies such as distributed data monitoring, large-scale data processing, artificial intelligent (AI), cloud computing and automation control. There is a large amount of data in EIS coming from large amounts of sensors and devices connected into IoT [4]. However, the network architecture of IoT is very complex, and the demand for network services is also diverse [5]. Therefore, the data transmission of IoT is a huge challenge for EIS. Accurate traffic measurement provides critical references for enterprise to design and manage the network of IoT, such as network planning, routing planning, load balancing, resource allocation and network monitoring. There are many methods to measure the network, such as direct measurement, traffic estimation. For the direct measurement methods, such as sFlow, NetFlow, they need the support of devices and additional software to measure the network, however, they consume amount of storage and computing network resource.

Fine-grained traffic size information provides essential input for many network design and operation tasks, such as capacity planning, network provisioning, load balancing, and anomaly detection [6,7,8]. Flow size can be measured directly or inferred (indirectly) from sampled statistics. In large-scale networks, direct flow measurement can be challenging due to the exploding traffic volume, limited monitoring resources and the prohibitively large overhead imposed on the network components. An alternate approach is estimating the traffic matrix from a set of aggregated/end-to-end measurements using network inference techniques. AI has developed particularly fast in the last ten years, it imitates biological processes to learn the characteristics, infers users’ behavior, and makes decisions by simulating the process of artificial recognition. It has been widely used in the practices, such as smart medical, smartphone, smart vehicle, speech recognition, and Identification, and changed human behavior pattern and living style. However, AI is still a junior intelligent system, and need to add some knowledge and reasoning to the existing applications, data, and environment to make it more friendly, smarter and more sensitive to the environment. One key challenge of traffic prediction is how much to rely on prediction models that are constructed using historical data in real-time traffic situations, which may differ from that of the historical data and change over time.

Accurate and efficient network-wide traffic measurement is crucial for network management. SDN decouples the data plane and control plane, improve the scalability of the data plane. When a new switch enters the network, it will handshake with the controller and exchange the information with each other, so the controller knows all the devices in the network, and the network topology can scale easily independently. Recently, SDN has opened up new opportunities in network measurement and inference. One major challenge in predicting traffic is how much to rely on the prediction model constructed using historical data in the real-time traffic situation, which may differ from that of the historical data due to the fact that traffic situations are numerous and changing over time. SDN opens up many new opportunities for addressing network measurement and inference problems. On one hand, it can augment network inference with multi-resolution monitoring and online learning. On the other hand, with the global view of the network, the controller can optimize the network-wide allocation of measurement resources. We could dynamically determine what/when/where to measure.

Previous studies showed that depending on the traffic situation AI models, including Naive Bayesian classifier, Decision Tree classifier, and Nearest Neighbor classifier, play an important role to improve to performance of the traffic monitoring. There are a large amount of data in the communication system, which provide rich history data for the AI model training. The application of AI in communication system has attracted the interest of many researchers. Mata et al. have a comprehensive survey of AI-based optical networking, from low-level devices to high-level management [6], AI in the optimal network not only improve the utilization of the wavelength but also improve the management efficiency. Proietti et al. utilized machine Learning-aided Quality of Transmission (QoT) estimation for lightpath configuration of intra-inter-domain traffic [8], and obtain high accurate Optical Signal to Noise (OSNR) prediction. Hagos et al. present a robust, scalable and generic machine learning-based method which may be of interest for network operators that experimentally infers Congestion Window and the underlying variant of loss-based TCP algorithms within flow from passive traffic measurements collected at an intermediate node [9]. Sendra et al. investigated the application of AI over routing protocols in the SDN paradigm, especially in wireless sensor nodes. The AI has been improved the application performance in SDN and used to improve the performance of routing protocols in the SDN network [10]. Different artificial intelligence-based techniques have been applied to achieve an enhanced load balance, network security and intelligent network applications in SDN.

There are many methods to predict the features of flow traffic in different perspective, the one of the most important applications is anomaly detection. They can learn users’ some features from the history data set, and predict the users’ behavior in the future without outside help. It can prove the system performance significantly. Since the traffic in the network is very huge, and the network management is hard, and many kinds of applicants in the network, how to accurately measure the traffic is very important to improve the utilization of network resource of operators. Inspired by the AI-based traffic prediction and the adaptive flow traffic measurement, we propose the ALAMM by predicting the traffic characteristics to decrease the measurement overhead and keep the high measurement accuracy. ALAMM is a pull-based active flow measurement. The main contribution of this paper as follows:

(1) We measure some network data in SDN in the front and use them as the data set to train the AI model which is used to predict the network traffic.

(2) We use the trained AI-based model to predict the traffic in SDN, then we proposed an adaptive sampling scheme according to the prediction results. Based on the sampling sequence obtained by the adaptive sampling, we send the measurement primitives to switches to sample the network traffic in networks and obtain the coarse-grained flow traffic.

(3) We use interpolation method to fill the coarse-grained flow measurement results and construct an optimization function which has multiple constraints to optimal the obtained fine-grained measurement, then we obtain the fine-grained flow traffic results.

The organization of this paper as follows. Section II provides the architecture of ALAMM and analyzes the model of ALAMM. We make a simulation to verify the performance of ALAMM in section III. Finally, we conclude in section IV.

2 Problem statement

In this section, we introduce the measurement model with traffic matrix of SDN. Then, we present the architecture of ALAMM and its workflow in following.

2.1 The traffic measurement of SDN

In SDN architecture, the control plane of each network device is separated from the data plane, and the control plane is logically centralized [11]. SDN makes network management much easier and more flexible. The overview of SDN architecture as Fig. 1 shows. Compared with the traditional network, SDN has features that control plane and forwarding plane decouple, logically centralized, programmable. The controller knows all the statistics of the ports and links of switches in the network, so the controller has a global view of the network. The switches in SDN are only responsible for executing the instructions from the controller.

Fig. 1
figure 1

The overview of SDN architecture

Full size image

In SDN, the control plane runs in controllers which are separated from the data plane, and data processing in switches are programed by controllers, the system model of SDN is shown in Fig. 1. There are three planes of SDN in Fig. 1, applications, control plane, and data plane. Applications know all the information of network through the northbound interface. There are many kinds of controllers, such as Opendaylight, Ryu, Nox, Pox. In SDN, all OpenFlow-based switches connect into controllers in the logical, and controllers program the action of data processing of switches through the southbound interface with the Openflow messages. The controller provides two types of interfaces, northbound interface, and southbound interface [12]. The northbound interface is the interface between applications and controllers. It is mainly called by applications and services in the high plane to obtain the network resource from controllers. The southbound interface is used to exchange information between the controller and switches, and OpenFlow is one of the most widely used protocols in the southbound interface.

The controller is the core of SDN, which allows users to develop new functions and modules, and it is flexible and scalable for operators to manage the network. The new functions and modules must be compatible with the existing software defined measurement frameworks. The southbound interface is used by the controllers and switches to exchange message with the OpenFlow protocol. In the data plane of SDN, the switches are used to process the data process in the network, they are programed by the controllers. When flow enters switches, the switch will match the flow with the flow entries in flow tables. If the flow does not match all the flow entries in the flow tables, switch will send the Packet_In messages which contained the key information of the flow to controllers. Then, the controller configures action for the flow and install flow entries to switches through Flow_mod messages, and controllers can interaction with switches and read the counter of flow entry which records the statistics of packets and bytes by sending the Packet_out messages to switches [13].

Flow in the network are from origin node to destination node. As we all know that, the flow in the network has the features of high density and high dynamic over time, so it is very huge challenge for describing the network traffic accurately and in real time. Then, we use the sequence x and y to describe the traffic of flows and links, respectively.

$$ x=\left\{x(1),x(2),\dots, x(t)\right\} $$
(1)
$$ y=\left\{y(1),y(2),...,y(t)\right\} $$
(2)

In the data plane of Fig. 1, we consider a simple mesh network with a controller in the control plane and four switches {S1, S2, S3, S4} in the data plane. Each switch has at least one physical link which connects with the other switches, there are five links {L1, L2, L3, L4, L5}. So the active flows {f1, f2, f3} in the network are transmitted with the path f1 : S1 → S2 → S4, f2 : S1 → S2 → S3 and f3 : S2 → S3 → S4. The traffic of flows and links are {x1(t), x2(t), x3(t)} and {y1(t), y2(t), y3(t), y4(t), y5(t)}. If there are N nodes in the network, then so there will have at most N ∗ N flows in the network. Then, we express the traffic of flows as a vector that

$$ X={\left[{x}_1(t),{x}_2(t),...,{x}_{N\ast N}(t)\right]}_{N\ast N}^T $$
(3)

Link is the physical connection between two switches, therefore, the switch in the network has no less than one link. If there are total M links in the network, so the link load in the network can be written as

$$ Y={\left[{y}_1(t),{y}_2(t),\dots, {y}_M(t)\right]}_M^T $$
(4)

The routing matrix represents the relationship between links and flows. Each elements in the routing matrix indicates whether a flow goes through a link or not. If amn = 1, it means the flow n transmits data through the link m; otherwise, amn = 0. The routing matrix can be written as

$$ A={\left[{a}_{mn}\right]}_{M\times \left(N\ast N\right)},m\in \left[1,M\right],n\in \left[1,N\ast N\right],{a}_{mn}\in \left\{0,1\right\} $$
(5)

Then, the traffic matrix can be written

$$ Y= AX $$
(6)

In SDN, controller discovers active switch ports and links by periodically sending LLDP packets in the network, so controller has a global view of the network topology. Because of the controller has a global view of the network, so it is easily for use to obtain the routing matrix A from controller.

In SDN, the controller collects real-time flow statistics from the corresponding switches, merges the raw data and passes the results to the applications. Then, we can measure the traffic of links and flows by collecting statistics from switches. There are many methods to measure the network traffic, such as system sampling, random sampling. In this paper, we proposed an AI-based method to measure the traffic of links and flows.

2.2 The AI-based measurement architecture in controller

AI is considered to be a junior intelligent system, it can imitate human behaviors to make some simple judgments or do some predictions and do some actions. AI requires amount of knowledge, environment information and historical data to train it. In recent years, some researchers have investigated the applications of AI in the field of communications to predict the traffic in network. It is important for traffic engineering to obtain the fine-grained measurement in SDN, such as load balancing, route planning. However, the pull-based fine-grained measurement in SDN generates much overhead and consumes much computing resource and storages in network devices. So, measuring the network traffic with low overhead is one of the critical issues in traffic measurement. If we know the feature of flows, we can sample the flow traffic based on our requirement and recovery the fine-grained measurement more accurately.

In this paper, we provide that using AI to predict the traffic feature and adaptive sampling scheme to update the sampling frequency. AI can learn features from historical data and predict actions and trends in future based on the predictive model. When we accurately measure the network traffic, the flow traffic trend is very significantly important for us to determine the sampling frequency. We shows the components of the ALAMM as shown in Fig. 2, the prediction module, sampling frequency module, coarse-grained measurement and training module are core of AI which used to measure the traffic directly. The fine-grained recovery module are and the measurement agent module are used to understand the requirement of the applications and program the measurement instruction.

Fig. 2
figure 2

The AI-based measurement architecture in SDN

Full size image

Artificial Neural Network (ANN) is one of the most important methods of AI [14], it is an algorithmic mathematical model which mimics the behavioral characteristics of animal neural networks and performs distributed parallel information processing. ANN is a data-driven method, it should be trained by inputting sufficient of data and knowledge, and it is often used for solving multi-dimension traffic prediction issues, such as network traffic, vehicle traffic. ANN is different from the traditional statistics method, it has the learning ability and its structure is flexible, users can change the ANN structure based on their requirement. The ANN model as Fig. 3 shows, it consists of one input layer and J hidden layers and one output layer, it is a stack of many neurons. There are K units in input layer, K neurons in output layer and M neurons in each hidden layer. Each neuron in the hidden layer and output layer connects to all neurons of the front layer with weighted edges, and neurons on the same layer cannot be connected to each other.

Fig. 3
figure 3

The neural network model of network traffic prediction

Full size image

The output of neuron in the hidden and output layers is a statistics variable, it is a weighted sum of the previous layer of neurons, so the neuron in the hidden and output layers can be expressed as

$$ {H}_{1m}=f\left(\sum \limits_{k=1}^K{w}_{1 km}{x}_k\right),m=\left\{1,2,..,M\right\} $$
(7)
$$ {H}_{jm}=f\left(\sum \limits_{n=1}^M{w}_{jnm}{H}_{j-1n}\right),m=\left\{1,2,..M\right\},j=\left\{2,3,..,J\right\} $$
(8)
$$ {y}_k=f\left(\sum \limits_{n=1}^M{w}_{jn k}{H}_{jn}\right),k=\left\{1,2,...,K\right\},j=\left\{J\right\} $$
(9)

where xk and yk are the measured and the prediction of flow traffic, respectively. wjnm is the weight of the edge in layer j from neuron n in layer j − 1 to neuron m in layer j. f(⋅) is the activation function that processes the traffic flow measurements corresponds to a log-sigmoid transfer function of the form linear data patterns.

Through short time slot sampling, we obtain the instantaneous rate of flows and links. Instantaneous rate measurement requires the controller sends two Read-state messages at shorter intervals, which is also a challenge for the controller. For simplicity, we can program it as a special primitive for rate measurement. However, the traditional flow-based fine-grained network measurements in SDN require the controller to frequently send Read-state messages to OpenFlow-based switches, and also generate a large amount of report messages to the controller, which would consume much computing resource of the controller. So, we use coarse-grained measurement method to reduce measurement overhead. SDN provides Flow-based measurements, it makes traffic measurement much easier than measurements in traditional networks. The OpenFlow-based switches in SDN count the packets, bytes and duration of flows that successfully matches the flow entries in the flow tables. So, the traffic measurement of flows and links can be considered as collecting the statistics in OpenFlow-based switches. SDN provides two approaches for controller to retrieve statistics from OpenFlow-based switches, push-based [15] and pull-based [16]. The push-based is an active methods, the switches send their statistics to controller when the flow entry removal or expiry. And the pull-based approach is a passive method, it needs the controller send Read-state messages to switches to collect the flow statistics or port statistics information. The push-based approach must be supported by the network devices and is not flexible, and the pull-based approach can be programmed by the controller based on the requirement. So, with the pull-based approach, the controller can arbitrarily measure the traffic in OpenFlow-based switches based on the requirement of applications and services.

In SDN, the fine-grained measurements better describe flow traffic in network, but it requires the controller to frequently send Read-state messages to OpenFlow-based switches. In order to decrease the overhead of measurement, the sampling method is proposed for the network measurement [14, 17]. Figure 4 shows the actual traffic in five minutes, and the red circles are the sampling points. With the coarse-grained sampling points, we use the matrix filling methods to recovery the traffic in the network.

Fig. 4
figure 4

the actual traffic of a flow in front 5 min

Full size image

For the traffic of flows in SDN are {xn(t)| n ∈ [1, N ∗ N]}, we can obtain the coarse-grained sampling results \( {\hat{x}}_n(k) \) with the sampling sequence δ(k), then the sampling process can be written as

$$ {\hat{x}}_n(k)={x}_n(t){\delta}_n(k) $$
(10)

where δn(k) is the sampling sequence of flow n, and k is the sampling time sequence. So, how the obtain the sampling time sequence is the key issue that we should studied in depth.

There are many sampling approaches which widely used, such as system sampling (periodic sampling) [15], adaptive sampling [17]. We can know that the system sampling and adaptive sampling are changed with the time. The system sampling is also called as the periodic sampling, its update length of each step is stable. For the sampling method, the sampling sequence can be written as

$$ {\delta}_u(t)=\sum \limits_{k=1}^{K_u}\delta \left({t}_0+ k\varDelta {t}_u\right) $$
(11)
$$ {\delta}_a(t)=\sum \limits_{k=1}^{K_a}\delta \left({t}_0+ k\varDelta {t}_a\right) $$
(12)

where δu(t) and δa(t) are the sampling sequence for system sampling and adaptive sampling, respectively. t0 is the start of the sampling process. Δt is the updating step for each sampling point. So, the sampling sequence is very important for the accuracy and overhead of measurements.

$$ \delta \left({t}_n\right)=\left\{\begin{array}{l}1,R\left({t}_n\right)-R\left({t}_n-\varDelta t\right)\ge \theta \kern0.5em \\ {}0, otherwise\end{array}\right. $$
(13)

where δ(tn) is sampling or not, R(tn) is the flow traffic prediction at time t. If the flow prediction value changes bigger than thresholdθ, we send a sampling primitive; otherwise, we think the flow is stable, and not sampling the traffic of the flow.

In the measurement scheme, users and applications can program the sampling period of traffic rate and the measured traffic set by calling the periodic sampling module to obtain the coarse-grained measurement There are many traditional methods for measuring the traffic rate of links and flows, such as sFlow and Netflow, but they should use the specific software or hardware to measure per-flow traffic rate and link load. In SDN, there is a simple method to measure the traffic rate by collecting the statistic of flows and ports in OpenFlow-based switches. Then, we propose the sampling primate to measure the network traffic. The controller programs a flow entry with the custom hard timeout, and installs it into OpenFlow-based switch. They utilize the custom hard timeout expiry as the trigger for statistics collection. Users can set the hard timeout based on the changes in flow traffic. However, when the custom hard timeout is large, the controller needs to wait a long time to receive the Flow_Removed message. In order to increase the flexibility of the measurement, the authors add a customizable active measurement method. When the hard timeout of the flow entry is bigger than the measurement period, the controller will send the Read_state message to the switch to collect statistics.

Both of the above methods mainly use the push-based mechanism to measure the flow traffic, this measurement scheme is inflexible. Here, we use the pull-based mechanism to measure the traffic of flows and links. There are two measurement interval should be set, the measurement interval and the measurement periodic. The measurement interval is the time difference of two measurement checkpoints, and the measurement period is the interval between two traffic rate measurements. For the statistic of flows and links, instantaneous traffic rate is the increment between two measurement checkpoints. The controller is programmed to send the Read_state message to collect the statistic of flows and ports at checkpointt1 and t2, then, the OpenFlow-based switch will report the statistic S(t1) and S(t2) to the controller. So, S(t2) − S(t1) is the byte increment of the statistics collected twice, and t2 − t1 is the measurement interval. If the measurement interval t2 − t1 is small enough, we think that the traffic S(t2) − S(t1) is the instantaneous traffic at time t. So, the instantaneous traffic rate is

$$ R(t)=\frac{S\left({t}_2\right)-S\left({t}_1\right)}{t_2-{t}_1} $$
(14)

By collecting the statistics of flows and links from the OpenFlow-based switches and utilizing Eq. (14), we obtain the traffic instantaneous sampling rate of links and flows. We utilize the Packet_In and Flow_Removed messages in the OpenFlow protocol to measure the traffic to know about the active flow in the network. We measure the traffic rate of active flows and links in the network with (14), in order to decrease the overhead of measurement process, so we obtain the coarse-grained measurement with the AI-based method. Then, we must use some method to find the fine-grained measurement method with the coarse-grained measurement and matrix filling method.

2.3 Fine-grained matrix filling and optimization

The fine-grained measurement result of flow j is xj, it has been filled by the interpolation method, and the actual flow traffic of flow j is \( {\hat{x}}_j \). Due to the cubic interpolation is the smoothest possible approximations of actual flow traffic, and there is also a gap between the measurement results and the actual flow traffic. In order to obtain a high measurement accuracy, we optimize the filling data to decrease the gap between the measurement results and the actual results of flows. Then, we propose an optimal function as follows:

$$ \left\{\begin{array}{l}\min \kern1.32em {\left\Vert Y-A\hat{X}\right\Vert}_2+\lambda {\left\Vert X\right\Vert}_2\\ {}\kern1.32em s.t.\kern0.33em \\ {}\kern2.97em C1:\kern0.66em {Y}_i\ge {A}_i\hat{X},\kern0.99em A={\left({A}_1,{A}_2,\dots, {A}_M\right)}^T\\ {}\kern2.97em C2:\kern0.66em {x}_j\ge 0,{Y}_i\ge 0\\ {}\kern2.97em C3:\kern0.66em \sum \limits_{j=1}^N{x}_{ij}=\sum \limits_{j=1}^N{x}_{ji}\kern0.33em \end{array}\right. $$
(15)

where λ is a Lagrange multiplier. Contrast to Eq. (15), the equation can easily be solved by the heuristic algorithm proposed in this paper.

Constraint C1 represents the constraint between link load and flow traffic, it means that the total traffic of flows on each link is no more the link capacity; constraint C2 represents the flow traffic limitation of links, which means that the traffic of flows and links is non-negative. Then, using the optimal function (15), we can obtain better flows measurement results. C3 denotes that the traffic of each node is conservative, i.e. the traffic flowing into one node is equal to that flowing out of it. Under constraint C1, we know that link load and flow traffic mapping relationship matrix A has M rows and N columns,

and M ≥ N when multiple flows transmission through a link. Then, the routing matrix A is an underdetermined matrix and there are infinite solutions for the linear constraint C1.

As we know the function (15) is very hard to solve it, then we use the Lagrange method to relax the constraints in the objective function (16), so the function (16) can be written as

$$ {\displaystyle \begin{array}{l}F(x)={\left\Vert Y-A\hat{X}\right\Vert}_2+\lambda {\left\Vert X\right\Vert}_2+\alpha \left({Y}_i-{A}_i\hat{X}\right)\\ {}\kern3em +\beta \left(\sum \limits_{j=1}^N{x}_{ij}-\sum \limits_{j=1}^N{x}_{ji}\right)\end{array}} $$
(16)

where α and β are the Lagrange multipliers. Then, we use the iteration method to find the fine-grained measurement which has been optimized. In the following, we propose the algorithm 1 to train the model of AI to predict the traffic of flows and links as shown in Table 1, and obtain the coarse-grained measurement based on the measure control with the AI as the algorithm 2, then we use the matrix filling method to recovery the fine-grained measurement and decrease the measurement error with the optimization method, the algorithm is shown in Table 2.

Table 1 Training the model of AI to predict the traffic of in SDN
Full size table
Table 2 The algorithm of ALAMM
Full size table

3 Simulation result and analysis

3.1 Simulation environment

In order to evaluate the performance of the measurement scheme proposed, we built a SDN test platform and wrote the measurement module in python. We use Ryu [18] as the controller and simulate the switches, hosts and their links by Mininet [19], and the OpenFlow-based switches is simulated by Open vSwitch in Mininet. In OpenFlow-based switch, flows match the flow entries in flow tables by the source IP, destination IP, source MAC, destination MAC and port, then we generate multi flows from one host to the other hosts. We use the video playing flow, and set the maximum link load as 10 Gbps. We conduct a simulation platform with the core Intel.

i5–3230 and 6G RAM, the operating system is Ubuntu 16.04TLS. The software version in the simulation platform as Table 3 shows. In the training process of the AI model, we have use 6 hidden layers, and use the front 500 time slots as the training data set to train the model.

Table 3 The version of software used
Full size table

For simplicity, we simulate a simple topology consisting of a controller, six switches and six hosts in the topology as Fig. 5 shows. We import the measurement module programmed into the controller, then, call the Periodic Sampling module to send Read-state message to OpenFlow-based switches through OpenFlow protocol.

Fig. 5
figure 5

The network topology in the simulation

Full size image

3.2 Simulation metrics

To verify the performance of the measurement scheme proposed, we introduce some common error evaluation metrics, such as Relative error (RE) and Relative Mean Square Error (RMSE).

The smaller the RE, the more accurate the measurement. The RE is the ratio of measured absolute error to actual value, reflecting the credibility of the measurement. The RE can be expressed as

$$ R{E}_i=\frac{\left|{x}_{ij}-{\hat{x}}_{ij}\right|}{{\hat{x}}_{ij}} $$
(16)

RMSE is usually used to evaluate the measurement accuracy in simulation experiments. It reflects the average deviation between the measured and actual values.

$$ RMSE=\frac{1}{N}\sum \limits_{t=1}^N\frac{{\left\Vert x-\overline{x}\right\Vert}_2}{{\left\Vert x\right\Vert}_2} $$
(17)

3.3 Simulation evaluation

In the simulation, we compare our ALAMM proposed with the measurement scheme of system sampling and random sampling, and the estimation method of Principal Component Analysis (PCA). ALAMM, R60, R240, U60, U240 are the same measurement scheme with different sampling frequency. ALAMM uses the adaptive sampling interval based on AI. U60 and U240 mean the system sampling scheme with the sampling interval 60 and 240, respectively. R60 and R240 replace the measurement scheme with the mean value of the random sampling is 60 and 240, respectively.

The network topology as Fig. 5 shows has 6 hosts, 6 switches in the network. We transmit the videos in network, so each user can retrieve media data from servers. All the links in the network is set as the duplex transmission mode. There are 100 OD pairs and 8 links between switches in the network. We make 20 flows in the network. By analyzing traffic of flows in the network, we find that traffic of each flow are similar with each other, then we select two flow randomly as examples to make analysis. Figure 6 show the confusion of the actual traffic and the measurement of different methods. The x axis are the actual traffic, and y axis is the measurement result. There is a diagonal in each sub-figure which represents y = x, it means the measurement equal to the actual traffic. When the dots at the upper of the diagonal, it means the measurement is bigger than the actual traffic, otherwise, the measurement is smaller than the actual traffic. The (a), (b), (c), (d), (e), (f) of Fig. 6 represent the measurement of ALAMM proposed, PCA, system and random sampling, respectively. We find that our ALAMM method is the better than the other measurement method, and the PCA method is the worst.

Fig. 6
figure 6

The traffic comparison of measured and actual

Full size image

Figure 7 shows the RE cumulative distribution function (CDF) of flow f2 and f4, respectively. From Fig. 7a, we can see that the about 80% RE of the ALAMM and Uniform sampling method are less than 0.3, while the performance of PCA method is the worst. Figure 7b has similar trend with Fig. 7a. However, the average RE of ALAMM is smaller than Uniform60 and larger than Uniform240 and PCA, the curves show that the ALAMM is a slightly inferior to Uniform60 but better than U240 and PCA. This mean that the measurement scheme based on sampling method is feasible, and the ALAMM which has the AI-based adaptive sampling interval is better than other sampling scheme. PCA has the worst measurement performance. This is mainly because PCA is an estimation measurement.

Fig. 7
figure 7

The measured overhead for different measurement methods

Full size image

Figure 8 shows the curve of RMSE of each method, it reflects the average RE of those measurements. We can see that the curve of measurement result of ALAMM, U60, U240, R60, R240 are most in 0.12 to 0.2, this mean that the measurement scheme with the sampling is stable. And the RMSE of ALAMM is much steeper than the other methods, this means the average measurement RE of ALAMM is much more stable than other methods.

Fig. 8
figure 8

The RMSE of the relative for different measurement methods

Full size image

We compare the difference between the overhead and RE of measurement scheme with different sampling model and interval. Figures 9 and 10 are the average measurement overhead and RE by repeating 100 simulations. Figures 9 and 10 shows the measurement overhead and average RE of different measurement, respectively. While the bar means the measurement overhead of each method, the blue line represents the measurement of ALAMM, and the black line at the top of bar are the standard deviation of measurement overhead. The bars in Fig. 10 represent the average RE of different measurement scheme in the simulation. We note that the measurement overhead of R60 and U60 scheme are higher than the other sampling scheme because there sampling granularity is smaller the other sampling scheme. The sampling interval of the ALAMM is similar with the U240, and it sampling overhead is a little bigger than the U240. But measurement RE of ALAMM is smaller than the U240. Though the measurement RE of ALAMM, R60 and U60 are the similar with each other, but the measurement overhead of R60 and U60 are much more than the ALAM. This donates our ALAMM scheme is feasible and with little overhead.

Fig. 9
figure 9

The measured overhead for different measurement methods

Full size image
Fig. 10
figure 10

The measured relative errors for different measurement methods

Full size image

Figures 11 and 12 show the average RE and average overhead of ALAMM for different measurement interval update step-length and sampling threshold. From Fig. 11, we note that for the same sampling threshold, the measurement RE increases with the sampling update step. And for the same sampling update step, the measurement RE increases along with the sampling threshold. At the same time, we find the sampling overhead in Fig. 12 decrease with the sampling interval update step length. When the sampling interval step beyond 20 slots, the measurement RE and overhead are both updating slowly. Therefore, we can change the sampling granularity based on our requirement of the accuracy and measurement overhead.

Fig. 11
figure 11

The average relative errors for different measurement threshold

Full size image
Fig. 12
figure 12

The average sampling number for different measurement threshold

Full size image

4 Conclusions

EIS aggregates a large amount of data of companies from devices in IoT, then the real-time reliable data transmission and data processing are very important for EIS. Accurate flow-based network measurement has a great impact on network traffic management in SDN. In this paper, we propose a novel ALAMM for flow traffic of SDN to decrease the measurement overhead with the high measurement accuracy. Inspired by the AI widely used in the practice. Then, we proposed the ALAMM for SDN to decrease the overhead and improve the accuracy. Firstly, we use the AI to predict the flow traffic in the network and with the traffic prediction result to obtain the sampling frequency adaptively. Secondly, we send the measurement primitives to switches to measure the coarse-grained traffic of flows and links from switches. Finally, the matrix filling and optimization method are proposed to recovery the fine-grained measurement and optimize the measurement result. Simulation results show that our approach can obtain the network traffic with low overhead and high accuracy.