1 Introduction

The backhaul network is composed of dedicated lines connecting the base station and the core network, which is an important part of the 5G mobile cellular network. Millimeter wave (mm-Wave) has been paid more attention in the applications of cellular network due to large available communication bandwidth. However, the effective transmission distance of mm-Wave is usually around 200 m because of its line-of-sight propagation characteristic [1]. Therefore, the concept of the mm-Wave small cell (SC) is proposed to cope with the inefficiencies of the frequency band by mm-Wave and to cope with ultra dense deployment by small cells. Based on this, the idea of using mm-Wave technology into wireless backhaul of future 5G system is naturally proposed, namely mm-Wave backhaul network. However, some new challenges are put forward for the mm-Wave backhaul network in terms of the metric parameters of the 5G system, such as the peak rate is at least 10 Gbps, end-to-end delay is limited in 1 ms and so on. In addition, the network topology is becoming irregular due to the plug-and-play deployment of the SC. There may be some isolated or broken ‘point‘ if simply one-hop backhaul is used. Finally, very high transmission rates of the users may be required in a short time, and no data of users may be transmitted in a long time. This shows that the traffic of SC is abrupt and dynamic changed. Therefore, the mm-Wave backhaul network is a network whose traffic is transmitted to the core network with mm-Wave communication in some possible way.

The work related to mm-Wave backhaul are mainly divided into two types, from the perspective of network architecture, i.e. centralized style and distributed style [2]. In the centralized backhaul network, traffic is transmitted to the macro base station (MBS) in one hop with mm-wave, where MBS is connected to the core network with fiber. The backhaul routing need to be controlled by the center in the centralized backhaul, which is also not conducive to the expansion of the network [3]. In the distributed backhaul style, traffic is transmitted into the sink nodes with mm-Wave wireless communication in a distributed multi hop manner, so as to form a mm-Wave self backhaul network (mW-SBN). Only a few cooperations between adjacent SCs are required. The whole network is extensible, and is also suitable for the ultra dense deployment of 5G [4].

One of the key for multi-hop mW-SBN is the design and optimization of the backhaul routing protocols [5]. Most existing protocols are designed for networks with which omnidirectional antennas have been equipped in each node. However, all of these may not be applicable to beamforming based mW-SBN [5]. A simple backhaul scenario in mW-SBN is shown in Fig. 1, where the node represents SC. Assume that D is the sink connected to core network by fiber. Nodes S, \(R_1\), \(R_2\), and \(R_3\) are general nodes communicating with mm-Wave. Note that S and D are not in the one-hop communication range of each other, if there is some traffic of S with certain requirement of rate to be transmitted to D, the relay nodes \(R_1\), \(R_2\), and \(R_3\) need to be selected to implement multi-hop backhaul. The connection between the adjacent nodes represents a mm-Wave beam link, and the number on the connection stands for number of remaining channels available for data transmission on the beam link. Note that if the traffic transmission rate of each channel is given, the number of the channels can be used to quantitatively character the maximum link rate of the link supported. In this paper, the maximum achievable link rate is defined as the remaining bandwidth of the link. A non-remaining bandwidth single-path routing (NBSR) scenario is shown in Fig. 1(a), where the traffic/ack flows between S and D are transmitted on a single path and the available remaining bandwidth of each node for backhaul is not known by adjacent nodes. A non-remaining bandwidth multi-path routing (NBMR) scenario is showed in Fig. 1(b), where traffic/ack flows between S and D are transmitted on multiple paths but the available remaining bandwidth of each node for backhaul is not known by adjacent nodes. A remaining bandwidth based multi-path routing (RBMR) scenario is showed in Fig. 1(c), where traffic/ack flows between S and D are transmitted on multiple paths and the available remaining bandwidth of each node for backhaul is known by adjacent nodes. How the transmission performance of mW-SBN is affected by multi-path and remaining bandwidth in Fig. 1 is compared as follows.

Fig. 1
figure 1

Remaining bandwidth based multi-path routing scenario

Full size image

1.1 Comparison between Fig. 1(a) and (b)

In Fig. 1(a), there is only one path to D from S, i.e. \(S \rightarrow R_2 \rightarrow D\), so only one path can be selected by S to backhaul traffic. While in Fig. 1(b), there are multiple paths to D from S, i.e. \(S \rightarrow R_1 \rightarrow D, S \rightarrow R_2 \rightarrow D\) and \(S \rightarrow R_3 \rightarrow D\). So the first traffic flow of S may be sent to \(R_1\). Then the second and the third traffic flows may be sent to \(R_2\) and \(R_3\), and the first traffic flow may be forwarded by \(R_1\) at the same time, so traffic of S will be sent on multi-paths. Of course, the ack flow from D to S can also be transmitted with the same style. The number of concurrent links and throughput of the network are increased by this multi-path strategy. Note that the multi-path strategy is not applied to traditional sub-6G network due to S, \(R_1\), and \(R_2\) are within carrier sensing range of each other. The comparison clearly shows that the mW-SBN performance is more likely improved by multi-path backhaul strategy than single path ones.

1.2 Comparison between Fig. 1(b) and (c)

It is assumed that the remaining bandwidth of the link between node S and \(R_1\), \(R_2\), \(R_3\) in Fig. 1(b), (c) is 0, 10, 30, which is called the link remaining bandwidth. The link remaining bandwidth between node D and \(R_1\), \(R_2\), and \(R_3\) is 30, 20, and 10. In Fig. 1(b), although there are multiple paths to the destination node D in S, i.e. \(S \rightarrow R_1 \rightarrow D, S \rightarrow R_2 \rightarrow D,S \rightarrow R_3 \rightarrow D\), the remaining bandwidth between adjacent nodes is not known to each other, so that the path bandwidth of each path is not known by S, the value of which is the minimum link bandwidth between all adjacent nodes on the path. When the heavily loaded path, such as the path \(S \rightarrow R_1 \rightarrow D\), is selected by S, the remaining bandwidth of the selected path can not necessarily meet the transmission requirements of S, which will result in a decrease in terms of network throughput. In Fig. 1(c), not only multiple paths to D, i.e. \(S \rightarrow R_1 \rightarrow D, S \rightarrow R_2 \rightarrow D,S \rightarrow R_3 \rightarrow D\), is known by S, but also the remaining bandwidth of each path is know by S, i.e. the path remaining bandwidth of \(S \rightarrow R_1 \rightarrow D\) is 0, the remaining bandwidth of \(S \rightarrow R_2 \rightarrow D\) and \(S \rightarrow R_3 \rightarrow D\) is 10. Note that the remaining bandwidth of each path is obtained through bandwidth information interaction between all adjacent nodes on the path and the remaining bandwidth based route discovery. At this time, the paths that meet its bandwidth requirement for data transmission, such as \(S \rightarrow R_2 \rightarrow D\) and \(S \rightarrow R_3 \rightarrow D\) are selected by S. The comparison between Fig. 1(b) and (c) shows that the path remaining bandwidth information contributes to path selection and effective backhaul.

In summary, if multi-path and remaining bandwidth information are obtained by nodes, efficient routing and transmission of the traffic are enhanced at some degree. It shows that the design of remaining bandwidth based multi-path routing is a key problem for the realization of mW-SBN. Therefore, a remaining bandwidth based multi-path routing protocol (RBMR), which is operated in distributed manner and based on dynamic source routing, is proposed in this paper. From the protocol classification, RBRM is an improved variant of the AODV (Ad hoc On-demand Distance Vector Routing) protocol. However, when compared with the traditional AODV related protocol, the remaining bandwidth is introduced to realize the route discovery and maintenance, making it perfectly integrated with the mW-SBN with directional transceiver. The main ideas of RBMR includes the following points. (1) The route discovery process is initialized by S when there is not any path that satisfied bandwidth requirements. (2) Routes in routing table is reserved and maintained when the lifetime of the path is lower than the threshold. (3) One route is removed when the source detects that the total current effective path bandwidth exceeds the threshold of the load. Our contributions are four aspects and summarized as follows.

  • The concept of remaining bandwidth is described from the perspective of channel division.

  • RBMR is designed to meet the highly dynamic requirements of transmission bandwidth and the reliability of the traffic.

  • The upper bound of average multi-path for network nodes is analyzed.

  • Performance of RBMR and its three variations, which are NBSR, NBMR, and RBSR, are simulated contrastively, and the corresponding result is analyzed.

The rest of this paper are organized as follows. The related work and the system model is introduced in Sects. 2 and 3. In Sect. 4, the operation of RBMR is introduced in detail and the upper bound of average multi-path for network nodes is analyzed theoretically. We compare the simulation results in Sect. 5. Finally, Sect. 6 summarizes the paper with work to be carried out in the future.

2 Related work

Recently, work related to mm-Wave backhaul has gained more attention. The two types of backhaul schemes with mm-Wave are analyzed in [6] firstly. The feasibility, problems and corresponding underlying method for mW-SBN has been surveyed in [7] especially. All follow-up related studies on backhaul network performance analysis and backhaul protocol design are based on the two scheme and architecture.

The backhaul path planning algorithms based on C-RAN or SDN for centralized mm-Wave backhaul networks are proposed by most of the literatures [8, 9]. Then the scope was expanded to the joint optimization problem including backhaul path planning, node deployment, resource allocation and so on [10, 11]. Among them, there is no lack of the ideas of clustering [12]. These problems are usually modeled as mixed integer programming [13], multi-objective optimization [14], etc., and the optimal solution is solved through optimization theory [13, 14].

There are few works on distributed multi-path routing in mW-SBN. Mello et al. [15] first demonstrates that there are significant gains in improving network reliability and coverage and reducing network operation costs with multi-path backhaul scheme through the system-level simulation. A routing protocol based on greedy forwarding algorithm according to the Mesh routing properties created by Delaunay is designed in Ref. [16]. Ref. [17] assumes that each node can flexibly adjust its own beam width according to the number of nodes in the network. Based on this, a novel beam width adaptive elastic routing protocol is proposed, where the next hop forwarding distance of the node can be flexibly adjust according to node’s beam width. Their simulation results show that the proposed protocol is better than shortest path routing protocol. Ref. [18] proposes a node-disjoint multiple path routing protocol based on dynamic source routing for 60 GHz mm-Wave networks. The network performance, such as routing overhead, throughput and so on, is compared and analyzed in the paper, where three different types of route request broadcast mechanisms and route request forwarding mechanisms are used in a combination manner. In Ref. [19], Seppanen et al. proposes a multi-path routing method with fault diagnosis and recovery. The algorithm generates a primary path and multiple backup paths between source and destination according to optimizing the set of primary paths. In Ref. [20], Al-Saadi et al. proposes a machine learning based cognitive heterogeneous routing protocol. The controller in the protocol selects the appropriate backhaul scheme based on the parameters of each layer of the network. In Ref. [21], Liang et al. designs a multi hop backhaul protocol with maximized throughput for the randomly deployed network, and proves that multi-hop heterogeneous backhaul is more energy efficient than traditional single-hop backhaul through simulation. In Ref. [22], a cluster-based multi-hop backhaul scheme is proposed. The cluster head elected method is given first. Then a route protection strategy for balancing backhaul traffic between cluster heads is designed to minimize the re-route overhead caused by cluster head reselection. An effective operation for mm-Wave heterogeneous multi-hop backhaul network controlled is proposed in Ref. [23]. Two functions are included in the operation, namely, multiplexed backhaul routes of overload SC and changed ON/OFF state of underload SC. A self-organized backpressure routing mechanism for dynamically deployed SC networks is proposed in Ref. [24, 25]. When traffic is routed, queue backlog and geographic information are combined in the mechanism to mitigate network congestion. Performance of Backpressure Multi-Radio (BP-MR per-packet), Backpressure Multi-Radio (BP-MR per-flow) of the paper are compared with OLSR.

However, these works can only be used in some specific scenarios. To the best of our knowledge, there is no work that has focused on how to design a effective backhaul multi-path routing protocol from the aspect of the remaining bandwidth of the backhaul path to solve the problem of the effective dynamic backhaul of traffic in mW-SBN.

3 System model

As aforementioned, the link remaining bandwidth information is beneficial to multiple path establishment and the efficient traffic backhaul according to the qualitative comparison between Fig. 1(b) and (c). In this section, the directional transceiver model is introduced first (given in the Sect. 3.1), then the remaining bandwidth is described in detail from the perspective of channel division (given in the Sect. 3.2).

3.1 Directional transceiver model

The directional communication reduces the interference between links, and concurrent transmissions (spatial reuse) can be exploited to greatly improve network capacity. At mm-Wave frequency, highly directional transmission using antenna arrays is an effective technique to overcome its heavier path-loss compared to the microwave systems. With SBSs fixed, the directional LOS transmission between SBSs can be achieved with the locations of SBSs adjusted appropriately.

We denote the distance between the transmitter s and the receiver r by \(d_{sr}\). We also denote the maximum directional gains of the transmitting end and the receiving end as \(G_t(s,r)\) and \(G_r(s,r)\), which can be achieved when the beam between them is aligned. In practical applications, the values of \(G_t(s,r)\) and \(G_r(s,r)\) depends on many factors, such as the number of antenna elements, multi-path angular scattering and the precision of the channel estimation. For analytical tractability, the actual antenna patterns are approximated by a sectorized antenna model, which is widely used in Ref. [26, 27], for physical layer, medium access control (MAC) layer, and radio resource control (RRC) layer studies. This simple model captures directivity gains, front-to-back ratio, and half-power beamwidth, which are considered as the most important features of an antenna pattern. In ideal sector antenna pattern, the gains are a constant for all angles in the main lobe, and equal to another smaller constant in the side lobe. Due to reciprocity, it can be assumed that the directional transmitting (TX) and directional receiving (RX) gain are the same. Then considering the path loss and signal dispersion over distance, the received power \(P_r(s,r)\) at r from s on beam k can be calculated as

$$\begin{aligned} P_{r}^{k}(s,r)=\theta G_t(s,r)G_r(s,r)d_{sr}^{-n}P_t(s,r) \end{aligned}$$
(1)

where \(\theta \) is a constant coefficient and proportional to \(\left( \frac{\lambda }{{4\pi }}\right) ^2\) ( \(\lambda \) denotes the wavelength), n denotes the path loss exponent, and \(P_t(s,r)\) denotes the transmission power. In mm-Wave networks, since each SBS is usually deployed on a better location, such as on the roof of the building or on the light pole of the road, and the communication beams is made extremely thin (usually \(5^{\circ }\)–\(10^{\circ }\)) according to massive MIMO, the inter-beam interference can be ignored, so the SINR can be substituted by the SNR [6, 9, 19, 28]. Then according to Shannon channel capacity, the link remaining bandwidth of the link from s to r on beam k can be estimated as

$$\begin{aligned} {R_{r}^{k}(s,r)} = \eta W{\log _2}\left( {1 + \frac{{{\theta }{G_t}(s,r){G_r}(s,r)d_{sr}^{ - n}{P_t}(s,r)}}{{{N_0}W}}} \right) \end{aligned}$$
(2)

where W is the bandwidth, and \(N_0\) is the onesided power spectra density of white Gaussian noise. Note that all the parameters except \(\eta \) of the above Shannon formula are deterministic in a static mesh network. \(\eta \in [0,1]\) is the characterization factor of the efficiency for the transceiver design (such as antenna elements, modulation, coding etc.) and the transmission conditions (such as blocking effect, outrage effect etc.) [29]. However, due to it is too difficult to model the factor in the actual environment, the characterization factor can be explained with the statistically analyzed packet loss rate at a large extent [30]. Therefore, the characterization factor is expressed herein as the statistical non-packet-loss-rate of the network for a period of time, which is named statistics time, and the period of the RHEL timer relative to the statistical time can reflect the accuracy of the channel state.

3.2 Remaining bandwidth

It is assumed that there are N ordinary SCs and M special SCs, which is always called sink node, in the mW-SBN. The sink nodes are connected to the core network through fiber and are communicated with the ordinary SCs through mm-Wave. The coverage of each SC is divided into K beam links (i.e. K RF chains is used). The traffic of SCs transmitted to the sink node in a multi-hop cooperative manner with mm-Wave communication. We assume that the neighbor discovery protocol in Ref. [31] is used, where the initial neighbor node table, which stores all of the neighbor SC around itself, for every nodes is established with the beam scanning. So the basic information of the neighbor nodes can be known by the nodes clearly, such as unique identification number of the neighbor node, the beam number of the neighbor node located, and the number of the established mm-Wave links. Each mm-Wave beam link between adjacent nodes is divided into multiple channels, and these channels are divided into traffic channels and backhaul channels. The traffic channel is used for uplink and downlink transmission of the traffic for the SC, and the backhaul channel is used for the backhaul of the traffic. The number of available backhaul channels on the k-th mm-Wave beam link in the SC is denoted by \(x_k\), \(1 \le k \le K, k \in {\mathbb {Z}}^+\), and K is the total number of the beam links between the SC and its neighbor node. \(\tau \) is the transmission rate on each channel. If the total backhaul remaining bandwidth of the SC is denoted by \(\alpha \), then it can be expressed as:

$$\begin{aligned} \alpha = \sum \limits _{k = 1}^{K} {{x_k} \times \tau } \end{aligned}$$
(3)

If \(y^{i}\) is the number of beam links between the SC and i-th neighbor node, \(w_k^{i}\) is the number of available backhaul channels between SC and i-th neighbor on k-th mm-Wave beam link. If the remaining bandwidth between the SC and i-th neighbor node is denoted by \(\beta \), then it can be given as:

$$\begin{aligned} \beta = \sum \limits _{k = 1}^{y^{i}} {w_k^{i} \times \tau } \end{aligned}$$
(4)

Noted that the specific method of channel allocation is not considered in this paper. Channel allocation is only used to explain the concept of mm-Wave link remaining bandwidth.

4 Remaining bandwidth based multi-path routing (RBMR)

In this section, the remaining bandwidth based multi-path routing (RBMR) is introduced detaily. RBMR is mainly composed of the exchange of remaining bandwidth information between neighbor nodes (given in the Sect. 4.1), the multi-path discovery that meets bandwidth requirements of source node (introduced in the Sect. 4.2), and the maintenance of the routing table (presented in Sect. 4.3). Various formats of the route packets in the proposed protocol and the physical meaning of each field of the packets are given in Sect. 4.4. The upper limit of the number of multi-paths is derived by calculating the average number of the neighbor nodes in Sect. 4.5.

4.1 Remaining bandwidth interaction of the nodes

The available bandwidth of each SC in the network changes from time to time. Therefore, periodic packets interactions between adjacent nodes are needed to obtain the available remaining bandwidth information of each other. The interactions of the remaining bandwidth information are implemented by periodic exchange of the route hello (RHEL) packets between adjacent nodes in RBMR. Note that the interaction period of RHEL has an impact on network performance. For example, when the timer of RHEL is set too long, it may cause problems that the updating the remaining bandwidth information between adjacent nodes is delayed, which will have a large impact on path discovery and route maintenance. The format of RHEL is shown in Fig. 5.

During the interaction of the remaining bandwidth information, the node first check whether the timer for RHEL, which is denoted by \({\textit{timer}}({\textit{rhel}})\), is times out. If the timer is times out, new RHELs are generated in sequence according to the neighbor node table, which is denoted by \({\mathcal{T}}\)\(= [t_{1 \times l}]_i\), i.e. \(t_i\) is the i-th entry of \({\mathcal{T}}\), in which the relevant informations of the i-th neighbor node are stored. It is noted that the main field values of the RHEL are also need to set. For example, \({\textit{RHEL.SA}} = {\textit{selfid}}\), \({\textit{RHEL.DA}} = t_{i}.{\textit{NbrID}}\), \({\textit{RHEL.BW}}\_{\textit{LEFT}} = t_{i}.{\textit{Bw}}\_{\textit{Left}}\), \({\textit{RHEL.LINKS}}\_\)\({\textit{BW}}\_{\textit{LEFT}} = t_{i}.{\textit{LinksBw}}\_{\textit{Left}}\). Then the RHELs are sent to neighbor nodes and \({\textit{timer}}({\textit{rhel}})\) is reseted.

When the RHEL is received by the neighbor node, the main field information are extracted and the corresponding node entry in the neighbor table is updated according to these information, i.e. \(t_{{\textit{RHEL.SA}}}.{\textit{Bw}}\_{\textit{Left}}\)\( = {\textit{RHEL.BW}}\_{\textit{LEFT}}, t_{{\textit{RHEL.SA}}}.{\textit{LinksBw}}\_{\textit{Left}} = {\textit{RH}}\)\({\textit{EL.LINKS}}\_{\textit{BW}}\_{\textit{LEFT}}\).

4.2 Multi-path routes discovery

For ease of description, the SC that initiating the routing request is called as source node. sink node is called as destination node, and the other SCs are all called as intermediate nodes. The number of the neighbor node from which the packets are received is denoted by inbr, and the number of the neighbor node to which the packets are sent is denoted by onbr. The main routing packets in RBMR are route request packet (RREQ, used for path discovery), route reply packet (RREP, used for path response), route delete packet (RDEL, used for path delete) and route data packet (RDAT, used for data transmission). The structure of all the packets is shown in Fig. 5. The physical meanings of the fields in various packets are detailed in the Sect. 4.4. For the convenience of description, the operation for various types of packets by the source node, sink node and other intermediate node is called as source node operation, destination node operation, and intermediate node operation respectively from the perspective that various packets are operated in the protocol. Now the specific flow of the three operations are described separately.

Fig. 2
figure 2

Flow chart of the source node operations

Full size image

4.2.1 Source node operations

There are three types of packets that can be processed by the source node, i.e. RREP, DATA and RDEL. The flow chart of source node operations is showed as Fig. 2.

When there are some traffic of the source node to be transmitted according to a certain backhaul bandwidth requirement, which is denoted by \(bw\_req\). The source node first checks whether there is any route of which the path bandwidth meets backhaul bandwidth requirement and reach to the destination node in the routing table. If there is, the route, which is denoted by \(r_p\), and \(r_p.Bw \ge bw\_req\), is selected from the routing table, which is denoted by \( {\mathcal{R}} = [r_{1 \times k}]_p\). Similar to the relationship between \(t_i\) and \({\mathcal{T}}\), \(r_p\) is the p-th entry of \({\mathcal{R}}\) and stores the relevant information of the p-th route. Then the corresponding ID number in the source route is added to the header of the data packet to form an RDAT for transmission. If there is not any available route of which the path bandwidth satisfies the backhaul bandwidth requirement in the \({\mathcal{R}}\), the data packets are stored in the cache first and \({\mathcal{R}}\) is waited for updating. Note that the stored data packets are discarded when the buffer timer times out. After the data is stored, RREQs are generated by the source node to start process of the route discovery. Then the set of the neighbor nodes in which the RREQ will be sent to the node is determined as follow:

$$ \begin{aligned} {\textit{SET}}_{{\textit{on}}}&= \left\{ onbr | \begin{array}{l} t_{onbr}.Bw\_{\textit{Left}}\ge bw\_req \& \\ t_{onbr}.{\textit{LinksBw}}\_{\textit{Left}}\ge bw\_req \end{array} \right\} \\&\quad + \left\{ onbr | \begin{array}{l} t_{onbr}.Bw\_{\textit{Left}} = 0 \& \\ t_{onbr}.{\textit{LinksBw}}\_{\textit{Left}}\ge bw\_req \end{array} \right\} \end{aligned}$$

Then RREQs are sent to the nodes \(onbr \in {\textit{SET}}_{tn}\) one by one. After that, the remaining bandwidth information of the entry corresponding to \(t_{onbr}\) in \({\mathcal{T}}\) is updated timely, i.e. \(t_{onbr}.{\textit{LinksBw}}\_{\textit{Left}} = t_{onbr}.{\textit{LinksBw}}\)\(\_{\textit{Left}} - bw\_req\). Then the timers for the RREQs, which is denoted by \({\textit{timer}}( {\textit{rreq.seqno}},onbr,{\textit{LinksBw}}\_{\textit{Left}},\)\(bw\_req,C_{rreq})\), is set respectively. What needs to be emphasized is that the bandwidth information is reset as \(t_{onbr}.{\textit{LinksBw}}\_{\textit{Left}} = t_{onbr}.{\textit{LinksBw}}\_{\textit{Left}}+ bw\_req\) when the timer exceeds its given duration, which is denoted by \(C_{rreq}\). Because it is probably means that the RREQ sent to the onbr is discarded.

If the packet that the source node received from inbr is RREP, the destination address of the RREP, i.e. \({\textit{RREP.DA}}\), is extracted first to determine \({\textit{RREP.DA}}\) equals to self-ID. Second, the path stored in the header of the RREP is added into the \({\mathcal{R}}\) and the corresponding timer is also found. If \({\textit{timer}}({\textit{rreq.seqno}},inbr,{\textit{LinksBw}}\_\)\({\textit{Left}},bw\_req,C_{rreq})\) has not timed out yet, it will be cancelled and the valid flag of \(r_{inbr}\) in \({\mathcal{R}}\) is set as \(r_{inbr}.\)\(valid = 1\), then data from the cache is sent out according to the paths. This means that the found path satisfies the requirements of the transmission time and bandwidth for the source node, i.e. the path is fully available for the data backhaul of the node. If the timer timed out and \(t_{inbr}.{\textit{LinksBw}}\_\)\({\textit{Left}} \prec bw\_req\), the valid flag of \(r_{inbr}\) in \({\mathcal{R}}\) is set as \(r_{inbr}.{\textit{valid}} = 0\). It means that although a route with the hop number larger than TTL is found for the source node, the required backhaul bandwidth of the source node is not satisfied by path bandwidth now. Therefore, the path is unavailable at this time. However, it is possible for the path to be validated by the routing table maintenance operation.

4.2.2 Intermediate node operation

There are three types of packets that may be handled by the intermediate nodes, i.e. RREQ, RREP, and RDEL. The flow chart of intermediate node operation is showed as Fig. 3.

Fig. 3
figure 3

Flow chart of intermediate node operation

Full size image

When the packet the intermediate node received from inbr is RREQ, the node first check whether there is a stored sequence number whose responding source address and destination address equal to the ones of RREQ. If it is and the sequence number of the RREQ is greater than the stored one, the following steps are used to processes the RREQ. Otherwise, the packet is directly discarded.

Step 1 Check the TTL of the RREQ, i.e. \({\textit{RREQ.TTL}}\). If \({\textit{RREQ.TTL}}= 0\), it means that the number of the packet forwarded times has reached the upper limit, then the RREQ is discarded;

Step 2Check the passed route node fields of the RREQ, i.e. \({\textit{RREQ.STAMP}}\). If self ID has been contained in the field, then the packet is discard, so that the routing loops is avoided.

Step 3 Check the routing table \({\mathcal{R}}\). If there is at least one route reached to the destination node in \({\mathcal{R}}\), which is denoted by \(r_p\), the routes are selected from \({\mathcal{R}}\) according to the following two guidelines. The first one is that the available path bandwidth of the routes is larger than the bandwidth of the RREQ, i.e. \(r_p.Bw > {\textit{RREQ.BW}}\_REQ\). The second one is that life time of the route is effective, i.e. \({r_p}.{\textit{valid}}= 1\) and \(r_p.{\textit{LeftTime}}\). Then all the number of the forwarding node in the selected routes are written to the ’stamp’ of the newly generated RREP, i.e. \({\textit{RREP.STAMP}}_n = r_p.{\textit{STAMP}}_n\), and the RREPs are sent back to the source node. After that, the remaining bandwidth information of corresponding entries in T are update, i.e.\(t_{inbr}.Bw\_{\textit{Left}} = t_{inbr}.Bw\_{\textit{Left}} - {\textit{RREQ.BW}}\_REQ\), \(t_{inbr}.{\textit{LinksBw}}\_{\textit{Left}} \)\(= t_{inbr}.{\textit{LinksBw}}\_{\textit{Left}} - {\textit{RREQ.BW}}\_REQ\). Otherwise, if there is not any route reached to the destination node at currently, then goes directly to the next step.

Step 4 Forward the RREQ. The set of neighbors that can forward RREQs is determined as follows:

$$ \begin{aligned} SET_{tn}&= \left\{ onbr | \begin{array}{l} {t_{onbr}}.Bw\_{\textit{Left}} \ge bw\_req \& \\ onbr \in A_n \end{array} \right\} \\&\quad - \left\{ onbr | \begin{array}{l} onbr={\textit{RREQ.STAMP}}_n \\ onbr \in A_n \end{array} \right\} \end{aligned}$$

If \({\textit{SET}}_{tn}\) is empty, the RREQ is discarded, it means that the remaining bandwidth of the links between it and all of the adjacent nodes is not large enough to meet the bandwidth requirement of the RREQ. If \({\textit{SET}}_{tn}\) is not empty, the remaining bandwidth of inbr is updated as \(t_{inbr}.{\textit{LinkBw}}\_{\textit{Left}} = t_{inbr}.{\textit{LinkBw}}\_{\textit{Left}} - {\textit{RREQ.BW}}\_REQ\), and corresponding \({\textit{timer}}({\textit{rreq.seqno}},\)\(inbr,{\textit{LinkBw}}\_{\textit{left}},{\textit{rreq.bw}}\_req, C_{rreq})\) is set. Note that the remaining bandwidth information will be reset, i.e. \(t_{inbr}.{\textit{LinkBw}}\_{\textit{Left}} = t_{inbr}.{\textit{LinkBw}}\_ Left\)\(+{\textit{rreq.bw}}\_req\) when this timer expires. After that, the remaining bandwidth of onbr is also updated as \(t_{onbr}.{\textit{LinkBw}}\_{\textit{Left}} = t_{onbr}.{\textit{LinkBw}}\_{\textit{Left}} - {\textit{RREQ.BW}}\_REQ\), and the \({\textit{timer}}\)\(({\textit{rreq.seqno}},onbr,{\textit{LinkBw}}\_{\textit{left}} ,{\textit{rreq.bw}}\_req,C_{rreq})\) is set. The remaining bandwidth information will be also reset as \(t_{onbr}.{\textit{LinkBw}}\_{\textit{Left}} = t_{onbr}.{\textit{LinkBw}}\_{\textit{Left}} + {\textit{rreq.bw}}\_\)req when this timer expires. Last but not least, the lifetime of the RREQ is updated as \({\textit{RREQ.TTL}} = {\textit{RREQ.TTL}} - 1\) and the RREQs are sent to the node in \({\textit{SET}}_{tn}\).

If the packet the intermediate node received from inbr is RREP, \({\textit{timer}}({\textit{rreq.seqno}},inbr,{\textit{LinksBw}}\_{\textit{Left}},\)\({\textit{rreq.bw}}\_req,C_{rreq})\) is found out and cancelled first. Second, \({\textit{timer}}(rreq.\)\(seqno,onbr,{\textit{LinksBw}}\_{\textit{Left}}, {\textit{rreq.bw}}\_\)\(req,C_{rreq})\) is found out and checked then. If the timer does not expire, this timer is canceled and the RREP packet is sent to the next-hop node according to the specified path that stored in the ’stamp’ domain of the RREP. If the timer times out, the path in the ’stamp’ domain of the RREP is added to R, and the remaining bandwidth information of inbr is updated as \(t_{inbr}.Bw\_{\textit{Left}} = t_{inbr}.Bw\_{\textit{Left}} - bw\_rep\). This means that a route from the node itself to the destination node is established, the bandwidth of which is \(rrep.bw\_rep\).

If the packet the intermediate node received from inbr is RDEL, it is sent to the next node obnr according to ’stamp’ field of the RDEL. Then the remaining bandwidth information between the intermediate node and the neighbors node, i.e. inbr and onbr, is updated as \(t_{inbr}.Bw\_{\textit{Left}} = t_{inbr}.Bw\_{\textit{Left}} + bw\_rde\) and \(t_{onbr}.Bw\_{\textit{Left}} = t_{onbr}.Bw\_{\textit{Left}} + bw\_rde\).

4.2.3 Destination node operation

There are three types of packets that can be processed by the destination node, i.e. RREQ, RDEL, and RDAT. The flow chart of destination node operation is shown in Fig. 4.

Fig. 4
figure 4

Flow chart of destination node operation

Full size image

If the packet the destination node received from inbr is RREQ, a new RREP packet is generated, and a reverse path is generated and is written into the ‘stamp‘ field of the RREP according to the ‘stamp‘ field of the RREQ. Then the RREP is sent out according to its ‘stamp‘.

If the packet the destination node received from inbr is RDEL, the corresponding backhaul path entry in \({\mathcal{R}}\) is deleted according to the field in the RDEL, i.e. \({\textit{RDEL.PATHNO}}\), and then the packet is destroyed.

If the packet the destination node received from inbr is RDAT, then the RDAT is destroyed. It means that the data is transmitted to the core network.

4.3 Maintenance of \({\mathcal {R}}\)

In addition to establishing the multi-path routes that meet the backhaul bandwidth requirements of the traffic through the above route discovery, the routes need to be maintained periodically. The route entry \(r_p\) in \({\mathcal{R}}\) where \({r_p}.{\textit{valid}} = 0\) and \(t_p.{\textit{LinkBw}}\_{\textit{Left}} > r_p.BW\) is found out. If the remaining lifetime of the route, i.e. \(r_p.{\textit{LeftTime}}\), is less than the threshold, which is denoted by \(time_{th}\), then make \(r_p.{\textit{valid}} \) equals to 1. This means the route is effectively available. Otherwise, the route entry, whose \(r_p.{\textit{LeftTime}} \prec time_{th}\) and \(r_p.{\textit{valid}} = 0\), is found out from \({\mathcal{R}}\). Under these circumstances, RDEL is generated and sent out in order to release the path.

4.4 Packet type

The packet types involved in RBMR are shown in Fig. 5, namely RHEL, RREQ, RREP, RDEL and RDATA. RHEL is used to update the remaining bandwidth information of the link between adjacent nodes in real time. RREQ and RREP are used for route discovery and route confirmation, and RDEL is used for path deletion during the maintenance of the routing table. The fields in various types of packets are explained as follows: \({\textit{TYPE}}\) indicates the packet type, and different values of the \({\textit{TYPE}}\) indicates different packet. For example when the value equals to 0, it means the packet is RREQ, and so on. \({\textit{SEQNO}}\) stands for the packet sequence number, and each RREQ has a unique identification number. SA indicates the source node address, i.e. the address of the node that generating the packet. DA stands for the destination node address, i.e. the address of the sink node. \(BW\_{\textit{LEFT}}\) indicates the total remaining bandwidth of all the backhaul paths in \({\mathcal{R}}\); \({\textit{LINKS}}\_BW\_{\textit{LEFT}}\) indicates the sum of the remaining bandwidth of beam links between itself and the neighbor node. \({\textit{TTL}}\) refers to the time to live of the packet, which is mainly used in RREQ. \({\textit{STAMP}}\_{\textit{NUM}}\) refers to the numbers of path nodes that the packet is sent to the destination node. The initial the value of the domain is set as 0 by the source node, and the maximum value of the domain is set as 16. \({\textit{STAMP}}_n\) is the address of the n-th intermediate node on the path that the packet has passed. \({\textit{PATHNO}}\) is the serial number of the path which is determined by the destination of the route.

Fig. 5
figure 5

Packets formation in RBMR

Full size image

4.5 Multipath analysis

Multi-paths establishment based on remaining bandwidth is an important parameter for RBMR. In fact, multi-paths which are discovered for the source node in RBMR are mainly relied on the forwarding of neighboring nodes. Therefore, the number of disjoint paths which are routed to the sink node from the source node will not be greater than the number of its neighbor nodes. In other words, the number of disjoint paths from the source node mainly depends critically on the number of neighbor nodes. Therefore, as long as the average number of neighbor nodes of the nodes in the network is analyzed, the average number of disjoint paths of the source node can be analyzed. Now, the analytical method of the average number of neighbor nodes for a node in the network is given as follow. For ease of illustration and analysis, it is assumed that there are N SC nodes are randomly deployed in a 2-D rectangular plane, which is denoted by A. The effective communication distance between adjacent SC, which is denoted as d, is 120 m.

From the distribution area of the nodes, the number of neighbor nodes for the nodes which is at the edge of the area is the smallest. For example, the number of neighboring nodes of a node that is exactly at an area boundary is half of the number of neighboring nodes that are at the central node of the network. Therefore, we divide the area distributed by nodes into several sub-areas for analysis. As shown in Fig. 6, the area distributed by the nodes is divided into three types of areas, namely \(A_1\), \(A_2\), and \(A_3\). Therefore, solve the average number of neighboring nodes per node in the network is equivalent to solve the expectation of the average number of neighboring nodes each node in each subarea, i.e.

$$\begin{aligned} G=\sum \limits _{i=1}^3 p_i \times G_i \end{aligned}$$
(5)

where the average number of neighbor nodes of each node in the sub-area \(A_i ({\hbox {i}}=1,2,3)\) is denoted by \(G_i\). \(p_i\) represents the ratio between the area of \(A_i\) and the area of A. The area of A is denoted by S, so \(S=X^2\), where X is the side length of A. The areas of \(A_1\), \(A_2\), and \(A_3\) is denoted by \(S_1\), \(S_2\), and \(S_3\) respectively. As can be seen from Fig. 6, \(S_1 = (X - 2d)^2\), \(S_2 = (X - 2d) \times d\) and \(S_3 = d^2\). Therefore, according to the definition of \(p_i\), we have \(p_1 = \frac{(X - 2d)^2}{X^2}\), \(p_2 = \frac{(X - 2d) \times d}{X^2}\), \(p_3 = \frac{d^2}{X^2}\).

Fig. 6
figure 6

Division of network area

Full size image

In addition, the area that A is intersected with the circle, whose center is (x, y) in \(A_i\) and radius is d, is denoted by \({a_i}(x,y)\). The average value of \({a_i}(x,y)\) in \(A_i\) is denoted by \(\overline{a_i(x,y)}\). Therefore,

$$\begin{aligned} \overline{a_i(x,y)}=\frac{1}{S_i}\iint _{A_i}a_i(x,y)ds \end{aligned}$$
(6)

It is assumed that \(\lambda = \frac{N}{{{X^2}}}\) is the average number of nodes per unit area. According to the physical definition of \(G_i\), \(\overline{a_i(x,y)}\) and \(\lambda \), we have:

$$\begin{aligned} {G_i} = \lambda \times \overline{a_i(x,y)} \end{aligned}$$
(7)

Put Eqs. (6) and (7) into Eq. (5) to get:

$$\begin{aligned} G&= \sum \limits _{i = 1}^3 p_i \times G_i \nonumber \\&= \sum \limits _{i = 1}^3 p_i \times \lambda \times \overline{a_i(x,y)} \nonumber \\&= \sum \limits _{i = 1}^3 p_i \times \lambda \times \frac{1}{S_i} \iint _{A_i}a_i(x,y)ds \end{aligned}$$
(8)

In addition, from the perspective of the node distribution area in Fig. 6, we have:

$$\begin{aligned} {a_1}(x,y)&= \pi {d^2} \end{aligned}$$
(9)
$$\begin{aligned} a_2(x,y)&= \frac{1}{2} \cdot \pi \cdot {d^2} + \int _0^y 2 \sqrt{d^2 - (y - t)^2} dt \nonumber \\&= \pi d^2 + y \sqrt{d^2 - y^2} - d^2 \arccos \frac{y}{d} \end{aligned}$$
(10)
$$\begin{aligned} a_3(x,y)&= \int _0^{\sqrt{d^2 - y^2} + x}( \sqrt{d^2 - (t - x)^2} + y)dt \nonumber \\&= \frac{1}{2} \pi d^2 + \frac{y}{2} \sqrt{d^2 - y^2} - \frac{1}{2} d^2 \arccos \frac{\sqrt{d^2-y^2}}{d} \nonumber \\&\quad + \frac{1}{2} x \sqrt{d^2-x^2} - \frac{1}{2} d^2 \arccos \frac{x}{d} \end{aligned}$$
(11)

The Eqs. (9), (10), and (11) are brought into Eq. (8) to obtain:

$$\begin{aligned} G&= p_1 \times \lambda \times \frac{1}{S_1} \iint _{A_1} \pi d^2 ds \nonumber \\&\quad + p_2 \times \lambda \times \frac{1}{S_2} \iint _{A_2}\left( \frac{\pi d^2}{2} + \int _0^y 2 \sqrt{(d^2 - {(y - t)}^2)}dt\right) ds \nonumber \\&\quad + p_3 \times \lambda \times \frac{1}{S_3} \iint _{A_3}\left( \int _{0}^{\sqrt{d^2-y^2}+x}(\sqrt{d^2-(t-x)^2}+y)dt\right) ds \nonumber \\&\mathop = \limits ^{(a)} \frac{p_1 \lambda }{{(X - 2d)}^2} \int _d^{X - d} \int _d^{X - d} \pi d^2 dxdy \nonumber \\&\quad + \frac{p_2 \lambda }{(X - 2d)d} \int _d^{X - d} \int _0^d\left( \frac{\pi d^2}{2} + \int _0^y 2\sqrt{d^2-(y-t)^2}dt\right) dxdy \nonumber \\&\quad + \frac{p_3 \lambda }{d^2} \int _0^d \int _0^d \left( \int _0^{\sqrt{d^2 - y^2} + x} (\sqrt{d^2 - (t - x)^2} + y)dt\right) dxdy \end{aligned}$$
(12)

where \(\mathop = \limits ^{(a)}\) stands for turning the double integral into quadratic integral. After simplification, the average number of neighbor nodes per node in the network can be expressed as:

$$\begin{aligned} G&= X^{-4}\left\{ N \left\{ d^3 \left\{ \arcsin (X-d/d)(X-d)-d\arcsin 1 \right. \right. \right. \nonumber \\&\quad \left. + \,d \sqrt{1-((X-d)^2/d^2)} \right\} - \frac{d\sqrt{2dX-X^2}}{3} \nonumber \\&\quad \left. \left. +\, \frac{d^3\pi (X-2d)}{2} \right\} \right\} + \frac{Nd^4\left( 3\pi + 11 \right) }{12X^4} + \frac{Nd^2\pi \left( X - 2d \right) ^2}{X^4} \end{aligned}$$
(13)

The comparison between the theoretical analysis results and the Monte Carlo simulation results is shown as Fig. 7. The number on the horizontal axis indicates the density of network nodes, i.e. \(\lambda \), the vertical axis represents the average number of neighbor nodes for the network node. As can be seen from the figure, when the effective communication distance and the deployment area of the SC are given, the number of the average neighbor nodes for the network node increases with the density of network nodes. However under the same condition, the theoretical analysis results basically coincide with the ones of Monte Carlo simulation, which verifies the theoretical analysis. It should be noted that the average number of multi-paths may be lower than the average number of neighbor nodes in actual network with the increases of the node density. This is because the RREQ will not be sent to the overloaded neighbor node by the intermediate node in multi-paths discovery.

Fig. 7
figure 7

Analysis for the average numbers of neighbor nodes

Full size image

5 Simulation and result

For the sake of estimating the performance of the proposed RBMR clearly, the NS2 is used to simulate RBMR and its three variations, i.e. RBSR, NBMR, and NBSR, and then the corresponding results are analyzed in this section. The main simulation scene, parameter settings and the performance metric are introduced in Sect. 5.1, the analysis of the simulation results are given in Sect. 5.2.

Table 1 Parameters and values used for simulation
Full size table

5.1 Simulation scene and parameter settings

The simulation area is set in a \(800 \times 800\) rectangular grid. The effective backhaul distance of adjacent SCs is set to 120 m as described in [31]. Note that the location problem of the sink node is not considered in this paper. The four sink nodes in the network, i.e., \({\textit{sink}}_1\), \({\textit{sink}}_2\), \({\textit{sink}}_3\), and \({\textit{sink}}_4\), are distributed at the four corners of the rectangular area. Since the interference between mm-Wave adjacent beam links can be neglected [6, 9, 19, 28, 32], the interference between adjacent millimeter wave links are not considered in this paper temporarily. According to the specifications of the 3GPP standard, the unit transmission time interval (UTI) on each mm-Wave beam link is 0.125 ms, there are 64 resource blocks (RB) in each UTI, i.e. the number of channels as described in Sect. 3. In order to facilitate the simulation, each beam link is full-band transmission, i.e. the number of channels per beam link is 1. The amount of data transmitted in the unit RB is 1460 byte, i.e. \(\tau =\) 1460 byte. In addition, as described in Sect. 4.1, the RHEL timer period may affect the update of bandwidth information, which in turn affects network performance. How to set the optimal value of RHEL timer is not in the consideration of our paper. The key parameters related to the RBMR are shown in Table 1.

The throughput of the sink nodes are denoted by as \(\gamma _i, i = 1,2,3,4 \) , the average throughput of the network is denoted by \(\gamma _a\), then it can be defined as:

$$\begin{aligned} \gamma _a = \frac{\gamma _1 + \gamma _2 + \gamma _3 + \gamma _4}{4} \end{aligned}$$
(14)

Data volume of RREQ, RREP, RHEL, RDEL and RDAT are denoted by \(\delta _{rreq}\), \(\delta _{rrep}\), \(\delta _{rhel}\), \(\delta _{rdel}\) and \(\delta _{rdat}\) respectively. The routing overhead in the network is denoted by \(\mu \), so it can be defined as:

$$\begin{aligned} {\mu } = \frac{{{\delta _{rreq}} + {\delta _{rrep}} + {\delta _{rdel}} + {\delta _{rhel}}}}{{{\delta _{rreq}} + {\delta _{rrep}} + {\delta _{rdel}} + {\delta _{rhel}} + {\delta _{rdat}}}} \end{aligned}$$
(15)

In addition, the network packet loss rate is explicitly defined as the ratio between the volume of the traffic that lost in the network and the volume of the traffic that generated by communication node. When the volume of the traffic received by the sink node is denoted by \(\nu _i, i = 1,2,3,4\) and the volume of the traffic generated by communication node is denoted by \(\nu _g\), the packet loss rate is denoted by \(\rho \), then it can be expressed as:

$$\begin{aligned} {\rho } = 1 - \frac{{{\nu _1} + {\nu _2} + {\nu _3} + {\nu _4}}}{{{\nu _g}}} \end{aligned}$$
(16)

5.2 Result analysis

For ease of description, the source nodes that generates backhaul traffic are called communication nodes. Here we have two different scenarios. The one is called Scenario 1, where the number of communication nodes is fixed but the backhaul traffic bandwidth of each communication node is change. The other one is called Scenario 2, where the backhaul traffic of each communication node is fixed but the number of communication nodes are changed. The throughput, routing overhead and packet loss rate of the four routing policies, i.e. RBMR, RBSR, NBMR and NBSR, are compared in each scenario. In order to verify the impact of the RHEL timer on network performance parameters as described in Sect. 3.1, we also compared the RBMR performance when RHEL timer period is set as 80 ms and 10 ms, which is called big period and small period respectively. What needs to be explained is that only the source path in the first RREP received is used as the route in the single-path routing strategies, i.e. NBSR and RBSR, when the route is established for the communication nodes. While the routing nodes in non-remaining bandwidth based routing strategies, i.e. NBSR and NBMR, perform the maximum capacity forwarding. When the link is saturated, the redundant packets are discarded.

5.2.1 Network performance analysis under scenario 1

In this scenario, the number of communication nodes that send data to the sink is set to 4, and the traffic bandwidth requirements of the four communication nodes increase from 0 to 1000 Mbps.

Fig. 8
figure 8

Network average throughput of scenario 1

Full size image

Figure 8 shows the relationship between the throughput of the four routing strategies and traffic bandwidth requirement of the node in scenario 1. As can be seen from the figure, the throughput of the four routing strategies keep increasing on the whole when the communication node’s traffic bandwidth continued to increase. The tending is stable when the throughput reaches saturation. When the communication traffic bandwidth of the communication nodes is small, there is no difference basically in the throughput of the four routing strategies. However, when the traffic bandwidth of the communication nodes is larger, the throughput of the RBSR is higher than that of the NBSR, which is up to 30% at maximum. The reason is that the traffic bandwidth of the communication nodes can be fully guaranteed by the route established on the remaining bandwidth of the link in RBSR. While some overloaded links may exists in the path established on non-remaining bandwidth of the link in NBSR, so the traffic bandwidth of the nodes may not be guaranteed. In addition, NBMR has a gain of 12% and 57% over NBSR and RBSR respectively in terms of throughput. Obviously, the throughput of RBMR remains the highest in all the four strategies due to the advantages of multi-path discovery and remaining bandwidth based path selection. In terms of the impact of RHEL timer period on RBMR, the throughput of small period is higher than that of big period, which is up to 7.6%.

Fig. 9
figure 9

Network routing overhead of scenario 1

Full size image

The relationship between the routing overhead of the four routing strategies and traffic bandwidth of the communication nodes in scenario 1 is shown in Fig. 9. It can be seen from the figure that the routing overhead of the four routing strategies generally shows a downward trend and tends to be stable when the communication traffic bandwidth of the communication nodes continues to increase. This is because the overall routing signaling overhead for a given topology is constant, and the routing overhead remains constant when the average throughput reaches saturation. When the communication node transmits a small amount of traffic, there are periodic neighbor information exchange overhead in both RBMR and RBSR. Therefore, the routing overhead of RBMR and RBSR are greater than that of NBMR and MBSR respectively. However, with the traffic bandwidth increased, the throughput of RBSR and RBMR increases rapidly and the difference of the routing overhead between the remaining bandwidth based paths selections strategies, i.e. RBSR and RBMR, and the non-remaining bandwidth based paths selections strategies, i.e. NBSR and NBMR, is significantly reduced. It is also shown that the remaining bandwidth based routing strategies is more suitable for the mW-SBN, which is characterized by high backhaul bandwidth and dynamic changed of traffic bandwidth. In terms of the impact of RHEL timer period on RBMR, the overhead under big period condition is higher than the one under small period condition.

Fig. 10
figure 10

Packet loss rate of scenario 1

Full size image

Figure 10 shows the relationship between the packet loss rate of the four routing strategies and traffic bandwidth requirement of the node in scenario 1. When the traffic bandwidth of the communication nodes continues to increase, the packet loss rate of all the four routing strategies generally increases. On the one hand, the packet loss rate in non-remaining bandwidth based routing strategies, i.e. NBSR and NBMR, is becoming more and more larger with the increase of the traffic bandwidth of the communication nodes. This is because the nodes in the route selected by the non-remaining bandwidth based routing strategies are delivered with the maximum capacity. When the transmission capacity of the intermediate node is insufficient to support the backhaul bandwidth requirements, the packet loss rate will increase. This problem is particularly prominent in NBSR. When the backhaul traffic bandwidth is small, especially when the total backhaul bandwidth of the communication node does not exceed the link capacity, the packet loss rate of remaining bandwidth based routing strategies, i.e. RBSR and RBMR, is low. However, when the backhaul traffic bandwidth is close to the link capacity, the packet loss rate of RBSR and RBMR increases dramatically. The reason is that link capacity of the established paths in RBSR and RBMR can not meet the backhaul traffic bandwidth of some communication nodes. On the other hand, as there are multiple available paths for traffic backhaul, the packet loss rate of the multi-path routing strategies, i.e. NBMR and RBMR, is a lower that of the single-path routing strategies, i.e. NBSR and RBSR. In terms of the impact of RHEL timer period on RBMR, the packet loss rate under big period condition is lower than the one under small period condition, which is up to 9.3%.

5.2.2 Network performance analysis under scenario 2

In this scenario, the traffic rate of each communication node is set at 100 Mbps, and the number of communication nodes in the network is increased from 2 to 32.

Fig. 11
figure 11

Network average throughput of scenario 2

Full size image

The relationship between the throughput of four types of routing strategies and the number of communication nodes in scenario 2 is shown in Fig. 11. It can be seen that the network throughput increases with the number of pairs of communication nodes increases,and the tends to be stable when the throughput reaches saturation. On the one hand, multi-path routes is selected for traffic backhaul in NBMR and RBMR. Therefore, the throughput of NBMR and RBMR are higher than NBSR and RBSR about 28–34% when the number of communication nodes are the same. On the other hand, the bandwidth requirement of the backhaul traffic are fully guaranteed routes selection of RBSR and RBMR, so the throughput of RBSR and RBMR are higher than that of NBSR and NBMR nearly 13–18% when the number of the communication nodes are equal. Obviously, RBMR always outperforms the other three strategies in terms of average network throughput. It also can be seen that the throughput gap between the big period condition and the small period condition is small in terms of the impact of RHEL timer period on RBMR.

Fig. 12
figure 12

Network routing overhead of scenario 2

Full size image

Figure 12 shows the relationship between the routing overhead and the number of communication nodes of the four routing strategies in scenario 2. It can be seen from the figure that the network overhead of the four routing strategies generally shows a downward trend with the number of communication nodes continuing to increase. When the number of communication nodes is small, the amount of traffic that needs to be backhauled in the network is also relatively small. Periodic bandwidth information interactions between adjacent nodes are required to establish routes that meets the requirements of the backhaul traffic. So compared to the NBSR and NBMR, a certain amount of overhead are generated in RBSR and RBMR when the number of communication nodes is small. With the increase of the number of communication nodes, although there is a slight increase in route signaling, the effective amount of data backhaul to the Sink node of all the four routing strategies will be greatly increased, and therefore the network overhead is reduced accordingly. When network backhaul traffic tends to be stable, network overhead tends to be stable. In addition, given the system running time, due to the large number of RHEL generated under small period condition, the network overhead under small period condition is higher than that under large period condition, which can also be clearly seen in Fig. 12.

Fig. 13
figure 13

Packet loss rate of scenario 2

Full size image

The relationship between the packet loss rate and the number of communication nodes for the four routing strategies in scenario 2 is shown in Fig. 13. It can be seen from the figure that the packet loss rate of the four routing strategies shows an upward trend with the increase of the number of communication nodes. However, the packet loss ratio of multi-path routes is lower than that of single-path route approximately 47–55%. This is because a path with a large link load is easy to be selected in single-path routing due to the increase of the number of communication nodes. So the traffic is unable to be transmitted on demand, the packet loss rate is also increased. In addition, the packet loss rate of remaining bandwidth based routing strategies is reduced by 12–16% compared to that of the non-remaining bandwidth based routing strategies. This is because that the route establishment of remaining bandwidth based routing strategies completely guarantees the bandwidth requirement of the traffic. While in the route discovery process of the non-remaining bandwidth based routing policy, each intermediate do the traffic forwarding as best as possible. It is easy to make the node with a large load to be selected as an intermediate forwarding node. Leading to the result that packet loss rate is increased during traffic backhaul process. It also can be seen that the packet loss rate under big period condition is lower than the one under small period condition in terms of the impact of RHEL timer period on RBMR, which is up to 36%.

6 Conclusion and future work

In this paper, a remaining bandwidth based multi-path routing (RBMR) protocol is designed to realize the distributed backhaul for the mW-SBN. The protocol is mainly composed of the interaction of remaining bandwidth information between neighboring nodes, the multi-paths discovery and establishment and the routes maintenance and removal. According to the RBMR, multi-path routes that meet the data transmission bandwidth requirements can be established and maintained in real time. The RBMR is simulated with NS2. Simulation results show that RBMR has obvious advantages in network average throughput, route overhead , and packet loss rate when compared with NBSR, RBSR, and NBMR. RBMR is so much suitable for the mW-SBN with a large amount of traffic and dynamic changes in bandwidth requirements.

The work that needs to be further carried out in the future mainly includes two aspects. on the one hand, the concept of the remaining bandwidth is considered on the determined radio resources. However, the method of resource determination, i.e. the channel allocation method which is always belonged to the MAC layer, has not been considered in this paper. This is exactly what we need to further study to form a system and complete cross-layer communication protocol for the mW-SBN. On the other hand, the acquisition of the remaining bandwidth information depends on RHEL. However, the optimal value setting of RHEL should be dynamically adaptive, i.e. it forms a closed-loop feedback system with factors such as the specific scene and the change of the channel condition in the network. Therefore, the other future work of this paper is forming a mechanism of the dynamic adaptive setting for RHEL optimal value.