Keywords

1 Introduction

Wireless communication, specifically wireless sensor, and actuator networks (WSANs) will play a key role in the future of Internet of Things (IoT), Industrial Internet of Things (IIoT), Industry 4.0. In a WSAN many sensors and actuators are embedded in the same physical environment to measure different parameters and actuate on it according to some feedback control strategy [1]. In the case of vehicular sensor-actuator network-based control in railway systems, physical parameters such as temperature of brake disks, temperature and humidity inside the vehicle, the temperature of heating pipes, speed, direction, and location, are measured by sensor devices and a recent trend is to have them sent to a controller over wireless links.

When using WSANs, train vehicular applications pose additional requirements such as generality, reliability, fault tolerance, predictability, and security. They need to support generality because they will be used in different countries regarding different traffic rules and even different legal frequency bands [2]. Reliability is naturally an extremely important factor. These systems need a reliable and low latency wireless communication network, for example to exchange physical parameters like location and speed in a direct communication between vehicles without third party involvement [2]. Fault tolerance is another important aspect that needs to be accounted to increase systems reliability and safety. Predictability is also fundamental to achieve safety and have the vehicles control performing adequately. Concerning the wireless communication, it means avoiding or preventing channel fading, channel interference, channel congestion, channel jamming and channel assignment [3].

In one of the latest reports released by Roll2Rail project [4], many standards such as IEEE 802.15.4 and protocols like, ZigBee, Industrial WLAN, WirelessHART and Ultra Wide Band (UWB) were analyzed to support Vehicle-to-vehicle (V2V) communication. The IEEE 802.15.4 standard, which defines the Physical and Data link layers of several protocols, has several disadvantages such as low reliability, unbounded packet delays, no protection against interference and jamming [1, 5, 6] and thus it is not a good option for a direct V2V wireless communication system. However, recently, IEEE has released the 802.15.4e amendment that extends the original IEEE 802.15.4 standard to better support the emerging needs of embedded real-time applications and improves reliability and latency by Time-Slotted Channel-Hopping (TSCH).

In the concept of TSCH networks, successful-hopping is defined as a hopping strategy that improves communication performance by changing channel every slot according to a predefined sequence. However, it is also possible to disable hopping and remain in the same channel. This can be better if the current channel is in a good state. The work in [7] highlights the effectiveness of channel hopping. But, the authors of [8] observed that random channel hopping communication for unstable zone can be counter-productive. To the best of our knowledge the problem of deciding whether to do hopping or not, has not received enough attention. This is our contribution in this paper, in which we propose a simple multi-agent based way of deciding whether to do hopping or not using a game theoretic approach, namely the Nash equilibrium.

The reminder of this paper is organized as follows. Section 2, is an overview of IEEE 802.15.4 and new amendment. In Sect. 3, we present previous works. Section 4 discusses Virtual Coupling and the reason of using IEEE 802.15.4e standard. In Sect. 5, we present our multi-agent model. In Sect. 6, we define some metrics to evaluate channel performance. Section 7 discusses Nash equilibrium and our game-theoretic description of successful hopping. In Sect. 8, we propose a solution based on Multi-agents and zero-sum game model. Finally, we conclude in Sect. 9.

2 The IEEE 802.15.4 and IEEE 802.15.4e

The IEEE 802.15.4 standard is a standard for low-rate, low-power, and low-cost Personal Area Networks (PANs). This standard, defines the Physical and Data link layers of several protocols but, it has several disadvantages such as low reliability, unbounded packet delays, no protection against interference and jamming [4, 10, 11]. Thus, it is not a good option for a direct V2V wireless communication. Recently, IEEE released the 802.15.4e amendment that extends the original standard to better support the emerging needs of embedded real-time applications and improves reliability and latency. In the IEEE 802.15.4e amendment, there are several Medium Access Layer (MAC) behavior models to support industrial automation application. In this work, our focus will be on Time Slotted Channel Hopping (TSCH) feature. TSCH is introduced to improve packet reliability and increase the probability of the joining time in the wireless communication networks. TSCH combines time slotted access with multi-channel and channel hopping capabilities. In the TSCH network, each node communicates with another node through a link. This link will reserve a time slot and channel for the supposed communication and will translated into the physical channels, by the Eq. 1.

$$ Channel = HSL\left[ {\left( {ASN + Channel_{offset} } \right)\,\bmod \,\left| {HSL} \right|} \right] $$
(1)

where ASN is the Absolute Sequence Number and HSL is the Hopping Sequence List.

3 Previous Works

In the work of [9] a Model-based Beacon Scheduling (MBS) algorithm has been proposed to autonomously select the links to use for advertising Enhanced Beacons (EBs) and, minimizes the average Joining Time by provided optimal EB schedule through MBS in TSCH networks. For that, they used a Discrete Time Markov Chain to provide a methodology to calculate the average joining time as it occurs in a different state with a set of discrete transitions which probability of the next transition is dependent on the previous transition. Then, the authors have concluded the MBS algorithm to provide an optimum EB by defining a minimization problem based on Mixed-Integer Non-Liner Programs class over the average joining time.

The authors of [10] proposed a solution that divides TSCH’s Slotframe into two parts: the advertisement plane and communication plane. This separation helps to reduces effects of flexible scheduling Enhanced Beacon (EB) broadcast on the overall operation of TSCH network and speed up joining time. In [11] the Authors designed two different EB scheduling algorithms to speed up the joining phase in an IEEE 802.15.4e network: Random Vertical filling (RV) and Random Horizontal filling (RH). In RV the coordinator transmits EBs on a random channel offset and at the first slot of the multi-Superframe structure. In RH, the coordinator transmits EBs at a random slot of the multi-Superframe structure with Channel offset = 0. The performance of both solutions is the same. The authors of [12] used fuzzy logic and the idea of dividing slotframe into two parts which were earlier discussed in [10] to speed up joining time and reduce energy consumption. In [6] the authors show how Routing Protocol for Low-Power (RPL) and Medium Access Control (MAC) they affect negatively over network performance by creating inefficient redundant paths. It happens because RPL always creates a routing topology without a priori knowledge about the topology that was created by MAC layer.

The authors of [13] introduced a heuristic blocking solution to exclude poor channels from the hopping list by creating a blacklist of very noisily channels and they called this solution A-TSCH. In [14] works they try to use a Bayesian frequency hopping game model based on Nash equilibrium to improve wireless sensor network’s resistant against of attack. In this paper, the authors have considered that there is an intrusion detection procedure which detects the attack and triggers channel hopping request. There are several other works that have been done in this area by using the concept of game theory such as [15,16,17,18].

4 Virtual Coupling and IEEE 802.15.4e Standard

A fundamental principle of railway signaling has always been that following trains must be separated by a sufficient margin to ensure each train is capable of braking to a stop before reaching the last known position of the train in front. A wireless communication link between the trains could ensure that if the leading train starts to break, the following train will do the same and maintain separation as the two trains slow together. This concept is referred to as Virtual Coupling or Virtually Coupled Train Sets (VCTS) and can provide maximum use of limit capacity in railway systems [19]. This includes achieving a more competitive and resource-efficient European transport system with a view to addressing major societal and technical issues such as rising traffic demand, congestion, safety, flexibility of timetables, no time-intensive coupling processes, reduction of aerodynamics through coupled wagons and increased capacity, also it provides more flexibility in train operations namely in depot. The concept of VCTS is based on the idea of using modern electronics and data transmission to run several self-propelled units one behind the other without physical contact but at distances as short as mechanically coupled trains. The trains could automatically join or leave when they reach a junction. See Fig. 1.

Fig. 1.
figure 1

Virtually coupling distance between 3 trains

Full size image

New technologies, such as radio systems, satellites, signaling and real-time communications and currently amiable systems like the European Railway Traffic Management System (ERTMS), European Transport Control System (ETCS) enables the application of train coupling and sharing concepts with dynamic joining and splitting of module trains in motion to so-called Virtually Coupled Train-Sets. Having a VCTS needs highly reliable and fast, real-time wireless V2V direct communication. systems. The IEEE 8028.15.4e standard can be a good candidate for this need [4].

5 Multi-agent Model

VCTS by nature needs dynamic and collaborative communication environment. Each train should be able to predict the best time to hop or maximize reliability, safety and, security of communication by not-hopping and using the current channel. By this explanation reliability, safety and, security of communication should be considered as a crucial desire for each train. Multi-Agent Systems (MASs) are well prepared to cope in a dynamic environment with intelligent entities. In our model, each agent must be able to evaluate communication channels performances and decide to use hopping technic or not, which hereafter called “hop” or “not-hop” actions. To have better understand, let denote Alice, Bob, and Trudy as agents with a common environment. This environment is a V2V direct communication through IEEE 802.15.4e based protocols. Physical parameters are speed and location of each train. While Alice and Bob are trying to maximize their communication performance, Trudy is trying to maximize his chance to jam their communication channel. There are different models and different solutions to design a zero-sum game for this scenario. But, what makes our work different than similar works [14, 17, 18] is the way we look at the problem. We will relax Trudy from the targeted model and we will focus on Alice and Bob. By this assumption, our model will have just two agents. Trudy is an agent outside of our system which will have a negative influence on communication performance. Alice and Bob are two trains under VCTS schema. The Fig. 3 shows a sequence diagram of Alice and Bob`s communication. Alice and Bob can exchange some messages in a formal language like the Foundation for Intelligent Physical Agents-Agent Communication Language (FIPA-ACL) to update their latest situation. This communication will help them to change their current behaviors and take future decisions in a cooperative fashion to get into a Nash equilibrium with a high probability. Using MAS and game theory to approach TSCH problem with a high probability of best hop or not-hop decision is discussing for the first time in this work (Fig. 2).

Fig. 2.
figure 2

The sequence of communication between Alice and Bob.

Full size image

6 Metrics

Based on [6, 8, 20] we will define channel performance ratio by using Expected Transmission Count (ETX). The ETX is a greedy approach to estimate the number of required transmissions needed before the neighbor (Alice/Bob) correctly receives data. Packet Delivery Ratio (PDR) is a ratio between the number of acknowledgments received and packets sent. Note that, λ is an upper-bound for ETX and can be tuned in a supervised or unsupervised way.

$$ ETX = Min\left( {\frac{1}{PDR},\lambda } \right)\,\,\left| {\,PDR = {{N_{ack} } \mathord{\left/ {\vphantom {{N_{ack} } {N_{sent} }}} \right. \kern-0pt} {N_{sent} }}} \right. $$
(2)

Where N ack and N sent are, respectively, the number of acknowledgments received and the number of packets sent. The studies in literature review approve that this estimation can provide strong evaluation about the performance of channel in wireless sensor networks (Table 1).

Table 1. Symbols used in the game model
Full size table

7 Nash Equilibrium

In game theory, a non-cooperative game is a game in which there is no collaboration or communication between players and each player acts independently. In 1950 John Nash proved that a finite non-cooperative game always has at least one equilibrium point [21]. Nash Equilibrium (NE) is a solution concept for predicting how a non-cooperative game should be played. To have a better understand, consider an n-person game with a finite set of strategies, denoted by S, for each player (person) and, corresponding to each player, i, a payoff function, P i , which maps S i into a real number.

$$ \left( A \right)\,\forall i,\,s_{i} \in S:p_{i} \left( {s_{i}^{*} ,s_{ - i}^{*} } \right) \ge p_{i} \left( {s_{i} ,s_{ - i}^{*} } \right)\,\,\,\,\,\,\,\,\,\left( B \right)\,\forall i,\,s_{i} \in S:p_{i} \left( {s_{i}^{*} ,s_{ - i}^{*} } \right) > p_{i} \left( {s_{i} ,s_{ - i}^{*} } \right) $$
(3)

When each player chooses a single or mixed strategy he obtains payoff. Equation (3) shows strict and possible Nash equilibrium. In our proposed game model, we allow two players (Alice and Bob) to have a communication just to build a rapid feedback against of channel unreliability. In fact, Alice and Bob will achieve an NE only when they settled in the most optimum channel, denoted by ch *. That means they will always do hopping if the probability of successful-hopping is higher than current state. value is higher than staying in the current channel with related ETX value. The Eq. 4 is showing this condition.

$$ \exists \,ch \in CH\left| {{}_{h}P^{{ETX^{ \uparrow } }} >_{nh} P^{{ETX^{ \downarrow } }} \vee_{h} P^{{ETX^{ \uparrow } }} <_{nh} P^{{ETX^{ \downarrow } }} } \right. $$
(4)

Now, let`s assume that the hopping strategy will be based on a randomized method similar to RV or RH as proposed by [11].

$$ \left\langle {\begin{array}{*{20}c} {\alpha = {}_{h}P^{{ETX^{{}} }} ,} & {\beta = {}_{nh}P^{{ETX^{{}} }} ,} & {\gamma = \frac{1}{{\left| {CH} \right| - \left| {UC} \right|}} < 1} \\ \end{array} } \right\rangle $$
(5)
$$ {}_{h}P^{{ETX^{{}} }} = {1 \mathord{\left/ {\vphantom {1 {\left( {\frac{{\sum\limits_{i \in CH} {ETX_{i} } }}{{\left| {CH} \right|}}} \right)}}} \right. \kern-0pt} {\left( {\frac{{\sum\limits_{i \in CH} {ETX_{i} } }}{{\left| {CH} \right|}}} \right)}} = \frac{{\left| {CH} \right|}}{{\sum\limits_{i \in CH} {ETX_{i} } }}\,\,\,\,\,\,\,,\,\,\,\,{}_{nh}P^{{ETX^{{}} }} = Max\left( {PDR_{i} ,\varepsilon_{i} } \right) $$
(6)

Based on the above equations we can define a payoff matrix for hopping or not-hopping as shown in Table 2. In another word, α is a ratio between 1 and average of all channel’s ETX. In each time-slot α is calculated by each player to accounted for future decisions. To have a better understanding about α let’s assume the value of ETX for each channel is equal to 1. Then, the probability of settling in a better channel or improving the current PDR by hopping to another channel is equal to α and, α is equal to 1. While the probability of settling in a better channel by using just a random mechanism will be equal to γ. Note that, α is a raw estimation of channel communication improvement and it does not guarantee the probability of assigning an optimum channel. The Eq. 6 shows that the probability of settling in ch* (optimum channel/channel with better performance) by hopping under α condition is higher than γ or β conditions if and only if α is bigger than β and in the worst-case scenario, α is equal to γ.

Table 2. Payoff matrix for hopping or not hopping
Full size table

8 Our Model

Our proposed solution relies on a MAS environment and creates a zero-sum game to have a good estimation wheatear hop or not-hop. Initially, the IEEE 802.15.4e standard introduce 16 channels for TSCH without any scheduling or hopping strategy. But, to keep our model simple, we assume hopping between 1 and 6 channels. Let denote Alice, Bob and Trudy respectively train A, train B and noisy mode, attacker, jammer, or any other entity that can have a negative influence on communication channel performance. Figure 3 is presenting α for each channel between time-slot t i and t i+n .

Fig. 3.
figure 3

Alice, Bob, and Trudy are accessing to channels randomly, the value in each channel is presenting the related α

Full size image

In the immediate section, we are calculating related probabilities for the first two time-slots (t i and t i+1 ) by assuming ε = 0.50, γ < 1 and λ = 10. Note that, channel number 2 is already occupied by Alice and Bob with PDR = 2/4 = 0.50.

$$ \gamma = \frac{1}{3}\,\,and\,\,\alpha_{{t_{i} }} = {1 \mathord{\left/ {\vphantom {1 {\left( {\frac{{\sum\limits_{i \in CH} {ETX_{i} } }}{{\left| {CH} \right|}}} \right)}}} \right. \kern-0pt} {\left( {\frac{{\sum\limits_{i \in CH} {ETX_{i} } }}{{\left| {CH} \right|}}} \right)}} = {1 \mathord{\left/ {\vphantom {1 {\frac{29}{6}}}} \right. \kern-0pt} {\frac{29}{6}}} = \frac{6}{29} \approx 0.21 \Rightarrow P_{\alpha }^{{ch^{*} }} < P_{\beta }^{{ch^{*} }} $$
(7)
$$ \gamma = \frac{1}{5}\,\,and\,\,\alpha_{{t_{i + 1} }} = {1 \mathord{\left/ {\vphantom {1 {\left( {\frac{{\sum\limits_{i \in CH} {ETX_{i} } }}{{\left| {CH} \right|}}} \right)}}} \right. \kern-0pt} {\left( {\frac{{\sum\limits_{i \in CH} {ETX_{i} } }}{{\left| {CH} \right|}}} \right)}} = {1 \mathord{\left/ {\vphantom {1 {\frac{7}{6}}}} \right. \kern-0pt} {\frac{7}{6}}} = \frac{6}{7} \approx 0.86 \Rightarrow P_{\alpha }^{{ch^{*} }} > P_{\beta }^{{ch^{*} }} $$
(8)

Based on the Eqs. 8 and Eq. 9 the following payoff matrices are showing two different strategies for Alice and Bob over different timeslots (Tables 3 and 4).

Table 3. Payoff matrix for timeslot t i
Full size table
Table 4. Payoff matrix for timeslot t i+1
Full size table

As we conclude from the Eqs. 8 and 9 at the time-slot t i , the maximum value of β cannot exceed 0.50 so, channel hopping will not be a good strategy for Alice and Bob. But, if we repeat the same calculation for the next time-slot (t i+1 ) then, channel hopping will be the best strategy for Alice and Bob and will keep them under NE.

Our proposed solution shows that if we keep ETX for each channel up to date then we can come up with a nice approximation based on zero-sum game to estimate probability of successful-hopping and settling in a better communication channel while the other works that introduced in the literature review are not capable of providing any decision-making capacity for hopping or not hopping.

9 Conclusion

In this paper, we have investigated strategies that attain cooperation in Time-Slotted Channel-Hopping without coordination or offline scheduling. We have developed an analytical model based on Multi-agents and game theory. Relying on this model, we have formulated a zero-sum game for successful channel hopping. We have showed this game has always a Nash equilibrium.

Finally, we have defined two basic and simple probabilities to estimate probability hopping or not-hopping decision-making process. In particular, we show how Nash equilibrium and agent based system can be used to build a predictable environment to improve random channel hopping communication to not be counter-productive for unstable zone.