RETRACTED ARTICLE: CMDP-based intelligent transmission for wireless body area network in remote health monitoring

Abstract

Remote health monitoring is one kind of E-health service, which transfer the users’ physiological data to the medical data center for analysis or diagnosis. Wireless body area network (WBAN) is a promising technology to achieve physiological information acquiring and delivering and thus has been widely adopted in remote health-monitoring applications. For WBAN, energy consumption is the major concern which has been addressed in many researches. Different from existing works, this work studies a joint scheduling and admission control problem with objective of optimizing the energy efficiency of both intra- and beyond-WBAN link. The problem is formulated as constrained Markov decision processes, and the relative value iteration and Lagrange multiplier approach are used to derive the optimal intelligent algorithm. Simulation results show the proposed algorithm is capable of, in comparison with greedy scheme, achieving nearly 100% throughput improvement in various power consumption budgets. Moreover, the proposed algorithm can achieve up to 5.5× power consumption saving for sensor node in comparison with other scheduling algorithms.

1 Introduction

Thanks to the profound progresses in biomedical signal sensing [1], information processing [2], and wireless communication technology [3], the healthcare paradigm is currently experiencing a marked transition. In the classic healthcare system, patients need to visit hospitals or clinics to receive medical services. With the help of E-health (Electronic Health) [4, 5], patients can access to the healthcare services at any place and any time, which greatly improves the patients’ quality of live and also reduces the cost on healthcare [6]. Remote health monitoring is one kind of E-health service in which the users’ physiological data (e.g., blood pressure, electrocardiogram, electromyogram, electroencephalogram, blood glucose oxygen levels and motion data) are acquired in a real-time manner and forwarded to medical data center for analysis, diagnosis or monitoring (by doctor or artificial intelligent algorithm) and storage. Remote health monitoring can benefit in early diagnosis of chronic diseases, abnormality detection. Meanwhile, large amount of collected medical data can significantly facilitate the development of artificial-intelligence-based diagnosis, analysis, prediction, and treatment plan providing algorithms.

To enable remote health monitoring, wireless body area network (WBAN) is widely adopted as it is a promising technology to achieve physiological information acquiring and delivering [7, 8]. Typically, a WBAN consists of a gateway node and several sensor nodes. The sensor nodes are capable of continuously sensing the physiological signal and sending data to the gateway node using short-range communication technology. At the same time, the gateway node connects to the Wi-Fi or cellular network to forward the collected data to a medical center [9,10,11]. Consider Wi-Fi network suffers from limited coverage area, so it cannot ensure the ubiquitous e-health services. In this paper, we only focus on cellular network. Figure 1 shows an example of WBAN-based solution architecture for remote health monitoring.

Fig. 1

Illustration of a WBAN-based solution architecture for remote healthcare monitoring

WBAN is typically required to operate for a long period of time for healthcare monitoring, and frequent interruptions of power supply can expose the user in unfavorable or even life-and-death situation. Thus the energy consumption is the major concern for WBAN. Among all the operations, wireless communication is the major source of energy consumption. Therefore, it is vital important to design an energy efficiency transmission policy. Since the battery capacity of sensor node is usually small due to limited node size, most works for WBAN energy efficiency design focus on only intra-WBAN communication (the transmission between the sensor node and the gateway) with the assumption of a resource-rich gateway. Compared to sensor node, gateway has larger battery capacity, but it needs to carry out many heavy tasks (long-range wireless communications, display, computation and so on), so energy consumption also pose a challenge for gateway node. Thus, in this paper, we study a joint scheduling and admission control problem and aims to optimize the energy efficiency at both intra- and beyond-WBAN link. In beyond-WBAN link, we improve the throughput with constraint of average power consumption budget which is decided by many factors such as battery capacity, planned working duration or residual battery situation. Meanwhile in intra-WBAN link, we focus on reducing the power consumption of the sensor node.

The main contribution in this paper is introduced as following. We propose a WBAN-based intelligent transmission algorithm to jointly optimize the energy efficiency of intra- and beyond-WBAN wireless link. In particular, an adaptive modulation scheme is applied to the gateway node to schedule the beyond-WBAN communication, and thus the throughput–power consumption trade-off on the gateway node can be optimized. In addition, a traffic admission control is used to decide the amount of generated data packets at sensor node allowed to be transmitted to the gateway node. Thus, energy consumption of wireless transmissions for sensor node can be considerably reduced. In this algorithm, the scheduling and admission control actions should be intelligently decided depending on the system states, thus an optimal policy should be derived. To this point, the joint optimization problem is formulated as a constrained Markov decision processes (CMDP) and is solved by using the relative value iteration algorithm. CMDP is a powerful decision-making tool to optimize a target system value by defining and analyzing the system state space, action space, reward model, and system transition probability distribution. To evaluate the performance of the proposed algorithm, extensive simulations are conducted, and the results show that the proposed algorithm, in comparison with greedy scheme, can achieve nearly 100% throughput improvement in various power consumption budget. In addition, in comparison with other scheduling algorithm, the proposed algorithm can achieve up to 5.5× power consumption saving for WBAN sensor node.

The remainder of the paper is organized as follows. Section 2 provides an overview of related works. Section 3 gives the system description. Section 4 focuses on the problem formulation and Lagrangian multiplier approach. Section 5 provides the simulation results and performance comparison. Finally, Sect. 6 offers concluding remarks and suggestions for future work.

2 Related work

2.1 Remote healthcare monitoring using WBAN

With the increasing need for ubiquitous e-health, WBAN has been widely used in healthcare-monitoring applications. Abawajy et al. [12] propose a pervasive patient health-monitoring (PPHM) system infrastructure. PPHM adopts cloud computing and Internet-of-Things technologies to enable a flexible, scalable, and energy-efficient system for remote healthcare-monitoring applications. A case study for real-time monitoring of a patient suffering from congestive heart failure using ECG demonstrates the effectives of the proposed framework. Ghanavati et al. [13] propose a cloud-based WBAN framework for real-time health monitoring. The proposed framework use cloud technology to facilitate the management and analysis of the WBAN data in huge quantity. In this framework, the physiological data acquired by WBAN sensors are transmitted to a mobile phone to receive initial process, and then forwarded to Cloud for further analysis and management. An EMG remote monitoring application is presented as case study for the proposed framework. A similar study in [4] proposes a framework for outpatients’ chronic diseases monitoring using WBAN and cloud technology. In this framework, the biomedical reading captured by WBAN sensors is fed to a mobile application to receive initial analysis. The data are further forwarded to a cloud to allow easy access by physicians. A case study is presented to show the effectiveness of the proposed framework. Hussein et al. [14] propose cloud-based health-monitoring system for analyzing the HRV (heart rate variability) data. This system uses WBAN devices to acquire ECG signal and forward the collected signal to a cloud for HRV analysis. With the help of this system, the people living in the remote areas can receive the best healthcare-monitoring services.

2.2 Energy-efficient transmission for WBAN

Highly energy-efficient transmission is important for WBAN, and thus it has attracted a lot of attentions from academic community. Nia et al. [15] address the challenge of restrict requirement for energy consumption in WBAN-based long-term continuous health monitoring. To enable energy efficiency, authors propose schemes for sample aggregation, anomaly-driven transmission, and compressive sensing to reduce the wireless transmission time. Analytical results show that significant energy saving is achieved. Zang et al. [16] proposed a gait-cycle-driven transmit power control scheme (G-TPC) for WBAN. The G-TPC exploits the periodic channel fluctuation in the walking scenario using accelerometer reading to arrange transmission at ideal channel condition in each gait cycle. The transmission power is adjusted according to the reading of received signal strength indication (RSSI). Experiment demonstrates 25% energy saving is achieved compared to traditional transmission power control schemes. Su et al. [17] propose a battery-aware time-division multiple access (TDMA) protocol for wireless body area monitoring network. This protocol takes use of battery recovery effect to maximize the lifespan of the network node. The works in [18, 19] introduce an energy-efficient WBAN MAC layer protocols which focus on reducing idle listening, overhearing, unnecessary beacon transmissions, and collisions. Argyriou et al. [20] proposes a new WBAN architecture that use capacitive body-coupled communication to relay the data from the sensor node whose wireless link is blocked due to body shadowing. However, these above-mentioned works as well as most other researches in this field limit their focus on intra-WBAN communication. To my best knowledge, only research in [21] considers the communication energy optimization problem for both intra- and beyond-WBAN. In this study, an optimal packet payload size solution is presented. The problem is formulated as geometric programming problem with the constraints of throughput and time delay which is solved by using numerical method. This study is different as our work since we aim to optimize the throughput and power consumption trade-off on gateway node by adaptively setting transmission power and modulation level, and reduce the power consumption for sensor node using traffic admission control scheme.

3 System description

In this paper, we consider a WBAN-based remote health-monitoring scenario. In this scenario, physiological data packets are generated at the sensor nodes in a constant rate and transmitted to gateway using short-range wireless transmission technology (for example, Zigbee [22] or Bluetooth [23]). At the gateway node, the data packets are temporarily hold in a queue waiting for the transmissions to the base station. The maximum queue length is determined by the maximum delay requirement in the upper layer application. The data packets may become worthless in the sense of real-time diagnosis if the waiting time exceeds the delay limit, and it is thus removed from the queue. In this work, we do not consider any packet drop due to unreliable wireless channel condition. Gateway node is subjected to a power consumption budget, and thus a transmission scheduling scheme is used to improve the throughput (packets per time slot). At the same time, a traffic admission control scheme is applied to adapt the traffic rate of intra-WBAN to the achieved throughput at the beyond-WBAN. By doing this, wireless transmitting overhead for WBAN sensor node is reduced which leads to a noticeable energy saving.

In this work, time is divided into time slots with equally size. At each time slot, the data packets are generated once, and the gateway node receives the updated system states information (intra- and beyond-WBAN channel stat, queue level). An intelligent transmission algorithm is running on the gateway node to carry out actions (the amount of packets transmitted to base station and the amount of packets transmitted from the sensor node to gateway) depending on the system states. Consider the fact that the number of mobile users is now huge, and the cellular networks are commonly short in bandwidth resources, resulting in very limited network capacity allocated to the remote health-monitoring service. Thus beyond-WBAN communication has to be carried out in a small time duration in each time slot. Figure 2 shows the system architecture.

Fig. 2

System architecture

4 Problem formulation

4.1 Equations formulation as a constrained Markov decision problem

As we mentioned, the scheduling and admission control actions should be intelligently decided depending on the system states, and thus an optimal policy should be derived. To this point, we formulate the considered problem as a CMDP. In the considered problem, the system state consists of intra- and beyond-WBAN channel state, and queue state and are denoted as $s^{n} = \left[ {h_{I}^{n} ,h_{B}^{n} ,q^{n} } \right]$, at each time slot n ($n = 0,1,2, \ldots$) where $h_{I}^{n} ,h_{B}^{n}$ represents the intra- and beyond-WBAN channel state, and $q^{n}$ represents queue state at the gateway node. We now discuss the system components in detail.

Wireless channel is naturally continuous, but the available communication module is usually limited and discrete, and thus in our work the channel condition at time slot $n$ is represented by $h_{I}^{n} ,h_{B}^{n}$ selected from the state space denoted as ${\mathcal{H}}_{I}$ and ${\mathcal{H}}_{B}$. Besides, it is assumed the intra- and beyond-WBAN channel state remains unchanged in a time slot, and makes a transition from $i$th state to $j$th state at the next time slot with transition probability $p_{I}^{h} \left( {h_{I,j} |h_{I,i} } \right)$ and $p_{B}^{h} \left( {h_{B,j} |h_{B,i} } \right)$.

Let $q^{n} \in {\mathcal{Q}} = \left\{ {0, \ldots ,B} \right\}$ denote queue state (length), where $B$ are maximal queue size and ${\mathcal{Q}}$ is the state space. According to Little’s Law, length of data queue is equivalent to the time delay of the buffered data, thus $B$ are determined by the maximal time delay limit (taking into account of further transmission delay). Let $r^{n}$ denote the number of data packet arriving at the time slot $n$. In this paper, we consider periodic monitoring service which means the data arrival rate and the packet size are constant.

The system state space is countable set ${\mathcal{S}} = {\mathcal{H}}_{I} \times {\mathcal{H}}_{B} \times {\mathcal{Q}}$, where × represents Cartesian product. At each time slot, actions are carried out. The actions are defined by 2-tuple $x^{n} = \left( {x_{I}^{n} ,x_{B}^{n} } \right)$ at time slot $n$ where $x_{B}^{n}$ corresponds to the amount of packets at the queue that can be transmitted to the base station and $x_{I}^{n}$ represents the amount of packets allowed to be send to the gateway node. Thus, we have

$$ x^{n} = \left( {x_{I}^{n} ,x_{B}^{n} } \right) \in U\left( {s^{n} } \right) = \left\{ {\left( {x_{I}^{n} ,x_{B}^{n} } \right)|0 \le x_{I}^{n} \le r^{n} ,0 \le x_{B}^{n} \le q^{n} } \right\} $$

(1)

where $U\left( {s^{n} } \right)$ denotes the feasible set of action when the system state is $s^{n}$ at time slot $n$. Then, the queue length evolves at each time slot as follows:

$$ q^{n + 1} = \hbox{min} \left( { q^{n} + x_{I}^{n} - x_{B}^{n} ,B} \right) $$

where $n = 0,1, \ldots$.

The system state evolution is thus given as:

$$ P\left( {s^{n + 1} |s^{n} ,x^{n} } \right) = p_{I}^{h} \left( {h_{I}^{n + 1} |h_{I}^{n} } \right) p_{B}^{h} \left( {h_{B}^{n + 1} |h_{B}^{n} } \right){\mathbf{I}}\left( {q^{n + 1} = \hbox{min} \left( { q^{n} + x_{I}^{n} - x_{B}^{n} ,B} \right)} \right) $$

(2)

where $I\left( \cdot \right)$ is the indicator function whose value is 1 if the event inside the bracket is true and 0 otherwise. $P\left( {s^{n + 1} |s^{n} ,x^{n} } \right)$ is the probability that the system will go to the state $s^{n + 1}$ if system is currently at $s^{n}$ and action $x^{n}$ is carried out at time slot $n$.

Physiological data contain useful medical information, thus delivering one data packet receives one utility. In addition, we take into the account the power consumption of sensor node by introducing a penalty cost multiplied by a weight coefficient $k$ representing a trade-off between throughput and energy consumption of sensor node. So we construct the utility function as:

$$ u^{n} = x_{B}^{n} - k \cdot x_{I}^{n} \cdot P_{s} \left( {h_{I}^{n} } \right) $$

(3)

where $P_{s}$ is the power consumption of sensor node given intra-WBAN channel state. Besides, the beyond-WBAN transmission of $x_{B}^{n}$ queued data packets under the channel state introduces a power consumption $c^{n}$ at gateway node. $c^{n}$ consists of transmission power $P_{tx}$ and circuit power $P_{on}$. $P_{tx}$ depends on $x_{B}^{n}$, $h_{B}^{n}$ and target bit error rate $BER$, and $P_{on}$ is a constant value. Thus, $c^{n}$ is calculated as

$$ c^{n} = P_{tx} \left( {h_{B}^{n} ,x_{B}^{n} ,BER} \right) + P_{c} . $$

(4)

The objective of this work can be then expressed as maximizing the long-term average utility with the restriction of average power consumption budget. To this point, an optimal stationary policy which is a function mapping system state to actions have to be driven. We denote the policy as

$$ \pi : S \to U\left( {s^{n} } \right) $$

(5)

With policy $\pi$, we denote long-term average utility and average power consumption as:

$$ U = \mathop {\lim \inf }\limits_{N \to \infty } \frac{1}{N}E\left[ {\mathop \sum \limits_{n = 1}^{N} u\left( {\pi \left( {s^{n} } \right)} \right)|s^{0} } \right] $$

(6)

$$ E = \mathop {\lim \sup }\limits_{{{\text{N}} \to \infty }} \frac{1}{N}E\left[ {\mathop \sum \limits_{n = 1}^{N} c^{n} \left( {h_{B}^{n} ,\pi \left( {s^{n} } \right)} \right)|s^{0} } \right] $$

(7)

Note that the system space is countable and discrete and the action space is finite, thus according to [24, Th 6.2.10] an optimal stationary deterministic policy exists. In this paper, we only focus the stationary policy. In addition, all the admissible policies induce a unichain MDP and the utility function and cost function is bounded, and thus the bounded average utility and bounded average power consumption with a unichain MDP are not dependent on the initial system state and $s^{0}$ can be dropped in Eqs. (6) and (7).

The objective of this work is now formally written as:

$$ \mathop {\hbox{max} }\limits_{\pi \in \varPi } U $$

(8)

such that

$$ E \le \bar{E} $$

(9)

where $\varPi$ is the set of all feasible policy and $\bar{E}$ is the average power consumption budget.

4.2 The Lagrangian approach

It has been proved that solving the constrained MDP is the same as solving the unconstrained MDP and its Lagrange dual problem [25]. The constrained MDP with average power consumption constraint can be transformed into an unconstrained MDP by introducing the Lagrange multiplier. This result is supported by Theorem 1 presented as follows.

Theorem 1

The optimal utility of the constrained MDP problem (8) can be computed as

$$ U_{{\bar{E}}} = \mathop {\hbox{max} }\limits_{\pi \in \varPi } \mathop {\hbox{min} }\limits_{\lambda \ge 0} J_{\pi ,\lambda } + \lambda \bar{E} = \mathop {\hbox{min} }\limits_{\lambda \ge 0} \mathop { \hbox{max} }\limits_{\pi \in \varPi } J_{\pi ,\lambda } + \lambda \bar{E} $$

(10)

where

$$ J_{\pi ,\lambda } = \mathop {\lim \inf }\limits_{N \to \infty } \frac{1}{N}E\left[ {\mathop \sum \limits_{n = 0}^{N} u^{n} \left( {s^{n} ,x^{n} ;\lambda } \right)} \right] $$

(11)

If the policy $\pi^{*}$ is optimal, then

$$ U_{{\bar{E}}}^{*} = \mathop {\hbox{min} }\limits_{\lambda \ge 0} \left\{ {J_{{\pi^{*} ,\lambda }} + \lambda \bar{E}} \right\} $$

(12)

The proof of Theorem 1 is given in [25]. In (11), $u^{n} \left( {s^{n} ,x^{n} ;\lambda } \right)$ is Lagrange utility with a given Lagrange multiplier $\lambda$ ($\lambda \ge 0$), and is defined as

$$ u^{n} \left( {s^{n} ,x^{n} ;\lambda } \right) = u^{n} \left( {s^{n} ,x^{n} } \right) - \lambda c^{n} \left( {h_{B}^{n} ,x_{B}^{n} } \right) $$

With a given $\lambda$, the long-term average Lagrange utility and Lagrange average power consumption is thus denoted as:

$$ U_{{\pi_{\lambda }^{*} }} = \mathop {\lim \sup }\limits_{N \to \infty } \frac{1}{N}E\left[ {\mathop \sum \limits_{n = 1}^{N} u\left( {s^{n} ,\pi_{\lambda }^{*} \left( {s^{n} } \right)} \right)} \right] $$

(13)

$$ E_{{\pi_{\lambda }^{*} }} = \mathop {\lim \inf }\limits_{N \to \infty } \frac{1}{N}E\left[ {\mathop \sum \limits_{n = 1}^{N} c^{n} \left( {h_{B}^{n} ,\pi_{\lambda }^{*} \left( {s^{n} } \right)} \right) } \right] $$

(14)

For a given $\lambda$, the maximal value of (11) is denoted as $J_{\lambda }^{*}$ and can be solved using the Bellman’s optimality equation as:

$$ J_{\lambda }^{*} \left( {q,h_{I} ,h_{B} ;\lambda } \right) = \mathop {\hbox{max} }\limits_{\pi \in \varPi } \left[ \begin{aligned} & u\left( {x^{n} ;\lambda } \right) + \mathop \sum \limits_{{h_{I}^{'} }} p_{I}^{h} \left( {h_{I}^{'} |h_{I} } \right) \mathop \sum \limits_{{h^{\prime}}} p_{B}^{h} \left( {h_{B}^{'} |h_{B} } \right) \\ & \quad \cdot I\left( {q^{\prime} = \hbox{min} \left( { q + x_{I} - x_{B} ,B} \right)} \right) \\ & \quad \cdot J_{\lambda }^{*} \left( {q^{\prime},h_{I}^{'} ,h_{B}^{'} ;\lambda } \right) \\ \end{aligned} \right] - J_{\lambda }^{*} \left( {\dot{q},\dot{h}_{I} ,\dot{h}_{B} ;\lambda } \right) $$

(15)

for any arbitrary but fixed state $\left( {\dot{q},\dot{h}_{I} ,\dot{h}_{B} } \right)$. $J_{\lambda }^{*}$ can be solved by using the well-known relative value iteration algorithm (RVI) [26]. Accordingly, the optimal scheduling policy with Lagrange multiplier $\lambda$, denoted as $\pi_{\lambda }^{*}$ can be derived as:

$$ \pi_{\lambda }^{*} = \mathop {\arg \hbox{max} }\limits_{{x^{n} }} \left[ \begin{aligned} & u\left( {x^{n} ;\lambda } \right) + \mathop \sum \limits_{{h_{I}^{'} }} p_{I}^{h} \left( {h_{I}^{'} |h_{I} } \right) \mathop \sum \limits_{{h^{\prime}}} p_{B}^{h} \left( {h_{B}^{'} |h_{B} } \right) \\ & \quad \cdot I\left( {q^{\prime} = \hbox{min} \left( { q + x_{I} - x_{B} ,B} \right)} \right) \\ & \quad \cdot J_{\lambda }^{*} \left( {q^{\prime},h_{I}^{'} ,h_{B}^{'} ;\lambda } \right) \\ \end{aligned} \right] $$

(16)

However, we still have the problem of calculating the Lagrange multiplier $\lambda$. It has been proved that $E_{{\pi_{\lambda }^{*} }}$ is a convex function of $\lambda$, and thus the optimal Lagrange multiplier $\lambda^{*}$ can be found by using the following update:

$$ \lambda^{n + 1} = \lambda^{n} +\epsilon \left({E_{{\pi_{{\lambda^{n}}}^{*}}} - \bar{E}} \right) $$

(17)

where $\epsilon$ is the convergence rate. Due to the convexity, formula (17) can converge to $\lambda^{*}$.

We now describe how to construct the optimal policy. It is demonstrated in [27] that the optimal policy is combined randomly by two pure policy. Supposed the $\lambda^{*}$ is already found, $\lambda^{*}$ is perturbed by $\beta$ to get $\lambda^{ - } = \lambda^{*} - \beta$ and $\lambda^{ + } = \lambda^{*} + \beta$. Then, we can obtain the corresponding pure policy $\pi_{{\lambda^{ - } }}^{ *}$ and $\pi_{{\lambda^{ + } }}^{ *}$. The optimal policy is combined randomly by $\pi_{{\lambda^{ - } }}^{ *}$ and $\pi_{{\lambda^{ + } }}^{ *}$ with randomize factor $q$. That mean at each transmission round, the algorithm selects $\pi_{{\lambda^{ - } }}^{ *}$ at probability $\alpha$, while it selects $\pi_{{\lambda^{ + } }}^{ *}$ at probability $\left( {1 - \alpha } \right)$. The randomize factor $\alpha$ is derived as:

$$ \begin{aligned} & E_{{\pi_{{\lambda^{ - } }}^{*} }} + \left( {1 - \alpha } \right)E_{{\pi_{{\lambda^{ + } }}^{*} }} = \bar{E} \\ & \quad \Rightarrow \alpha = \frac{{\bar{E} - E_{{\pi_{{\lambda^{ + } }}^{*} }} }}{{E_{{\pi_{{\lambda^{ - } }}^{*} }} - E_{{\pi_{{\lambda^{ + } }}^{*} }} }} \\ \end{aligned} $$

(18)

The process of calculation of the optimal policy is shown in Algorithm 1.

5 Simulation results

5.1 Experiment setup

In this section, we use a MATLAB-based simulator to evaluate the performance of the proposed algorithm. Table 1 summarizes the parameters used in the simulation. We assume the gateway node use M-QAM (quadrature amplitude modulation) scheme and can transmit 1 to 8 packets in a time slot by adjusting the modulation level. The corresponding transmission power function $P_{tx}$ can be found in [3]. In addition, we assume block-fading wireless channel for both intra- and beyond-WBAN link. In other words, the intra- and beyond-WBAN channel process $\left\{ {h_{I}^{n} } \right\},\left\{ {h_{B}^{n} } \right\}$ is independent and identically distributed (i.i.d) with distribution probabilities $P_{B}$ and $P_{I}$ which are specified as $P_{B} = \left[ {1,1,2,3,3,2,1,1} \right]/14$ and $P_{I} = \left[ {1,1,1} \right]/3$, respectively. This experimental setting is typical for most WBAN-based remote health-monitoring applications.

Table 1 Simulation parameters

To benchmark the proposed algorithm, we have considered following three schemes:

• (1) Greedy scheme Greedy scheme transmits as many queued data packets as possible with the constraint of power consumption budget at each time slot. If the power budget is not used up, the gap will be compensated at the next transmission round. This greedy scheme does not adopt any traffic control scheme.

• (2) Scheduling without traffic admission control (SOAC) SOAC scheme is similar as the proposed algorithm, but no traffic admission control is adopted.

• (3) Scheduling with native traffic admission control (SNAC) SNAC scheme is similar as the proposed algorithm, but a naïve traffic admission control is adopted. Naïve traffic admission control allows as many data packets as possible only if the queue has enough remaining capacity.

In this section, we refer the proposed algorithm as scheduling with traffic admission control (SAC).

5.2 Results

Figure 3 shows the throughput and average power consumption trade-off with the various average power consumption budgets ranging from 20 to 60 mw. We observe that the achieved throughput is increased as the average power consumption increases in all schemes since with a higher power consumption budget more power can be used to deliver the data packets. Furthermore, it is easy to observe that the greedy scheme shows a much worse performance in throughput compared to that of all scheduling schemes; almost 100% throughput can be increased by using scheduling. This is because the greedy scheme dose not exploit the dynamic channel state and queue state information, and blindly transmit the queued packets as long as the power budget allows. The result also shows that all three scheduling schemes have similar performance in terms of throughput since an intelligent transmission policy at gateway node can use the power consumption budget in a more efficient way by analyzing the dynamic characteristics of the system states.

Fig. 3

Throughput and average power consumption budget trade-off of the four schemes

Figure 4 demonstrates the impact of traffic admission control on power consumption of sensor node at different average power consumption budget. Three scheduling schemes are considered. (It is no need to consider SOAC scheme and greedy scheme together as both of them do not consider any traffic admission control.) The result shows that if no traffic admission control is adopted (SOAC scheme), the power consumption of sensor node keeps at the highest level (5.7 mw) as every generated data packet is allowed to the gateway node. The power consumption of sensor node can be significantly cut down even if a naïve traffic admission control is used because it can effectively reduce the power consumption waste caused by queue overflow. By using proposed SAC, the power consumption of sensor node can be further reduced (50% when average power consumption budge is set as 30 mw). This traffic admission control of proposed SAC not only guarantee that no queue overflow would occur and also try to allow the sensor node’s wireless transmission at time when the intra-WBAN channel are in better states. In addition, the results in Fig. 4 show the performance gap diminishes as average power consumption budge getting higher. This is because that the optimal traffic admission control tends to use greedy manner when average power consumption budge is getting sufficient since a high achieved throughput at gateway node can digest all generated data packets.

Fig. 4

Power consumption of sensor node of the three scheduling schemes with different power consumption budget

The results in Fig. 5 explain how the proposed SAC can outperform other schemes in terms of power consumption of sensor node. In Fig. 5, the average queue size with different power consumption budget is given. It is observed that greedy scheme has the highest average queue size, while the proposed SAC scheme has the lowest average queue size when power consumption budget is lower than 40 mw. A high average queue size means a high queue overflow rate. The queue overflow is due to the reason that the power consumption budget cannot handle the traffic rate. As the power consumption budget increasing, the average queue size of greedy scheme is reduced, but still at a high level. While the average queue size of SOAC scheme and SNAC scheme is reducing in a fast rate, the SNAC scheme results in a lower average queue size than SOAC as a simple traffic admission control is used. For the proposed SAC, the resulted average queue size resides at around 5.8 no matter how does power consumption budget varies, rather than keep reducing as the other schemes do. This is because the proposed SAC tend to keep a moderate average queue size which not only can guarantee no queue overflow occur and also allow the intra-WBAN transmissions at satisfied channel states as much as possible in the premise of without compromising the achieved throughput.

Fig. 5

Average queue size of the four schemes with different power consumption budget

As we mentioned, the power consumption of sensor node is taken into account by introducing a penalty cost (which is the power consumption of sensor node) with a weight $k$ in the utility function. To investigate the impact of parameter $k$ to the proposed SAC algorithm, we evaluate the power consumption of sensor node and throughput with different value of $k$. Figure 6 shows the results. As we can observe in Fig. 6a that higher value of $k$ results in lower power consumption of sensor node as a more heavy weight is put on the penalty cost in utility function. On the other hand, as we can see in Fig. 6b that higher value of $k$ negatively affects the throughput as the higher weight of power consumption of sensor node let the policy tend to sacrifice the throughput so as to obtain an optimal long-term utility. It is noted that the varying range in throughput is small (from 1.285 to 1.36), it is because the generated rate of data packet is small (2 packets in 1 s) in our considered scenario. Thus it is reasonable to set a relative high value of $k$ (for example 0.1) to achieve lower power consumption for WBAN sensor node.

Fig. 6

Impact of $\varvec{k}$ on a the power consumption of sensor node and b throughput of beyond-WBAN link

6 Conclusions

In this paper, we have studied the joint scheduling and admission control problem for WBAN-based remote health-monitoring applications. By using constrained Markov decision processes approach, an intelligent transmission algorithm is proposed to jointly optimize the energy efficiency of gateway node and WBAN sensor node. Simulation results are provided to demonstrate that the proposed algorithm significantly outperforms the greedy scheme (in terms of throughput) and other scheduling schemes which do not consider the intra-WBAN link (in terms of power consumption for WBAN sensor node). Possible topic for future work is to apply reinforcement learning approach [28] which do not require any priori statistical knowledge and consider multiply heterogeneous WBAN senor nodes. In this case, the amount of system states can be huge, thus structural knowledge [29] should be studied and exploited to reduce the algorithm complexity and accelerate the convergence rate.