Keywords

1 Introduction

Internet of Things (IoT) technologies are increasingly used in smart homes, smart cities, etc. Among these technologies, wireless sensing has attracted much attention from both the academic and industrial fields, due to its advantages of non-intrusive and privacy-preserving.

There are a variety of wireless signals in our daily life, among which Wi-Fi is the most common one in indoor environments. Cisco Annual Internet Report (2018–2023) shows that the number of public Wi-Fi hotspots will be nearly 628 millions by 2023, 4 times more than that in 2018. Therefore, wireless sensing based on the Wi-Fi signal is promising to achieve truly seamless access, attracting a great deal of research and leading to lots of breakthroughs. In 2000, Bahl et al. [1] proposed to use RSS for indoor localization and implemented Radar based on Wi-Fi for the first time. Youssef et al. [21] designed a high-precision positioning system named Horus, whose average positioning error reached 0.6 m. However, RSS is inaccurate and susceptible to multi-path effects, and thus cannot be used for fine-grained sensing. In 2011, Halperin et al. [3] released the CSI Tool, which enables the extraction of CSI (i.e., Channel State Information) from commercial Wi-Fi devices (e.g., the Intel 5300 NIC). Compared with RSS, CSI is more fine-grained and sensitive to the environment, and has been widely used for human behavior sensing, such as trajectory tracking [4, 8, 18, 20, 23], respiration sensing [7, 9,10,11,12, 17, 23,24,25,26,27,28], gait detection [13, 15, 19, 22], gesture recognition [2, 5,6,7, 16].

However, most existing Wi-Fi sensing systems are still in the lab stage and there are some major issues preventing their practical deployment. One of the issues is that the sensing range of Wi-Fi based systems is very limited, not even effectively cover a room, as the target’s reflection signal is quite weak. Another issue is location dependency, i.e. the sensing performance degrades greatly once the target’s location varies. Thereby, how to effectively overcome the above problems and implement a robust Wi-Fi sensing system is a prominent challenge. To this end, we take advantage of the fact that Wi-Fi devices are commonly equipped with multiple antennas and introduce beamforming to enhance the target’s reflection signal, which help improve the system’s sensing ability in long-range scenario. Meanwhile, to address the location dependency problem, we propose an sub-carrier selection algorithm based on the dynamic signal energy ratio.

The main contributions of this paper are as follows:

  • We built a beamforming-based Wi-Fi sensing platform using commercial Wi-Fi devices, based on which both delay and sum beamforming algorithms were implemented.

  • To address the location dependency problem, we proposed an optimal sub-carrier selection algorithm based on the dynamic signal energy ratio.

  • Based on the built Wi-Fi sensing platform, we implemented a robust respiration sensing system, which outperforms the state-of-the-art baseline methods.

The rest of the paper is organized as follows. In Sect. 2, we discuss the relevant work briefly. Then the beamforming algorithm and the respiration sensing system are presented in Sect. 3 and Sect. 4, respectively. In Sect. 5, we evaluate the performance of the proposed system. Finally, the paper is concluded in Sect. 6.

2 Related Work

This section presents recent advances in the field of respiration detection based on wireless sensing.

Sensing Range: Generally, the solutions to the limited sensing range can be divided into two categories according to the processing stage [14]. One is to strength the signal at the source. For example, LoRa, a signal used for long-range communication in the IoT, was used for sensing in [27]. The other is to enhance the received signal with proper processing. For instance, FarSense [26] assumed that the time-varying random phase shifts are similar at different antennas on the same receiver, and eliminate the noise by dividing the CSI readings. However, the CSI obtained from either receiving antenna contains target motion information, leading to the destruction of phase information after dividing. EMA [23] proposed the sensing signal-to-noise ratio (SSNR) to measure the sensing ability of received signal, and combined the CSIs on multiple antennas by the optimal weight vector that maximize SSNR to enhance them.

Robustness: FullBreath [24] analyzed the widely existing location dependency problem using only magnitude or phase information, and proposed to use the complementary of magnitude and phase to solve the problem. Zhang et al. presented a new idea of using the curvature curve of target reflection signal to solve the location dependency problem in [27], which also alleviated the problem of insufficient spatial resolution due to the limited number of antennas. MultiSense [25] took the difference of sensing ability of sub-carriers into account, and performed a sub-carrier filtering and merging algorithm based on respiration energy to improve the robustness.

3 Beamforming

Beamforming, also known as spacial filtering, is to adjust the amplitude and phase of multiple signals so that the signals interfere with each other in the desired direction to be enhanced or weakened. There are many ways to implement beamforming, the most common of which is based on the antenna array.

3.1 Preliminary

Antenna array is a set of antennas arranged according to certain rules. The simplest of them is the equally spaced line array (hereafter referred to as line array) in which the array elements are arranged at equal intervals on a straight line, as shown in Fig. 1.

Fig. 1.
figure 1

Application scenario and structure of equally spaced linear array.

Full size image

Let the antenna interval of the n-element line array be d, the wavelength of the signal be \( \lambda \), and the angle of arrival be \( \theta \). In the far field, the received signals can be expressed as

$$\begin{aligned} \begin{aligned} y_1&={} \delta (t) e^{j \phi (t)} (h_{s,1} + s e^{-j \frac{2 \pi (1 - 1) d \sin {\theta }}{\lambda }} + \varepsilon _1) \\ y_2&={} \delta (t) e^{j \phi (t)} (h_{s,2} + s e^{-j \frac{2 \pi (2 - 1) d \sin {\theta }}{\lambda }} + \varepsilon _2) \\&\qquad \qquad \qquad \vdots \\ y_n&={} \delta (t) e^{j \phi (t)} (h_{s,n} + s e^{-j \frac{2 \pi (n - 1) d \sin {\theta }}{\lambda }} + \varepsilon _n) \\ \end{aligned} \end{aligned}$$
(1)

where \( h_{s, k} \) is the static path signal, s and \( e^{-j \frac{2 \pi (k - 1) d \sin {\theta }}{\lambda }} \) are the complex coefficient including amplitude and common phase, and the phase difference of the target reflected signals, respectively, \( \varepsilon _k \) is the additive Gaussian white noise (AWGN), \( \delta (t) \) is the automatic gain noise (AGN), and \( \phi (t) \) is the random phase offset.

To facilitate the description, we have the following equation

$$\begin{aligned} \boldsymbol{Y} = \delta (t) e^{j \phi (t)} (\boldsymbol{H_s} + \boldsymbol{A} s + \boldsymbol{N}) \end{aligned}$$
(2)

where \( \boldsymbol{Y} = [y_1, y_2, \cdots , y_n]^\textrm{T} \), \( \boldsymbol{H_s} = [h_{s,1}, h_{s,2}, \cdots , h_{s,n}]^\textrm{T} \), \( \boldsymbol{N} = [\varepsilon _1, \varepsilon _2, \cdots , \varepsilon _n]^\textrm{T} \), \( \boldsymbol{A} = [1, e^{ -j \frac{2 \pi d \sin {\theta }}{\lambda }}, \cdots , e^{ -j \frac{2 (n - 1) \pi d \sin {\theta }}{\lambda }}]^\textrm{T} \).

3.2 Delay and Sum Beamforming (DSB)

In order to enhance the signals, what we need to do is to find a proper weight vector \( \boldsymbol{W} = [w_1, w_2, \cdots , w_n]^\textrm{T} \) to combine the CSIs. The most comprehensible one is Delay and Sum Beamforming, which is based on the vector addition principle i.e. \( \mid \boldsymbol{a} + \boldsymbol{b}\mid \le \mid \boldsymbol{a}\mid + \mid \boldsymbol{b}\mid \), and the equal sign holds when and only when \( \boldsymbol{a} \) and \( \boldsymbol{b} \) are in same direction. Therefore, in order to enhance the signal, we need to perform the following steps: 1) align the signal; 2) sum them up.

Note that it is the target reflection signal that we want to enhance. As can be seen from Eq. 2, the phase differences of the target reflection signals are determined by \( \boldsymbol{A} \), so the simplest way to align them is to multiply them by \( \boldsymbol{A}^\textrm{H} \). Then all we need to do is sum them up. In general, we can take the weight vector as

$$\begin{aligned} \boldsymbol{W_{DSB}} = e^{-j\alpha } \cdot \boldsymbol{A}^\textrm{H} = e^{-j\alpha } \cdot [1, e^{j \frac{2 \pi d \sin {\theta }}{\lambda }}, \; \cdots \;, e^{j \frac{2 (n - 1) \pi d \sin {\theta }}{\lambda }}] \end{aligned}$$
(3)
Fig. 2.
figure 2

A simple example of Delay and Sum Beamforming. (Color figure online)

Full size image

The following is a simple example to illustrate the effect of the beamforming algorithm. As shown in Fig. 2, two original CSIs are shown on the left and the beamformed one is on the right, where green arrows are the static path signals, red ones are the target reflection signals, and purple ones are the ambient noises. The two CSIs on the left side are \( y_1 = h_{s, 1} + h_{d, 1} + \varepsilon _1 = (6+3i) + ( -1+4i) + (-1.41-1.41i) \) and \( y_2 = h_{s, 2} + h_{d, 2} + \varepsilon _2 = (-4-8i)+(5+i)+(2i) \), where \( h_{d, 2} \) and \( h_{d, 1} \) has a phase difference of \( \delta \phi = 92.7^{\circ } = 1.61 rad \). According to Eq. 2, we have the weight vector \( \boldsymbol{W_{DSB}} = [1, e^{j \frac{2 \pi d \sin {\theta }}{\lambda }}] = [1, e^{j \frac{2 \pi d \sin {1.61}}{\lambda }}] \), which rotates the second CSI counterclockwise by 1.61 rad in the complex plane, thus aligning the dynamic vectors of CSIs. Then we sum the vectors and get \( y_{comb} = h_{s, comb} + h_{d, bomb} + \varepsilon _{comb} = (14-0.61i) + (-2.24+8.95i) + (-3.41-1.51i) \), where \( h_{d, comb} \) is 2.24 times as large as \( h_{d, 1} \), \( h_{s, comb} \) is 2.08 times as large as \( h_{s, 1} \) and \( \varepsilon _{comb} \) is 1.87 times as large as \( \varepsilon _1 \), respectively. Obviously, the dynamic vector increases most significantly, which means that the dynamic path signal is enhanced and our approach works.

3.3 Beam Nulling

As shown in Eq. 2, the actual CSI also contains time-varying random phase shifts such as Carrier Frequency Offset (CFO), Sampling Frequency Offset (SFO), and automatic gain noise, components that invalidate the sensing methods based on CSI timing. FarSense [26] in which the CSI of the two antennas are divided effectively eliminate these noises, but related studies [25, 27] demonstrated that the phase of the sensed signal is corrupted due to the dynamic information contained in the denominator. In the following, we present a beamforming-based method (hereinafter referred to as Beam Nulling) to eliminate the above noise without destroying the phase information.

In the previous subsection, we used a weight vector to enhance the target reflection signal by the DSB algorithm. Here, we can also use another weight vector to achieve our goal, but note that here we are tring to nulling it instead of enhancing. Therefore, we can describe the Beam Nulling problem as follows.

$$\begin{aligned} \begin{aligned}&y_{null} = \delta (t) e^{j (\phi _c + \phi _s)} (\boldsymbol{W_{null}} \boldsymbol{H_s} + \boldsymbol{W_{null}} \boldsymbol{A} s + \boldsymbol{W_{null}} \boldsymbol{N})\\&\boldsymbol{s.t.} \quad \boldsymbol{W_{null}} \boldsymbol{A} = 0 \end{aligned} \end{aligned}$$
(4)

Analytical solution of the above problem cannot be found because the unknown dynamic path signal is only a part of the CSI and is unknown. However, we can find an approximate solution instead. To obtain an approximate solution, we introduce a metric named dynamic signal energy ratio as an indicator to solve the problem using a stochastic optimization algorithm.

Definition 1

Dynamic signal energy ratio (DSER) is the ratio of the energy of the dynamic path signal to the total energy in the spectrum. The calculation consists of two steps:

  1. 1.

    Perform Fast Fourier Transform (FFT) within a time window to obtain the spectrum of different signal components;

  2. 2.

    Calculate the ratio of the energy of the dynamic components to the total energy.

According to the above definition, the Beam Nulling problem can be transformed into a dynamic signal energy minimization problem. The algorithm is described as follows

figure a

After obtaining the approximate most weight matrix \( \boldsymbol{W_{null}}^{*} \), a reference signal without contain dynamic information and applied to eliminate random phase shift and automatic gain noise, that is

(5)

3.4 Verification of Beamforming

After the above processing, we enhance the target reflection signal and eliminate the random phase shift and auto gain noise to obtain a signal with higher sensing ability. In this subsection, we verify the effectiveness of the beamforming and beam nulling algorithms by a sliding experiment.

Figure 3(a) and Fig. 3(b) are the experimental scenario and setting for slide experiment, respectively. In this experiment, we put the slide with a length of 1 m on the vertical bisector of the transceiver at 5–6 m, and let the slider with the thick metal plate move slowly and uniformly from 5 m to 6 m and collect CSI packets. The change of the length of the reflection path \(\varDelta x = 2 \times (\sqrt{6^2 + 1.5^2} - \sqrt{6^2 + 1.5^2}) = 1.929\) m. According to the Fresnel zone model, it is known that the metal plate theoretically crosses \( \frac{\varDelta x}{\frac{\lambda }{2}} = 74.4 \) boundaries of the Fresnel zone and corresponding to a waveform with 37.2 periods, where the wavelength of the 5785 MHz signal is \( \lambda = \frac{3 \times 10^8}{5785 \times 10^6} = 0.0519\) m. Then we take 1000 of the 5990 CSI packets collected, which corresponding to a signal with \( \frac{1000}{5990} \times 37.2 = 6.2 \) periods theoretically, and process them with our proposed approach. Below we give the experimental results.

Fig. 3.
figure 3

Experimental scenario (a) and setting (b) for slide experiment.

Full size image

As shown in Fig. 4, it is obvious that the amplitude increases significantly in Fig. 4(b) and Fig. 4(c), which indicates that the beamforming enhances the intensity of the target reflected signal. In addition, the patterns of the signals in Fig. 4(b) and Fig. 4(c) are identical to those in Fig. 4(a) and have 6 full periods included, which is consistent with the theoretical value.

Fig. 4.
figure 4

Effect of beamforming on signal amplitude, (a), (b), (c) are the waveforms of the amplitude of the original CSI, beamformed CSI and beam-nulled CSI, respectively.

Full size image

We also analyze the effect of beamforming on phase, as shown in Fig. 5. Unlike the amplitude results, the phase waveforms of both the original and beamformed signals are quite heterogeneous and do not reflect any motion pattern, but the beam-nulled signal in Fig. 5(c) shows an obvious periodicity as amplitude waveform in Fig. 4(a).

Fig. 5.
figure 5

Effect of beamforming on signal phase, (a), (b), (c) are the waveforms of the amplitude of the original CSI, beamformed CSI and beam-nulled CSI, respectively.

Full size image

The above analysis leads us to the following conclusions. Our proposed beamforming algorithm can effectively enhance the target reflection signal while maintaining the pattern and periodicity of the original signal. What’s more, the beam nulling algorithm in this paper can recover the phase information of the signal from the noisy original signal.

4 Respiration Sensing System

This section first introduces the mechanism and solution of position dependency, and then introduces the respiration sensing system proposed in this paper.

4.1 Location Dependency

Currently, most of the respiration sensing algorithms are based on the amplitude. Considering the original signal phase information in the previous section, this phenomenon is not difficult to understand. However, this approach suffers from a serious location dependency problem, i.e., the sensing ability of the signal declines severely when the target is only at different locations which causes the same signal change.

Figure 6 uncovers the reason for position dependency problem. Although both the static path signal (green arrow) and target reflected signal (red or yellow arrow) changes are the same in the above figure, but the amplitude of the fluctuations of the synthesized signal (blue arrow) varies greatly. The amplitude fluctuation of the synthesized signal is the smallest when the change range of the target reflection signal is symmetric about the static path signal while the amplitude fluctuation of the synthesized signal is the largest when the change range of the target reflection signal is symmetric about the vertical line of the static path signal.

Fig. 6.
figure 6

The mechanism underlying the location dependence problem. (Color figure online)

Full size image

The fundamental reason is that the synthetic signal used for sensing is influenced by the static path signal in addition to the target reflected one. Therefore, a possible way to solve this problem is to remove the static path signal from the synthesized signal. To remove the static component, we do band-pass filtering in frequency domain, retaining only the frequency components in the respiration band, thus solving the location dependency problem by using the location-independent target reflected signal features.

4.2 Respiration Sensing

We depict the respiration sensing process in Fig. 7, which is divided into three main steps, i.e. pre-processing, respiration feature extraction and target detection.

Fig. 7.
figure 7

The process of respiration sensing algorithm.

Full size image

Pre-processing The purpose of pre-processing is to enhance the target reflection signal, reduce noise and select the optimal sub-carrier.

For each sub-carrier of the original CSI, a angle range from \(-80^{\circ }\) to \(80^{\circ }\) is scanned in steps of \(1^{\circ }\). At each scanning step, a weight vector of DSB algorithm is computed, and then the CSIs are weighted and summed to obtain the synthetic signal. After that, we take 30 s as the time window, 0.1 ~0.5 Hz as the respiration belt, using Algorithm 1 to approximate the optimal weights and constructing a reference vector to eliminate time-varying errors.

Breathing Feature Extraction. We found that the sensing ability of different sub-carriers are quite different through a large number of experiments. Thus DSER is used as an indicator to select the sub-carrier with the strongest respiration energy for the following steps to ensure the quality of respiration features.

In order to extract the respiration features, we perform band-filtering on the optimal sub-carrier, and only retain the energy of the components in the respiration frequency band, so as to obtain the frequency-direction spectrum as in Fig. 8(a) and the filtered signals. As Fig. 8(a) shown, there is a target in the \(5^{\circ }\) direction with a respiration frequency of 0.3 Hz.

Fig. 8.
figure 8

An spectrum example of respiration sensing. (Color figure online)

Full size image

Target Detection. To avoid the effect of side lobes, our target detection algorithm is divided into three steps, namely frequency scanning, direction scanning and timing detection, which means that we simultaneously utilize the information of three dimensions of frequency, space and time.

When scanning the frequency, we extract the maximum energy of a certain frequency in all directions within the respiration range as the energy of the frequency. Then a reasonable threshold that is calculated with the maximum and minimum energies is set to filter out the energy peaks greater than it to obtain candidate frequencies. For example, we get a candidate frequency of 0.3Hz in Fig. 8(b), where the green line is the threshold.

For each candidate frequency, the next step is to scan each direction. We extract the energy-direction curve from the direction-frequency spectrum and perform the same way as frequency scanning to obtain candidate directions. We get a candidate direction of \(5^{\circ }\) in Fig. 8(c).

For each candidate direction of each candidate frequency, we take out the filtered signal and use the short-term auto-correlation function to obtain the corresponding respiration rate.

After performing the above steps, our system will output the target’s respiration rate.

5 Evaluation

5.1 Experiment Setup

Our system uses a TP-Link wireless router as the transmitter, whose frequency is set to 5785 MHz, the receiver is a PC equipped with an Intel 5300 NIC and three omnidirectional antennas, and CSI Tool in [3] is used to collect CSI. During the experiment, both the transmitter and receiver are at a height of 85cm, and subject breathes naturally with a metronome.

The experimental parameters were set as follows unless otherwise specified. The sampling frequency is set 100 Hz, the distance between transceiver is 3 m, and the subject is located at 4 m on the vertical bisector of the transceiver. We collect 150 s of CSI at each sampling point and process it in segments.

In order to evaluate the system, the indicators used are described here. Target distance is half of the sum of the distance between the target and the transceiver, i.e. \(x = \sqrt{d^2 + \frac{LoS}{2}^2}\). The mean absolute error (MAE) is the average of the absolute errors of the predicted respiratory rate for all segments, and accuracy is \( (1 - \frac{MAE}{BPM_{truth}}) \times 100\% \).

5.2 Comprehensive Experiment

As Fig. 9 shown, in this set of experiments, let the target located at a distance d, where d is taken as 3 m, 4 m, 5 m, 6 m, and 7 m. To evaluate our method, we use the raw CSI, CSI quotient and beamformed CSI as inputs, respectively, and calculate the subject’s respiration rate with the same period detection algorithm. Finally, we have the following results.

Fig. 9.
figure 9

Experimental scenario and experimental setup for comprehensive experiment.

Full size image

From the Fig. 10, it can be seen that the MAE of both beamformed CSI and CSI ratio is much lower (or the accuracy is much higher than that of original CSI), indicating that our method is as effective as CSI ratio in enhancing signal sending ability. The MAE of beamforming is 0.445 BPM (breath per minute) when the target is located at 3 m on the vertical bisector of the transceiver, Though, when the distance increases to 7 m, the MAE of the original CSI exceeds 1 BPM while that of beamforming in this paper is still far less than 1 BPM ans is 0.729 BPM, which well meet the accuracy requirements in application scenarios.

Fig. 10.
figure 10

Experimental results of comprehensive experiment

Full size image

5.3 Other Experiments

We also set up experiments to investigate the effect of transceiver distance and target orientation on sensing performance, and the experimental setup is shown in Fig. 11(a) and Fig. 11(b).

Fig. 11.
figure 11

Experiment setup of different transceiver distances and target orientations

Full size image

Effect of Transceiver Distance. In this set of experiments, we explore the impact of LoS path signals on system performance by changing the transceiver distance from 1.2 m to 3.6 m in steps of 0.6 m, as shown in Fig. 11(a).

Fig. 12.
figure 12

Experimental results with different transceiver distances

Full size image

As shown in Fig. 12, it can be seen that both MAE and accuracy tend to be a random distribution, which means that the LoS signal strength does not affect the performance of the sensing system. Such experimental results is consistent with the analysis in [23].

Effect of Target Orientation. As shown in Fig. 13, the system has the smallest average absolute error and the highest accuracy when the front of the body faces the transceiver, while the MAE is relatively large (or the accuracy is slightly lower) when the side and back face the transceiver. This result is expected because the displacement of the front side of a human chest is the largest when breathing, which leads to a more obvious variation on the signal, making the accuracy higher; in contrast, the displacement of the back and side of the chest is smaller, making the accuracy lower.

Fig. 13.
figure 13

Experimental results of target orientation

Full size image

6 Conclusion

In this paper, to achieve robust respiration detection, we introduced the beamforming technology as well as a metric named dynamic signal energy ratio into the field of Wi-Fi sensing. Specifically, the commonly used delay and sum beamforming algorithm is implemented and its practical effects is verified, based on which a Wi-Fi beamforming-based sensing system is developed. The performance of the proposed system is evaluated with a number of experiments, and results show that the system achieves high accuracy and robustness, compared with baseline methods.