Introduction

Since it was first proposed by Tylor [1] in 1993, φ-OTDR has become a hot topic in the field of large-scale facility monitoring due to its high sensitivity and outstanding response speed. But with the development of optical communication technology, the traditional φ-OTDR results in a low SNR at any point and encounters a technology bottleneck [2,3,4,5,6] because coherent fading.

Therefore, in recent years, researchers also focus on how to suppress system noise, to improve the overall SNR of the system and extend the sensing distance of the system. In 2018, Liu F [7] adopt self-adaptive wavelet threshold algorithm to suppress noise, they construct a low-loss measurement system in a low-mode optical fiber, which can improve 3.72 dB compared to the traditional method. In 2019, Wu Y [8] added simulated annealing algorithm based on wavelet transform, they adopt 2-D edge detection method to locate vibration events, the SNR at the far end of sensing fiber is improved by about 4.3 dB. Although the data types collected by this method are highly representative, there are many parameters required by this method, and it is also complicated to constantly update them, and that leads to poor real-time performance. In 2020, Wang P F [9] adopt a vibration detection method based on deep learning, the multi-dimensional characteristics of the sensor data are utilized to optimize the performance, thus improving the adaptability and noise reduction ability of φ-OTDR. However, the used sensor fiber is short, which cannot represent noise reduction of the weak signal at the far end over long distance. In 2021, Khurram N [10] adopt SVD algorithm based on weighted processing to suppress random noise in φ-OTDR, they constructed Rayleigh backscattering (RBS) signals into 2-D matrix, they used a weighted singular value decomposition algorithm to filter random noise in 2-D matrix. But from the current research status, the distributed optical fiber sensing system based on φ-OTDR still faces many problems hindering its development, such as

  1. 1.

    Most researchers focus on the local vibration study at the near end of sensing fiber, but in fact, as the length of the sensing fiber increases, the signal strength at the far end become weak enough to be undetectable by the detector. Therefore, the improvement of far-end local SNR or overall SNR will be more practical.

  2. 2.

    Most of the simulative vibration events are single frequency, which means there are few disturbance types. Although they are relatively stable, they still cannot fully represent various vibration frequency events in the actual complex environment.

To realize the improvement of SNR in the whole sensing fiber distance, this paper we adopt three methods to suppress the random noise of the system and improve the overall SNR. Finally, two technical indicators, SNR and RMSE are used to compare and analyze the results obtained by the above methods.

Principle and design

φ-OTDR system detects signals by using Rayleigh scattering, Rayleigh from England proposed that the position relationships between molecules are broken due to thermal motion, hence subharmonics emitted by molecules are no longer coherent and that is where Rayleigh’s scattered light comes from [11]. Assuming that we divide the optical fiber which distance is L into N segments evenly, and there are M scattering points on the optical fiber of L/N which scattering light polarization states are all the same, then the superposition vector sum of the scattered photoelectric field of these M scattering points is calculated as follows [5]:

$$r_{k} \exp \left( {j\varphi_{k} } \right) = \sum\nolimits_{i = 0}^{M} {a_{i} \exp \left( {j\xi_{i} } \right)}$$
(1)

\(r_{k}\) and \(\varphi_{k}\) represent the amplitude and phase of M scattering points in the k section of the fiber, respectively, \(a_{i}\) and \(\xi_{i}\) represent the amplitude and phase of \(i_{{{\text{th}}}}\) scattering points in the k section of the fiber, respectively. Assuming that the four parameters are independently mutually, \(\xi_{i}\) is uniformly distributed in \(( - \pi ,\pi )\). Then, the probability density function of optical initial phase \(\psi\) and scattering phase r of RBS is shown in the following function [8]:

$$\left\{ \begin{gathered} P\left( r \right) = \frac{r}{{\sigma^{2} }}\exp \left( { - \frac{r}{{2\sigma^{2} }}} \right),r > 0 \hfill \\ P\left( \psi \right) = \frac{1}{2\pi }, - \pi < \psi < \pi \hfill \\ \end{gathered} \right.$$
(2)

r and \(\psi\) satisfy the Rayleigh distribution and uniform distribution, respectively, \(\upsigma\) only represents one order of magnitude which means 10−7, and it has no other practical meaning or unit quantity; the electric field of Rayleigh scattered can be inferred as follows [11]:

$$E_{i} = E_{0} \exp \left( { - 2\alpha L_{i} } \right)\sum\nolimits_{k = 1}^{M} {r_{k} \exp \left( {j\varphi_{k} } \right)}$$
(3)

\(\alpha\) represents attenuation coefficient; the length of the front \(i_{{{\text{th}}}}\) section of the optical fiber as \({L}_{i}\); \({E}_{0}\) represents the amplitude of the electric field of a light wave; assuming that the scattering coefficient at each scattering center point is all the same which denoted by T. Then, the intensity of the light signal detected by the photodetector is shown as follows [12]:

$$\begin{gathered} I_{i} = \left| {E_{i}^{2} } \right|\;\sum\limits_{k = 1}^{M - 1} {\sum\limits_{{k^{\prime } = k + 1}}^{M} {E_{0}^{2} T\exp \left( { - 4\alpha L_{i} + V} \right)} } \hfill \\ + 2E_{0}^{2} T^{2} \sum\limits_{k = 1}^{M - 1} {\sum\limits_{{k^{\prime } = k + 1}}^{M} {\cos \left( {T_{k}^{\prime } - T_{k} + V} \right)} } \hfill \\ \end{gathered}$$
(4)

In the above equation, V represents the probability function of the phase of reflection points, which is uniformly distributed in (0,2Π). Since φ-OTDR scattering points are randomly distributed, the φ-OTDR waveforms usually show random oscillations as shown below. When simulating the event with no disturbance, the light intensity value in the OTDR system will decrease with the increase of sensing distance. As can be seen from the simulation diagram, the optical signal intensity of φ-OTDR will be reduced with the increase of optical fiber length, and the SNR will be low in some certain positions due to the coherent fading noise, so how to reduce the effect of noise on system is the focus of this paper (Fig. 1).

Fig. 1
figure 1

The simulation result of φ-OTDR with disturbance

Full size image

Implementation methods

Collection of experimental data

The operating principle of φ-OTDR adopted in this paper is shown in Fig. 2. The ultra-narrow linewidth laser emits a 1550 nm laser. The coupler 1 is used to divide the light wave into two optical signals. One signal is modulated by an AOM (Acousto-optic modulator), AOM is a device that can modulate a laser beam. Its specific function is that the modulated signal is acted on the AOM in the form of electrical signal (amplitude modulation), and then transformed into the wave field in the form of electrical signal, so that when the light wave passes through the AOM, the optical carrier is modulated and becomes the intensity modulated wave that “carries” information. And the other signal is attenuated by a variable optical attenuator (VOA). Erbium-doped fiber amplifiers (EDFA) are used to amplify and enhance modulated signals. The function of the circulator is to integrate the amplified signal with the sensor signal collected by the sensing fiber, and the output signal is multiplexed to the filter, and the function of the filter here is to filter out the additional noise introduced into the system by EDFA amplification. The above attenuated optical signal and the output signal of filter are combined into one optical signal by 50:50 coupler. Then, the photodetector is connected to receive the transmitted light wave signal and output it into the data acquisition card (DATA ACQ). The DATA ACQ is used to collect optical fiber sensing data signals. Pulse generators are used to generate synchronization signals to DATA ACQ and AOM drivers. The laser wavelength issued by the laser source with a narrow line width is about 1550.12 nm, the pulse width is 100 ns, and the repetition frequency is 2 kHz, the sampling frequency of DATA ACQ is about 2000 Hz.

Fig. 2
figure 2

Schematic diagram of φ-OTDR system

Full size image

When collecting data on the spot, we mainly laid optical fiber for a location in campus, which is about 3 m away from the pedestrian road, the system equipment used and field data collection are shown in Fig. 3.

Fig. 3
figure 3

Field experiment collected data: a φ-OTDR system and test fiber, b Walking, c Cycling, d Climbing, e Raining, f Knocking

Full size image

Space division multiplexing

When φ-OTDR is used for signal measurement, the occurrence of coherent fading will be inevitable if the fiber used is a single (multi) core fiber. Thus, in this paper, space division multiplexing (SDM) is used to reconnect optical fibers, where SDM refers to the multi-space reuse of optical fiber in space (physical level), and the construction principles are as follows: The multi-core fiber used in this paper is seven-core fiber which is similar to other multi-core fibers. The specific connection modes are as follows: label each fiber core of the seven-core fiber, where the left (right) side of the figure is marked with 1–7 \(\left( {1^{\prime } - 7^{\prime } } \right)\), other ports are connected through jumpers in the sequence of \(1 - 1^{\prime } - 2^{\prime } - 2 - 3 - 3^{\prime } - 4^{\prime } - 4 - 5 - 5^{\prime } - 6^{\prime } - 6 - 7 - 7^{\prime }\). Port 1 serves as input and port \({7}^{\mathrm{^{\prime}}}\) serves as output (Fig. 4).

Fig. 4
figure 4

Schematic diagram of reconstructed seven-core optical fiber

Full size image

It should be noted that although there will be seven different signal reception moments for fiber optic sensing of the same location point, and there will be a time interval when the seven optical fibers transmit signals, the interval between the seven sensing signals is very short because the light propagation speed in the fiber is 2*108 m/s, which is extremely fast. Therefore, the SDM reconfiguration connection proposed in this paper can not only realize multiple and multi-dimensional optical fiber signal sensing, but also for the seven times obtained data at the same position, if we take the inverse of the adjacent times first, and then calculate the difference average of the weighted combination. The sensor signal obtained finally can not only suppress the coherent fading noise of φ-OTDR, but also can obtain the information of disturbance events as time varies.

As we can see from the data, two technical indicators are used in this paper to evaluate the effectiveness of noise reduction: the first is SNR, SNR represents the overall quality of φ-OTDR sensor signal, which is specifically reflected in the detection ability of φ-OTDR vibration events, and its definition formula is as follows [13]:

$${\text{SNR}} = 10*\log \left( {\frac{{P_{{{\text{signal}}}} }}{{P_{{{\text{noise}}}} }}} \right) = 20*\log \left( {\frac{{V_{{{\text{signal}}}} }}{{V_{{{\text{noise}}}} }}} \right)$$
(5)

\({P}_{{\text{signal}}}\) represents useful signal power, \({P}_{{\text{noise}}}\) represents noise signal power, and \({V}_{{\text{signal}}}\mathrm{ and }{V}_{{\text{noise}}}\) represent useful signal amplitude and noise signal amplitude, respectively. SNR is a key indicator of φ-OTDR, which not only reflects the stability of the system, but also affects the reliability of the system positioning accuracy. SNR reflects the noise reduction ability of the method used, and the higher the SNR is, the better the noise suppression effect of the algorithm is. The second is root-mean-square error (RMSE), which directly reflects the similarity between the reconstructed signal and the original signal after noise reduction, and the smaller the RMSE is, the higher the restoration degree of the signal is, which reflects the noise reduction quality from the side. The calculation formula for RMSE is shown in 6:

$${\text{RMSE}} = \sqrt {\frac{1}{n}\sum\limits_{k = 1}^{n} {\left[ {y\left( n \right) - x\left( n \right)} \right]^{2} } }$$
(6)

y(n) represents the reconstructed signal after noise reduction, x(n) represents the original signal, and n is the length of signal. RMSE is the deviation between the result value and the true value, and its calculation method is calculating the square root value after summing and averaging the difference between the result value and the true value. Under normal circumstances, if RMSE is less than 2, the signal reduction degree is fine. After calculation, when appropriate weighted values are set, it can calculate the overall SNR of the 1 km optical fiber which is 8.41 dB, while the overall SNR of the original signal is 5.35 dB. Therefore, the SNR increased by this method is 3.06 dB, and the RMSE is approximately 1.94.

Wavelet threshold decomposition

Wavelet transform (WT) is a multi-resolution time–frequency analysis method, which is evolved based on Fourier transform [14]. WT replaces the infinite triangular basis function in Fourier transform with the finite wavelet basis function (WBF), and then changes the scale factor and shift factor of WBF to realize the time–frequency transformation of signals under different circumstances. In the conventional WT, this paper selects the threshold noise reduction method to reduce the noise of the data, and the overall flow chart is shown as follows (Fig. 5).

Fig. 5
figure 5

Improved wavelet threshold decomposition noise reduction flow chart

Full size image

Among them, WBF is the most diverse and critical. The WBF should be designed with the aim to make detection relatively easy, accurate, and less affected by noise. Among many WBFs, dbN has the characteristics of orthogonality, compact support, low complexity, and it has no linear expression. Especially, when the number of N layers is selected as 3, it is very suitable for WT of vibration data.

In essence, WT is a filtering process, and each of the decomposed layers (DL) is equivalent to a filtered extraction of data, so the number of sampling points will be reduced by half after each layer. Therefore, if the decomposed layers selection of wavelet transform is lower, the wavelet coefficients of the decomposed signal are very susceptible to noise interference. Hence, when choosing the DL, it should start from the original SNR of the signal, and if the original SNR is low, then it is better to choose more DLs. The DL selected in this paper is 3, and its decomposition process is shown below. Firstly, the input two-dimensional matrix signal is set as x(t), and the first layer decomposition is performed on x(t) to get a detail signal CD1 and an approximate signal CA1, where CD1 refers to the noise signals containing many high frequencies, while CA1 is mainly the useful signals of low frequencies, and the second-layer decomposition is mainly for CA1, which is decomposed to get again a CD2 and a CA2, and the third layer is the same (Fig. 6).

Fig. 6
figure 6

Three-layer decomposition diagram

Full size image

In the design of threshold function, the common threshold function includes soft threshold function and hard threshold function, but both have much limitations [15]. When WT is used to decompose vibration signals, the signal strength decreases with the increase in sensing distance, and the noise distribution at various scales also changes differently with the increase of DL selected, so that the amplitude and quantity of wavelet coefficients occupied by useful signals will become larger. If the above two kinds of threshold function are used, vibration signal distortion or excessive noise retention will inevitably happen. Therefore, to adapt to the distribution of wavelet coefficients at various scales, and adopts an adaptive threshold function for threshold processing, whose function expression is shown as follows:

$$y = x - \lambda + \frac{2\lambda }{{1 + e^{{\frac{2\lambda }{x}}} }}$$
(7)

λ represents the wavelength which is usually 1550.12 nm. This function can change according to the change of the decomposition scale, and the continuity of this function is fine which will not exist the above oscillation phenomenon, at the same time, it can also process most of the wavelet coefficients, avoiding the useful signal being filtered out, which is very suitable for the processing of such vibration signals. At the waveform level, to make a more intuitive comparison with the results obtained by others, the noise reduction results here will be described in Sect. "Comparison of noise reduction results". In terms of technical index, the SNR after WT is 10.05 dB, which increases SNR of the system by about 4.7 dB. And what mainly realized is suppression to a variety of system noise, and RMSE is 1.80.

Singular value decomposition

In 1970, Golub [16, 17] proposed singular value decomposition (SVD) for the first time. The method was mainly applied in matrix operation of linear algebra in the beginning. In noise reduction, SVD was mainly applied in nonlinear and non-stationary signals, matrix transformation was taken as the basic idea of operation, and matrix processing method based on feature vectors was used to process signals and noises. Meanwhile, the data of signal subspace is extracted by zeroing out the value of signal subspace. This idea will not be affected by the spectrum distribution of noise. The noise reduction steps of two-dimensional matrix data adopted by SVD in this paper are as follows:

  1. 1.

    Let \({\widetilde{S}}_{t}={S}_{t}+{m}_{s}={S}_{t}^{*}+{w}_{t}+{m}_{s},t=\mathrm{1,2}...,{\text{n}}\), where \(\left\{{S}_{1},{S}_{2},...{S}_{n}\right\}\) is the sequence after zero averaging; \({{\text{m}}}_{{\text{s}}}\) is sequence mean; \(\left\{{w}_{1},{w}_{2},...{w}_{n}\right\}\) is a white noise sequence which mean is 0 and variance is \({\upsigma }_{{\text{w}}}^{2}\); \(\left\{{S}_{1}^{*},{S}_{2}^{*},...{S}_{n}^{*}\right\}\) corresponds to the true components in sequence \(\left\{{S}_{1},{S}_{2},...{S}_{n}\right\}\). \(\left\{{w}_{1},{w}_{2},...{w}_{n}\right\}\) and \(\left\{{S}_{1}^{*},{S}_{2}^{*},...{S}_{n}^{*}\right\}\) are mutually independent.

  2. 2.

    The signals of interference optical path collected by data acquisition card are constructed as Nxh dimension matrix.

    $$\prod { = }\left[ {\begin{array}{*{20}c} {S_{1} } & {S_{2} } & \ldots & {S_{h} } \\ {S_{2} } & {S_{3} } & \ldots & {S_{h + 1} } \\ \vdots & \vdots & {} & \vdots \\ {S_{N} } & {S_{N + 1} } & \ldots & {S_{n} } \\ \end{array} } \right]_{N \times h}$$
    (8)

    In the above matrix, \(h = \left[ {\left( {n + 1} \right)/2} \right]\); \(N = n - h + 1\).

  3. 3.

    We perform SVD on the constructed N*h matrix to obtain singular value output matrix.

    $$\prod = U\Gamma V^{T}$$
    (9)

    Make singular value decomposition on the above N × h dimensional matrix \(\prod\), where \(U\in {R}^{N\times N} and V\in {R}^{h\times h}\) are orthogonal matrices and R is an arbitrary constant. The construction form of the intermediate matrix \(\Gamma\) is as follows:

    $$\Gamma { = }\left[ {\begin{array}{*{20}c} {\Delta_{q\; \times \;q} } & {O_{{q\; \times \;\left( {h - q} \right)}} } \\ {O_{{\left( {N - q} \right)\; \times \;q}} } & {O_{{\left( {N - q} \right)\; \times \;\left( {h - q} \right)}} } \\ \end{array} } \right]_{N \times h}$$
    (10)

    where \(\Delta_{{{\text{q}}\; \times \;{\text{q}}}} = {\text{diag}}\left( {\delta_{1} ,\delta_{2} ,\delta_{3} \ldots \delta_{n} } \right),\delta_{i} \left( {i = 1,2, \ldots ,q} \right)\) is the singular value of matrix \(\prod\), and \({\text{q}}\) is the rank of the matrix \(\prod\).

  4. 4.

    Make threshold control on the singular value output matrix to obtain the threshold output matrix, and the equation of artificially setting a filter threshold is (0 < \(\eta\)<1):

    $${{\sum\limits_{i = 1}^{r} {\delta_{i}^{2} } } \mathord{\left/ {\vphantom {{\sum\limits_{i = 1}^{r} {\delta_{i}^{2} } } {\sum\limits_{i = 1}^{q} {\delta_{i}^{2} } }}} \right. \kern-0pt} {\sum\limits_{i = 1}^{q} {\delta_{i}^{2} } }} = \mu$$
    (11)

    r is the eigenvalue of the matrix \(\prod\) and the absolute value of the difference with \(q\), set the \(q-r\) small singular values to 0, and reconstruct the intermediate matrix:

    $$\Gamma^{\prime } = \left[ {\begin{array}{*{20}c} {\Delta_{r\; \times \;r} } & {O_{r\; \times \;(h - r)} } \\ {O_{(N - r)\; \times \;r} } & {O_{(N - r)\; \times \;(h - r)} } \\ \end{array} } \right]_{N\; \times \;h}$$
    (12)

    here, \(\Delta_{{r\; \times \;{\text{r}}}} = {\text{diag}}\left( {\delta_{1} ,\delta_{2} , \ldots \delta_{r} } \right)\), and \(O\) is another representation of the \(\Delta\) matrix.

  5. 5.

    The threshold output matrix is filtered out to obtain the SVD filtering output matrix which is reconstructing the output matrix after the threshold control into a new matrix:

    $$\prod^{\prime } = U\Gamma^{\prime } V^{T} = \left[ {\begin{array}{*{20}c} {S_{1}^{\prime } } & {S_{2}^{\prime } } & {...} & {S_{h}^{\prime } } \\ {S_{2}^{\prime } } & {S_{3}^{\prime } } & {...} & {S_{h + 1}^{\prime } } \\ {...} & {...} & {...} & {...} \\ {S_{N}^{\prime } } & {S_{N + 1}^{\prime } } & {...} & {S_{n}^{\prime } } \\ \end{array} } \right]_{N\; \times \;h}$$
    (13)

    The elements in \({\prod }^{\mathrm{^{\prime}}}\) are averaged to obtain a sequence of SVD filtered output values.

The larger (smaller) singular values correspond to signal components with large (small) information. When the useful signal is not covered by noise, the singular value corresponding to the noise signal component is small, which will only cover the component with small information of the original signal. Therefore, by appropriately discarding the signal component corresponding to the small singular value and using the large singular value to restore signals, the influence of noise signal can be eliminated, while the main information of the original optical path signal can also be retained which realizes a great estimation of the original optical path signal. The SNR after SVD is 13.67 dB, and the improvement to the system OSNR is 8.32 dB, and the RMSE is 1.59.

Comparison of noise reduction results

Firstly, the comparison between the original waveform diagram under a single trace line and after the suppression by the three methods is plotted as shown below.

Figure 7a represents the comparison between the original signal and SDM under a single track, Fig. 7b represents the comparison of Wavelet, and Fig. 7c represents the comparison of SVD. The abscissa of both images represents the fiber length which is represented by m. The ordinate represents the light intensity power which is represented by mW. As can be seen from the above two figures, the original signal contains a large amount of noise which result in high signal distortion. After wavelet transform, the signal becomes much smoother, and the signal fading points are also significantly reduced. It can be seen from Fig. 7c that compared with the original signal or the waveform diagram after wavelet transform or SDM, the waveform diagram after SVD has better effect in terms of no matter smoothness or number of signal fading points. Then, 2000 trace were overlaid to draw diagrams to compare the noise reduction effects of the two methods which is shown as follows (Fig. 8).

Fig. 7
figure 7

Waveform diagram of a single trace: a SDM b Wavelet c SVD

Full size image
Fig. 8
figure 8

Multi-trace diagram: a Original signal, b SDM, c Wavelet, d SVD

Full size image

The abscissa represents the length of optical fiber which is represented by m, and the ordinate represents the amplitude of light intensity which is represented by V. It can be seen from the figure that, although there is more than one vibration event, the maximum vibration location cannot be accurately located due to the large amount of noise. The noise passing through the wavelet transform signal is significantly reduced, and the position of the vibration point also tends to be obvious. It can be observed that disturbance events happen at many sensing position points through the trace superposition diagram of wavelet transform, but the maximum vibration position point cannot be accurately located. As can be seen from Figure d, the system noise is significantly suppressed through SVD, and more importantly, it is very easy to locate the maximum disturbance position at about 420 m. Therefore, from the perspective of waveform diagram, SVD is the one with better effect.

In the end, the original SNR, the type of suppressed noise and other aspects are combined to draw Table 1 which is shown as follows.

Table 1 Comparison of technical indexes of noise reduction methods
Full size table

It can be seen from the above table that SVD is the one with better effect no mater in terms of SNR or in terms of RMSE.

Conclusion

In this paper, the original signal collected by φ-OTDR system has an overall SNR of 5.35 dB over the whole 1 km distance. Firstly, the coherent fading noise of the system can be reduced by space division multiplexing technology on the physical layer, and the overall SNR is finally improved 3.06 dB, and the overall SNR reaches 8.41 dB, with the RMSE of 1.94. The advantages of this method are simple and fast, but the SNR is not significantly improved. Secondly, the wavelet transform method at the algorithm level is used to reduce various random fading noises of the system, and the SNR is improved 4.7 dB, and the overall SNR reaches 10.02 dB, with the RMSE of 1.80. Finally, SVD is used to reduce various random noises in the system (mainly coherent fading), and the overall SNR is finally improved 8.32 dB, and the overall SNR is 13.67 dB, with the RMSE of 1.59. The noise suppression algorithm based on SVD is the best noise reduction method in this paper.