Deep emulational semi-supervised knowledge probability imaging method for plate structural health monitoring using guided waves

Abstract

Deep networks can obtain the structural state features and optimize the parameters of the feature layer according to the training labels. The training data including the damage signals are quite helpful for detection model training, but sometimes the industrial damage signals are difficult to obtain, especially in airplane skin and other large structures. In this paper, a deep emulational semi-supervised probability imaging algorithm is proposed to present the damage state in the absence of damage samples. A promising signal generation method for simulated damage was implemented through signal encoding, ReLU activation and reconstruction with disturbance, and its effectiveness was verified in metal plate structures and anisotropic composite plate structures. The experiment results illustrate that the proposed method can detect the damage only using normal state signals, presents a good materials generalization in both aluminium plate and composite plate, and has better performance than other state-of-art methods.

1 Introduction

Piezoelectric ultrasonic guided waves technology [1,2,3,4,5,6] has obvious advantages in energy conversion efficiency, sensitivity, signal-to-noise ratio, response frequency range and portability, which have been identified as a promising technology in structural health monitoring. To reduce the negative effects of the dispersion and multimodal of piezoelectric ultrasonic guided wave signals, many scholars proposed innovative theories and new detection methods in the signal processing of piezo-ultrasonic guided wave signals. At present, artificial intelligence algorithm and image characterization have been improved in guided wave signal processing, which further improves the practicability of guided wave structural state monitoring technology. These damage identification methods based on artificial intelligence can be mainly divided into two categories, machine learning-based methods and deep learning-based methods.

Machine learning-based ultrasonic guided wave structural monitoring method is mainly refers to using Artificial Neural Network, Fuzzy theory, Support Vector Machine (SVM), Compression sensing, Gaussian mixture model, Bayesian method and other pattern recognition methods. It aims to obtain information about implicit damage status in ultrasonic guided wave signal, which improves detecting accuracy and efficiency. Khan [7] introduced the application of machine learning technology in quantification and location of intelligent structural damage detection, he pointed out that the features inputting effectively into the machine learning model are the key to damage assessment. Mardanshahi [8] proposed a machine learning classification model for detecting matrix cracks automatically in composite materials. The model implemented a linear discriminant analysis method to improve the classification accuracy. Wang [9] proposed a guided wave detection method based on compressive sensing to detect the damage in pipeline structures. The method reduced the amount of guided wave detection data as well as maintaining the accuracy of defect identification. Peng [10] proposed a pipe protection framework based on distributed fiber sensing technology and machine learning technology. The framework is used to detect wave propagating and scattering in pipe structure including straight pipe and elbow pipe. Jimenez [11] proposed a new method for wind turbine blade reliability monitoring based on a nonlinear autoregression system and linear autoregression model. Harley [12] used compressed sensing and sparse representation technology to study guided wave dispersion curves. Machine learning-based structural health monitoring method has made abundant achievements and has been widely used in all kinds of structure condition monitoring in the study. But most machine learning algorithms still need to select manually the mode of ultrasonic guided wave signal and rely heavily on prior knowledge.

Deep learning-based guided wave monitoring technology can extract damage information from original guided wave signals by multi-layer nonlinear mapping. It could replace the artificial feature extraction in traditional machine learning with hidden layer representation. Training samples were used as baseline signals to update the weights and biases of neural networks, the hidden layer, which represents the similarity between test signals and training samples, is used to evaluate the current structure state in the monitoring process. Liu [13] proposed a Generalized Regression Neural Network Based on Time Difference Mapping to improve the damage localization accuracy in the plates. Ebrahimkhanlou [14] adopted and compared two deep learning methods (stacked autoencoder and convolutional neural network) in the acoustic emission source localization task. Xu [15] extracted several features from the first wave on multiple monitoring paths, and input them into the convolutional neural network to identify the length of fatigue cracks. Alguri [16] combined digital surrogates with transfer learning to improve a wavefield imaging method. Su [17] used Fourier transform to extract frequency domain features of signals and establish a damage classification and recognition model by using spectrum with damage characteristics and corresponding damage modes as input and output of convolutional neural network, respectively. Zhang [18] explored a deep transfer learning method to use the monitoring model of one structure to monitor another structure. When the model structure is complex enough, the training data set is sufficient, and the computing power is enough, deep-learning-based structure status monitoring method can effectively learn the relationship between multi-source monitoring data and structural state. But in the incomplete set, some label sample data are missing. The existing training set samples are not enough to train a good model to meet the monitoring accuracy requirements. Moreover, the re-labeling of enough samples will consume a lot of human and material resources and even is impossible. The incompleteness makes a different distribution between the original training samples and the new test samples, so the dataset should be compensated before model training. The data enhancement method mainly refers to the expansion of data samples through sample generation, so as to achieve the purpose of enhancing the training effect. Mao [19] proposed an abnormal data identification method by combining Generative Adversarial Networks (GAN) with autoencoder and verified the accuracy and robustness of the method in incomplete bridge data sets. Lei [20] proposed a data generation method based on a Deep Convolutional Generative Adversarial Network to solve the problem of lost data caused by sensor failure or transmission failure. The method generated virtually the lost signals by using the original signals as input to the deep learning model. Xiong [21] proposed a new method for missing signal reconstruction based on Generative Adversarial Networks. The applicability of this method in missing signal reconstruction is verified in structural health monitoring of nuclear power plant. For piezoelectric ultrasonic guided wave signals, the data-based method can be implemented by adding noise, translation, phase space reconstruction and other ways. Incomplete set knowledge transfer is a challenging frontier field in structural health monitoring. The key lies in the transfer and utilization of related ultrasonic guided wave knowledge to enhance the universality and generalization ability of the ultrasonic guided wave monitoring model. However, the research of structural health monitoring algorithm based on knowledge transfer is still in the initial stage [22], its application needs further exploration and research in the industry.

To further visually indicate the shape and position of the damage, researchers improve many imaging algorithms in guided wave signal, such as Computerized Tomography (CT), delay-and-sum imaging and probability tomography. Huthwaite [23] proposed a thickness estimation method based on dispersion characteristics for the generation of guided wave thickness diagrams, realizing flat plate corrosion tomography. Hay [24] used embedded sensors and Lamb wave ultrasonic tomography to detect material loss of components on the underside of aircraft wings. Prasad [25] studied the key influencing factors of ultrasonic guided wave tomography in a multilayer anisotropic composite plate. The experiments proved that truncating guided wave signal, optimizing sensors placement and normalizing the imaging features can effectively improve the imaging quality. Khodaei [26] proposed a numerical model of ultrasonic propagation in composite laminates and improved a delayed stacking imaging algorithm, which was verified by impact damage imaging in reinforced aluminum plates and composite plates. The key step of ultrasonic guided wave imaging method is to obtain imaging characteristics, which means that the appropriate signal processing method is particularly important to obtain effective damage indicators. Commonly, it is necessary to select the first wave or a specific mode for construction state analysis. After structural identification using the artificial intelligent method, the ultrasonic guided wave imaging algorithm can avoid the feature selection by using modal analysis and time/frequency domain analysis. However, there are few algorithms that combine imaging and deep learning.

Based on the above analysis, the application of artificial intelligence algorithm still has the following key problems in ultrasonic guided wave structural monitoring. First, the deep learning method can replace the manual process with layer extraction of the neural network, but it is difficult to train the model when the labeled real damage samples are missing. Moreover, the imaging algorithms combined with deep learning have not received the deserved attention. In this paper, a novel deep emulational semi-supervised knowledge probability imaging (DESK-PI) method is proposed to monitor the state of the structure with an incomplete set of missing damage samples and realize automatic extraction and imaging characterization of multi-sensor damage indicators. Experiments on aluminum plate and composite plate show that DESK-PI method can obtain the damage index of structure effectively. The experimental result of damage imaging shows that the proposed method has good generalization performance in different samples. Comparison results also show that the monitoring performance of DESK-PI is superior to other methods. The method relies less on prior knowledge and applies the ultrasonic guided wave intelligent monitoring technology to structures without enough damage samples.

The remainder of this paper is organized as follows: Section 2 presents the process of the proposed approach. Section 3 analyses the experimental results. Finally, Sect. 4 concludes the whole works.

2 Deep emulational semi-supervised knowledge probability imaging method

In this paper, a novel deep emulational semi-supervised knowledge probability imaging method is proposed to monitoring the health state of the plate structure. The DESK-PI method, which can be divided into four stages as Fig. 1, can automatically capture the damage index and visually present the damage.

Fig. 1

The framework of DESK-PI method

In the first step, the ultrasonic guided wave signals obtained by the sensor array need to be standardized to improve the consistency of signal samples. Secondly, the deep features of input signals are extracted using a locally connected network, and the signals would be reconstructed later. Third, the hidden representation of normal state signals is used to generate analog abnormal signals, and different types of signals are identified through the triplet network. The corresponding damage index can be obtained by comparing the normal state signal, positive and negative disturbance signals and test signals. Finally, probabilistic imaging technology is implemented to visualize the damage state.

2.1 Signal standardization

For an ultrasonic guided wave monitoring system with Np sensors, the ultrasonic guided wave data set of the monitoring structure in the normal structural state $\Omega_{{{\text{health}}}} = \left\{ {{\mathbf{x}}_{{{\text{health}}}}^{n} } \right\}_{n = 1}^{{N_{s} }}$ and unlabeled test data set (before standardization) $\Omega_{{{\text{testing}}}} = \left\{ {{\mathbf{x}}_{{{\text{testing}}}}^{n} } \right\}_{n = 1}^{{N_{t} }}$ were obtained. $N_{s}$ and $N_{t}$ are the numbers of samples in the training set and test set, respectively. The signal normalization process is divided into sensor channel normalization and sample normalization.

Sensor channel normalization is used to reduce the differences caused by different sensor installation positions when multiple receiving sensors are used, such as to correct the arrival time of received signals affected by the installation position. The threshold segmentation method is used to standardize the input signal. Meanwhile, to obtain the complete first wave mode, the segmentation position of each sensor is aligned with the nearest sensor. All samples are divided according to the following equation.

$$S_{i} = \min \left[ {{\text{where}}\left( {X^{i} > X_{\max } \times T_{\max } } \right)} \right]$$

(1)

$$S_{i} \leftarrow S_{i} - \min (S_{i} ),$$

(2)

where Xⁱ is the signal sample received by the ith sensor, X_max is the maximum of the training set, S_i is the start point of the segmentation signal, T_max is a percentage threshold in signal standardization, which is set as 10% in this work.

Sample normalization is used to avoid the training differences of signals with different amplitudes during the training of deep network models. The signal amplitudes of different receiving sensors vary greatly, so the neural network model tends to reconstruct the signals with higher amplitudes. Therefore, each training sample should be normalized, the formula is as follows:

$$X_{i} \leftarrow \frac{{X_{i} - \min (X)}}{\max (X) - \min (X)} \times N_{{\text{f}}} - \left( {N_{{\text{f}}} /2} \right),$$

(3)

where X_i is the ith element in the training sample, N_f is the normalization factor, max(X) and min(X) are the maximum and minimum of the training sample. As for the Standardization factor N_f, Standardization factor can be matched with the learning rate to improve the fitting effect. The usual practice is to set N_f to 1 and adjust the learning rate. We used the Adam optimizer's default learning rate (0.001) with N_f set to 10.

2.2 Data expansion mechanism for analog damage signal generation

The generation method of simulated structural damage signal can be expressed by the following formulas:

$$R_{{}}^{H} = {\text{Encoder}}\left( {X_{{{\text{Normal}}}} } \right)$$

(4)

$$X_{{{\text{simdamage}}}} = {\text{Decoder}}\left( {\hat{R}_{{}}^{H} } \right),$$

(5)

where X_normal is the normal ultrasonic guided wave signal, X_simdamage is the simulated structural damage signal, R^H is the hidden layer representation of ultrasonic guided wave signal, $\hat{R}_{{}}^{H}$ is the hidden layer representation after disturbance; Encoder and Decoder are locally connected encoding network and decoder reconstruction network, respectively.

By adding disturbance into the hidden representation, the simulated auxiliary training signal is generated. The decoder can generate a reconstructed signal that is different from the normal state signal. Different auxiliary training signals can be obtained by controlling the amplification ratio parameter R_d and the random disturbance D_a, and the random disturbance D_a obeys the normal distribution of N(0,1). And then the simulated hidden representation can be amplified by:

$$R_{i}^{H} = R_{i}^{H} \times \left( {1 + D_{{\text{a}}} } \right),$$

(6)

where R_i^H is the ith hidden representation element, which meets the amplification condition. Different disturbance coefficients are set to generate a simulated damage sample for every simulated normal sample. For simulated damage samples, the disturbance size is set to between 0.5 and 10% of the hidden representation amplitude. For simulating normal samples, the disturbance is set to less than 10^–4%.

In deep learning networks, the autoencoder network is a typical fusion network, which can obtain the fusion features of ultrasonic guided wave signals through encoding and decoding. After signal standardization, locally connected networks are used to obtain depth features by dimensionality reduction of training signals. The biggest difference between locally connected networks and fully connected networks is that locally connected networks are used to obtain local features of signals and reduce network parameters. As shown in Fig. 2, a three-layer auto-encoder contains input layer, hidden layer and an output layer. The mapping from input layer to the hidden layer is called encoder while the mapping from the hidden layer to the output is a decoder.

Fig. 2

The structure of locally connected auto-encoder

For the locally connected networks with N_L encoding layers, the calculation result of the first layer of the original signal X_i is:

$$\begin{gathered} h_{i} = \sum\limits_{{}}^{{}} {\left( {X_{s*i - 1 + j} \times W_{i,j} + b_{i,j} } \right)} , \hfill \\ \begin{array}{*{20}c} {} & {i \in \left[ {1,l/s} \right]} \\ \end{array} ,j \in [1,s] \hfill \\ \end{gathered}$$

(7)

$$\hat{Y}_{i} = {\text{Relu}}(h_{i} )$$

(8)

where h_i is the ith element of the hidden representation, X is the input signal, s is the stride step of locally connected networks, l is the input signal length, W_i,j is the jth element of the ith weight. The main features can be obtained and the signal dimension will be reduced after the locally connected calculation. In order to preserve waveform features, a leaky Relu function that produces a non-zero slope for negative values is used in the dimensionality reduction process:

$$Y_{i} = \left\{ {\begin{array}{*{20}c} {X_{i} } & {X_{i} > 0} \\ {X_{i} /P_{{{\text{relu}}}} } & {X_{i} < 0} \\ \end{array} } \right.$$

(9)

$$Y_{i}^{\prime } = \left\{ {\begin{array}{*{20}c} 1 & {X_{i} > 0} \\ {1/P_{{{\text{relu}}}} } & {X_{i} < 0} \\ \end{array} } \right.,$$

(10)

where the Y_i is the activated value, X_i is the weighted sum in the neuron, P_relu is the slope parameter of the activation function.

After the second layer, the result of the N_l layer can be calculated by the following formula:

$$\begin{aligned} h_{i}^{{(N_{l} )}} & = \sum\limits_{{}}^{{}} {\left( {h_{s*i - 1 + j}^{{(N_{l} )}} \times W_{i,j}^{{(N_{l} )}} + b_{i,j}^{{(N_{l} )}} } \right)} , \\ & i \in \left[ {1,l/s^{{N_{l} }} } \right],j \in [1,s] \\ \end{aligned}$$

(11)

$$\hat{Y}_{i} = {\text{Re}} {\text{lu}}\left( {h_{i} } \right).$$

(12)

The decoder can realize the reconstruction of original input through:

$$\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{X}_{m} = g\left( {W^{T} \times h_{m} + b_{2} } \right),$$

(13)

where $\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{X}_{m}$ is the reconstruction of the input signal, $g( \cdot )$ is the activation function of the output layer, W^T is the weight matrix of the decoder and b₂ is the bias of the output layer. Due to a large number of parameters in the fully connected network reconstruction, L₂ regularization was adopted to avoid overfitting in the reconstruction process.

$$l_{{L_{2} }} = \left\| W \right\|_{2} = \sum\limits_{i} {w_{i}^{2} }$$

(14)

L₂ can reduce the coefficient of some features to 0, thus indirectly realizing feature selection.

To make $\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{X}_{m}$ closely approximate $X_{m}$, the reconstruction error, which is used to update the weight matrix and the bias, can be calculated by the mean square error (MSE) loss function:

$$L\left( {X_{m} ,\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{X}_{m} } \right) = c\left\| {X_{m} - \overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{X}_{m} } \right\|^{2} + l_{{L_{2} }} ,$$

(15)

where c is the constant factor of the loss function.

The network parameters can be updated by the backpropagation algorithm, and the mean square error is utilized as a loss function to calculate the difference between the reconstruction and the input signal:

$$W^{T}_{{{\text{new}}}} = W^{T} - \lambda \frac{\partial L}{{\partial W^{T} }}$$

(16)

$$b_{{2{\text{new}}}} = b_{2} - \lambda \frac{\partial L}{{\partial b_{2} }}.$$

(17)

2.3 Anomaly monitoring model construction

Semi-supervised learning is a rapidly developing field in deep learning [27,28,29,30]. It utilizes potential variables learned through unsupervised training to improve the performance of supervised learning. This matching pattern is widely used in structural health monitoring, especially in the abnormal analysis of he structural state.

Triplet network uses three samples to form a training group to fit the loss function, whose basic structure is shown as Fig. 3.

Fig. 3

The structure of triplet network

The triplet network consists of three networks with the same feedforward (shared parameters), with one master sample and two auxiliary samples as inputs, and the network outputs the Euclidean distance between the master sample and the auxiliary samples. The three inputs are represented as x, x + , and x−, and the mapping layer of the network is represented as M_w(X). In simple terms, the input signal of the triplet network consists of a baseline sample and the corresponding positive and negative samples. The baseline sample is randomly selected from the training data set, the positive samples belong to the same class as the reference and the negative samples do not belong to the same class. They are randomly selected to form a triad. The relationship between them is measured by Euclidean distance, and the training parameters make x close to x + and away from x−, so as to achieve the recognition task.

Given a baseline signal X_n, a positive signal X_n+ and a negative signal X_t, the mapping function M_w() with sharing parameters can transfer the inputs to a high-level space. And then the distance E_w of the mapped signal can be calculated by some distance metric. This distance can be utilized as the deviation value from the testing signal to the normal signal. For each element (sample) in the triplet, a parameter-sharing network is trained to obtain the feature expression of the three elements, denoted as:

$$M_{w} (x), \, M_{w} (x - ), \, M_{w} (x + ).$$

(18)

The Triplet network uses L-2 normal form distance to calculate the difference of each sample:

$${\text{TripletNet}}(x,x - ,x + ) = \left[ \begin{gathered} \left\| {M_{w} (x) - M_{w} (x - )} \right\|_{2} \hfill \\ \left\| {M_{w} (x) - M_{w} (x + )} \right\|_{2} \hfill \\ \end{gathered} \right] \in R_{ + }^{2} .$$

(19)

The comparator is to process the vector mentioned above. Through training, the distance between x + and x feature expression should be as small as possible, and the distance between X− and X feature expression should be as large as possible. There should be a fixed distance between X + and X and the distance between X− and X, which is generally chosen as 1.

$${\text{Loss}}\left( {d_{ + } ,d_{ - } } \right) = \left\| {(d_{ + } ,d_{ - } - 1)} \right\|_{2}^{2} = {\text{const }}\,d_{ + }^{2}$$

(20)

$$\begin{aligned} d_{ + } & = \frac{{e^{{\left\| {M_{w} (x) - M_{w} (x + )} \right\|_{2} }} }}{{e^{{\left\| {M_{w} (x) - M_{w} (x + )} \right\|_{2} }} + e^{{\left\| {M_{w} (x) - M_{w} (x - )} \right\|_{2} }} }} \\ d_{ - } & = \frac{{e^{{\left\| {M_{w} (x) - M_{w} (x - )} \right\|_{2} }} }}{{e^{{\left\| {M_{w} (x) - M_{w} (x + )} \right\|_{2} }} + e^{{\left\| {M_{w} (x) - M_{w} (x - )} \right\|_{2} }} }}. \\ \end{aligned}$$

(21)

Triplet network model allows learning by comparing samples rather than direct data labels, so it can be used as an unsupervised learning model. Triplet model is used as training samples, which performs much more accurately in simple classification tasks with less data and achieves better classification results on multiple data sets. After the network training, the testing triplet would be input into the network to assess the damage index. The testing signals from different sensors are standardization, and one model can predict the damage index of testing signals from all these sensors to assess the structural state.

2.4 Damage imaging mechanism based on multipath anomaly characteristics

As a multi-sensor imaging method, probabilistic imaging assigns damage probability to the imaging points and improves the imaging performance through a damage probability distribution. Damage index and time-dependent imaging weight are the most important features in probabilistic imaging, and the damage probability of each imaging discrete point can be calculated by these two parameters:

$$P(x,y) = \sum\limits_{r = 1}^{{N_{P} }} {p_{r} (x,y)} = \sum\limits_{r = 1}^{{N_{P} }} {{\text{DI}}_{r} \times } W_{r} (x,y),$$

(22)

where P_r(x,y) is the estimated damage probability in the rth path, N_P is the number of the transmission path of all sensors, W_r(x,y) is the linear attenuation imaging weight.

The damage index was obtained by the triplet network, and the linear attenuation weight W_r(x,y) was determined according to the arrival time of the damage signal. After setting the percentage threshold, the arrival time of scattered signals and normal state signals can be obtained. In the anisotropic structure, the ultrasonic guided wave velocity needs to be determined according to the Angle, so the damage path cannot be defined directly by the theoretical equation. Instead, the damage possibility of discrete imaging point is measured by the time difference between the reference point and the damage point, which can be obtained by distance and sound velocity in each direction, as shown in Fig. 4. The calculation formula of time coefficient C_T is as follows:

$$C_{T} = \left( {\Delta t^{\prime} - \Delta t} \right)/t_{0} = \left[ {\left( {t_{1}^{^{\prime}} + t_{2}^{^{\prime}} - t_{0} } \right) - \left( {t_{1} + t_{2} - t_{0} } \right)} \right]/t_{0}$$

(23)

Fig. 4

Schematic diagram of damage detection

A large C_T value means that the larger the distance between the reference point and the damage point, the lower the damage probability. When C_T is 0, the reference point and damage point are considered to coincide. During the imaging process, W_r(x,y) is calculated by C_T and imaging weights of linear decay:

$$W_{r} (x,y) = \left\{ \begin{gathered} 1 - C_{T} /\gamma ,{\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} C_{T} < \gamma \hfill \\ 0,{\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} C_{T} \ge \gamma {\kern 1pt} {\kern 1pt} {\kern 1pt} \hfill \\ \end{gathered} \right.,$$

(24)

where $\gamma$ is the scaling coefficient of linear attenuation weight W_r(x,y), which is set as 2 in this work.

After calculating the damage probability value point by point, the damage image of the plate structure can be obtained, and the position with the highest total damage probability is the most possible location of the damage.

3 Experimental verification

3.1 Data description

In this section, aluminum plate and composite plate are introduced as verification structures for semi-supervised condition monitoring. Multi-sensor imaging can combine ultrasonic guided wave data from multi-sensor arrays, which can effectively improve the accuracy of structure monitoring. A piezoelectric ultrasonic guided wave sensor is installed on the monitoring structure to stimulate and receive signals. When there is damage in the structure, the ultrasonic guided wave propagation in the monitoring structure will change and be acquired by the receiving sensor. Damage localization of non-isotropic structures like the composite plate is a more challenging problem due to the inconsistency of ultrasonic characteristics in different directions.

• Case 1. Aluminium plate damage detection

The structural size of aluminum plate is 200 × 100 mm and the thickness is 7 mm, as shown in Fig. 5. The test platform includes an arbitrary waveform generator, amplifier, piezoelectric sensors and data acquisition card. The sensors are all circular piezoelectric transducers with a size of 10 × 1 mm. Artificial damage is made on the back of the aluminum plate, and the damage depth is gradually increased from 1 to 6 mm.

Fig. 5

Test specimen (aluminum)

A 5-peak 200 kHz sinusoidal signal was used as an excitation signal, with a sampling frequency of 5 MHz and a sampling length of 2.5 K. Thirty samples were collected for each damage degree, including 30 normal samples and 180 damage samples. Ultrasonic-guided wave signals of different damage degrees are shown in Fig. 6. In the experiment, only one receiver PZT was arranged to verify the validity of the proposed damage index.

Fig. 6

Receiving signals (After standardization)

• Case 2. Composite plate damage detection

The size of the composite plate is 600 × 600 mm and the thickness is 3 mm, as shown in Fig. 7a. In our previous study [31, 32], the velocity of this composite material in all directions has been calculated, and the results are shown in Fig. 7b. The sensors were the same as the aluminum plate experiment. A 200 kHz 5-peak sinusoidal signal was adopted with a sampling frequency of 10 MHz and a sampling length of 5 K. Total of 16 PZT sensors were used in the 250 mm radius ring, with one PZT sensor installed at 22.5 degrees intervals. In the experiment, 4 damage conditions of 5 mm, 10 mm, 15 mm and 20 mm were set. For each damage, 450 ultrasonic guided wave samples were collected by 15 PZT sensors (30 samples each PZT).

Fig. 7

Test specimen (composite)

3.2 Reconstruction results

Traditional autoencoder networks can reduce information redundancy, but the hidden layer representation obtained by fully connected mode is a global feature. Local connection network can extract local features of ultrasonic guided wave signals and generate abnormal signals by slightly changing local features. In this step, the simulated abnormal signal reflects the change of local ultrasonic guided wave mode, which is similar to the characteristics of the real damage signal. The local connection encoder structure adopted in this paper is shown in Fig. 8.

Fig. 8

The structure of locally connected autoencoder network

The initial parameters were determined by the trial and error method, and the optimal parameters were determined according to the training error. The network hyperparameters of encoder E and decoder D are shown in Table 1. The encoder consists of five local connection layers and one flatten layer. The input length of the encoder is set to 512, which is about the length of the first wave. In particular, multiple kernel parameters are employed to extract features from the original signal, which is equivalent to a multi-channel local connection network. In the coding process, local connection size is set to 2, and leakage Relu function with a slope parameter of 0.3 is implemented to preserve waveform characteristics. The mean square error function was used for network training, and all networks were optimized with a learning rate of 0.001.

Table 1 The structures of encoder E and decoder D

• (1) Case 1: Aluminium Plate

The reconstruction results of the input signal are shown in Fig. 9a. It can be seen that the reconstruction error between the reconstructed signal and the original signal is very small. This means that the ultrasonic guided wave characteristics in the original signal are well extracted. Local characteristics of the original signal are obtained by local connection, the waveform and envelope characteristics of the original signal are obtained by reducing network parameters and computation.

Fig. 9

Reconstruction signal of noisy signal

It indicates that the waveform features will change the reconstructed signal when the disturbance is added into the hidden representation, thus constructing the simulated abnormal signal. Random disturbance is added to the hidden representation as shown in Fig. 9b, and part of signal points with hidden features are changed. The reconstructed analog abnormal signal is shown in Fig. 9c. Through the comparison with the input signal, it is found that part of the amplitude of the reconstructed signal changes with the random disturbance, which is similar to the real damage mechanism shown in Fig. 9. (The OS means the original signal, the RS means the reconstructed signal, the PR means the positive representation, the NR means the negative representation and the DS means the damage signal.)

• (2) Case 2: Composite Plate

Figure 10a shows the original signal and its reconstructed signal in the composite plate. There is little difference between the original signal and the reconstructed signal, which proves the proposed method is effective. The hidden representation is good at extracting the characteristics of the OS after encoding.

Fig. 10

Reconstruction signal of the noisy signal

The hidden representations with different levels of disturbance are shown in Fig. 10b. The sampling points of the hidden representation are randomly selected to change by the program. There are the reconstructions shown in Fig. 10d. The reconstruction of the positive representation and negative representation simulate signals with different levels of damage. It can be seen in Fig. 10c that the constructions are very close to the real damage signals. Meanwhile, it can be seen from Fig. 10b, d that the disturbance makes the hidden representation locally changed. This change makes a difference between the original signal and simulated damage signals.

Simulated damage signals can be got by adding disturbance obeying a normal distribution to the hidden layer. The reconstructed signals are presented, which shows the influence of random disturbance has certain regularity. It is concluded that partial waveform and amplitude would change after reconstruction. The validity of the proposed simulated damage signal is verified, integrating the results of the signal in aluminum plate and composite plate.

3.3 Damage index and imaging results

The structural parameters of triplet networks are shown in Table 2. Four layers are employed in the triplet network, and finally the values of normal and abnormal state signal in a dense layer will be contrasted to obtain the Euclid distance. The initial segment of the signal is zero signal, and no signal distortion occurs in the hidden layer. At the end of the signal, an incomplete hidden signal may be reconstructed into a complete wave packet. Therefore, the input length of the triplet network is set to 1024, and the input signal would contain several modes of the monitoring signal. The binary cross entropy is used for triplet network training, and the ‘adam’ optimizer with a learning rate of 0.001 is utilized.

Table 2 The structures of triplet networks

• (1) Case 1: Aluminium Plate

As the damage index, the Euclidean distance was obtained after the Triplet network training the test signals and base signals. Test signal samples are classified according to damage index, as shown in Fig. 11a. The output of the triplet networks has two values, the first value representing the difference from the normal state and the second value representing the difference from the damage signal. The orange line represents the damage index and the blue line represents the difference with damage signal. In Fig. 11b, the box diagram is employed to represent the distribution of the damage index of testing samples corresponding to different damage degrees, where the length of the box represents the range of the calculated damage index, and the red line represents the median. The outliers represent the misrecognition points. The result shows that the distribution of the damage indices of test samples corresponding to different damage degrees is consistent with the previous linear diagram.

Fig. 11

The damage index in aluminium plate

The average damage index of each 30 sampling points is shown in Fig. 11c. The result shows the validity of the proposed method. The damage index reflects the damage degree of test signals, they are positively correlated in general with less influence of guide wave multi-modes. The damage index increases more and more slowly with the enlargement of the damage degree. The damage index of 2 mm depth of damage is slightly higher than 1 mm.

• (2) Case 2: Composite Plate

The structural characteristics of composite plates are more complex than aluminium plate. On the one hand, the propagation velocity of a guided wave in a composite plate is anisotropic. On the other hand, multilayer structures accentuate the multimodal and dispersion characteristics of guided waves, which makes waveform overlap more serious and damage waveforms more difficult to identify. To overcome these difficulties, a multi-sensors fusion monitoring system is proposed to obtain more information about damage, which makes contributions to structural state imaging. Figure 12a shows the damage index of the experimental signal in a composite plate. The average value of each 30 sampling points is shown in Fig. 12b. As same as Fig. 11, the damage index is increased with the increase of damage level for each sensor. It proves the ability of the network to recognize damage in multi-sensors fusion monitoring system. The sensors of large damage index are PZT2, PZT9, PZT16, which forms the paths sensitive to damage by transmitting PZT1.

Fig. 12

The damage index of composite plate signal

Substitute the damage index and the time coefficient to Eqs. (22), (23) and (24) to calculate, the damage probability of each point on the plate can be obtained as the pixel value of plate structure state imaging. The damage location is most likely to be in the place with the maximum damage probability. Different degrees of damages image obtained by the proposed are shown in Figs. 13, 14, 15 and 16a. Figures 13, 14, 15 and 16b shows its result after 97% threshold processing. As can be seen in the damage image, the localization of different degrees damage is close to reality. The image shape is axisymmetric with respect to the sensing path PZT1-PZT9. Because PZT1-PZT9 is the sensing path which leads the wave through the damage front. When the damage probability threshold was set to 97% of the maximum, the damage area image was more concentrated and closer to the actual damage area. It shows that the proposed method behaves well in damage monitoring of composite.

Fig. 13

The 5 mm damage image of proposed method

Fig. 14

The 10 mm damage image of proposed method

Fig. 15

The 15 mm damage image of proposed method

Fig. 16

The 20 mm damage image of proposed method

3.4 Method comparisons

To show the superiority in damage feature extraction, the proposed method is compared with the traditional autoencoder network in simulated damage sample generation. The results of using a traditional autoencoder network to reconstruct the signals in a composite plate are shown in Fig. 17. Comparing Fig. 17a with Fig. 10a, it can be seen that both methods can reconstruct the original signal well. But as for the first wave reconstruction, the result of the traditional method produces considerable errors.

Fig. 17

Reconstruction signal of the noisy signal

It shows that the traditional method can extract the global feature of the original signal by adopting fully connected networks. However, there are some shortcomings in the local feature extraction of the signal, which brings errors to the reconstructed signal. It can be seen in Fig. 17b, c that the global shape of the signal is changed when the hidden layer representation is disturbed. It shows that the global reconstructed signal is greatly affected by the extracted local feature when using the traditional method to reconstruct the disturbed feature. The change of a single feature point can cause a change in the global signal. The traditional method brings local error to the signal without disturbance and brings global influence to the signal when a disturbance is applied. This may not cover the difference caused to the signal by damage. The proposed feature extraction method can improve the problem with better practicability.

The original damage image and threshold damage image obtained by elliptic probability imaging are shown in Fig. 18. As can be seen from the original damage image, several elliptic bright rings representing the damage probability value are formed. With the excitation PZT1 and each receiving PZT as the focus, the elliptic bright rings converge and intersect to form a highlighted damage region obtained by the algorithm. The result shows that the distance between the calculated damaged area and the actual damaged area is large, and the positioning accuracy is not enough. It is because the elliptic probability imaging method uses the average velocity of the guided wave group to calculate the damage probability, which causes errors in the calculation of the time coefficient. After the superposition of each channel, the positioning error becomes larger. The location errors of the two methods are shown in Table 3 under different degrees damage. The results illustrate that the proposed method improves the anisotropy of velocity and improves positioning accuracy.

Fig. 18

The damaged image of comparison methods

Table 3 The location error of different degree damage

4 Conclusion and feature work

To monitor the structural state of sparse data set with missing damage samples, a novel deep emulational semi-supervised knowledge probability imaging method was proposed, which realized automatic extraction and imaging characterization of multi-sensor damage indicators. According to the actual requirements of monitoring applications, the proposed method can use only the normal guided wave signals to train the monitoring model and effectively identify the abnormal state. By adding normal random disturbance to the ultrasonic guided wave encoding signal, the simulated abnormal state signal is generated. The encoding mapping distance between the simulated abnormal state signal and the original signal is used as the damage index. The proposed method can apply the data-driven monitoring technology to the structure without damage data set, and develop a deep learning imaging technology with more generalization and less dependence on prior knowledge. It also promotes the learning ability of the monitoring model by simulating the damage sample, recognizes the damage state when the sample is missing features and achieves the multipath damage imaging in plates. In the future work, the author will also pay more attention to the model transfer under different monitoring environments to further improve the generalization performance and practicability of the deep learning structure monitoring model.