Keywords

1 Introduction

High-resolution range profile (HRRP) is the coherent summations of complex echoes from target scatters in each range cell [1]. It represents the projection of target scattering centers on radar line of sight. Since HRRP is easy to be obtained and contains much structure information of targets, it has attracted intensive attentions from the radar automatic target recognition (RATR) community [2, 3].

After a long term of research, the US Army Missile Command thought that target recognition based on HRRPs is the most promising identification method in the existing technology [4]. This viewpoint made a rapid development of this method. Zyweck and Bogner [5] proposed a method to solve the problem of aspect sensitivity of HRRP. It uses multi-template matching to carry out target recognition with HRRPs of different aspect. Du et al. [6] proposed a method for calculating the Euclidean distance in higher-order spectra feature space. It avoids calculating the higher-order spectra directly. Botha proposed an identification method based on neural network in 1996 [7]. He thought that, compared with other recognition mehtods, neural networks can extract more unknown features from HRRP and provide a very high target correct recognition rate if the amount of training data is large enough. However, the neural networks he used are simple perceptrons. More complex and useful neural networks are not applied because of the lack of theory or hardware performance.

Now, with the development of hardware technology, deep neural networks have applied in various fields. In 2012, Geoffrey Hinton [8] won the competition of ImageNet image recognition by using deep neural network. He reduced the error rate of “top 5” to 15.315%. The same year, Graves A. proposed improved long short-term memory (LSTM) networks [9], which can effectively process the timing issues. For HRRP, deep neural networks can extract more features from it for identification and LSTM networks can process the dependence between sampling points. We believe the target recognition ratio based on HRRP can be improved via deep neural networks.

However, the information of target provided by HRRP is limited. Although the deep neural networks can extract more features, the correlation of these features is too high to improve the recognition rate further. We need to use more other features to do that. Nowadays, the reconnaissance system is mostly composed of multi-source sensor networks. The whole system can work effectively with the fused information obtained by different sensors (visible light sensors, infrared sensors, and radar sensors). It is believed that the target recognition ratio can be improved via the fusion feature of different sensors.

In this paper, we analyze the performance of different recognition methods based on HRRP. The result shows the superiority of neural networks to Bayesian classifier and SVM in classification. Meanwhile, we apply multi-source feature fusion to target recognition based on neural networks. We establish a multilayer perceptron to carry out target recognition with fusion feature obtained by color images, grayscale images, and HRRPs. The results show that, in sunny day, the target recognition ratio using fusion feature is higher than using HRRP only.

The structure of this paper is organized as follows. In Sect. 2, we propose the methods of getting fusion feature of images and HRRPs. Section 3 introduces the neural networks. In Sect. 4, we conduct the training of neural network and the simulation of target recognition. Finally, Sect. 5 summarizes our investigation.

2 Feature Fusion

First of all, we need to extract some certain features from the images and HRRP of target to form the fusion feature.

2.1 HRRP

HRRP cannot be used for target recognition or feature extraction directly. We must do some processing about it. The amplitude of HRRP obtained by radar sensor is defined as:

$$\begin{aligned} R\;=\;[r(1),\;r(2), \ldots ,r(N)]\end{aligned}$$
(1)

N is the number of the distance units.

The processing of HRRP is power transformation and normalization. Botha et al. [7] think these processing can improve the target recognition rate. Power transformation is defined as:

$$\begin{aligned} H(n)\;=\;r(n)^\alpha \quad 0<\alpha <1 \end{aligned}$$
(2)

H(n) is the amplitude of HRRP’s sample after power operation.

The next step is normalization. After that, we can calculate the length and the change rate according to [10]. Then, we can obtain a feature vector:

$$\begin{aligned} F_{radar}\;=\;[L,\;t]\end{aligned}$$
(3)

L is the length of target, and t is the change rate.

2.2 Image Feature Extraction

We can obtain color images and grayscale images of target from visible light sensor and infrared sensor, respectively. It is difficult to use the entire images for fusion. So, we need to extract some typical features from these images.

For color image, we extract its color moments [11] for target recognition. The mathematical definitions of three color moments are:

$$\begin{aligned} u_i\;=\;\frac{1}{N}\sum _{j=1}^Np_{i,j} \end{aligned}$$
(4)
$$\begin{aligned} \sigma _i = \left( \frac{1}{N}\sum _{j=1}^N(p_{i,j}-u_i)^2\right) ^\frac{1}{2} \end{aligned}$$
(5)
$$\begin{aligned} s_i = \left( \frac{1}{N}\sum _{j=1}^N(p_{i,j}-u_i)^3\right) ^\frac{1}{3} \end{aligned}$$
(6)

\(p_{i,j}\) represents the probability of the appearance of pixel with a gray level of j in color channel i. N represents the number of pixels in the image.

According to three color channels of RGB image, we can obtain a feature vector of color moments:

$$\begin{aligned} F_{color}=[u_R,\;\sigma _R,\;s_R,\;\mu _G,\;\sigma _G,\;s_G,\;\mu _B,\;\sigma _B,\;s_B]\end{aligned}$$
(7)

For grayscale image, we can extract several features based on gray histogram as follows:

$$\begin{aligned} m\;=\;\sum _{i=0}^{L-1}z_ip(z_i) \end{aligned}$$
(8)
$$\begin{aligned} \sigma \;=\sqrt{\;\sum _{i=0}^{L-1}(z_i-m)^2p(z_i)} \end{aligned}$$
(9)
$$\begin{aligned} e\;=\;-\sum _{i=0}^{L-1}p(z_i)\log _2\left( p(z_i)\right) \end{aligned}$$
(10)

m, \(\sigma \), and e represent the average brightness of texture, the average contrast of texture, and the entropy of gray histogram, respectively. L is the number of the gradation, \(z_i\) represents the \(i\) \(th\) gradation and \(p(z_i)\) represents the probability of the appearance of pixel with a gradation of \(z_i\).

Then, we can obtain another feature vector for fusion:

$$\begin{aligned} F_{gray}=[m,\;\sigma ,\;e]\end{aligned}$$
(11)

Finally, we can get the fusion vector:

$$\begin{aligned} F_{fusion}=[F_{gray},\;F_{radar},\;F_{color}]\end{aligned}$$
(12)

3 Neural Network

After getting the features, we begin to introduce the neural network. Here, we only briefly introduce two types of neural networks used in the research.

3.1 Multilayer Perceptron

Multilayer perceptron is the most common neural network. It is composed of input layer, hidden layers, and output layer. The structure of multilayer perceptron is shown in Fig. 1.

Fig. 1
figure 1

The structure of multilayers perceptron

Full size image

For multilayer perceptron, we make the entire HRRP or feature vectors as input data. Output is set as the types of targets. We can change the number of features to be extracted by changing the number of hidden layers or the nodes in each layer.

3.2 LSTM Network

LSTM network is a variant of recurrent neural network. It can be used for learning long-term dependence information [9]. Each LSTM unit is composed of cell, input gate, output gate, and forget gate. The structure is shown in Fig. 2.

Fig. 2
figure 2

The structure of LSTM networks

Full size image

In LSTM unit, the first step is to compute what information should be thrown away from the cell state according to the output of last state and current input. The next step is to compute the new information which should be stored in the cell state. After that, updating the old cell state. Finally, computing the output based on current input, current cell state, and the output of last state.

HRRP is a variant of radar echo. And it can be considered as a time series vector. We believe it is reasonable to use LSTM networks to conduct recognition based on HRRP.

4 Simulation

The echo data in the simulation obtained by the radar simulation software BSS. In this research, F-15, Tu-16, and AH-64 are selected as targets to be recognized. We get 300 sets of HRRPs, color images, and grayscale images for each kind of aircraft. The attitude angle is transformed from 3 to 30\(^\circ \) with the step size of 3\(^\circ \). The number of sampling points of each HRRP is 320.

After processing the data, we can acquire HRRP and \(F_{fusion}\) of targets. Then, the rest work is divided into two parts:

  1. 1.

    Performance analysis of different recognition methods based on HRRP (simple perceptron, multilayer perceptron, LSTM networks, Bayesian classifier, and SVM).

  2. 2.

    Performance analysis of recognition method based on fusion feature.

4.1 Performance Analysis of Different Recognition Method

This part is to analyze the performance of five different recognition methods. First, we build three different neural networks (LSTM networks, multilayer perceptron, and simple perceptron). The parameters of them are shown in Tables 1, 2, and 3.

Table 1 Parameters of LSTM networks
Full size table
Table 2 Parameters of multilayer perceptron
Full size table
Table 3 Parameters of simple perceptron
Full size table

In Tables 2 and 3, \(W_i\) is the shape of weight matrix of hidden layer i. And \(W_{out}\) is the shape of weight matrix of output layer.

HRRPs are used as training data of three neural networks. For LSTM networks, We need to reshape the HRRP to (16,20) to fit the input size and time step. The number of training times for all neural networks is 100,000, batch size is 20, the loss function is cross entropy and the optimizer is Adam optimizer.

Then, we add a Bayesian classifier [10] and a SVM to compare the performance of different recognition methods based on HRRP.

In testing, each time we change the SNR, we will conduct 600 tests and calculate the average recognition rate of three aircrafts. The curves of SNR and recognition rate are shown in Fig. 3.

Fig. 3
figure 3

The recognition rate of different methods based on HRRP

Full size image

Figure 3 shows the performance of five different classifiers based on HRRP. It is obvious that neural networks are superior to other classifiers. Because they can extract more features from HRRP to conduct target recognition. From neural networks, simple perceptron has the worst performance because of the fewer nodes, fewer layers, and fewer features. So, we believe that the performance can be improved by adding the number of layers and nodes. We can see that the recognition rate based on LSTM networks is very close to that of multilayer perceptron. Although LSTM networks have advantages in timing processing, multilayer perceptron have more layers or nodes and can extract more features from the target. So, for target recognition rate, they almost have the same performance.

4.2 Performance Analysis of Recognition Method Based on Fusion

It can be learned from the above that neural networks can provide higher target correct recognition rate than that of SVM and Bayesian classifier. In this part, we establish another multilayer perceptron to conduct target recognition based on fusion vectors \(F_{fusion}\). The parameters are shown in Table 4.

Table 4 Parameters of multilayer perceptron based on fusion feature
Full size table

In testing, we simulate a rainy scene by adding Gaussian blur filter to the parts of images. And we make the rest of the images as test data on sunny day. Then, we can obtain two sets of \(F_{fusion}\) to conduct target recognition. Meanwhile, we add two neural networks based on HRRP (LSTM networks and multilayer perceptron) to analyze the performance. The testing results are shown in Figs. 4 and 5.

Fig. 4
figure 4

The recognition rate of different methods in sunny day

Full size image
Fig. 5
figure 5

The recognition rate of different methods in rainy day

Full size image

From Fig. 4, we can see that the correct recognition rate based on fusion is better than that of HRRP only when the SNR changes from 8 to 13 dB. It is believed that, on sunny day, color images and grayscale image can provide important information of targets. Using fusion feature is conducive to the recognition.

Figure 5 shows the performance of method based on fusion is worse than the others on rain day. Because in this case, the resolution of image is too low to contribute to the recognition. The features extracted from images are useless and the effective features are only the length and the change rate extracted from HRRP. It is difficult to get a high recognition rate based on only two features. So, it is proved that the strategy of recognition should change with the environment.

5 Conclusion

In this paper, we analyzed the performance of different recognition methods based on HRRP. The result shows the superiority of neural networks to SVM and Bayesian classifier. The next, we established a multilayer perceptron to carry out target recognition with fusion feature obtained by color images, grayscale images, and HRRPs. The results show that, in sunny day, when the resolution of the images is high, the target recognition ratio using fusion feature features is higher than using HRRP only.