Preliminary Study on the Method of Generating Infrared Imaging Guidance Simulation Image Based on Deep Learning Algorithm

Abstract

With the development of infrared imaging guidance technology, how to quickly and efficiently test the abilities of flight control, mid-end guidance, target identification, anti-interference strategy and precise strike of guided weapon in various complex background and interference becomes one of the issues that need to address. This paper summarized the main infrared scene simulation image generation technology, and analyzed its advantages and disadvantages. In response to the problems of traditional methods, deep learning is introduced to solve the problems existed in the simulation systems.

1 Introduction

Precision-guided weapons have shown excellent performance in all the wars since the Gulf War, and are the main weapons of information warfare under the future high technology conditions. In recent years, accurate strike has become a very promising subject of military research among various countries. Countries are competing to develop and improve the guidance mode and precision of guided weapons to meet the needs of future wars [1].

With the rapid development of infrared imaging technology, more and more infrared imaging equipment are put into use. In the development process of infrared imaging equipment, a large number of infrared scenes need to be used to verify the equipment. Due to the limitation of conditions, the acquisition of some real infrared scenes is difficult to complete, as an alternative, using computer-generated infrared scenes is an effective way to solve this problem. Compared with obtaining real infrared scenes, using computer generation can not only save a lot of unnecessary efforts and resources, but also produce infrared images not available under harsh natural and geographical conditions.

2 Research on Infrared Scene Simulation Image Generation Technology

For the past few decades, Infrared image generation system commonly adopt Modeling and Database and Image Generator mechanism. With advanced computer processing techniques, especially mature computer image generation technology, to process the geometric visibility, perspective, color calculation. The corresponding infrared radiation calculation is performed simultaneously. The infrared feature is mapped into a gray level, and the output of the infrared view image is completed.

A series of mature commercial software has been developed abroad, such as DIRSIG, SHIPIR, SE-RAY-IR, MuSES, Vega and SE-Workbench-IR, etc. [2]. Among them, SE-Workbench-IR software has been widely used in military research in Britain, France and Germany as well as some Asian countries. Infrared image generation software can be run on an SGI high-grade graphics workstation including texture material mapping and atmospheric modeling, etc. With the powerful function of SGI advanced graphics hardware subsystem, infrared images of battlefield are generated simultaneously. Therefore, it can be seen that mechanism effectively solves the real-time and effectiveness problems and meets the requirements of infrared vision simulation.

2.1 Traditional Infrared Imaging Simulation Process

The environment of the simulated infrared guided missile involves calculations of the target background temperature, infrared radiation, atmospheric radiation transmission, infrared human interference and other factors. Connections between the links as shown in Fig. 1, infrared image reflects the target background temperature field and infrared radiation distribution, target background temperature is affected by terrain, weather, atmosphere, and other factors [3]. The target background radiation depends on its own temperature, also by the surface thermodynamics characteristics, such as ground material impassivity and reflectivity. When the radiation energy leaving the target background and passes through the atmosphere to reach the missile, the attenuation effect of the atmosphere on the target background radiation and the superposition effect of the atmospheric thermal radiation should also be considered [4].

Fig. 1.

Infrared imaging simulation process

2.2 The Disadvantages of Traditional Imaging Methods

In recent years, the requirements of equipment test and identification are very strict. Due to the expense and other limitations, it is difficult to obtain sufficient test data. It is necessary to use simulation test technology to verify the complex battlefield environment, boundary limits and actual combat assessment of Precision-guided weapons, so as to find out the combat performance and efficiency of equipment [5]. However, the traditional infrared scene generation method still has many problems, such as complex procedures, complicated modeling process, many calculation parameters. In order to improve the speed of calculations, objects with complex and irregular geometry are usually simplified in modeling, and a simplified model is established to solve it, which leads to distortion of simulation results, low background image resolution, and unable to effectively generate highly correlated infrared images with environmental characteristics. The existing infrared imaging guidance simulation system has a limited number of generated targets, interference and background images, low fidelity, incomplete coverage and adequacy, which can not meet the requirements of performance verification of the imaging guidance system. In addition, with the continuous improvement of the detector resolution, the original low-resolution real samples cannot meet the training requirements of the simulation and recognition algorithm. The traditional image stretching enhancement will cause image blur and mosaic, serious areas and blocks will not be able to truly represent the optical properties, in the enhancement process is filling but destroy the details. Therefore, more efficient and accurate infrared scene generation methods need to be explored to improve the authenticity and diversity of infrared imaging simulation.

3 Infrared Image Generation Method Based on Deep Learning Algorithm

Deep learning is a model, which uses iterative approximation to replace the direct method, and is extremely adapted to the above problems. At present, image processing by using Generative Adversarial Networks (GAN) in deep learning algorithm is the main research focus. In the field of image processing, GAN, as a new generation model, is mainly used for the modeling and generation of real data distribution, including image classification, generation, transformation, prediction and other processing. GAN can rely on the adversarial generation mechanism to generate a large number of data samples to compensate for the insufficient training data in machine learning.

3.1 Generative Adversarial Networks

Inspired by the idea of two-man zero-sum game in game theory, GAN mainly consists of two parts: generator and discriminator. The purpose of the generator is to generate new samples with the sample data to fool the discriminator; and to identify whether the input image comes from the real data distribution. The two confront each other and constantly adjust the parameters. The ultimate goal is to make the discriminator cannot successfully distinguish whether the output of the generator is true, so that the output of the generator reaches the degree of authenticity, thus giving the generator the ability to generate diverse and high truth samples [6].

The overall structure of the generated infrared image model is shown in Fig. 2, and the GAN consists of two parts: generator and discriminator. The function of generator G is to learn the distribution of sample data, map an input random noise z into an image, generate new samples with the same distribution as the sample data to deceive the discriminator. The discriminator D distinguishes whether the input image is a real image or a pseudo image generated by the generator, and identify whether the input image comes from the real data distribution. The optimization process of the whole network is a binary mini-max game problem, that is, the training target of the generated network generates more real images, and the training goal of the identification network is to identify the generated image as pseudo images as far as possible. Through adversarial training, the performance of the generator and discriminator keeps improving, eventually reaching Nash equilibrium.

Fig. 2.

Generates the overall structure of the IR image model

Generative adversarial network based on deep neural network is one of the best methods to generate image quality in the generative model. According to the different input of the generator, image generation based on generative adversarial network is mainly divided into two methods: image generation based on random vector and image generation based on image conversion. Image transformation based on GAN mainly includes four parts: design of objective function, design of generator architecture, design of discriminator architecture and design of training algorithm.

3.2 Generate an Adversarial Network Training Process

The ultimate goal of GAN is to minimize the distance between the generated data distribution and the real data distribution, and there are two ways to measure the distance, namely, the f-divergence and the integral probability measure. An adversarial mechanism is used to fix the discriminator parameters when trained. When training the discriminator, fix the generator parameters. The training process iterates alternately, and both the generator and the discriminator train to maximize each other’s errors. Eventually, the generator generates a distribution closer to the real sample data, and the discriminator will be better discriminatory. In general, the generator G has learned the true data distribution when the identification power of D increases to a certain extent, and the data source cannot be correctly identified [7]. The training process of adversarial generative networks is a process of alternately optimizing the generator and the discriminator. First, fix the generator G, training the discriminator D. At this point, the network is equivalent to a dichotomy model, and the model loss function is the cross-entropy, defined as follows:

$$\begin{aligned} \textrm{V}(\textrm{G}, \quad \textrm{D})=E_{x \sim p_{\text{ data } }}[\log D(x)]+E_{z \sim p_{G}}[\log (1-D(z))] \end{aligned}$$

(1)

Formula: x is the sampling of the input data distribution. z is the sampling of a given data distribution. E (.) is the expectation. Through the loss function, unlike the general binary classification problem, the input data of the discriminator comes from the real data and the generated data. Given the generator G, optimize D, the optimization objective is to maximize the formula. On a continuous space, the above loss function is defined as:

$$\begin{aligned} \textrm{V}(\textrm{G}, \quad \textrm{D})=E_{x \sim p_{\text{ data } }}+E_{z \sim p_{G}} \end{aligned}$$

(2)

$$\begin{aligned} \begin{aligned}&E_{x \sim p_{d a t a}}=\int _{x} P_{d a t a}(x) \log D(x) d x \\&E_{z \sim p_{G}}=\int _{z} P_{G}(z) \log (1-D(z)) d z \end{aligned} \end{aligned}$$

(3)

The optimal solution of the generator G is:

$$\begin{aligned} D^{*}(x)=\frac{P_{\text{ data } }(x)}{P_{\text{ data } }(x)+P_{G}(z)} \end{aligned}$$

(4)

Since the generator and the discriminator train to maximize each other’s errors, D (x) should approach to 1 when the discriminator input comes from real data, and D (G (z)) should approach to 0 when the discriminator input comes from the generator. At this point, the goal of G is to bring D (G (x)) closer to 1. Thus, the loss function of the generator is:

$$\begin{aligned} \begin{gathered} G^{*}=\arg \min _{G} \max _{D} V(G, D) \\ P_{G}(z)=P_{d a t a}(x) \end{gathered} \end{aligned}$$

(5)

The optimization procedure of GAN can be described as a minimal-maximum optimization problem. At that time, G achieved the optimal value. V (G, D) measures the difference in the probability distribution between the generated data and the real data. The training goal of the discriminator is to maximize this difference, and the discriminator can maximize the identification of the data from the real data or the generated data. The generator training objective is to minimize this maximum difference, so that the probability distribution of the generated data is as consistent as possible with the probability distribution of the real data. Generative adversarial network has been widely used in image generation, style conversion, noise reduction repair and other fields due to its good automatic feature extraction and image reconstruction ability. Adopting generative adversarial networks can enable the generation of high-fidelity and high-resolution samples.

3.3 Image Generation Method Based on Generative Adversarial Networks

Generative adversarial network is a popular direction of deep learning in recent years, and this algorithm has very good results in generating images. Generative adversarial network is a large class of frameworks, composed of generator also called generative model, and discriminator also called discriminate model, as shown in Fig. 3. The generative model is responsible for generating the required image from a random vector, and the discriminating model judges the real image and the generated image [8]. The generator and the discriminator choose the constitutional neural network or other functional network structure according to the task requirements. As the generative model and the discriminate model play games with each other, the image generated by the generator is infinitely close to the real image, reaching the degree of authenticity.

Putting a picture or feature vector into high dimensional space can be regarded as a point. Images of the same type have the characteristics of similar characteristics, which have a close distance in high dimensional space and roughly obey a certain distribution. The generative model then receives a random vector, and maps the random vector to the image space through the generator to generate the image satisfying this distribution. Through the discriminator comparing the real image and the generated image, it determines whether the image generated by the generator is close to the distribution of the real sample data.

The training process of the network starts from the generator to generate a random image, and sends the random image generated by the generator into the discriminator together to adjust the discriminator weight parameters, so that the discriminator can well distinguish between the generated image and the real image. Later, the parameters of the discriminator are fixed, and only the generator is trained, so that the images generated by the generator cannot be identified by the current discriminator, so the above steps are repeated. In this process, the performance of the generator and the discriminator will improve alternately, and finally the image generated by the generator will approximate the real image (Fig.   4).

Fig. 3.

Infrared imaging simulation process

Fig. 4.

Diagram of the discriminator working principle

4 Summary

By applying deep learning algorithm to infrared imaging guidance simulation image generation, can use generated network will expand a small number of real sample data to high fidelity large sample data, the low resolution image generate high resolution image, and can add different interference in the original image to generate new images such as different functions. By introducing deep learning algorithm into infrared imaging technology, it can effectively solve the problems of insufficient sample size and low fidelity of combat environment scenes of target and background in the simulation system of the existing system.