Metallographic Grade Recognition and Data Analysis Based on 6G Industrial Internet

Abstract

With the development of modern image processing technology, the introduction of image processing and analysis technology into metallographic microstructure analysis has become a key research direction. Through the effective analysis and measurement of metal microstructure and structure, and even the prediction, application and design of material properties, the quantitative analysis of gold phase can be realized. Using machine learning algorithm can partially replace the task of manual rating, and make up for the shortcomings of high intensity and poor repeatability of human detection fees. In this paper, in order to solve the problems such as high requirements for identifying grain size and large amount of retained information in metallographic atlas, we use up sampling and convolution operations to restore and preprocess the image, build a convolution neural network, predict and segment each pixel separately, and determine the maximum pooling and average pooling proportion in the metallographic atlas sampling process, so as to retain more feature information in the process of bottom feature extraction. Aiming at the problems of resource isolation, high transmission delay, low data fusion efficiency and low analysis accuracy of metallographic atlas in the process of industrial Internet data analysis, a federated learning-driven industrial Internet based on 6G enabled metallographic atlas data collaboration, fusion and analysis method is proposed. Based on the cross layer aggregation high-precision data analysis method of integrated learning, the aggregation strategy of synchronization and hierarchical link between layers within the layer is constructed to realize the metallographic map recognition and data analysis system, and improve the accuracy of intelligent decision-making of industrial Internet business based on 6G.

1 Introduction

1.1 A Subsection Sample

Metal has always played a very important role in human evolution and played a huge role in social production and life. Metal materials such as steel are widely used in construction, machinery, transportation, home appliances, shipbuilding and other industries. The strength, toughness, plasticity and other mechanical properties of metal materials have a great impact on the actual industrial and civil use. Generally, we hope that metal materials have stronger strength and better toughness. It is generally believed that there are many factors affecting the mechanical properties of metal materials, one of which is the grain size of metal. Generally speaking, the smaller the grain size, the higher the strength, plasticity and toughness of the metal material, and the better the mechanical properties of the material. There are many ways to express grain size, which can be expressed in terms of length, area, volume or grain size index. Common methods include average diameter, average intercept, nominal grain diameter, grain Fritt diameter, average grain section area, number of grains per unit volume and grain level index G [1].

The evaluation of grain size is generally based on the comparison method, which relies on experienced staff to observe the grain and grain boundary line at the appropriate magnification through a professional metallographic microscope and compare it with the national standard rating chart. Such rating methods not only require the inspector to undergo more strict training, but also the inspection time cannot be controlled, and due to the personal subjectivity of the inspector, The rating results are also uncertain.

With the development of modern image processing technology, the introduction of image processing and analysis technology into metallographic microstructure analysis has immediately become a key research topic. Through effective analysis and measurement of metal microstructure and structure, and even prediction, application and design of material properties, quantitative metallographic analysis can be achieved. In the direction of grain size grading technology, a large number of image processing software algorithms have also emerged. Through the combination of image processing and feature engineering, machine learning algorithms can partially replace the task of manual grading to a certain extent, which can well make up for the shortcomings of time-consuming and laborious human detection and poor repeatability. For example, the classification algorithm in machine learning can achieve fast and accurate classification for different types of objects, so it can be better applied to the evaluation of steel metallographic grade. In recent years, the field of metallographic analysis has also gradually transited to the use of digital image processing technology, especially in the fields that can not be directly determined by human experience and micro fields, with the help of machine vision, it can be better and more convenient to detect.

Some scholars at home and abroad regard grain boundaries as edges, and then use edge detection operators such as Canny operator and Sobel operator to improve the division of grain boundaries. S. Journaux et al. used Canny operator to find wave crest line extraction filter, and used wavelet transform to construct directional filter, achieving the expected results [2]. Peregrina-Barreto et al. [3] are using median filtering and threshold for edge detection and image segmentation. Banerjee et al. [4] put forward an automatic system based on image processing to accurately extract the closed contour of grains with the edge as the guide. G. Dom í nguez-Rodr í guez et al. recognized edges by using Sobel operator, normalized edges by different Gaussian filters (intensity, roughness or both), and discretized edges by using threshold. The image recognition accuracy was improved by 8% [5]. For the tasks of silicon particle segmentation and grain boundary detection, Ming Chun Li and others designed a multi-task learning network based on contour detection (RCF), and fine-tuned the RCF grain boundary detection results by using the combination of feature matching loss to generate the confrontation network, so that the segmentation accuracy and boundary accuracy have reached the advanced level [6].

At the same time, some scholars use mathematical morphology algorithm in digital image processing to solve the problem of grain boundary extraction. Yinghui Zhang et al. analyzed the samples with different grain boundary microstructures in BFe10–1–1 Cu–Ni alloy, studied the connectivity of high-energy random grain boundaries, and obtained the conclusion that the combination of grain boundary microstructure and quantitative material structure-performance relationship model can better judge the grain boundary [7]. Ma et al. [8] proposed a propagation grain boundary detection algorithm based on 3D information as domain knowledge to solve the problem of grain boundary blurring, and used a local propagation method based on overlapping block strategy to solve the problem of grain boundary missing. Roberto Perera et al. proposed an optimized machine learning framework to independently and efficiently characterize pores, particles, grains and grain boundaries from a given microstructure image. Compared with traditional methods, the proposed framework has significantly improved the analysis time [9].

Segmentation methods in image processing technology have been highly concerned by scientific researchers. Kotas et al. [10] proposed to use spectral and threshold methods to segment metallographic images. Onchis et al. proposed an image segmentation method based on FCM algorithm. Metallographic image segmentation experiments show that the FCM fuzzy clustering algorithm has good accuracy and convergence speed in image segmentation [11]. In recent years, new segmentation methods have emerged continuously. Nie Kang et al. proposed a metallographic image segmentation method based on improved region item CV model [12] by replacing the region item of energy function in the traditional CV model with the reciprocal cross-entropy threshold selection criterion due to the disadvantage that the traditional CV model cannot accurately segment images with large gray changes and has many iterations. Xu Zhenying et al. used marker-based watershed algorithm to pre-segment the metallographic image, and then used the improved mean shift algorithm to automatically extract the grain boundary. A high-speed and high-precision grain boundary extraction algorithm based on region separation was presented [13]. Bulgarevich et al. [14] used a random forest statistics algorithm based on machine learning technology to segment the three-phase steel under the optical microscope, accurately segmenting the ferrite, pearl and bainite in the metallographic image. Thimm et al. [7] proposed a metallographic image segmentation method combining optical measurement using digital image correlation (DIC) technology. Wittwer et al. [15] proposed an image segmentation method based on directional reflectance microscopy, using multi-angle optical microscopy technology to achieve full automatic and reliable particulate segmentation of polycrystalline surfaces. R. Podor et al. calculated and constructed a segmented image of grain boundary from an oblique image set and developed a semi-automatic method for segmenting grain boundary, SEraMic algorithm [16].

Most of the metallographic microstructures also present various shapes and complex textures in the images, which seriously impede the accurate segmentation of the components. Han et al. [2] proposed a hybrid algorithm based on a Gaussian filter and an average shift method to segment and classify metallographic images with complex textures. Lai et al. accurately segmented the metallographic images of titanium and ceramics using machine learning and complex network methods. On the one hand, the image of titanium alloy is segmented based on pixel-level classification by feature extraction and machine learning algorithm. On the other hand, the ceramic image is segmented by complex network theory [17]. However, the metallographic image still has some drawbacks, such as multi-noise, low contrast and difficult to segment. In order to solve this problem, Dutta et al. [18] proposed a new method to remove noise from the microscopic image, which can better segment the ferrite-martensite metallographic image. Luo Xunqin et al. performed a basic histogram processing on the melt metallography for the features of blurred boundary of the metallographic image, confused internal grains, irregular distribution of grain morphology and high noise of the cutting surface image, and proposed an improved watershed algorithm and HOG algorithm for image segmentation and internal feature information extraction [19]. Mohamed Abbas Hedjazi et al. proposed an image repair algorithm for multi-objective generative countermeasure network, which uses LBP-based loss function to minimize the difference between the generated texture and the real texture of the ground, and enhances the texture details to improve performance and increase efficiency [10]. In order to achieve accurate image segmentation, Shao et al. [20] proposed an improved adaptive weighted mean filter algorithm to effectively eliminate the influence of image noise on the metallographic image segmentation.

In this paper, in order to solve the problems such as high requirements for identifying grain size and large amount of retained information in metallographic atlas, we use up sampling and convolution operations to restore and preprocess the image, build a convolution neural network, predict and segment each pixel separately, and determine the maximum pooling and average pooling proportion in the metallographic atlas sampling process, so as to retain more feature information in the process of bottom feature extraction. Aiming at the problems of resource isolation, high transmission delay, low data fusion efficiency and low analysis accuracy of metallographic atlas in the process of industrial Internet data analysis, a federated learning-driven industrial Internet enabled metallographic atlas data collaboration, fusion and analysis method is proposed. Based on the cross layer aggregation high-precision data analysis method of integrated learning, the aggregation strategy of synchronization and hierarchical link between layers within the layer is constructed to realize the metallographic map recognition and data analysis system, and improve the accuracy of intelligent decision-making of industrial Internet business.

2 Metallographic Pattern Recognition and Data Analysis System

We use convolution network to identify metallographic maps with high accuracy, and use the federal learning driver to complete the study of metallographic data collaboration, fusion and analysis methods, to achieve the development of metallographic map recognition and data analysis system. Solutions to these problems are presented. The relationship structure of metallographic pattern recognition and data analysis system is shown in Fig. 1.

Fig. 1

Metallographic map identification and data analysis system

• (1) Extraction of crystal boundary from metallographic image and characterization of characteristic variables

In order to solve the problem of extracting and feeding metallographic data from intelligent manufacturing process enabled by industrial Internet, this paper puts forward a method of data collection of metallographic map based on model building, parameter design and comparative analysis, uses convolution neural network to improve the crystallinity of metallographic map extract, analyses the simplified algorithm of machine vision system, and on the basis of artificial neural network classification algorithm. Establish a set of analysis system devices and image recovery algorithms. The image classification methods based on H moments and SVM, support vector machine and artificial neural network verify the accuracy and efficiency of the proposed methods/algorithms.

• (2) Fusion and analysis of massive multisource heterogeneous metallographic data based on the Federal Learning Framework

Based on the Federal Learning Framework, this paper fuses and analyzes the massive multisource heterogeneous data of metallographic maps to achieve high-efficiency data compression and unified structure of heterogeneous data, so as to give full play to the role of large industrial data. The data preprocessing algorithm for metallographic map recognition distributed computing is studied. An efficient unsupervised learning model is proposed, which does not require a large number of label samples to be trained in advance and relies only on local data characteristics. Redundant data is eliminated by clustering and non-dominant sorting algorithm, local data quality is optimized and classified according to certain characteristics, data value is improved and data volume is solved. On this basis, based on the high-order semantic characteristics of pre-processed data, the efficient fusion algorithm of heterogeneous data is studied, and a deep learning framework based on tensor decomposition theory is proposed. The complexity of multisource heterogeneous data is modeled using tensor decomposition theory, and the unstructured pre-processed data is fused by deep learning to solve the heterogeneous problem caused by multisource data.

• (3) Metallographic pattern recognition and data analysis system simulation verification

In order to solve the problem that non-uniformly distributed microstructures of metal materials require microscopy to measure the field of view with low efficiency and large error, this paper introduces computer image intelligent recognition technology, develops a hardware and software system for storing and analyzing the metallographic data through image information processing algorithms such as historic metallographic map, field metallographic map, metallographic map standard, etc. The microstructural properties of cast-forge-weld hot-processed metal materials in the process of industrial intelligent manufacturing are simulated and verified by experiments.

3 Metallographic Map Recognition Based on Neural Network

The convolution neural network algorithm is developed on the basis of deep neural network. Compared with traditional image classification methods, convolution neural network has the ability of feature extraction and self-learning, and reduces the number of neurons required for the full connection layer by weight sharing, simplifies the network structure and reduces the computational load. Convolution Neural Network eliminates the full connection layer of deep neural network, restores the image to its original size through up-sampling and convolution operations, and predicts each individual pixel for the purpose of semantic segmentation. Based on the characteristics of metallographic maps, a new downsampling method is proposed. The relationship function determines the maximum pooling and average pooling ratio in the downsampling process. This method can protect more edge details, thus retaining more feature information in the process of feature extraction at the bottom level. Also, the logical operation processing is combined with the join operation in the upsampling process. Join processing adds more features to the high-level features, and logical operation processing adds more boundary information to the low-level features. Compared with traditional image classification methods, convolution neural network has the ability of feature extraction and self-learning, and reduces the number of neurons required for the full connection layer by weight sharing, simplifies the network structure, reduces the amount of computation, and significantly improves the accuracy of the metallographic micro-organization map in the identification process. The basic structure of convolution neural network is shown in Fig. 2. There are mainly convolution layer, activation layer, pooling layer and full connection layer.

Fig. 2

Basic structure of convolution neural network

The role of the activation layer is to nonlinearize the neural network, increasing its complexity so that it can learn very complex data models. The idea comes from synapses in the brain, where information is processed through the nerves and synapses are activated, and then transmitted backwards. Activator layer is similar to synapse and its structure is an activation function, among which Sigmoid function, tanh function and ReLU function are the three most common activation functions.

The Sigmoid function is the first and most commonly used activation function, and its formula is:

$$\partial \left( x \right) = \frac{1}{{1 + \exp \left( { - x} \right)}}$$

(1)

The advantage is that the input can be compressed between 0 and 1, so the data is not easy to diverge and derive during transmission, but gradient diffusion is easy to occur during transmission. Therefore, the sigmoid function is generally used in the final output layer, where the values from 0 to 1 are used to represent the probability.The tanh function is a variant of the sigmoid function and its formula is:

$$\partial \left( x \right) = \frac{{e^{x} - e^{ - x} }}{{e^{x} + e^{ - x} }}$$

(2)

The center point of the tanh function is at the origin, which solves the problem of computing complexity of the sigmoid function center point on (0, 0.5). In contrast, the tanh function makes the neural network converge faster and reduces the training time.The ReLU function is the most popular activation function now, and its formula is:

$$\partial \left( x \right) = \max \left( {0,x} \right)$$

(3)

There are some problems with the Sigmoid and Tanh functions: they are very saturable and the gradient changes very slowly in the saturated region, resulting in a slow convergence rate, especially when the number of layers is large, the gradient disappears easily. As a result, many scholars have introduced the ReLU activation function. When x > 0, its gradient is constant, which solves the problem of gradient disappearance, and because the gradient is constant, the amount of calculation is greatly reduced. When x ≤ 0, the ReLU function changes the output of the neuron to zero, sparses the entire network, reduces the overall number of parameters, and reduces the possibility of fitting. Many advantages make the ReLU function mostly used for the remaining active layers except the last ones.

Another important component is the pooling layer. The biggest problem of in-depth learning is that there are too many parameters, too much calculation, and fitting is easy to occur. The introduction of pooling layer can effectively solve this problem. The pooled layer is a kind of non-linear downsampling, mainly Max pooling, average pooling, L2-norm pooling and so on. Pooling is a dimension reduction method in deep neural network and is widely used in classification networks. The common pooling operations are maximum pooling and average pooling. Global maximum pooling can usually be used to reduce dimension. In the semantic segmentation of metallographic images, in order to better protect edge details, this topic combines maximum pooling and average pooling to obtain more edge details. Formula (3) is the total algorithm.

$$\min (\;) = \alpha \max (\;) + \left( {1 - \alpha } \right)average(\;)$$

(4)

Form (3) In order to use the extended weight function as an excitation function, in the original logic function, since the logical function gradient in the interval subunit is large, the excitation effect is large only in this area, but the gray value of the metallographic image changes from 0 to 255, with a large change interval. Therefore, the excitation interval of the logical function is enlarged and expanded, and the logical function is changed according to Formula (4).

$$\alpha = sigmoid\left( {\frac{w * x}{4}} \right)$$

(5)

In formula, x is the pooled area, W is the convolution core, and is an important initialization parameter. This parameter can be adjusted and learned. By adjusting several parameters, the mixed pooling parameters can be obtained in formula (5).

$$\min (\;) = f\left( {\frac{w * x}{4}} \right)\max (\;) + \left( {1 - f\left( {\frac{w * x}{4}} \right)} \right)average(\;)$$

(6)

From chain derivation:

$$\begin{aligned} \frac{\partial E}{{\partial x}} & = \frac{\partial E}{{\partial mix(\;)}} * \frac{\partial mix(\;)}{{\partial x}} \\ & = \delta * \left\{ {f\left( {\frac{w * x}{4}} \right) * \left[ {1 - f\left( {\frac{w * x}{4}} \right)} \right]\left[ {\frac{\max }{i}x_{i} - \frac{1}{N}\sum\limits_{i}^{N} {x_{i} } } \right]} \right\} \\ \end{aligned}$$

(7)

Formula (6) is a backward propagation gradient derived from the loss function. Finally, according to the back propagation algorithm, the weight parameters can be updated, and the best result weight parameters can be obtained to update the whole model parameters. This combination operation can protect the edge details information in the feature extraction process of metallographic features to the maximum extent and is the optimal processing method.

Full-junction layer is the most important structure of deep neural network (DNN), which is based on multilayer perception. Each neuron in the full-junction layer is connected with each neuron in the former and the latter layers. Full-connected layers generally appear in the last few layers of the network. The output extracts the high-dimensional features, which are directly connected to the output layer to classify the high-dimensional features through the output layer. As shown in Fig. 3, the implied layer is the fully connected layer.

Fig. 3

Full connection layer structure

The main function of the full connection layer is to aggregate the local features extracted from the previous layer. This method has too many parameters for the full connection layer, which basically accounts for 70–80% of the network parameters. So now a lot of research has started to design other network structures to replace the full connection layer. A more popular alternative is to use Global Average Pooling (GAP) instead of the full connection layer.

Sampling algorithms on multiscale feature fusion have more advantages on feature map replication preservation and fusion strategies, because adding semantic information in high-level feature maps and adding low-dimensional feature strategies in low-level feature maps can greatly improve the accuracy of semantic segmentation, but single fusion or single logical operation processing are both defective. We use dynamic fusion methods.

In order to increase more feature channels, more high-order semantic information is added to the feature map, which is more helpful for semantic segmentation. In low-order feature maps, the logical addition operation strategy is selected. Because the image size is large, the fusion strategy means more calculation parameters and computational load, and the low-order feature maps are more low-level feature information, which can be better preserved by the logical addition operation. By using the high-order and low-order classification strategies, not only the computational amount and parameters can be reduced, but also the semantic information and low-order feature information can be well preserved, which can improve the semantic segmentation to a certain extent. The overall structure of the improved U-net network is shown in Fig. 4.

Fig. 4

Multi-scale feature fusion grain boundary extraction network

The semantically segmented image is obtained from the collected image after an improved U-net neural network. However, the semantically segmented image also has some drawbacks. In some impurity areas, the local area is considered to be the same as the boundary, and the carbon body area is determined to be semantically as the boundary. However, the real segmented image should theoretically be segmented as the carbon body, as shown in Fig. 5.

Fig. 5

Semantic segmentation image enhancement method based on morphology

Due to the limitations of semantic segmentation, which interferes with the determination of crystal boundary, an image enhancement algorithm based on morphology and logical operation is proposed. After U-net semantic segmentation, deeper regions are obtained by threshold segmentation. The temporary boundary area image is obtained by subtracting deeper regions from the original image. The boundary of the disturbance area is obtained by subtracting the corroded large deep value area from the large deep value area. The total boundary is obtained from the boundary of the interference area plus the temporary boundary area.

We propose a attention mechanism for feature extraction module to focus on learning important information for deep reasoning and capture the remote dependencies of deep reasoning tasks. A multilayer U-Net network is proposed to regularize the cost volume, downsample the cost volume, extract context information at different scales and neighboring pixel information to filter the cost volume, and then generate a final refined estimation depth map by regression. The deep fusion network model is composed of the original image feature extraction, the grain boundary image feature extraction, and the fusion feature grain size discrimination module.

Different algorithms are used to test and verify the recognition of metallographic spectra based on machine vision. Iteration method is combined with image processing to improve the adaptability of the algorithm. The first step is to recover the degeneration through metallographic pattern recognition such as regularization algorithm.

The process of metallographic map degradation can be expressed as (8) (Fig. 6):

$${\text{h}}\left( {a,b} \right) = l\left( {a,b} \right) * g\left( {a,b} \right) + o\left( {a,b} \right)$$

(8)

Fig. 6

Multi-modal data grain size recognition algorithm architecture for deep learning of multi-data fusion

The degraded image represented by Form (8) is the original image, the fuzzy kernel, and the additive noise. Considering the enormous amount of space deconvolution computation, fast Fourier transform is required to convert the image signal to frequency domain for solution (8). The core problem is to perform frequency domain inverse Fourier transform:

$${\text{u}}\left( {a,b} \right) = l^{ - 1} \left( {\frac{{h\left( {\upsilon ,m} \right)}}{{g\left( {\upsilon ,m} \right)}}} \right) + l\left( \frac{h}{g} \right)$$

(9)

U(a, b) is the approximate solution of the frequency domain of l(a, b), h(v, m) is the frequency domain expression of the degenerated image h(a, b), and the frequency domain expression of the fuzzy kernel estimation is shown in the expression (10).

$${\hat{\text{g}}}\left( {{\text{v}},{\text{m}}} \right) = g\left( {\upsilon ,m} \right) + \Delta g\left( {\upsilon ,m} \right) + \prod\limits_{{o_{p} }}^{u = 0,i = 1} {\left( {\upsilon ,m} \right)}$$

(10)

4 Metallographic Map Data Fusion Based on Federal Learning

Considering the problem of time delay and energy consumption caused by massive data transmission and centralized processing of each element in the intelligent manufacturing process of the industrial Internet, the metallographic map intelligent recognition terminal of each element in the industrial Internet can independently complete the local data preprocessing task through distributed deployment and transmission method. Federal learning drives data fusion with high computational efficiency, solves problems such as low data fusion and analysis efficiency, eliminates honorary data by clustering and non-dominant sorting, gives full play to data value, and solves the unstructured problem of metallographic organization multi-source map.

First, the data of one of the metallographic spectrum intelligent recognition terminals is defined as, where n is the number of samples of the data, and D is the sample dimension. This topic decomposes the data into k subspaces $\left\{ {S_{i} } \right\}_{i = 1}^{k} \left( {i = 1, \ldots ,k} \right)$ through subspace clustering, and obtains the data sample coefficient matrix $Z \in R^{n \times n}$ to obtain the affinity between each sample, as shown in Formula (11)

$$Z = \left[ {z_{1} ,z_{2} , \ldots ,z_{n} } \right] = \left( {\begin{array}{*{20}l} 0 \hfill & {z_{12} } \hfill & \cdots \hfill & {z_{1n} } \hfill \\ {z_{21} } \hfill & 0 \hfill & \cdots \hfill & {z_{2n} } \hfill \\ \vdots \hfill & \vdots \hfill & \ddots \hfill & \vdots \hfill \\ {z_{n1} } \hfill & {z_{n2} } \hfill & \cdots \hfill & 0 \hfill \\ \end{array} } \right)$$

(11)

where z_i is the ith column of the coefficient matrix Z, and z_ii = 0 is restricted to avoid invalid solutions.

Secondly, in order to eliminate redundant data, the project stores the nonzero subscript column of each data sample in the coefficient matrix Z in the nested list N, which is different from the traditional definition of nearest neighbor. The sample nearest neighbor is defined as the nearest neighbor sample layer that considers both the number of public neighbors and the size of affinity coefficient, and retains the nonlinear relationship between each other in N through the non dominated sorting algorithm. At the same time, the coefficient matrix between samples after removing redundant data is represented as $Z^{*} \in R^{n \times n}$. Finally, in order to represent the retained strong correlation data samples according to certain characteristics, the project will define the number of clusters through spectral clustering algorithm, and obtain the optimization matrix according to the following formula, so as to obtain the best clustering results and give full play to the data value.

$$W^{*} = \frac{1}{2}\left( {Z^{*} + \left( {Z^{*} } \right)^{T} } \right)$$

(12)

In view of the fact that the existing data fusion methods in the industrial Internet cannot take into account the lack of fusion accuracy and model training time when dealing with the heterogeneous problems caused by multi-source data, the project plans to propose a deep learning scheme based on semantic features and tensor decomposition theory, which will efficiently fuse the pre-processed heterogeneous data Y, and output the distributed fusion data set D, so as to facilitate the subsequent data prediction and resource management based on federated learning.

According to the different characteristics of heterogeneous data, the preprocessed data Y is divided into four types: audio, video, image and text data, and is represented as a sub tensor model T based on its high-level semantic features. For example, the sub tensor of video data can be expressed as:

$$T_{{{\text{video}}}} = R^{{I_{f} \times I_{w} \times I_{o} \times I_{c} }}$$

(13)

I_f, I_w, I_o and I_c represent the time, width, height and three primary colors of the video respectively.

Finally, the higher-order tensor is input into the higher-order back-propagation neural network for training, and the training results are output to the distributed fusion data set D. The higher order back propagation neural network calculates the partial derivative through two key steps: forward propagation and back propagation. In the forward propagation stage, the input and output data of the hidden layer and the output layer are calculated respectively. In the back propagation stage, the partial derivatives of the loss function to the higher-order tensor are reconstructed by calculating the residuals of each neuron, hidden layer and output layer. Among them, the training is planned to adopt layer-by-layer training and random training. The layer-by-layer training from bottom to top limits the ownership value and offset to a certain parameter space to prevent random initialization and reduce the quality factor of each hidden layer. This process can achieve accurate fusion of features of heterogeneous data; Randomly select a sample to update the model parameters each time for random training, which can accelerate the training speed of the model, thus reducing the time of data fusion and improving the fusion efficiency.

5 Simulation Results

Metallographic texture image analysis uses a special camera to observe specific samples, and connects the camera to the metallographic microscope through a specific interface to obtain the metallographic sample composition image, and saves it in the database. Before quantitative analysis, sample preparation and image internal structure acquisition are also very important. When preparing the sample, there are five steps in total. From sample extraction to subsequent processing, the final sample should have: representative organization, no false images, real organization, no wear marks, pitting or water marks, etc. The specific steps are shown in Table 1.

Table 1 Table captions should be placed above the tables

Figure 7 shows the image restored using Wiener filter, and then the restored image is evaluated using the peak signal-to-noise ratio and SSIM respectively. The abscissa is the recovery process using different NSR values as recovery parameters, and the ordinate is the evaluation results of the peak signal-to-noise ratio and SSIM respectively. The specific evaluation value is shown in Fig. 7.

Fig. 7

Image quality evaluation curve after Wiener filter restoration

Figure 8a is the evaluation curve obtained after adding different types of noise (white noise, Gaussian noise, etc.) and using Wiener filtering algorithm to restore the image. Figure 8b is the optimized evaluation curve. The abscissa is the evaluation result after adding different types of noise to the standard original image, and the ordinate is the image quality evaluation result curve obtained after using different algorithms to restore.

Fig. 8

Baboon evaluation curve