Keywords

1 Introduction

Steels are the most important materials used in industry, as oil refineries, chemical industry, power engineering industry and petrochemical domains, because of their excellent mechanical properties [1, 2]. Based on their chemical compositions, steels can be categorized into four groups: Carbon, Alloys, Stainless and Tool steels. The steel microstructures have different appearances, influenced by alloying elements, rolling process, cooling rates, heat treatment and further post-treatments [3]. These manufacture processes induce various microstructures, with different micro-constituents such as ferrite, cementite, austenite, pearlite, bainite and martensite. The steel performances is highly depend on the distribution, shape and size of phases in the microstructures [3]. Carbon group is the most important commercial steel alloy; it exhibits a ferrite/ pearlite microstructure. Duplex stainless steels (DSS) are dual phases, comprise equivalent proportions of ferrite (\(\alpha \)) and austenite (\(\gamma \)) and exhibit excellent integration of mechanical and corrosion properties [4]. Austenitic alloys are mono phase with austenitic matrix.

Traditionally, steel microstructures are classified by comparing the microscopy images with reference series. Especially for steel and its complex microstructures, the comparison with reference series is strongly dependent on the expert’s subjective opinion. Furthermore, it is tedious and time consuming task [5].

Recently, computer vision and image processing have great applications in materials science where many works are devoted to automatize some important tasks [5]. According to the study in [6] those works can be mainly subdivided on two groups:

(i) Image processing techniques that are based on the properties of the image itself (such color intensity, shape, texture). The segmentation step is crucial in this approach for further processing such as the morphological parameter computations that helps to quantitatively describing each microstructures. The simplicity, efficiency and accessibility of this approach have made it ideal candidate to be part of many works. For example, image thresholding was used in [7, 8], Region Growing in [9, 10], and Variational models in level set framework in [11, 12]. as the material micrographic images exhibit repeated local patterns, the texture analysis technique is employed as descriptors. In this context, authors in [13] used Local binary descriptor and several combinations of morphological parameters in support vector machine (SVM) to classify the microstructure components (martensite, pearlite and bainite). They concluded that the texture features showed fewer correlations with each other, which is one of the great advantages over the other two parameter groups.

(ii) Learning-based approach that is based on learning a model from the data to be treated. Nowadays, it is widely introduced in micrographic analysis. In [14] authors classified the 13CrMo4-5 steel damages, using the geometrical coefficients resulting from the SEM digital images and their classification methodology uses artificial neural networks (ANN). S. M. Azimi et al. [15] worked on microstructural classification of low Carbone steel by Deep Learning method, with 93.94% classification accuracy. In the study of Vitalii et al. [16], authors have developed an algorithm to automate metallographic metals analysis based on artificial intelligence technologies. Beskopylny et al. [17] applied a method based on non-destructive test, to evaluate the indentation characteristics that correlate with the material properties to classify steel grades. The main contributions of this paper are:

  1. 1.

    The image processing stage describes an algorithm that aims to classify three steel classes based on their micrographic images. To achieve such goal several steps are considered: (i) The images are prepossessed then segmented to extract the MGs, (ii) the MG morphological is described base on its shape features (orientation, circularity, elongation) and phase rate (iii) The Local Oriented Optimal Pattern (LOOP) is used to describe the microstructure texture. (iiii) The extracted features are fed to the Decision Tree (DT) algorithm to classify the input image onto the corresponding grade.

  2. 2.

    Metallic Materials Dtataset we propose, for this study, a new small dataset that contains 90 microstructure images of three steel grades (austenitic, carbon, and duplex stainless). The steel samples performing and image acquisitions (using the light optical microscope Nikon-Eclipse) were done at the Mechanics and Materials Development laboratory of Research Center in Industrial Technologies -CRTI- https://www.crti.dz/. The dataset is available in public domain github platform, Microstructure Images for Metallic Materials at https://github.com/Yamina77/Microst.Images_MetallicMaterialsDataset

2 Material and Dataset

The materials analyzed for this study were acquired from the industry machinery elements. The three types of steel are mainly used to fabricate typical products, as tubes, pipes, plates for pressure vessels, boilers and piping procedures. The different specimens were identified as S1, S2 and S3, to refer to Austenite steel, Carbone steel, and Stainless steel, respectively. The samples were prepared with an established procedure, providing high surface quality. First, specimens with a 20 \(\times \) 20 mm\(^{2}\) square surface were cut from the received plates. The obtained samples were then, prepared for metallography by a conventional polishing, on turning disks with grinding papers of different grit size in six steps (P320, P500, P800, P1000, P1200, and P2400). The last polishing was performed by adding an alumina standard suspension onto the rotating disk to obtain a mirror surfaces. The polished samples were washed with ethanol and dried with air; a subsequent etching with adequate chemical reagent is performed to reveal the different microstructures.

The images acquisition was done by the means of a light optical microscope (Nikon-Eclipse), equipped with objective lenses and camera for maximum optical resolution. The images are saved in .JPG format, with different observation scales (20 \(\mu m\), 50 \(\mu m\), 100 \(\mu m\)), and resolutions (\(640 \times 480\), \(2560 \times 1920\)). In this work, a total of 90 images are used that are regrouped in three folders according to their grade (30 images for carbon steel, 30 images for Stainless steals, and 30 images for Austenite).

3 Proposed Materiel Classification Model

Through the above mentioned discussion in Sect. 1, we propose the framework shown in Fig. 1 where the classification is based on both grain’s morphology features and texture analysis using local binary pattern. Mainly the model includes three steps where the following subsection details each one.

Fig. 1.
figure 1

Micrographic identification based on Morphology and texture in decision tree classifier

Full size image

3.1 Segmentation and Morphological Features Computation

The segmentation is the crucial step in this task. In this work, we have used the Chan-Vese in level set framework optimized via Split Bregman method for fast convergence. The Chan and Vese [18] constructed the energy functional by minimizing the square error between the gray value and the mean value of the pixel points inside and outside the curve.

(1)

where \(i=1, \dots N\) represents the \(i^{th}\) channel of the original image \(u_0\), generally \(N=3\) for color images. \(\lambda ^{in}\) and \(\lambda ^{out}\) are constant vectors that penalize energy inside and outside curve in each channel i.

(2)

The \(c^{in}_i\) and \(c^{out}_i\) are the constant vectors that represent the average intensities inside and outside the curve. They are defined as follows:

(3)

For \(c_{1}\) and \(c_{2}\) fixed, the according Euler-Lagrange equation that allows the evolution of the curve is given by the following Eq. (4)

(4)

To achieve fast convergence of the Chan-Vese model, we have adapt the Split Bregman method for the minimization process. The details of its implementation is done in [19].

The segmented image is then used to calculate a set of grain morphological features that allows describing the structural of microstructure. In our work, we have chosen four morphological features so that each one give a best grain characterization as follows:

  • Phase Fraction that is computed as the percent of the phase surface divided by the total image surface. In the case of austinitic grade this feature is very low since it is a monphase steel.

  • Elongation Elo and Orientation \(\theta \): those features describe well the duplex steel, where phases are longer and flattened than in other steels (austinitic and carbon).

  • Circularity Cir: this feature is used to caracterise the carbon’s phases.

3.2 Texture Analysis

Local binary descriptors have been shown to be effective encoders of repeated local patterns for robust discrimination in several visual recognition tasks [20]. In literature exist a large variety of texture descriptors that are derived from the first and popular one named Local Binary Pattern LBP [20]. In the present work, we use the Local Oriented Optimal Pattern (LOOP) [21] that was introduced to overcomes the disadvantages of classic LBP.

Let \(i_c\) be the image intensity I at pixel \((x_c, y_c)\) and \(i_n\) (\(n = 0, 1, \dots , 7\)) be the intensity of a pixel in the \(3\times 3\) neighborhood of \(i_c\) excluding the center pixel \(i_c =(x_c, y_c)\). Also, the 8 responses of the Kirsch masks noted by \(m_n\) corresponding to pixels with intensity \(i_n\), (\(n=0,\dots , 7\)). Each of these pixels are assigned an exponential \(w_n\) (a digit between 0 and 7) according to the rank of the magnitude of \(m_n\) among the 8 Kirsch mask outputs. Then the LOOP value for the pixel (\(x_c, y_c\)) is given by [21]

$$\begin{aligned} LOOP(x_c,y_c) \, =\, \sum _{n=0}^{7} \, s(i_n - i_c).2^{w_n} \end{aligned}$$
(5)

where

$$\begin{aligned} s(\mathrm {\textbf {x}} ) = \, \left\{ \begin{array}{ccc} 1 &{} if &{} \mathrm {\textbf {x}} \ge 0 \\ 0 &{} {} &{} otherwise \end{array} \right. \end{aligned}$$
(6)

As reported in [21] the LOOP descriptor has many advantages (i) encodes rotation invariance into the main formulation, (ii) negates the empirical assignment of the value of the parameter k in a variety of local binary descriptors (iii) less susceptible to noise than the traditional LBP operator, (iiii) LOOP allows gains in time complexity.

Three images are randomly selected from each grade to be used as reference for it. Let \(H_{aust}\), \(H_{carb}\), and \(H_{duplex}\) be the histograms of the resulting LOOP images for austinitic, carbon and duplex, respectively. Let H be the histogram of the image to be classified. The Mean Square Error MSE is then computed for each grade as follows

(7)

where D is the length of H. Such the image is classed to the grade that corresponds to the minimum average squared distance between the H and reference \(H_i\).

3.3 Microstructure Identification Based on Morphological and LOOP Descriptor in Decision Tree

The framework of the decision tree based method for microstructure identification is shown in the second bloc in Fig. 1. It is proposed through the following analysis. The class of austinitic steels is characterized by a monphase structure thereby the phase rate is small. Consequently the identification of this class is the combination between the austinitic local binary descriptor and fraction phase.

The grains, in duplex class, are horizontally laminated, thereso this class is strongly described by a very weak orientation, non circularity and large elongation shape features such its morphology parameters are set to \([\theta =false, Cir=false, Elo=true]\), incorporate with texture descriptor.

The carbon steel class generally has polygonal grains, such it is classified based on circularity and elongation morphology parameters associated to the LOOP descriptor with Or logic operation.

4 Experimental Evaluations and Discussion

In this section, we present comparisons of the steel micrographic images segmentation using the proposed morphological-local binary pattern model against some widely used methods. Furthermore, the evaluation is performed to show the advantages of combining the morphological and local texture computation to increase the classification accuracy. In addition, all experiments are done on the dataset described in Sect. 2.

The algorithm is implemented using Matlab2018a, on a computer equipped with \(CPU~Intel(R)~Core~(TM)~i7-10700F CPU @ 2.90GHz\) and \(16,0 \,Go\) of RAM. Furthermore, for all experiments the level set is automatically initialized to a rectangle binary function, where \(\varPhi =+1\) inside curve and \(\varPhi =0\) outside (as shown in the first raw of Fig. 2(a)).

Fig. 2.
figure 2

Demonstration example where input image, in first row, refers to duplex stainless micostructure. The second row presents the segmentation bloc: from the left to right initial zero level set function (yellow line), the convergence of this function, and the segmented image. The third row shows the LOOP image and corresponding histogram.

Full size image

The experiment in Fig. 2 shows the first bloc outcomes for an example of Duplex stainless image displayed in first raw and column. The curve initialization is represented, via yellow solid line on the image domain, in the second column and the corresponding level set on last column. The second row represents, from the left to the right the convergence of curve (solid yellow line), the level set at the convergence and the binary segmented image, respectively. The last row shows the LOOP image and its histogram. The algorithm performs well the segmentation where the two phase in the microstructure are correctly extracted. In addition, The LOOP image exhibits exceedingly the pattern in the image.

4.1 Segmentation Performance Compared with the Other Methods

The morphological parameters are directly computed from the segmented image, thereby it is primordial to use algorithm able to deal with high performance on microstructures. In this subsection, we compare the segmentation results using proposed method with some widely used methods. The first row of Fig. 3 displays, an arbitrary selected images from each considered material grades. The second, third and fourth rows show the segmentation results for three class of methods, as follows:

  • Thresholding methods represented by Ridler algorithm and Kapur algorithm;

  • Clustering methods represented by kmeans and fuzzy kmeans (FCM);

  • Variationnal (or active contours) methods represented by Local Binary Fitting Energy LBF model [22] and the Picewise Constant Chan-Vese (PC) model optimized using Split Bregman method (adopted in our work).

As the Austenitic grade is a monphase steel, the segmented image should be one region. This is obtained by Ridler, kmeans and PC algorithms. However, Kapur, FCM and LBF have extracted two regions. The carbon is a biphase steel, its microstructure is characterized by small two regions. A good performance are obtained using Kapur, FCM and PC methods. The PC model out performs all others methods in the case of the Duplex steel grade, where the two regions are correctly extracted.

Fig. 3.
figure 3

Testing of the segmentation outcomes on three images for the three grades. The first row: the original images. The \(2^{nd}\) to \(4^th\) rows represents obtained binary segmentation using Thresholding-based methods, Clustering-based methods and Variational-based methods.

Full size image

4.2 Classification Performance Evaluation

The classification in our work is performed using the classic decision tree algorithm, where there is no learning stage (see Sect. 3). To evaluate the classification rate of proposed algorithm with and without combination of features, we use the accuracy rate formulation done in Eq. 8. The obtained values are displayed on Table 1. These results shows clearly the importance of the combination between morphological and texture features to classify steels microstructures, specially for the carbon and duplex grades where accuracy is 100% and 93.33%, respectively.

(8)
Table 1. Classification accuracies obtained for each grade in the three cases.
Full size table

5 Conclusion

The proposed image analysis algorithm involves image segmentation, morphological feature extraction, and local texture pattern synthesis. The obtained set of parameters are incorporated in a decision tree algorithm to classify the input image to its corresponding steel grade (Carbon, Austenitic and Duplex stainless). The developed method allows to determine the grade and steel quantitative parameters (ratio Ferrite/Perlite, grain amount, etc.), that are very useful for metallurgy.

The proposed method was shown a higher accuracy using the combination of morphological and texture features compared to using them separately. However, the study was done on small dataset thereby it can not be generalized. In addition, a comparative study with the state-of-art methods should be done. Both of these points are the goals of further work.