1 Introduction

Concurrent and reliable monitoring of flames in fossil-fuel-fired furnace plays an important role in the improved understanding of energy conversion, pollutant formation and subsequent optimization of combustion processes. However, most of the practices for flame monitoring are largely limited to indicating whether the flame is present for safety reasons. In recent years, the trend of using low-quality fuels and fuel blends from a variety of sources has aggravated the problems of flame stability, poor combustion efficiency, high emissions and other operational problems. To meet the stringent standards on energy saving and pollutant emissions, advanced monitoring and characterization of flames have become highly desirable in the power generation industry as well as the research laboratory [1].

The physical characteristics of a flame and fire image can be identified by its geometry, luminance and thermodynamics parameters. In recent years, fire safety engineering research is mostly emphasized on the image processing-based system and increasingly used in flame and fire detection systems [2,3,4,5,6]. Moreover, the image processing-based techniques are found more reliable as compared to the conventional flame detector techniques [7,8,9,10], due to their capability of remote detection of small-area fire, small-sized flame and other merits [11].

Edge detection is one of the important steps in fire and flame image processing which provide the foundation for analysis and monitoring of the system. It is due to the fact that the flame and fire edges form a basis for the qualitative analysis of the parameters such as area, size, shape, location and stability. Moreover, the edges can reduce the amount of redundant data and filter out the unwanted information such as background noise and ambiguities within the images. Furthermore, edges can be used to segment a group of flames which can be helpful for multiple flame monitoring in the industry [12].

In the last decades, several techniques have been proposed to identify the fire and flame image edges [13,14,15]. Bheemul et al. [16] presented an effective technique to extract the flame contours by detecting the changes of brightness in the horizontal direction line to line over the flame image. However, this technique is suitable only for steady and simple flame images. Toreyin et al. [17] proposed an edge detection algorithm based on the hidden Markov models and wavelet transform. Further, Wang et al. [18] presented a Chan–Vese active contour model for flame images, particularly obtained from the power plant. Jiang and Wang [19] improved the Canny edge detector which has been used to detect moving fire area in larger region fire images. Recently, Qiu et al. [20] proposed an autoadaptive edge detection algorithm for flame and fire image processing. It is based on the modified Canny edge detection technique and iterative threshold adjustment. Moreover, Gupta and Gaidhane [21] developed a simple and robust approach for flame image analysis, which is based on the local binary patterns (LBP) and double thresholding techniques. In this technique, the main features of an image are obtained by local binary patterns, and two-level thresholding is used to make the edges visible and clear. Although all these methods have its own advantages for flame and fire detection, they have some limitations such as unclear and discontinuous edge detection. Therefore, for detecting the size, shape and other geometric characteristic, and to obtain the clear, continuous and closed edges of the flame and fire images, an effective and efficient technique still remains a challenge for image processing researchers.

In this paper, some existing techniques have been tested to study the effectiveness in fire and flame image analysis. It is observed that the response of these techniques is affected by fragmentation and the edges of the fire and flame images are melted. To overcome these problems, an efficient and effective technique for fire and flame image analysis is proposed. It is based on the LBP operator and the double thresholding method. The double thresholding is carried out by the Levenberg–Marquardt (LM) optimization technique.

The rest of the paper is organized as follows: A brief overview of existing techniques is explained in Sect. 2. Sections 3 and 4 describe the proposed algorithm for fire and flame image edge detection and analysis, respectively. Section 5 presents the experimental results for various flame and fire images. Finally, concluding remarks and future research scopes are given in Sect. 6.

2 Overview of edge detection techniques

The tasks such as process, analysis and monitoring of flame images are of paramount importance in the power generation industry. It is increasingly used in temperature, flame and fire detection systems. It is due to the fact that the temperature calculation of visible area is able to provide additional information such as fire region, effects, gases [22,23,24]. In these techniques, edge detection is one of the important steps and acts as the foundation for other processing.

Generally, the edges of an image are represented by first- and second-order derivatives of the 2D image \( f\left( {x ,y} \right) \). Mathematically, it can be defined by a gradient vector as [25]

$$ \nabla f\left( {x\text{,}y} \right) = \left[ {\begin{array}{*{20}c} {G_{x} } \\ {G_{y} } \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {\frac{{\partial f\left( {x\text{,}y} \right)}}{\partial x}} \\ {\frac{{\partial f\left( {x\text{,}y} \right)}}{\partial y}} \\ \end{array} } \right] $$
(1)

where \( G_{x} \) and \( G_{y} \) are the gradient in the x and y coordinates, respectively. For 2D digital image [25],

$$ G_{x} = f\left( {x + 1\text{,}y} \right) - f\left( {x\text{,}y} \right)\;{\text{and}}\;G_{y} = f\left( {x\text{,}y + 1} \right) - f\left( {x\text{,}y} \right) $$
(2)

The magnitude of the gradient vector is given by

$$ {\text{mag}}\left( {\nabla f} \right) = \sqrt {G_{x}^{2} + G_{y}^{2} } = \sqrt {\left( {\frac{{\partial f\left( {x,y} \right)}}{\partial x}} \right)^{2} + \left( {\frac{{\partial f\left( {x,y} \right)}}{\partial y}} \right)^{2} } $$
(3)

The direction of the gradient vector is represented by the angle \( \alpha \) as

$$ \alpha \left( {x\text{,}y} \right) = { \tan }^{ - 1} \left( {\frac{{G_{x} }}{{G_{y} }}} \right) $$
(4)

Based on above mathematics and first-order derivative, Roberts, Prewitt, Sobel and Canny are presented in the literature and these methods are commonly used to extract the edges of an image. The second-order derivative-based method is Laplacian of Gaussian (LOG) also called as Mexican Hat or Marr–Hildreth operator. The Gaussian function is a nonlinear function given as

$$ G\left( {x,y} \right) = e^{{ - \frac{{x^{2} + y^{2} }}{{2\sigma^{2} }}}} $$
(5)

where \( \sigma \) is a standard deviation called as space constant. Then the Laplacian of Gaussian (LOG) function can be written as [25]

$$ \begin{aligned} \nabla^{2} G\left( {x\text{,}y} \right) & = \frac{{\partial^{2} G\left( {x\text{,}y} \right)}}{{\partial x^{2} }} + \frac{{\partial^{2} G\left( {x\text{,}y} \right)}}{{\partial y^{2} }}\text{,} \\ \nabla^{2} G\left( {x\text{,}y} \right) & = \left[ {\frac{{x^{2} + y^{2} - 2\sigma^{2} }}{{\sigma^{4} }}} \right] \, e^{{ - \frac{{x^{2} + y^{2} }}{{2\sigma^{2} }}}} \\ \end{aligned} $$
(6)

In this method, the edges are detected by searching a second-order derivative over the entire image and search of zero crossing of LOG. However, these methods have limited performance as compared to Canny edge detection method.

In the last decade, some edge detection-based techniques have been proposed and applied in various applications for flame or fire image analysis [11,12,13,14,15]. Recently, Qiu et al. [20] presented an autoadaptive edge detection approach and applied to the flame and fire image analysis. Further, Habiboğlu et al. [2] proposed a fire and flame detection method based on the covariance matrix approach. This method works very well when the fire is close and visible. However, it poorly performs for small fire area. Moreover, it is observed that the above existing techniques are not enough suitable for flame and fire image analysis. Therefore, an efficient and simple approach is proposed for fire and flame image analysis.

3 Proposed algorithm for flame and fire image analysis

Generally, a flame region has a higher luminance as compared to its ambient and continuous background. Therefore, it is required to enlarge the difference among the flame and background. The basic idea is to extract the principle edges and remove the irrelevant edges for image analysis. To carry out this process, a new approach is proposed which is summarized in the following steps.

  • Step 1: Extracting and adjusting the gray level

In this step, the color image is converted to the grayscale image. It can be done by simple averaging of RGB components or separate average of colors prominent wise. Then the gray level of an image is adjusted according to its statistical distribution.

If \( I\left( {x\text{,}y} \right) \) is the intensity value of a point \( \left( {x\text{,}y} \right) \) in the original image \( f\left( {x\text{,}y} \right) \), then the edge value is given as [26, 27]

$$ V\left( {x\text{,}y} \right) = \frac{1}{m \times m}\sum\limits_{i = - m}^{m} { \, \sum\limits_{j = - m}^{m} {I\left( {x + i,y + j} \right)} } $$
(7)

Then the intensity \( u\left( {x\text{,}y} \right) \) and local information \( v\left( {x,y} \right) \) are given as

$$ u\left( {x\text{,}y} \right) = \frac{{I\left( {x\text{,}y} \right) - I_{ \hbox{min} } }}{{I_{ \hbox{max} } - I_{ \hbox{min} } }} $$
(8)
$$ v\left( {x\text{,}y} \right) = \left\{ {\begin{array}{*{20}c} { - \frac{1}{2}\frac{{\left[ {V\left( {x,y} \right)} \right]^{\beta } }}{{V_{ \hbox{min} } }}\text{,}} & { V\left( {x,y} \right) < 0} \\ {\frac{1}{2}\frac{{\left[ {V\left( {x,y} \right)} \right]^{\beta } }}{{V_{ \hbox{max} } }}\text{,}} & {V\left( {x,y} \right) < 0} \\ \end{array} } \right. $$
(9)

where \( \beta \) is enhancement factor \( 0 \le \beta \le 1 \).

The multi-peak generalized histogram enhancement is calculated as

$$ g\left( {x\text{,}y} \right) = f\left( {u\left( {x\text{,}y} \right)\text{,}v\left( {x\text{,}y} \right)} \right) $$
(10)

The density function is given as

$$ p\left( {g\left( {x\text{,}y} \right)} \right) = h\left( {f\left( {u\text{,}v} \right)} \right) $$
(11)

The value of the histogram \( H\left( p \right) \) can be calculated from the density function. Finally, the output image is obtained using the piecewise linearization of \( H\left( p \right) \) as \( \hat{f} = T\left( p \right) \).

  • Step 2: Smoothing of the image

Next step is to smoothen the image. This is done by filtering out the noise from an image before finding its edges. A Gaussian filter can be exploited for this process. The Gaussian function can be represented as

$$ G\left( {x,y} \right) = \frac{1}{{2\pi \sigma^{2} }}e^{{ - \frac{{x^{2} + y^{2} }}{{2\sigma^{2} }}}} $$
(12)

where \( \sigma \) represents the standard deviation. Based on Eq. (12), a Gaussian mask can be obtained. As the width of the Gaussian mask is increased, the clarity of the edges is also improved. After certain tests and evaluation, the Gaussian mask with a standard deviation \( \sigma = 1.4 \) is used in the implementation.

$$ G_{\text{mask}} = \frac{1}{159}\left[ {\begin{array}{*{20}c} 2 & \quad 4 & \quad5 & \quad 4 & \quad 2 \\ 4 & \quad 9 & \quad {12} & \quad 9 & \quad 4 \\ 5 \quad & {12} & \quad {15} & \quad {12} & \quad 5 \\ 4 \quad & 9 & \quad {12} & \quad 9 & \quad 4 \\ 2\quad & 4 \quad & 5 \quad & 4 \quad & 2 \\ \end{array} } \right] $$
(13)

However, further increase in the Gaussian mask also increases the local error and it becomes very difficult to distinguish between the false and true edge.

  • Step 3: Local Binary Patterns (LBP)

After smoothing, LBP are applied to the preprocessed image. In LBP, each pixel in \( 3 \times 3 \) neighborhoods is replaced by calculating values using Eq. (14). This process is repeated for all the pixels of an image [28].

$$ {\text{LBP}}_{P,R} = \sum\limits_{p = 0}^{p - 1} {s\left( {g_{p} - g_{c} } \right) 2^{p} } $$
(14)

where

$$ s\left( x \right) = \left\{ {\begin{array}{*{20}c} {1\text{,}} & {x \ge 0} \\ {\text{0,}} & {x < 0.} \\ \end{array} } \right. $$
(15)

\( P \) number of equally spaced pixels, \( R \) radius of circle with center \( g_{c} \). \( g_{c} \) gray value of the center pixel of a local neighborhood, \( g_{p} \) gray value of a sampling point in an evenly spaced circular neighborhood.

The LBP operator of size \( 3 \times 3 \) is shown in Fig. 1. From Fig. 1, the binary code using LBP operator is 01011111 and its equivalent decimal code is 95. Thus, the center pixel is replaced by 95 pixel values. Averaging of the RGB components, smoothing and LBP operation on the colored images handle the spatial and temporal information together.

Fig. 1
figure 1

Example of LBP operator. Binary equivalent = 01011111; decimal equivalent = 95

Full size image

  • Step 4: Thresholding of the LBP image

It is observed that the best results on LBP processed image can be obtained by selecting the higher threshold values \( k_{2}^{ * } \) and lower threshold value \( k_{1}^{ * } \). For this, range of pixel value between \( k_{2}^{ * } \) and \( k_{1}^{ * } \) is selected. However, the lower value of \( k_{1}^{ * } \) results into false edges and higher \( k_{2}^{ * } \) value eliminates the actual valid points. Therefore, the section of valid and correct threshold is an important task to process the LBP image. An optimal value of these thresholds is used in this paper. These multiple threshold values can be calculated using the global thresholding technique as given below.

The mean intensity of the pixels in the region \( C_{1} \, \left( {{\text{pixels }}\left\{ {0, \, k} \right\}} \right) \) and \( C_{2} \)\( \left( {{\text{pixels}}\left\{ {k + 1,L - 1} \right\}} \right) \) is given as [25]

$$ m_{1} = \frac{1}{{P_{1} }}\sum\limits_{i = 0}^{k} {i \cdot P_{i} } $$
(16)
$$ m_{2} = \frac{1}{{P_{2} }}\sum\limits_{i = k + 1}^{L - 1} {i \cdot P_{i} } $$
(17)

where \( P_{1} \) and \( P_{2} \) are the probabilities of the region \( C_{1} \) and \( C_{2} \), respectively. The global intensity value can be calculated as

$$ m_{G} = \sum\limits_{i = 0}^{L - 1} {i \cdot P_{i} } $$
(18)

For double thresholding between-class variance can be obtained as

$$ \sigma_{B}^{2} = P_{1} \left( {m_{1} - m_{G} } \right)^{2} + P_{2} \left( {m_{2} - m_{G} } \right)^{2} $$
(19)

Then the optimal thresholds for \( k_{1}^{ * } \) and \( k_{2}^{ * } \) can be computed as

$$ \sigma_{B}^{2} \left( {k_{1}^{ * } \text{,}k_{2}^{ * } } \right) = \mathop { \hbox{max} }\limits_{{0 < k_{1} < k_{2} < L - 1}} \sigma_{B}^{2} \left( {k_{1} \text{,}k_{2} } \right) $$
(20)

At this stage, the preliminary image \( \tilde{f}\left( {x,y} \right) \) with the edges is obtained from the original flame images. This image is processed further for closed edge extraction.

  • Step 5: Obtaining the closed edges

The LBP-based preliminary image \( \tilde{f}\left( {x\text{,}y} \right) \) has thicker and unrelated edges wider than a pixel. Therefore, the non-maxima suppression technique is exploited for thinning down the broader edges.

The magnitude of the gradient of \( \tilde{f}\left( {x\text{,}y} \right) \) intensity can be calculated as

$$ \Im \left( {x,y} \right) = \sqrt {\tilde{f}_{x}^{2} + \tilde{f}_{y}^{2} } , $$
(21)

and the direction is given by

$$ \theta \left( {x,y} \right) = { \tan }^{ - 1} \left[ {\frac{{\tilde{f}_{x} }}{{\tilde{f}_{y} }}} \right] $$
(22)

The edge magnitude of two neighborhood edge pixels perpendicular to the edge direction is considered, and the one with a lesser edge magnitude is neglected. This non-maxima suppressed image may still contain few false edges. To remove these false edges, two thresholds \( T_{1} \) and \( T_{2} \) are chosen to create two different edge images \( \tilde{f}_{1} \) and \( \tilde{f}_{2} \), where \( \tilde{f}_{1} \) is contained most of the false edges and \( \tilde{f}_{2} \) has all continuous edges. The threshold selection algorithm starts with the edge points in \( \tilde{f}_{2} \), linking the adjacent edge points in \( \tilde{f}_{2} \) forming a contour, and the process continues till no more adjacent edge points are available. In order to speed up the process of finding the thresholds \( T_{1} \) and \( T_{2} \), a Levenberg–Marquardt (LM) optimization algorithm is used which optimizes a mean square error-based cost function [29, 30]. In this step, \( T_{1} \) and \( T_{2} \) need to be autoadjusted. Initially, the value of threshold \( T_{1} \) and \( T_{2} \) will be given based on the initial results. Further, \( T_{2} \) can be adjusted to different values assuming \( T_{1} \) fixed at initial value. The mean square error deviation \( E\left( {\mu_{m} } \right) \) of a contour start from point \( C_{\text{start}} \left( {x_{s} ,y_{s} } \right) \) and end at \( C_{\text{end}} \left( {x_{e} \text{,}y_{e} } \right) \) is used as a cost function to judge the new value of \( T_{2} \). Suppose \( E\left( {\mu_{m} } \right) \) small enough, the computation process terminates. If \( E\left( {\mu_{m} } \right) \) is still greater than the desired value, a further error computation process is applied to \( C_{\text{start}} \left( {x_{s} \text{,}y_{s} } \right) \). Thus, the cost function \( \chi \left( {\mu_{m} } \right) \) can be written as

$$ \chi \left( {\mu_{m} } \right) = \sum\limits_{i = 1}^{N} {\left[ {E\left( {\mu_{m} } \right)} \right]^{2} } $$
(23)

where

$$ E\left( {\mu_{m} } \right) = C_{\text{start}} \left( {x_{s} \text{,}y_{s} } \right) - C_{\text{end}} \left( {x_{e} \text{,}y_{e} } \right) $$
(24)

The second-order Taylor series results in updated initial guess

$$ \begin{aligned} \chi \left( {\mu_{m + 1} } \right) & = \chi \left( {\mu_{m} + \Delta \mu_{m} } \right)\text{,} \\ & \quad \approx \chi \left( {\mu_{m} } \right) + g_{m}^{T} \Delta \mu_{m} + \frac{1}{2}\Delta \mu_{m}^{T} A_{m} \, \Delta \mu_{m} \\ \end{aligned} $$
(25)

where \( g_{m} = \nabla \left. {\chi \left( \mu \right)} \right|_{{\mu = \mu_{m} }} \) and \( A_{m} = \nabla^{2} \chi \left( \mu \right) \). If we calculate the gradient of the above equation with respect to \( \Delta \, \mu_{m} \) and set it equal to zero, then another equation can be obtained as

$$ \begin{aligned} \chi \left( {\mu_{m + 1} } \right) & = \chi \left( {\mu_{m} + \Delta \mu_{m} } \right)\text{,} \\ \Delta \, \mu_{m} & = - A_{m}^{ - 1} \, g_{m} \\ \end{aligned} $$
(26)

Thus, the Newton’s method can be defined as

$$ \mu_{m + 1} = \mu_{m} - A_{m}^{ - 1} g_{m} $$
(27)

The Levenberg–Marquardt algorithm is a variation of Newton’s statistical method which is designed for minimizing the sums of squares of nonlinear functions. Here, \( \chi \left( {\mu_{m} } \right) \) is a sum of square functions as given in Eq. (23). The gradient of \( \chi \left( {\mu_{m} } \right) \) can be written as [30]

$$ \nabla \chi \left( {\mu_{m} } \right) = 2J^{T} \left( {\mu_{m} } \right) E\left( {\mu_{m} } \right) $$
(28)

where \( J\left( {\mu_{m} } \right) \) is the image Jacobian matrix. Similarly, Hessian matrix can be obtained as

$$ \nabla^{2} \chi \left( {\mu_{m} } \right) = 2J^{T} \left( {\mu_{m} } \right) J\left( {\mu_{m} } \right) $$
(29)

Then the updated guess value can be calculated as

$$ \begin{aligned} \mu_{m + 1} & = \mu_{m} - \left[ {2J^{T} \left( {\mu_{m} } \right)J\left( {\mu_{m} } \right)} \right]^{ - 1} 2J^{T} \left( {\mu_{m} } \right)E\left( {\mu_{m} } \right)\text{,} \\ \mu_{m + 1} & = \mu_{m} - \left[ {J^{T} \left( {\mu_{m} } \right)J\left( {\mu_{m} } \right)} \right]^{ - 1} J^{T} \left( {\mu_{m} } \right)E\left( {\mu_{m} } \right) \\ \end{aligned} $$
(30)

This Eq. (30) is called as Gauss–Newton method. Further, Hessian matrix \( \left( H \right) \) can be approximated as \( G = H + \lambda_{m} I \). This leads to the LM algorithm

$$ \begin{aligned} \mu_{m + 1} & = \mu_{m} - \left[ {J^{T} \left( {\mu_{m} } \right) \, J\left( {\mu_{m} } \right) + \lambda_{m} I} \right]^{ - 1} J^{T} \left( {\mu_{m} } \right) \, E\left( {\mu_{m} } \right)\text{,} \\ \Delta \mu_{m} & = - \left[ {J^{T} \left( {\mu_{m} } \right)J\left( {\mu_{m} } \right) + \lambda_{m} I} \right]^{ - 1} J^{T} \left( {\mu_{m} } \right) \, E\left( {\mu_{m} } \right) \\ \end{aligned} $$
(31)

where \( \lambda_{m} \) is a positive scalar parameter and \( I \) is the identity matrix. The free parameter \( \lambda_{m} \) works as an adaptive parameter of the step length during the iterations. The choice of this parameter has a great impact on the convergence of an error. This algorithm has very important features such that as \( \lambda_{m} \) increases, it approaches the steepest descent algorithm and as \( \lambda_{m} \) decreases to zero, the algorithm becomes Gauss–Newton.

Initially, the LM algorithm starts with \( \lambda_{m} = 0. 0 1 \). If a step does not reach to smaller value for \( E\left( \mu \right) \), then the step is repeated by multiplying \( \lambda_{m} \) by the factor \( \tau \ge 1 \). This approaches to the steepest descent, but results in slow convergence. On the other hand, if a step produces a smaller value for \( E\left( \mu \right) \), then \( \lambda_{m} \) is divided by \( \tau \ge 1 \) for the next step so that the algorithm will move toward the Gaussian–Newton which provides the faster convergence.

  • Step 6: Analysis of flame image

After successful clear and continuous edge detection the various parameters such as area, mean, standard deviation, entropy are calculated to analyze the flame and fire images.

The whole process of flame and fire image edge detection is summarized using the flowchart as shown in Fig. 2

Fig. 2
figure 2

Proposed concept of flame and fire image edge detection

Full size image

4 Flame image analysis

The various image parameters such as Area, Mean, Standard Deviation and Variance are used to analyze the edged flame and fire images. These parameters are described in this section.

4.1 Flame and fire image parameters

The Area is defined as the total number of nonzero pixels in an image. The principle of calculating the area is that as the area of the flame increases the size of the flame is also increasing. The area of edged images can be calculated as [25]

$$ A = \sum\limits_{i = 0}^{w - 1} {\sum\limits_{j = 0}^{h - 1} {I_{ij} } } \text{,}\quad \forall A \in \left\{ {0,w \times h} \right\} $$
(32)

and

$$ I_{ij} = \left\{ {\begin{array}{*{20}c} {0\text{,}} & {P\left( {i, \, j} \right) = 0} \\ {1\text{,}} & {P\left( {i, \, j} \right) > 0.} \\ \end{array} } \right. $$
(33)

where \( P\left( {x\text{,}y} \right) \in \left\{ {0\text{,} 2 5 5} \right\} \) represent the intensity of pixels at the position \( \left( {x,y} \right) \),\( x \in \left\{ {0,w - 1} \right\} \), \( y \in \left\{ {0,h - 1} \right\} \). Mean \( \left( \mu \right) \) is the overall average intensity of the pixels of the flame image. It can be represented as [25]

$$ \mu = \frac{1}{A}\sum\limits_{i = 0}^{w - 1} {\sum\limits_{j = 0}^{h - 1} {P\left( {i,j} \right)} } { ,}\quad \forall \mu \in \left\{ {0\text{,} 255} \right\} $$
(34)

Standard deviation \( \left( \sigma \right) \) and variance \( \left( v \right) \) give the measures of the spread of the distribution or data about the mean.

$$ v = \frac{1}{A}\sum\limits_{i = 0}^{w - 1} {\sum\limits_{j = 0}^{h - 1} { \left[ {P\left( {i, \, j} \right) - \mu } \right]^{2} } } \;{\text{and}}\;\sigma = \sqrt v $$
(35)

4.2 Formulation of relation between flame area and temperature

Image area plays an important role in flame and fire analysis. It is one of the simplest features and also acts as the foundation of other parameters computation. After various experimentations and careful analysis, it has been observed that the area of a red color component of the flame or fire images is directly proportional to the temperature. Moreover, after detailed study of literature, it has been observed that the maximum fuel efficiency and minimum environmental pollution can be obtained at 1299 °C combustion furnace flame temperature [4]. Therefore, in this paper a flame image with temperature 1299 °C has been considered as a reference temperature image for an efficient working of the furnace.

It is well proven in image processing that the graphical and visual presentations are more suitable for the human vision system as compared to statistical and mathematical analysis. Therefore, in this paper a graphical approach is also presented which predicts the relation between temperature and the area of the flame image. The proposed approach for estimation of the temperature of the furnace using the graphical analysis is summarized in the following steps:

  • Step 1: Obtain the edged contour using the proposed approach as given in Sect. 3.

  • Step 2: Calculate the area of edge contour using Eq. (32).

  • Step 3: Calculate the radius of the reference and test flame images as

    $$ r_{\text{ref}} = \sqrt {\frac{{A_{\text{ref}} }}{\pi }} \text{,}\;{\text{and}}\;r_{{{\text{test}}k}} = \sqrt {\frac{{A_{{{\text{test}}k}} }}{\pi }} \text{,} \quad \forall \, k = 1,2,3, \ldots ,n $$
    (36)

  • Step 4: The circles \( c_{\text{ref}} \left\{ {0,r_{\text{ref}} } \right\} \) and \( c_{{{\text{test}}k}} \left\{ {0,r_{{{\text{test}}k}} } \right\} \) can be drawn for reference and test flame image, respectively.

  • Step 5: The temperature of the furnace can be obtained by the relation

    $$ T = 10^{3} \times \left[ {\alpha_{0} \left( {r_{{{\text{test}}k}} - r_{\text{ref}} } \right) + \alpha_{1} } \right] \pm 5\,^\circ {\text{C}} $$
    (37)

    where \( \alpha_{0} = 0.1038 \) and \( \alpha_{1} = 1.3179 \) are the constants and can be calculated using the polynomial curve fitting analysis.

5 Experimental results and analysis

After implementing the proposed algorithm as described in Sects. 3 and 4, the numbers of flame and fire images are processed to evaluate the effectuality of the proposed approach. All the experimentations are carried out on a system equipped with Intel(R) Core (TM) i7-4500U, CPU @ 2.40 GHz, 8 GB RAM in a MATLAB R2013a software environment.

5.1 Edge detection using proposed approach

In this paper, the conventional edge detection techniques have been applied with the appropriate parameters to process the flame and fire images. For fare comparison, the image database reported by Qiu et al. [20] and Gupta and Gaidhane [21] is considered for the experimentation. These flame images are taken for propane Bunsen flames burning in open air with moving camera. Firstly, the existing edge detection techniques are tested on the flame images. Ideally, the extracted flame edges should be clear, continuous and uninterrupted. Although many parameters are adjusted in Sobel, Roberts, Prewitt and Laplacian operator techniques, flame edges are unclear and discontinuous. Therefore, it is not always possible to obtain ideal edges from real-life images using these existing techniques. Figure 3a–f shows the examples of results obtained by the various existing edge detection techniques. Among all these conventional methods, the Canny method gives the most clear, continuous and uninterrupted edges. However, the result cannot be called as optimum as it gives the multiple responses. Therefore, it is clear that none of the traditional methods are able to give the optimum edges. Therefore, in this paper an attempt has been made and a new approach is proposed for clear and continuous edge detection as explained in Sect. 3.

Fig. 3
figure 3

Comparison results of existing edge detection techniques. a Original image, b Sobel method, c Roberts method, d Prewitt method, e Laplacian method and f Canny method

Full size image

The effectuality of the proposed approach is evaluated on the refinery flame image. The LBP operator of size \( 3 \times 3 \) pixel and a Gaussian mask represented by Eq. (13) is considered in all these experiments. Moreover, the initial values of lower and higher threshold are set to \( T_{1} = 0.35 \) and \( T_{2} = 0.98 \), respectively. Assuming threshold \( T_{1} \) as fixed in order to adjust \( T_{2} \), the threshold optimization is initiated using the Levenberg–Marquardt (LM) algorithm. The LM algorithm is configured as one input layer, one hidden layer and one output layer. The LM optimization process starts with \( \lambda_{m} = 0. 0 1 \). The various experimentations are carried on refinery flame image. The intermediate stage and final edge detection results of the proposed algorithm are shown in Fig. 4. Figure 4b shows a LBP pattern image, Fig. 4c shows the LBP plus a threshold image, and Fig. 4d shows the edged image obtained using the proposed approach. It is observed from Fig. 4d that the proposed approach generates the clear, continuous and uninterrupted edges.

Fig. 4
figure 4

Illustration of proposed approach. a Original image, b LBP pattern image, c image after threshold and d edge detection using the proposed approach

Full size image

Recently, the new computing algorithms [20, 21] are presented in the literature which is based on the strategy to remove the redundant edges and identifying the principal edges using a Sobel operator and thresholding technique. On the other hand, the proposed approach is based on LBP operator \( \left( {3 \times 3} \right) \), double thresholding technique and LM-based threshold optimization which modifies the target image before the process of threshold adjustment and optimization of the edges. This makes the proposed approach more efficient than the existing approaches [20, 21]. In this experiment, the results of the proposed approach are compared with the autoadaptive edge detection technique. Figure 5 shows the edge detection results of autoadaptive technique [20] and proposed approach. It is clearly observed from Fig. 5 that the edges detected by the proposed approach are clearer and represent all the edge features of an image.

Fig. 5
figure 5

Comparison of results of autoadaptive [20] and proposed approach. a Original images, b autoadaptive technique, c proposed approach

Full size image

Further, the efficiency of the proposed approach is tested under the different noisy conditions. In this experiment, the different types of random noise such as salt and peppers \( \left( {\eta = 0.2} \right) \) and Gaussian noise \( \left( {\rho = 0.3} \right) \) are added to the reference flame images. Although the noise is added to the reference image, the edges obtained by the proposed approach are more clear and continuous than the existing method. The various results in the presence of salt and peppers and Gaussian noise are shown in Fig. 6.

Fig. 6
figure 6

Results in noisy conditions. a, e Original image; b with salt and pepper noise, f with Gaussian noise, c, g edged image using autoadaptive method [20] and d, h edge-detected image using proposed approach

Full size image

The computational complexity is an another evaluation parameter for any algorithm. Therefore, the computational time of the proposed approach is compared with the existing conventional methods. This experiment is carried out for a set of five different images. Table 1 shows the average execution time of five images for existing methods as well as the proposed approach. It is observed from Table 1 that the time taken by the proposed approach is little more than existing methods. It is due to the iterative nature Levenberg–Marquardt error optimization-based algorithm. This algorithm requires comparatively more time for error optimization. However, more computational time is compensated by the accuracy and effectiveness of the proposed approach.

Table 1 Time comparison of different exiting methods and the proposed edge detection
Full size table

In flame image analysis, it is very difficult to find clear and continuous edges of real and high resolution. Therefore, in this paper the effectiveness of the proposed approach is also tested on the high-resolution images. Figure 7a shows a wood gas flame image of high resolution of size \( 846 \, \times 1721 \) pixel taken from a flame gasifier using moving camera. Similarly, Fig. 7d shows a high-resolution campfire image of dimension \( 1200 \, \times 1600 \) pixel with static camera. In this experiment, the results of the proposed approach are compared with the existing Canny-based edge detection method. Figure 7b, e shows the Canny-based edge-detected images, whereas Fig. 7c, f shows the results of proposed approach. In Canny-based results (Fig. 7b, e) the edges are unclear and broken in certain areas, whereas the edges detected by proposed approach are comparatively clear and continuous. Thus, it is clear that the propose approach performs better than the conventional edge detection methods for static as well as moving camera.

Fig. 7
figure 7

Edge detection results for real images. a Wood gas flame image, b edge detection using Canny method, c edge detection using proposed approach, d campfire image, e edge detection using Canny method and f edge detection using proposed approach

Full size image

5.2 Edge detection based on red color component using proposed approach

In this experiment, the database [4] of flame images is used to detect the edges related to the red color component of RGB images. Further, these edged images are considered in the calculation of flame characteristics such as area, radius, mean, standard deviation, temperature. Figure 8 demonstrates an example of furnace combustion flame image database and their respective red color component-based edge detection results. It is well known that the red color component in an image contains most of the information about the temperature. Therefore, in this paper red color component is used for study and analysis of flame images. Figure 8a–d shows the furnace flame images with temperature \( 1197\text{,}\;1299\text{,}\; 1 3 9 7 \) and \( 1499\,^\circ {\text{C}} \), respectively. These images are taken from blackbody furnace using flame image detector-based Samsung SCC-833P color charged-coupled device (CCD) camera installed in different height of the furnace wall.

Fig. 8
figure 8

Edge detection of red color component of database images using proposed approach a–d 1197, 1299, 1397 and 1499°C, respectively, and e–h edge detection results using proposed approach

Full size image

In this experiment, a red color component is extracted to define the temperature of the flame image. From Fig. 8a–d, it is observed that the red color component increases as the temperature rises above the minimum level. In the literature [4], it is reported that the red color component starts growing above \( 1000\,^\circ {\text{C}} \) temperature in the furnace flame image. The temperature of these images can be obtained using the different methods [7,8,9,10]. However, these methods are more costly and require the dedicated instruments as well as the trained operator. On the other hand, image processing-based methods are reliable, simple and require less equipments. Therefore, in this paper an attempt has been made and a simple approach is proposed to calculate the temperature of a furnace as explained in Sect. 4. Here, the edges of the furnace flame images are extracted using the proposed approach as shown in Fig. 8e–h. It is observed that the edges of the red component of the flame image for temperature \( 1499 \, ^\circ {\text{C}} \) go beyond the expected area. It indicates the overheating temperature of the furnace and it can be used as a danger alarm in the system.

The area and radius of the different images can be obtained using Eqs. (32) and (36), respectively. Table 2 shows the area and radius of Fig. 8a–d. It is observed from Table 2 that as the temperature increases, the radius also increases. This property can be used to calculate the temperature of the furnace using image processing algorithms. However, the radius of Fig. 8d is infinite which can be used to show the danger alarm or overheating in the system.

Table 2 Variation of temperature, area and radius of the flame images
Full size table

5.3 Calculation of flame parameters

The different flame image parameters such as area, mean, standard deviation and entropy can be calculated using Eqs. (32)–(35). These parameters show the characteristics like stability, intensity and effects. Table 3 shows the calculated values of the various flame image features discussed in Sect. 4. It is observed from Table 3 that the area of the red component of the flame image is directly proportional to the temperature. Similarly, the mean and the standard deviation can be used to find the stability of the flame by calculating the coefficient of variance.

Table 3 Variation of flame parameter
Full size table

The values of mean and standard deviation increase with the increase in temperature. Moreover, it has been observed from Table 3 that the increase in temperature results into more entropy. Thus, these flame parameters can be used to predict the stability of the flame images.

5.4 Computation of flame temperature

The visual perception plays an important role in image processing and computer vision research area. Therefore, in this paper an attempt has been made and proposed a graphical approach for the comparison of furnace flame temperature. In this approach, furnace flame temperature \( 1299 \, ^\circ {\text{C}} \) is considered as a reference or the ideal temperature at which the system gives an optimum output [4]. Here, the main aim is to calculate the real time temperature of the furnace with respect to the reference temperature. Moreover, one can calculate the temperature of the furnace flame using the proposed technique as explained in Sect. 4. The graphical method can be used for visual analysis of the furnace temperature.

In this experiment, furnace flame image database [4] is considered for the analysis and calculation of the temperature. Figure 9 shows an example of the proposed graphical approach. In this experiment, two images with unknown temperature and one reference image with known temperature \( \left( {1299\,^\circ {\text{C}}} \right) \) are considered for analysis. The radius of each image is calculated using the proposed approach as given in Sect. 3 as well as Sect. 4. Then the respective circles are drawn as shown in Fig. 9. The middle circle shown in Fig. 9 with radius 22.5 is drawn for the reference image. Similarly, the outer circle and inner circle are drawn for unknown temperature with calculated radius 23.45 and 21.26, respectively.

Fig. 9
figure 9

Graphical comparison of the unknown temperature with the reference temperature

Full size image

Now, Eq. (37) can be used to calculate the unknown temperature of the flame image with respect to the reference image. After various experimentations and rigorous analysis the value of constant \( \alpha_{0} \) and \( \alpha_{1} \) are set to 0.1038 and 1.3179, respectively. The temperature for outer circle and the inner circle is obtained which is found to be 1416 and 1189 °C, respectively.

Further, the results of the computation of temperature for various flame images are validated using the database consisting of 100 different flame images with known temperature. The proposed graphical method is found suitable for visual inspection. If the temperature of the flame image under test is equal to or closed to the reference flame image, then the circles will overlap on each other. Thus, operator can predict the under-heating or overheating of the furnace by simple visual inspection. Moreover, it is observed that the proposed approach is comparatively effective and efficient as compared to the conventional methods. Therefore, it can be used to calculate the edges and unknown temperature of the flame images.

6 Conclusions and future scope

In this paper, a simple and efficient flame and fire image analysis approach has been proposed and evaluated in comparison with the conventional techniques. Experimental results have shown that the developed algorithm is effective in identifying the clear, continuous and uninterrupted edges and effective area of the flame and fire images. The clearly defined flame area plays an important role in quantification of flame parameters such as flame surface area, flame volume, flame or fire spread speed, temperature. Moreover, the proposed approach may be useful in detailed understanding and advanced image processing-based monitoring of the furnace combustion flame.