Visual Smoke Detection

Abstract

In this paper, we have proposed a novel and efficient visual smoke detection algorithm. Smoke detection in video surveillance is very important for early fire detection. Proposed algorithm uses an unique combination of features to detect smoke efficiently. These features use appearance, energy and motion properties of the smoke. Further analysis of past history of smoke increases the accuracy of the algorithm. These features are less complex and enable the algorithm for real time application. A general assumption is that smoke is a low frequency signal which may smoothen the background. We focused on the nature of the smoke (shape disorder, energy reduction and variability over time) and proposed a novel algorithm which requires no user intervention and prior data training. Due to the large variability in the feature values, we assigned the fuzzy membership to these features instead of hard thresholding to reduce classification errors. Simulation carried out with available dataset, show that smoke is accurately localized both in time and space via proposed approach.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

The ability to provide early and accurate alerts are the critical objective for any smoke detection system and achieving that objective is the primary challenge. The earliest possible warning of any potential smoke increases the time available for responders to take necessary action. This in turn minimizes risks to life and property. Video-based smoke detection techniques detect smoke anywhere within the field of view of the camera. Video based smoke detection techniques have many advantages such as fast response, non-contact, indoor and outdoor detection and ability to record video for smoke progress analysis.

Designing of a smoke detection algorithm is challenging due to following reasons: (1) Variability in density, lighting, diverse background, interfering non-rigid objects etc., (2) None of the image features such as intensity, motion, energy, and obscuration characterizes smoke well, and (3) Visual pattern of smoke is difficult to model.

The complex temporal behavior of the smoke makes the smoke detection very challenging. These behaviors are random shape and motion, change in color and density. Smoke detection algorithm has wide range of applications. It can be used for indoor (Industrial plants, Warehouse, Cold storage, Large boilers & equipments protection, and Laboratories) and outdoor (Forestry services, Mining, Oil fields, and Gas stations) surveillance. It can be used to protect valuable assets which require continuous monitoring and it can be programmed to release CO2 when smoke is detected.

2 Background

Many smoke detection algorithms are proposed in past few years. Mainly, these algorithms use appearance, energy, motion features and/or combination of these features (hybrid) for successful detection of smoke.

In the category of appearance features based algorithm; Wang et al. [1] proposed an appearance based real-time smoke detection algorithm. This algorithm uses modified CS-LTP (centre symmetric - local ternary pattern) texture features combined with color information for smoke detection. Difference in the histograms of CS-LTP features of background and foreground detects the smoke regions. Color information is used for further refinement. Relying only on the appearance, restrict the algorithm performance. Maruta et al. [2] proposed a novel smoke detection algorithm based on fractal property. Here, it is assumed that image information of smoke is self affine fractal and local Hurst exponent characterize this features. Value of Hurst exponent decides the presence of smoke, and temporal consistency in exponent gives further confidence in localizing the smoke regions. Chen et al. [3] proposed color and texture features based smoke detection algorithm. Inter frame difference and Local Binary Pattern (LBP) are used to analyze dynamic characteristics of the smoke. Later smoke history image is constructed to reduce false alarms.

In the category of motion features based algorithm; Brovko et al. [4] proposed a motion analysis based smoke detection algorithm. This algorithm uses dynamic and static features of the smoke. Here, optical flow analysis is used to separate out smoke region with the assumption that smoke grows in certain direction only. This assumption restricts the algorithm performance in outdoor scenarios where, smoke can grow in any direction due to wind. Wang et al. [5] proposed an early smoke detection algorithm based on swaying and diffusion features. First, moving regions are separated out using fuzzy integral. Later, smoke regions are determined using analysis of swaying nature of these regions based on centroid. Diffusion features are used for further refinement with the assumption that smoke diffuses over time. For outdoor scenarios, these swaying features generate false alarm when bushes are present in the scene. Li et al. [6] proposed a novel smoke detection algorithm based on fuzzy and mobility characteristics. Modified GMM is used for detection of moving region and characteristics of region mobility are used for final detection of smoke regions. Here, assumption is that, in smoke region displacement in the bottom is slower than that of central of the whole region. However, this assumption fails in many outdoor conditions.

In the category of energy based algorithms; Gonzalez et al. [7] proposed an energy features based smoke detection algorithm. Stationary wavelet transform is used for energy estimation with the assumption that smoke reduces the high frequency details in foreground. To rely on the potential smoke regions, area features are used. Here, it is assumed that the area of the smoke region increases continuously. However, sometimes it happens that one smoke region is partitioned into smaller regions and contradict this assumption.

In the category of hybrid features based algorithms; Li et al. [8] proposed a wavelet features and optical flow based smoke detection algorithm. First, moving areas are detected using Gaussian mixture model. Out of these moving regions, smoke regions are defined where energy reduces in foreground with respect to background. Optical flow is used for further refinement. Chen et al. [9] introduced a new concept of contrast image. On the basis of this contrast image wavelet analysis is performed on the background and foreground images. It is assumed that the smoke is low frequency component, ratio of high frequency and low frequency components is used to decide whether motion region is smoke or not.

Calderara et al. [10] proposed, a smoke detection algorithm by analyzing color and texture features of the moving objects extracted from mixture of Gaussians. Here, wavelet analysis is performed to study the change in energy in smoke regions. Then color analysis based on blending function is used to separate out candidate smoke regions. Avgerinakis et al. [11] proposed appearance and motion features based smoke detection algorithm. Histogram of gradients and optical flow (HOG-HOF) is used to localize smoke in time domain. This information is used along with the color features for classification of smoke region.

Toreyin et al. proposed [12] appearance and color features based spatio-temporal smoke detection algorithm. It is assumed that edges present in image start losing their sharpness over time. This behavior is analyzed using wavelet transform. Periodic behavior of smoke boundaries are analyzed by a Hidden Markov Model. Reduction in energy, periodic behavior in boundaries and reduction in chrominance values, all these clues are used to make final decision of smoke region.

2.1 Contributions

Prior art algorithms are based on the certain assumptions like color of smoke, direction of motion, and energy. It is observed that these features restrict the algorithm performance. However, effective combination of these features in spatial and temporal domain may resolve these issues. Proposed algorithm uses hybrid features for detection of smoke. Novelties in the proposed algorithms are as follows: Unique combination of features (energy, appearance (shape disorder and transparency) and motion (consistency and variability over time)) enables us to detect smoke effectively. Due to the large variability of smoke features, assignment of fuzzy membership helps us to detect smoke effectively. Distance based classification with past history increases the accuracy of the algorithm. No prior data training is required for classification. Algorithm is not using any color features, which makes it robust to detect smoke of any color. Computation of these features are less complex which confirms the real-time application of visual smoke detection. Algorithm analyzes the energy reduction and shape disorder in spatio-temporal domain, which reduces the false alarm significantly. Smoke is transparent in nature, algorithm analyzes the transparency (blending with background) which reduces false alarm. Algorithm analyzes smoke features in block as well as blob level, which reduces miss and false alarms. Proposed algorithm requires a buffer size of 5 frames, which is very low in comparison with prior art algorithms. Existing algorithms require 9 frames to 50 frames.

3 Our Approach

Our smoke detection algorithm analyzes the spatio-temporal behavior of the smoke. Block diagram of the proposed smoke detection algorithm is shown in Fig. 1. First we subtract the background using adaptive Gaussian mixture model (GMM) to detect all foreground (moving) objects present in the scene. Then we estimate the transparency of each moving objects. Semi-transparent moving objects are possible smoke regions. Feature analysis of these possible smoke regions is performed and distance based classifier separate out potential smoke regions and non-smoke regions.

Fig. 1.

Block diagram of the proposed smoke detection algorithm.

Adaptive Gaussian mixture model is a reliable method to approximate the background modeling. It models each pixel as a mixture of Gaussians and update the model using an online (Expectation-Maximization) EM algorithm. This model deals with illumination changes and repetitive motion from clutter efficiently. Background modeling by Gaussian mixtures is a pixel based process. Let x is a random process representing intensity value of a pixel at given time. Let p(x) represents a Gaussian mixture having K density components.

$$\begin{aligned} p(x) = \sum _{k=1}^{K}w_k N(x;\mu _k,\sigma _k) \end{aligned}$$

(1)

where, $w_k$ are weights and $N(x;\mu _k,\sigma _k)$ is normal distribution with mean $\mu _k$ and covariance $\sigma _kI$. GMM estimates these parameters over time to model background. Parameters are initialized with $w_k = w_0$, $\mu _k = \mu _0$ and $\sigma _k = \sigma _0$, if there is a match

$$\begin{aligned} \frac{\parallel x-\mu _i\parallel }{\sigma _i}<\tau \;\;\text {for some}\; i\;\epsilon \;[1,...,K] \end{aligned}$$

(2)

where, $\tau$ is threshold. These parameters are updated as follows:

$$\begin{aligned} \begin{aligned} w_k = (1-\eta _1)w_k(t-1)+\eta _1M_k(t)\\ \mu _k = (1-\eta _2)\mu _k(t-1)+\eta _2x\\ \sigma _k^2(t) = (1-\eta _2)\sigma _k^2(t-1)+\eta _2\parallel x-\mu _k(t)\parallel ^2 \end{aligned} \end{aligned}$$

(3)

where, $\eta _1$ and $\eta _2$ are learning parameters and $M_k(t)$ is 1 for matched components otherwise 0. If there is no match then lowest weight components are reinitialized. The K distributions are sorted in ascending order by $\frac{w_i}{\sigma _i}$. This ordering moves the most probable background and low variance at the top. The first B Gaussians which exceeds the threshold ($\rho$) is retained for a background defined as

$$\begin{aligned} B = \underset{x}{\mathrm {argmin}}(\sum _{k=1}^{K}w_k >\rho ) \end{aligned}$$

(4)

If the value of threshold ($\rho$) is small, the background model is uni-model and is multi-model for high value of threshold. Intuitively, Gaussians with high probability of occurrence and low variability are most likely to be background. Once updated background image is available, we subtract the current frame with the corresponding pixels at the same position on the background image and extract all moving object blobs.

Then for the detection of the possible smoke blobs, transparency of the each moving object is calculated. Here, assumption is that smoke regions are more transparent than non-smoke regions. Current frame $I_t$ is modeled as a combination of foreground $FG_t$ and background $BG_t$ using blending parameter $\lambda$ ($\lambda \; \epsilon \; [0,1]$) as mentioned in following equation.

$$\begin{aligned} I_t=\lambda FG_t+(1-\lambda )BG_t \end{aligned}$$

(5)

For transparent region $\lambda$ is 0 and for opaque region $\lambda$ is 1. For semitransparent region like smoke, value of $\lambda$ lie within [0, 1]. It is claimed [4] that optimum value of $\lambda$ for smoke is 0.38. Once we have background image $BG_t$ and $\lambda$, we can get the foreground $FG_t$ using Eq. (5). Possible smoke candidate region mask for each pixel can be obtained using following condition.

$$\begin{aligned} Mask^{ij} = {\left\{ \begin{array}{ll} \text{ Smoke }\;(1), &{} \text{ if } FG_t^{ij}>T \\ \text{ Non-smoke }\;(0), &{} \text{ otherwise } \end{array}\right. } \end{aligned}$$

(6)

where, T is a pre-determined threshold. With this transparent object detection, we separate out smoke regions and other moving objects. For further refinement of the smoke regions, we examined the features of each possible smoke region. We have analyzed the smoke features at block as well as blob level in spatio-temporal domain. It is noted that we have analyzed only those blocks and blobs which support both moving and transparent objects criteria. For block level analysis, we subdivided the image into non overlap blocks and estimated following features for each block.

• 1. Reduction in energy ($\alpha$)

• 2. Consistency of the energy reduction over time ($\beta$)

Here, $\alpha$ and $\beta$ are analyzed in spatial and temporal domain respectively. For blob level analysis, we estimated the following features for each candidate blob.

• 1. Shape disorder ($\gamma$)

• 2. Variability of the shape disorder over time ($\delta$)

Here, $\gamma$ and $\delta$ are analyzed in spatial and temporal domain respectively. Detail descriptions of these smoke features are as follows:

3.1 Reduction in Energy ($\alpha$)

It is noted that smoke creates blur in the scene or reduces the sharpness. As edges get blurred, we can assume that smoke reduces the energy of the background. We have computed the energy using wavelet transform. Wavelet transform is typically computed by applying a separable filter bank to the image, given by

$$\begin{aligned} \begin{aligned} cA = (L_x*(L_y*I)\downarrow _{2,1})\downarrow _{1,2}\\ cH = (H_x*(L_y*I)\downarrow _{2,1})\downarrow _{1,2}\\ cV = (L_x*(H_y*I)\downarrow _{2,1})\downarrow _{1,2}\\ cD = (H_x*(H_y*I)\downarrow _{2,1})\downarrow _{1,2}\\ \end{aligned} \end{aligned}$$

(7)

where, I is the original image and $*$ denotes convolution operation in two dimension. $\downarrow _{x,y}$ denotes decimation by x and y in corresponding dimension. L and H are low and high pass filters respectively. Coefficient cA is obtained by low pass filtering in both directions and termed as approximation image. cH, cV and cD are details coefficients obtained via high pass filtering in one or more directions. Due to the sub-sampling involved in this estimation, the total number of coefficients is equal to the pixels present in the original image.

Energy of each block is represented as follows:

$$\begin{aligned} E = \frac{cH^2+cV^2+cD^2}{cA^2} \end{aligned}$$

(8)

For smoke region, there is a reduction in energy (since details get blurred), If $E_{curr}$ and $E_{bg}$ are the energies of a block in current and background image then reduction in energy ($\alpha$) is estimated as follows:

$$\begin{aligned} \alpha = \frac{E_{curr}}{E_{bg}} \end{aligned}$$

(9)

Reduction in energy ($\alpha$) for smoke and non smoke region is shown in Fig. 2. It is observed that smoke reduces the sharpness of the scene which result in the reduction of energy. For stationary background or non smoke region, there is no change in energy. The value of $\alpha$ is low for smoke block compared to non-smoke block.

Fig. 2.

Reduction of energy with respect to background for smoke and non smoke region.

3.2 Consistency of the Energy Reduction over Time ($\beta$)

With time, smoke gets denser and energy at that region keeps on reducing. Since, for smoke block there is a consistent decrease in energy, whereas for rigid moving object energy reduction is not consistent and has fluctuations. As shown in Fig. 2, over time as the smoke density increases, decrease in the reduction in energy continues. Consistency of the energy reduction is estimated by counting number of times $\alpha$ is low (below a certain threshold) for a particular block location over time. Consistency of the energy reduction ($\beta$) is estimated as follows:

$$\begin{aligned} \beta = \left[ \sum _{i=0}^{n-1}{H[\tau - \alpha _i}]\right] \end{aligned}$$

(10)

where, H is the heaviside step function, whose value is 0 for negative argument and 1 for positive argument. n is the number of frames and $\tau$ ($\tau < 1$) is a threshold. Value of $\beta$ is more for smoke region compared to non smoke region. Maximum value $\beta$ can attain is n i.e. the number of frames under consideration.

3.3 Shape Disorder ($\gamma$)

Disorder or irregularity in shape discriminates the smoke region from other objects [12]. Generally man made objects have simple shape or less disorder in comparison with smoke.

Fig. 3.

Shape disorder analysis, shape of a (a) smoke region, and (b) man made object, boundary distance vector of (c) smoke region, and (d) man made object. (axes are adjusted for display).

We sampled the shape boundary of each region into N linearly separated points and computed the distance of these points from the centroid of the shape. We got a distance vector d[n] of size $1\times N$. We performed 1D wavelet decomposition of this distance vector. Wavelet decomposition gives low band approximation signal and high band detail signal. Distance vector is decomposed into low band a[l] and high band b[l] sub-signal using following equations:

$$\begin{aligned} \begin{aligned} a[l] = \sum _{n}h[2l-n]d[n]\;\;\;\;\text{ where, }\; h[l] = \{0.25,0.5,0.25\}\\ b[l] = \sum _{n}g[2l-n]d[n]\;\;\;\;\text{ where, }\; g[l] = \{-0.25,0.5,-0.25\} \end{aligned} \end{aligned}$$

(11)

Shape disorder ($\gamma$) is calculated as follows:

$$\begin{aligned} \gamma = \frac{\sum _l b[l]}{\sum _l a[l]} \end{aligned}$$

(12)

It is observed that value of $\gamma$ is more for smoke region compared to non-smoke region. Figure 3 shows the shape and corresponding distance vector of smoke region and man made object. It is evident from the figure that smoke region is having more disorder compared to man made object.

3.4 Variability of the Shape Disorder over Time ($\delta$)

For the estimation of shape disorder variability, we estimated the variation in $\gamma$ over time. $\delta$ is estimated as follows:

$$\begin{aligned} \delta = \frac{1}{n-1}\left[ \sum _{i=0}^{n}x_i^2-\frac{1}{n}\left( \sum _{i=0}^{n-1}x_i\right) ^2\right] \;\;\;\text{ where, }\;\;x_i = \gamma _i - \frac{1}{n}\left( \sum _{i=0}^{n-1}\gamma _i\right) \end{aligned}$$

(13)

Fig. 4.

Variability of the shape disorder over time. Shape disorder over time for (a) smoke region, (b) man made object or non smoke region, Histogram of (c) signal shown in (a), and (d) signal shown in (b). (axes are adjusted for display).

where, $\gamma _i$ is the shape disorder of a particular block in $i^{th}$ frame and n is the number of frames under observation. Time profile of shape disorder signal and its corresponding histogram (for zero mean signal) is shown in Fig. 4. It is evident from the signal profile and its corresponding histogram that value of $\delta$ is high for smoke region compared to non-smoke region.

3.5 Fuzzy Membership Assignment to Features

These estimated features ($\alpha$, $\beta$, $\gamma$, and $\delta$) have varying range and very fuzzy in nature. Pre-determined threshold won’t be sufficient to discriminate smoke and non-smoke regions. We have assigned a membership to these features. Triangular and S shaped memberships are used for these features.

Triangular membership is denoted as

$$\begin{aligned} f(x;a,b,c) = {\left\{ \begin{array}{ll} 0; &{} x\le a \\ \frac{x-a}{b-a}; &{} a\le x\le b\\ \frac{c-x}{c-b}; &{} a\le x\le b\\ 0; &{} c\le x\end{array}\right. } \end{aligned}$$

(14)

S Shaped membership is denoted as

$$\begin{aligned} f(x;a,b) = {\left\{ \begin{array}{ll} 0; &{} x\le a \\ 2\left( \frac{x-a}{b-a}\right) ^2; &{} a\le x\le \frac{a+b}{2}\\ 1-2\left( \frac{x-b}{b-a}\right) ^2; &{} \frac{a+b}{2}\le x\le b\\ 1; &{} x\ge b\end{array}\right. } \end{aligned}$$

(15)

an example of triangular membership (trimf) for f(x; 3, 6, 8) and S shaped membership (smf) for f(x; 1, 8) is shown in Fig. 5.

Fig. 5.

Membership functions, (left) Triangular membership for f(x; 3, 6, 8), (right) S shaped membership for f(x; 1, 8).

• 1. Triangular membership for $\alpha \mathbf{:}$ Due to smoke, there is a reduction in energy at a particular block with respect to the background. However, other moving objects may also increase or decrease the energy. Triangular membership function put a cut-off for abrupt energy reduction and increase. Here, assumption is that change in energy (both reduction and increase) due to rigid moving objects is very large. This fuzzy feature is termed as $f_\alpha$ where, $f_\alpha \; \epsilon \; [0, 1]$.

• 2. S shaped membership for $\beta \mathbf{:}$ For a smoke block, energy reduction remains consistent, but other moving objects give fluctuation in energy reduction which reduces the consistency of energy reduction. S shaped membership function put a lower bound cut-off to give low membership to other moving objects. This fuzzy feature is termed as $f_\beta$ where, $f_\beta \; \epsilon \; [0, 1]$.

• 3. S shaped membership for $\gamma \mathbf{:}$ Shape disorder is more in case of smoke in comparison with other moving objects. Value of shape disorder is high for smoke and low for non-smoke or other moving objects. S shaped membership function fulfills this requirement. This fuzzy feature is termed as $f_\gamma$ where, $f_\gamma \; \epsilon \; [0, 1]$.

• 4. S shaped membership for $\delta \mathbf{:}$ Due to random motion and shape disorder, smoke shape keep on changing over time. Other objects have fixed size and their shape do not change over time. S shaped membership function fulfills this requirement. This fuzzy feature is termed as $f_\delta$ where, $f_\delta \; \epsilon \; [0, 1]$.

3.6 Past Smoke History (PSH)

We have used one more feature termed as Past Smoke History (PSH). Here, past smoke history is used to increase classification accuracy. Past smoke history is defined based on the previous frame classification decision. This confidence is denoted as the likeliness of the features to represent the assigned class (i.e. smoke and non-smoke).

Classification is performed at block level (non overlapping blocks). We have estimated $f_\alpha$ and $f_\beta$ at block level and $f_\gamma$ and $f_\delta$ at blob level. For block level classification, for each block $f_\gamma$ and $f_\delta$ are assigned according to the corresponding blob i.e. all blocks fall inside or at the boundary of the blob are given same $f_\gamma$ and $f_\delta$. Estimated feature vector for $b^{th}$ block of $i^{th}$ frame is denoted as $f_{bi} = [f_{\alpha i},\; f_{\beta i},\; f_{\gamma i},\; f_{\delta i},\; PSH_{b(i-1)}]$. For each block, $f_{\alpha }$, $f_{\beta }$, $f_{\gamma }$, and $f_{\delta }$ are calculated from the current frame whereas, PSH is calculated from the last frame classification decision (see Eq. (17)). Inclusion of past smoke history as a feature avoid the toggling in classification decision in successive frames. Here, assumption is that a block classified as class $t_k\; (k\;\epsilon \;\{1,2\})$ in particular frame will more likely to represent same class in next frame.

Target class for smoke feature can be represented as $t_1 = [1, 1, 1, 1, 1]$ (i.e. high fuzzy membership to each feature and ideally 1) and for non-smoke feature can be represented as $t_2 = [0, 0, 0, 0, 0]$ (i.e. low fuzzy membership to each feature and ideally 0).

Steps for class assignment are as follows:

Estimation: For every $b^{th}$ block of $i^{th}$ frame, estimate feature $f_{bi}$

Matching: Each block is classified by estimating feature $f_{bi}$ and choosing the closet target $t_k$ using following equations:

$$\begin{aligned} \underset{k}{\mathrm {argmin}}\parallel f_{bi}-t_k\parallel \end{aligned}$$

(16)

where, $\parallel . \parallel$ represents $l^2$ norm

Update: update smoke history of each $b^{th}$ block for next frame using following equations:

$$\begin{aligned} PSH_{bi}=min\left( max\left( \left( PSH_{b(i-1)}+(-1)^{k-1}e^{-d}\right) ,0\right) ,1\right) \end{aligned}$$

(17)

where, $PSH_{bi}$ is the past smoke history of a $b^{th}$ block of $i^{th}$ frame, $d = \parallel f_{bi}-t_k\parallel$, $t_k$ is the assigned target class ($k =1$ for smoke and $k =2$ for non smoke).

For every frame, we find out all the blocks supporting smoke target class and these blocks will support some blobs in the mask obtained after transparent object detection. All these supporting blobs are marked as the potential smoke regions. For display, we drawn the boundaries of these blobs on original image. Boundaries of these blobs are obtained by Moore-Neighbor tracing algorithm modified by Jacob’s stopping criteria.

4 Experimental Results and Discussion

We implemented algorithm in C using OpenCV library on a system with a 3.29 GHz Intel(R) Core(TM) i3 CPU and 3.40 GB of RAM running on Ubuntu platform. Algorithm is running at 25 frames per second (fps). For quad core processor, CPU utilization is $25\%$, which ensures system can process 4 camera streams simultaneously. For temporal analysis, we assumed a batch of 5 frames and for spatial analysis we assumed a block of size 8. Frames are resized to $256\times 256$ and each block is of size $8\times 8$, thus, we have 1024 blocks and $1024\times 5$ features ($\alpha$, $\beta$, $\gamma$, $\delta$, and PSH) for each frame. For the first frame, we initialized all blocks with PSH equal to 0.5 which means all blocks are equiprobable for smoke and non smoke.

Table 1. Description of the videos used for the experiment.

We performed the qualitative and quantitative analysis of our smoke detection algorithm in outdoor and indoor conditions. Unique combination of features with past history make algorithm robust to all conditions. Videos (used in the experiment) descriptions and challenges in smoke detection are mentioned in Table 1. We performed experiments on all possible scenarios of smoke videos like sparse and dense smoke, illumination variation and background similarity. Results of the intermediate steps of proposed algorithm are shown in Fig. 6. Results show that above mentioned features are sufficient for the detection of all potential smoke regions.

Fig. 6.

Intermediate results of proposed smoke detection algorithm. (a) & (f) Input frame, (b) & (g) detection of all moving objects, (c) & (h) possible smoke region (after transparency criterion), (d) & (i) output of feature extraction and classification (yellow rectangle shows smoke region and blue rectangle shows non-smoke region), (e) & (j) potential smoke region (final mask). (Color figure online)

Fig. 7.

Qualitative results of proposed smoke detection algorithm.

Qualitative results of proposed algorithm are shown in Fig. 7. Results show that proposed algorithm detects smoke regions effectively without producing any significant false alarm. Figure 7a shows smoke in outdoor conditions. Smoke is behind the fence and a person with smoke color shirt is moving. For smoke region, energy reduction and shape disorder are high compared to other moving object regions which enables us to detect smoke effectively without any false alarm. Figure 7b shows dense smoke in front of a flat surface. As flat surface of background and dense smoke have less energy thus reduction in energy property is less reliable. However, shape disorder and variability in shape disorder properties enable us to detect smoke successfully. Figure 7c shows very sparse smoke which covered entire field of view. Due to coverage of entire field of view, shape disorder and variability in disorder are unreliable features. As smoke is sparse and blurred the background thus reduction in energy and its consistency are strong enough to support smoke detection. Figure 7d shows smoke present at far distance. Shape disorder is easy to detect but due to far distance variation in shape disorder is very low. However, reduction in energy and its consistency over time are supporting features. Transparent object detection is also a plus in this case. Figure 7e shows variation in illumination. As smoke grows up, it becomes less visible in bright sunlight but still algorithm is able to detect smoke. In Fig. 7f background is similar to the smoke. With the adaptive background modeling, we are able to extract smoke regions. As we do not rely on color features thus background similarity is not an issue. Here, smoke and background both are flat which makes energy features unreliable but shape disorder is strong enough to support successful detection. Results verify that proposed algorithm modeled with unique features which make it robust to all real life conditions.

Table 2. Performance evaluation of smoke detection algorithms.

Quantitative analysis is performed in terms of the detection rate (DR) and error detection rate (EDR), where the former indicates the ratio of the number of detected smoke regions to the total number of smoke regions whereas the later stands for the ratio of the number of wrongly detected smoke regions to the total number of smoke regions. The results for smoke localization for the proposed and prior art algorithms are mentioned in Table 2. Proposed algorithm outperformed the prior art algorithms with an average detection and error detection rate of $\thicksim$93% and $1.33\%$ respectively for all different scenarios. Results confirm that proposed algorithm is effective, robust and has a earlier smoke alarm. Proposed algorithm robustly detects smoke in a video stream in real time at 25 fps.

5 Conclusion

In this paper, we have proposed a novel visual smoke detection algorithm. Proposed algorithm is robust to detect smoke of any color. Algorithm can easily detect smoke and non-smoke regions due to spatio-temporal analysis of energy and shape disorder features. Detection at modular level (first moving objects detection, then transparent objects detection, then feature analysis and finally past history analysis) increases the efficiency of the algorithm. Use of data driven fuzzy thresholds avoids user intervention to select hard thresholds for features discrimination between smoke and non-smoke. Features extraction and classification are less complex in comparison with prior art algorithms which ensures real-time performance. Space-time analysis of unique features combination ensures high efficiency in comparison with other existing algorithms. Proposed algorithm has wide range of surveillance applications in outdoor and indoor conditions.