Accepted Manuscript

A New K-medoids Clustering and Swarm Intelligence Approach to

Fire Flame Detection

Amin Khatami, Saeed Mirghasemi, Abbas Khosravi, Chee Peng Lim,

Saeid Nahavandi

PII: S0957-4174(16)30499-7
DOI: 10.1016/j.eswa.2016.09.021
Reference: ESWA 10885

To appear in: Expert Systems With Applications

Received date: 30 June 2015

Revised date: 12 September 2016
Accepted date: 12 September 2016

Please cite this article as: Amin Khatami, Saeed Mirghasemi, Abbas Khosravi, Chee Peng Lim,
Saeid Nahavandi, A New K-medoids Clustering and Swarm Intelligence Approach to Fire Flame De-
tection, Expert Systems With Applications (2016), doi: 10.1016/j.eswa.2016.09.021

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service
to our customers we are providing this early version of the manuscript. The manuscript will undergo
copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please
note that during the production process errors may be discovered which could affect the content, and
all legal disclaimers that apply to the journal pertain.
 ACCEPTED MANUSCRIPT

A New K-medoids Clustering and Swarm Intelligence

Approach to Fire Flame Detection

Amin Khatami1,∗

T
Institute for Intelligent Systems Research and Innovation
Deakin University, Geelong, VIC, 3217, Australia
e-mail: [email protected]

IP
Saeed Mirghasemi

CR
School of Engineering and Computer Science
Victoria University of Wellington, New Zealand
e-mail: [email protected]

Abbas Khosravi, Chee Peng Lim, and Saeid Nahavandi

US
Institute for Intelligent Systems Research and Innovation
Deakin University, Geelong, VIC, 3217, Australia
e-mail: abbas.khosravi, chee.lim, [email protected]
AN

Abstract
M

Automated computer vision-based fire detection has gained popularity in recent

years, as every fire detection needs to be fast and accurate. In this paper, a
ED

new fire detection method using image processing techniques is proposed. We

explore how to create a fire flame-based colour space via a linear multiplication
of a conversion matrix and colour features of a sample image. We show how
PT

the matrix multiplication can result in a differentiating colour space, in which

the fire part is highlighted and the non-fire part is dimmed. Particle Swarm
CE

Optimization (PSO) and sample pixels from an image are used to obtain the
weights of the colour-differentiating conversion matrix, and K-medoids provides
a fitness metric for the PSO procedure. The obtained conversion matrix can
AC

be used for fire detection on different fire images without performing the PSO
procedure. This allows a fast and easy implementable fire detection system. The

∗ Correspondingauthor
Email address: [email protected] (Amin Khatami)

Preprint submitted to Journal of LATEX Templates October 5, 2016

 ACCEPTED MANUSCRIPT

empirical results indicate that the proposed method provides both qualitatively
and quantitatively better results when compared to some of the conventional
and state-of-the-art algorithms.
Keywords: Fire detection; Particle swarm optimisation; K-medoids; Otsu’s

T
thresholding method; Contrast enhancement.

IP
1. Introduction

CR
Designing and deploying reliable and fast fire detection systems are crucial,
as billions of dollars are spent annually on fire detection systems to improve

US
home and office safety from the fire hazards. Recently, a variety of fire detection
systems have been developed based on sensors to detect smoke and fire (Gollner,
2016) (Ko et al., 2009) (Yu et al., 2005). However, there is always a significant
AN
delay by using sensors in fire detection systems because of the distance between
the fire area and the sensor location, especially in outdoor environments (Shen
et al., 2013). On the other hand, with regard to computer vision techniques,
M

advanced image and video processing techniques perform considerably faster

(Cho et al., 2008) (Wang et al., 2011) (Fleury et al., 2013). In general, a visual
fire detection system is defined on the basis of several perspectives, which include
ED

geometry, movement and colour pixel. In this manner, detecting fire flame pixels
as colour features is of great importance.
PT

Nowadays, researchers have developed many powerful techniques based on

colour analysis (Prema et al., 2014) (Chen et al., 2004). Chen et al. (Chen
et al., 2004) proposed a two-stage fire detection method. First, fire and smoke
CE

pixels are detected based on chromatic and dynamic features in RGB colour
space. Next, the fire pixels from the previous step are verified using disorder
AC

characteristic of flames and the growth of fire regions. In an another research,

Toreyin et al. (Töreyin et al., 2006) replaced the rules defined in (Chen et al.,
2004) with a temporal wavelet transform to detect the quasi-periodic behaviour
in flame boarders. Celik et al. (Celik et al., 2007) (Celik & Demirel, 2009)
(Celik, 2010) conducted some interesting investigations to define a new fire-

2
 ACCEPTED MANUSCRIPT

based colour space. In (Celik et al., 2007), they utilised statistical analysis
to extract foreground information using an adaptive background subtraction
algorithm, resulting in a real-time fire-detector technique. In (Celik & Demirel,
2009), the images are converted to YCbCr colour space to propose a new method

T
to segment fire pixels. This method accurately separates the luminance from the

IP
chrominance resulting in an effective fire-based colour space. Elmas and Yusuf
(Elmas & Sönmez, 2011) presented a detection system for forest fire images by

CR
investigating different neural network models. They utilised Artificial Neural
Network, Naive Bayes classifier, and Fuzzy Switching to propose a multi-agent
decision support system for fire detection.

US
Along with colour features, data mining techniques such as clustering and
classification methods have been deployed by researchers to better differentiate
between fire and non-fire components. Xuan and Kim (Xuan Truong & Kim,
AN
2012) proposed a useful method for automatically detecting fire in video frames.
Their proposed method consists of four stages. Firstly, an adaptive Gaussian
mixture technique is used to detect moving regions. Secondly, the Fuzzy C-
M

Means Clustering (FCM) method is used to choose candidate fire areas from
those regions. Thirdly, a discrete wavelet transform algorithm is used to extract
ED

the approximated and detailed coefficients of fire regions. Lastly, Support Vector
Machine (SVM) is applied to classify fire and non-fire pixels. Dai Duong et al.
(Dai Duong et al., 2011) presented a vision-based fire detection approach. They
PT

used an interval type-2 FCM algorithm to detect fire in video datasets. More-
over, Ho (Ho, 2013) used SVM to propose a new classification-based method
CE

to overcome the night-time restrictions for analysing smoke and fire related
video frames. In another research related to clustering, Chakraborty and Paul
(Chakraborty & Paul, 2010) utilised an unsupervised modified K-means clus-
AC

tering technique for detecting fire pixels. They used hierarchical and partition
clustering to provide a hybrid algorithm based on RGB and HSI colour spaces.
Zhao et al. (Zhao et al., 2015) also defined some rules to label flame pixels
of wildfire images based on a hierarchical detection system. The technique
gradually detects fire pixels using a range of features from low-level pixel-based

3
 ACCEPTED MANUSCRIPT

features to high-level video-based semantic features. FCM, SVM, and Bayes

classifier have been used for comparison in their approach.
As mentioned earlier, many studies have been conducted based on FCM and
K-mean techniques. Therefore, inspired by the effectiveness of applying such

T
techniques for designing fire flame detection systems, we employ K-medoids,

IP
which has some advantages over K-means and FCM, in this study. Accordingly,
K-medoids is utilised to distinguish between fire and non-fire pixels in a scheme

CR
that is described in the next section. Also, motivated by other multiplication-
based colour spaces such as YCbCr, we introduce a linear matrix multiplication
that results in a new colour space with colour-differentiating properties for fire
and non-fire regions.
US
This paper presents a new colour space to detect fire pixels in forest images.
The main aim is to achieve a conversion matrix describing a new fire flame-
AN
based colour space on the basis of colour properties. The proposed method is
mathematically simple both in feature extraction and colour space conversion.
The colour components of each pixel are directly extracted and used without any
M

further analysis. Also, a linear matrix multiplication forms the foundation of

the colour space conversion. Particle Swarm Optimization (PSO) is used as an
ED

effective tool to look for the weights of the conversion matrix. One interesting
property of the obtained conversion matrix is that it could be generalized to
other images. The aforementioned properties of the new algorithm make it
PT

simple and efficient for fire detection. Comparison to some other state-of-the-
art algorithms also shows that the proposed method is effective for fire flame
CE

detection.
The rest of the paper is organised as follows. Section 2 is devoted to a brief
description of the techniques we have used in this paper. A new algorithm to
AC

detect fire flame is proposed in section 3. In Section 4 the experimental results

and analyses are presented. Concluding remarks and future research directions
are provided in section 5.

4
 ACCEPTED MANUSCRIPT

2. Research Methods

Our fire detection algorithm utilizes PSO, K-medoids clustering, and Otsu’s
thresholding algorithms. Here we briefly describe these algorithms.

T
• K-medoids Clustering: the K-medoids clustering is a procedure that par-

IP
titions homogeneous data samples together based on feature(s) attributed
to each data sample, and a similarity metric. It optimises the absolute dis-

CR
tance between the data samples and the chosen centroid. The main reason
to choose this method over other clustering algorithms such as K-means,
is that K-medoids optimises the distance between the data samples by cal-

US
culating the L1 norm, while K-mean uses L2 norm. As such, K-medoids is
more robust to noise and outliers than K-means (Han et al., 2001). In con-
trast to K-means in which cluster centres are the average of the samples,
AN
K-medoids selects the data samples as centres.

An early method of K-medoids is called Partitioning Around Medoids

(PAM) (Kaufman & Rousseeuw, 1990). While PAM can perform well in
M

practical applications based on clustering, it is very time-consuming be-

cause of its structure. However, it does perform well with a small number
ED

of data points. In this study, we use PAM since the number of data points
in our fire detection algorithm is small. The following steps explain the
procedure of PAM:
PT

– Step 1: choose k data samples among n samples randomly as the

medoids, with no replacement.
CE

– Step 2: use the Euclidean metric to compute the distance for each
data sample to the nearest medoid.
AC

– Step 3: for each medoid, m, and non-medoid, o, by swapping m and

o step by step, calculate the total cost of the configuration. The total
cost among points is computed by the following equation:
c X
X n
Cost(X, V ) = d2 (xj , vi ) (1)
i=1 j=1

5
 ACCEPTED MANUSCRIPT

T
IP
CR
US
AN
M

Figure 1: Block diagram of the proposed hybrid intelligent algorithm for fire detection
ED

where X = {x1 , x2 , ..., xn } contains the data samples, n is the to-

tal number of data samples, c is the number of clusters. V =
PT

{v1 , v2 , ..., vc } contains the cluster centres, vi is the centre for the
ith cluster, and d2 (xj , vi ) is the distance measure between xj and vi .
CE

– Step 4: choose the best configuration according to the lowest error.

– Step 5: repeat the above steps until no change occurs.

AC

• Particle Swarm Optimization (PSO): PSO is an iterative optimization

technique originally proposed in (Eberhart & Kennedy, 1995) motivated
by social behaviour of mechanisms particularly birds flocking. It is mod-
elled by multi-dimensional particles in which each individual particle is
regarded as a potential solution for an optimisation problem, and is able

6
 ACCEPTED MANUSCRIPT

to move toward the best position with respect to a fitness function. The
fitness function determines the objective of the search procedure based on
four parameters; xi , vi , Pbest,i , and Gbest . xi and vi , i ∈ {1, 2, ..., n} are
the current position velocity vector, for the i − th particle, Pbest,i , in a

T
minimization problem, is the best position vector (the smallest value) for

IP
the i − th particle, which is calculated by:

CR
 Pt if f (xt+1 t
) > Pbest,i
t+1 best,i i
Pbest,i = (2)
 xt+1 if f (xt+1 t
) ≤ Pbest,i
i i

where f : <n 7→ < is the fitness function, and t = {0, 1, ..., N } denotes the

US
iteration number. The global best position, Gbest , is the best-fit particle
in the neighbourhood. Equation (3) shows how Gbest is computed at time
AN
step t.

t
Gbest = min{Pbest,i }, where i ∈ {1, 2, ..., n}, and n > 1 (3)
M

Each individual particle is updated by a velocity vector in order to change

its position. The velocity and position vectors are calculated by:
ED

t+1 t t t
vij = ω × vij + c1 r1j [Pbest,i − xtij ] + c2 r2j
t
[Gbest − xtij ] and
(4)
xt+1 t t+1
ij = xij + vij
PT

where
CE



 t
vij is the velocity vector of particle i, in dimension j, at iteration t;





 xtij is the position vector of particle i, in dimension j, at iteration t;




 c and c
AC

1 2 are positive numbers used for levelling the contribution


 between cognitive and social components;





 t
r1j t
and r2j are stochastic numbers in [0,1] at iteration t;




 ωt is interia weight.
(5)

7
 ACCEPTED MANUSCRIPT

The inertia weight, controls the impact of the previous velocities on the
current velocity, and changes through the following equation:

w = (winitialize − wf inal )(tmax − t)/tmax + wf inal (6)

T
where winitialize initializes the value of the weight vector, and wf inal is the

IP
final weight vector, t is the iteration number, and tkmax is the maximum
number of iterations.

CR
It should be taken into account that PSO is a comprehensive search, seek-
ing the optimal solution subject to the given constraints and fitness func-
tion defined in its search space.
US
• Otsu’s threshold technique: The Otsu’s threshold technique is an unsu-
AN
pervised algorithm for automatic image threshold. It was first proposed
by Nobuyuki Otsu (Otsu, 1975) to reduce a grey-level image to a binary
image, assuming that the image contains two class of pixels. The tech-
M

nique biases toward the class with a larger variance, and is equal to the
average of the mean of two classes divided by the variance. In other words,
the threshold operation separates the pixels of an image into two classes,
ED

at a grey-level threshold, T , calculated as follows:

PT

2
T = max{σB (T )}, f or 0 ≤ T ≤ L (7)

2
where σB is a measure to determine the separability of threshold T .
CE

2
σB (T ) = [µT (w(T ) − µ(T )]2 /{w(T )[1 − w(T )]} (8)
AC

where

T
X T
X L−1
X
f 2w(T ) = Pi , µ(T ) = iPi , µT = iPi , Pi = ni /N, Pi > 0
i=0 i=0 i=0

(9)

8
 ACCEPTED MANUSCRIPT

and L is 256, the total number of grey levels. Variables ni and N are
the number of pixels at level i, and the number of pixels in the image
respectively.

T
3. The Proposed Fire Detection Algorithm

IP
This study introduces a new conversion-based target-oriented colour space
using PSO and K-medoids clustering algorithm. The new algorithm is simple to

CR
apply, effective to detect fire pixels among non-fire ones in the background of an
image, and computationally efficient. Our algorithm obtains a conversion matrix
to convert an RGB colour image to a new colour space using a linear matrix

US
multiplication. To determine the weights of the conversion matrix, sample pixels
are required from the fire flame and non-fire flame regions. One challenging point
AN
in finding the conversion matrix is the wide range of fire flame colours, because
different materials burn with different flame colours. Sometimes the flame is
blue, especially in industrial locations, and sometimes it shows a structure of
M

nested rings of colours, changing from white in the core to yellow, orange, and
red in the border. To overcome this difficulty, initially, we restrict our study
to some specific datasets such as forest fire images, since, generally in forest
ED

environments, the flame colours are in the ranges of red and yellow.
Fig. 1 shows the block diagram of the proposed method. The main idea is
PT

to produce a conversion matrix that has colour-differentiating properties. We

use PSO to search for the proper weights of the conversion matrix, and we
use sample pixels from fire and non-fire flame regions to make the search to
CE

converge to a proper solution. K-medoids clustering takes part in performance

evaluation of the proposed conversion matrices throughout the PSO iterations.
The weights of the conversion matrix gradually evolve during the course of
AC

the PSO search in a way that the conversion leads to a differentiating colour
space. In fact, the differentiation is made between the colour components of two
limited sets of colour pixels that belong to fire and non-fire regions of an image.
In other words, PSO uses a limited set of pixels to produce a matrix that can

9
 ACCEPTED MANUSCRIPT

separate colour features via a conversion. Even though this conversion matrix is
obtained using colour features from a limited number of pixels, it has the ability
to be used to convert the whole image to the new colour space, and to show
differentiating properties. As we will show in the results section, the conversion

T
matrix obtained from one image in our image dataset could even be used for

IP
other images in the dataset. To intensify the effect of colour differentiating
on a test image, a non-linear contrast enhancement procedure is applied to

CR
the colour channels of the original image before conversion. Finally, to obtain a
binary image we apply the Otsu’s thresholding method to the contrast-enhanced
converted image. This hands in a threshold which through a hard thresholding

US
function produces a fire-classified image.

3.1. Feature Extraction

AN
Sample pixels are extracted for the construction of the conversion matrix.
We explain shortly that these sample pixels actually provide the PSO search
for the conversion matrix, with a metric to evaluate the goodness of a proposed
M

conversion matrix. To make the conversion more effective in terms of colour

differentiation, we manually select sub-images from different parts of fire and
ED

non-fire regions of an image. These sub-images form fire and non-fire sample
images (Fig. 2-b and 2-c). Next, colour components are extracted from the
pixels of these sample images to form two set of feature samples. Then, these
PT

sample features are used to determine the weights of the conversion matrix. In
this paper, sample images of both fire and non-fire are 25 × 25 pixels as depicted
in Fig. 2. Therefore, each sample image is a matrix of size 25 × 25 × 3, in which
CE

the third dimension represents the R, G, and B channels. Fig. 2-a shows the
original image from which sample images are extracted. Fig. 2-b,c are sample
AC

images of fire and non-fire respectively. Distinct blocks in each sample image
shows how they are collectively made of different regions of the original image.
Next, a feature matrix is formed to be used in the PSO search procedure. This
feature matrix is composed of the colour components of fire and non-fire pixels
from R, G, and B channels of the sample images. The feature matrix is of

10
 ACCEPTED MANUSCRIPT

T
IP
(a) (b) (c)

CR
Figure 2: (a) original target in RGB colour space. (b,c) feature samples

1250 × 3 size in which 1250 is the total number of pixels in fire and non-fire

US
sample images, and each column belongs to one of the colour channels of these
pixels.
AN
3.2. PSO Representation

In this study, PSO is used to search for the proper weights of the conver-
sion matrix. Therefore, the particles are potential solutions for the conversion
M

matrix. The conversion procedure that converts the sample features to the new
space is a linear matrix multiplication according to:
ED

y =x∗W (10)

where W is the conversion matrix, x represents the feature matrix in the RGB
PT

space, y is the corresponding feature matrix in the new colour space, and symbol
{∗} is the product operation. To keep the same number of colour channels in
CE

the new colour space, W has to be 3 × 3. This means that there are nine weights
of the W . Therefore, each particle, pi , is a nine-dimensional array containing
the weights of a conversion matrix as:
AC

pi = [xi1 , xi2 , ..., xi9 ] (11)

The values provided by a particle are used to convert the feature matrix to a
new space according to Equation 10. To evaluate how well a particle performs in

11
 ACCEPTED MANUSCRIPT

terms of colour separation, a metric that measures the differentiating properties

of that particular particle is formulated.

3.2.1. Fitness Evaluation

T
After a particle is used to convert the feature matrix to the new space, we
need to evaluate its colour differentiating properties. Fire and non-fire pixels

IP
occupy the upper and lower half of the joint feature matrix respectively, or vice
versa. Ideally, the converted feature matrix has to represents pixels in the new

CR
space in a way that the fire and non-fire pixels stay in their own half. The
K-medoids clustering is utilized to measure if there is any change of positions in

US
the new space. For this a two-class K-medoids clustering is performed on each
column of x, to obtain the class of each pixel from the sample sub-images before
conversion. As mentioned before, each column of x is the colour components
AN
from one of R, G, and B channels. This is followed by another two-class clus-
tering of columns of y to obtain the classes of the converted pixels. Therefore, a
well-separating conversion matrix should have the minimum number of classes
M

changes before and after conversion. Equation (12) and (13) show Equation 10
being substituted in the K-medoids objective function:
ED

n X
X c
J(U, V ) = kyk − vi k2 (12)
k=1 i=1
PT

n X
X c
J(U, V ) = k(x ∗ W )k − vi k2 (13)
k=1 i=1

where c is the number of clusters. V = {v1 , v2 , ..., vc } contains the cluster

CE

prototypes in which vi is the centre of the i − th cluster, and U contains all the
data samples. We use the Euclidean distance as the similarity measure between
AC

a data sample and a cluster centre.

Having the class label of pixels in x and y we calculate the number of mis-
classified pixels according to:

Error V alue = (en1 + en2 ) (14)

12
 ACCEPTED MANUSCRIPT

where Error Value is the number of misclassified pixels, en1 is the number of
pixels in the first half of the converted feature matrix with the wrong class label,
and en2 represents that of the second half of the converted feature matrix.
The Error Value actually measures the clustering error of K-medoids due

T
to malfunctioning of the linear conversion which is the result of inappropriate

IP
weights for the conversion matrix. Since we have three colour channels, the
Error Value is calculated for each channel. Then the highest value is returned

CR
as the fitness value to reflect the worst performance of the particle under con-
sideration. To have the most colour differentiating conversion matrix, the Error
Value should be as minimum as possible.

US
Having nine elements in the conversion matrix, and the Error Value that is
computed using K-medoids, it is useful to engage PSO to search for the optimum
solution.
AN
3.2.2. Summary of the PSO procedure
Overall, the following steps summarize how the PSO procedure performs:
M

• Step 1: Initializing the K-medoids and PSO algorithm parameters, which

include the population size, iteration number, dimension size, winitialize ,
ED

wf inal , and the minimum and maximum velocity of particles. The first
solution is suggested randomly.
PT

• Step 2: Using Equation (10) to convert the feature matrix to the new
colour space.
CE

• Step 3: Applying the K-medoids clustering procedure to x and y to obtain

the class of the sample pixels before and after conversion.
AC

• Step 4: Computing the misclassification in each colour channel of y using

Equation (14) as the fitness metric.

• Step 5: Using equation (4) to update the position and velocity vectors of
particles according to current Pbest and Gbest .

13
 ACCEPTED MANUSCRIPT

• Step 6: Repeating step 2 to 5 until PSO terminates and the optimal W is

found.

3.3. Contrast Enhancement

T
Enhancing an image provides better contrast and a more detailed image
as compared to a non-enhanced image. The techniques in this field are used

IP
to emphasize and sharpen image features for display and analysis. Contrast
enhancement is the process of manipulating an image intensity values so that

CR
the result is more suitable than the original for a specific application. Therefore,
depending on the application, an empirical design is required for the best results.

US
To have a high degree of colour differentiation in our approach, a contrast
enhancement method is performed on each colour channel of the original RGB
image before conversion to the new space. This colour map adjustment makes
AN
the difference between fire and non-fire even more distinctive after the conver-
sion. We take Power-Law Transformation (PLT) (Gonzalez & Woods, 2006)
as the contrast enhancement technique in this paper. Let’s assume the input
M

image grey levels ranges between [Ilow , Ihigh ], and we want to stretch them to
the range [Idown , Itop ] in the output image. PLT also known as ”gamma trans-
ED

formation” is the technique that performs this mapping through the following
formula:
PT

s = c × rγ (15)

in which i and j are the pixels in the input and output image respectively, and
CE

c and γ are positive constants. γ is a constant that determines the shape of

the curve, which is describing the relationship between the input and output
intensity values. Various values of γ results in different transformation curves,
AC

and different levels of enhancement as shown in Fig. 3 (Gonzalez & Woods,

2006). If γ is between 0 and 1, the transformation weights the mapping toward
higher (brighter) output values. If γ is greater than 1, the mapping is performed
toward lower (darker) output values. If the γ = c = 1, the mapping is a linear
one as depicted in Fig. 3.

14
 ACCEPTED MANUSCRIPT

T
IP
CR
US
Figure 3: Plots of the Equation 15 for various values of γ (c = l in all cases). All curves were
scaled to fit in the range shown (the Figure adopted from (Gonzalez & Woods, 2006)).
AN
In our contrast enhancement via PLT, for a sample image, the minimum and
maximum intensity values of each colour channel are termed as Imin and Imax
M

respectively. All the intensity values in the interval [Imin , Imax ] are mapped to
the interval [0, 255]. By default, 1% of the image is saturated at the low and
high intensities of the original image. Since, fire flame important information
ED

mostly relies on channel R, for a more effective colour differentiation in the new
colour space, R colour components have to be brightened, and G and B colour
PT

components have to be darkened through PLT. For this, γ is empirically fixed

at 1.5, 0.7, and 0.9 for R, G, and B channels respectively.
CE

3.4. Thresholding Based on Otsu’s Method

The contrast enhanced image is converted to the new colour space using the
PSO-evolved conversion matrix. The resulting image has good fire differentiat-
AC

ing properties. To provide quantitative evaluation, we need to have a binary

image in which each pixel comes with a label. Since the image is already colour
differentiated for fire flames, using a simple intensity-based thresholding could
provide us with a binary image. To obtain the threshold value, we use Otsu’s

15
 ACCEPTED MANUSCRIPT

method. Otsu’s thresholding algorithm easily provides a dynamic threshold ac-

cording to the intensity properties of a grey-level image. The image in the new
colour space is converted to a grey-level image, and then the Otsu’s thresholding
method is applied to it to obtain the proper threshold value. Next, we apply

T
a hard thresholding on the intensity values of the grey-level image using the

IP
obtained threshold value to generate a binary image.

3.5. Parameter Setting

CR
All the parameters related to PSO and PLT are depicted in Table 1.

Table 1: Parameter settings of the proposed method.

Parameter US Value
AN
Population size 50
Iteration number 120
Search space dimensionality 9
M

c1 and c2 in PSO 1
winitialize 1
wf inal 0.5
ED

c in PLT 1
γR (γ for channel R) 1.5
PT

γG (γ for channel G) 0.7

γB (γ for channel B) 0.9
CE

4. Experimental Result
AC

4.1. Dataset and Evaluation

The lack of standard datasets for fire flame detection limits us to some
particular cases. To cover a wide diversity of images, a dataset of more than
one hundred images of forest environments is formed. These images are collected

16
 ACCEPTED MANUSCRIPT

from different sources on the Internet. About fifty percent of these images are
of low quality, and contain a high degree of colour similarity. Since the ground-
truths are not available for this dataset, we have manually created them, in
order to validate the effectiveness of the proposed method quantitatively in

T
comparison to other methods. Fig. 4 shows some of the images from our

IP
dataset with the corresponding ground-truths. It is worth mentioning that
fire images in forest environments usually contain trees, land surfaces, grasses,

CR
and sometimes sky as the background. Therefore, it is not expected to see
complicated backgrounds in these images. Also, most of the images have fire
targets with low colour-similarity to other regions representing the background.

US
Nevertheless, to validate our proposed method under more difficult criteria,
50 images in our dataset have more complexity and a higher degree of colour
similarity compared to the rest of the images.
AN
During the experiments, one image from the dataset is first chosen to extract
the sample fire and non-fire images. Then, the obtained conversion matrix
using the sample images via the proposed algorithm, is generalised to other
M

images in the dataset for fire detection. The results are compared with three
other methods, the new colour spaces introduced in (Khatami et al., 2015b),
ED

(Khatami et al., 2015a), and SVM (Cortes & Vapnik, 1995). The fire detection
in (Khatami et al., 2015b) is the PSO and K-medoids procedure introduced in
this paper followed by Otsu’s thresholding for a binary image. (Khatami et al.,
PT

2015a) introduces a fire detection algorithm similar to (Khatami et al., 2015b)

but uses FCM clustering algorithm instead of K-medoids. The well known
CE

SVM as a powerful classification method has also been commonly used as a

comparison method for fire flame detection in recent researches (Zhao et al.,
2015) (Truong & Kim, 2012) (Habiboğlu et al., 2012).
AC

For quantitative comparison, we take into account three metrics. These

metrics are detection accuracy (DA), true positive rate (TP), and false positive
rate (FP). These measures are commonly used as evaluation metrics in fire
detection systems (Truong & Kim, 2012) and (Rong et al., 2013). DA is the
proportion of the number of truly classified pixels (fire and non-fire) to the total

17
 ACCEPTED MANUSCRIPT

T
IP
CR
US
AN
M
ED
PT

(a) (b) (c) (d) (e) (f)

Figure 4: Sample images from our dataset with the corresponding ground-truths. Columns
CE

(a, c, e) are the original images. Column (b, d, e) are the images with ground truths.
AC

18
 ACCEPTED MANUSCRIPT

number of pixels. The bigger the DA, the better the detection accuracy. TP is
the proportion of accurately detecting a real fire pixel as a fire pixel, and FP is
that of a real fire pixel as a non-fire pixel. Therefore, TP is the proportion of
the number of truly classified fire pixels to the number of all the fire pixels, and

T
FP is the proportion of the number of misclassified fire pixels to the number of

IP
all the fire pixels. Higher values of TP and lower values of FP indicate better
accuracies.

CR
4.2. Qualitative Results
Applying the proposed approach using the sample images in Fig. (2), the

US
conversion matrix in equation (16) is obtained. Then, this conversion matrix is
applied to all the images in our dataset to create new images in the new colour
space. This generalization ability of the proposed method changes its applica-
AN
bility to a much faster procedure that involves a PLT contrast enhancement,
and a linear matrix multiplication followed by a Otsu-based thresholding which
makes it computationally very efficient.
M

 
3.2753 1.9701 1.8017
 
 
W =  −0.0269 −0.0774 0.2938  (16)
ED

 
−3.0439 −1.9676 −2.3011
PT

To see the colour discriminating properties of this conversion matrix, Fig.

5 is provided. This figure shows the image in Fig. 2 and two other randomly-
CE

selected images in the original and the proposed colour spaces. Fig. 5-a shows
the original images. Fig. 5-b shows the converted images applying the proposed
contrast enhancement strategy and W in Equation (16). Fig. 5-c shows the
AC

results after applying the intensity thresholding to the converted images based
on Otsu’s method. This figure shows that the new space is able to divide the
images to two clusters with substantial colour differences between fire regions
and their background. Also, one could see that the idea of generalization to
other images works effectively.

19
 ACCEPTED MANUSCRIPT

T
IP
CR
(a) US(b) (c)
AN
Figure 5: Depiction of colour differentiating properties of the new colour space. (a) Images
in the original RGB colour space. (b) Contrast enhanced images in the new colour space. (c)
Final binary images after applying Ostu-based thresholding to the images in column (b).
M

Fig. 6 shows a comparison between the new colour space and some other
classic colour spaces, including YIQ, HSV, and YCbCr. This illustrates the su-
ED

perior abilities of the new colour space in fire flame differentiation in comparison
to other colour spaces.
To observe the distribution of colour components of pixels in the original
PT

colour space (RGB) and the new colour space, a 3D plot of the colour com-
ponents of pixels of Fig. 2(a) before and after conversion is depicted in Fig.
CE

7. The converted image could be observed in Fig. 5 (the top row of column
(b)). This depiction indicates that the new colour space has two important
attributes. Firstly, it can differentiate between the colour components of fire
AC

flame and non-fire flame regions. Secondly, the variation of colour distribution
in the fire flame and non-fire flame pixels is reduced.
Fig. 8 shows some more fire detected results of the proposed method on
different images from our dataset. Also, Fig. 9 shows different forest images with

20
 ACCEPTED MANUSCRIPT

T
IP
CR
US
AN
M
ED

(a) (b) (c) (d) (e) (f)

PT

Figure 6: Comparing the images in the new colour space with some classic colour spaces.
Column (a): the original images. Converted images to YIQ colour space (column (b)), HSV
colour space (column (c)), YCbCr colour space (column (d)), and the proposed colour space
CE

(column (e)). Column (f) is final detection results after applying the Otsu-based thresholding.
AC

21
 ACCEPTED MANUSCRIPT

T
IP
CR
(a) (b)

US
Figure 7: Distribution of colour components of the image in Figure 2 before and after conver-
sion. (a) Colour distribution of the original image. (b) Colour distribution of the converted
image as depicted in the up row of column b, Figure 5.
AN
no fire flame pixels, and the final detection results with no detected fire pixels.
This shows that the proposed method is effective only on images containing fire
flames.
M

Applying the same conversion matrix as in Eq. (16), Fig. 10 shows eight
randomly-selected images from the dataset for a visual comparison to other
ED

methods. It can be inferred from the detection results that the proposed method
performs mostly better than SVM and (Khatami et al., 2015a). Moreover, the
images depicted in Fig. 11 are some samples with higher colour similarity in the
PT

background. Visually analysing the results shows that SVM incorrectly detects
some non-fire regions such as smoke regions, and (Khatami et al., 2015a) fails
CE

to detect real fire pixels in a few cases. The proposed method could also be
problematic when there are fire-like regions in the image. For a further more
accurate comparison between the methods a quantitative evaluation is provided
AC

in the next sub-section.

4.3. Quantitative Results

This section presents some quantitative comparisons of the proposed method

to other methods. To compare the quantitative performance on individual im-

22
 ACCEPTED MANUSCRIPT

T
IP
CR
US
AN
M

(a) (b) (c) (d) (e) (f)

ED

Figure 8: Final detection results on more images from the dataset. (a,c,e) are the original
images. (b,d,f) are final images produced by the proposed method.
PT
CE
AC

(a) (b) (c) (d) (e) (f)

Figure 9: Some images with no fire flame regions, and the corresponding detection results
produced by the proposed method having no fire-detected pixels.

23
 ACCEPTED MANUSCRIPT

T
IP
CR
US
AN
M
ED
PT
CE
AC

(a) (b) (c) (d) (e) (f)

24
Figure 10: Comparison of the proposed method and those in (Khatami et al., 2015b),
(Khatami et al., 2015a), and SVM (Cortes & Vapnik, 1995) for some sample images in the
dataset. Column (a): the original images. Column (b): ground truth images. Column (c):
detection results from proposed method. Column (d): detection results from the (Khatami
et al., 2015b). Column (e): detection results from SVM. Column (f) detection results from
(Khatami et al., 2015a).
 ACCEPTED MANUSCRIPT

T
IP
CR
US
AN
M
ED
PT

(a) (b) (c) (d) (e) (f)

CE

Figure 11: Comparison of the proposed method and those in (Khatami et al., 2015b),
(Khatami et al., 2015a), and SVM (Cortes & Vapnik, 1995) for some sample high-colour-
similarity images in the dataset. Column (a): the original images. Column (b): ground truth
AC

images. Column (c): detection results from proposed method. Column (d): detection results
from the (Khatami et al., 2015b). Column (e): detection results from SVM. Column (f)
detection results from (Khatami et al., 2015a).

25
 ACCEPTED MANUSCRIPT

Table 2: Quantitative comparison of the proposed method, (Khatami et al., 2015b), (Khatami
et al., 2015a), and SVM (Cortes & Vapnik, 1995) for the images in Fig. 10

Proposed method (Khatami et al., 2015b) (Khatami et al., 2015a) (Cortes & Vapnik, 1995)
Fig. 10
TP% FP% DA% TP% FP% DA% TP% FP% DA% TP% FP% DA%

T
(row 1st ) 84.69 15.31 96.23 72.86 27.14 93.54 76.37 23.63 94.34 67.34 32.66 92.23
(row 2nd ) 98.87 1.13 97.57 97.77 2.23 98.3 88.27 11.73 98.28 99.41 0.59 84.59

IP
(row 3th ) 66.37 33.63 97.1 33.33 66.67 94.78 44.99 55.01 92.41 54.61 45.39 96.2
(row 4th ) 78.1 21.9 89.19 41.44 58.56 81.39 19.16 80.84 57.06 34.01 65.99 79.36
(row 5th ) 92.94 7.06 94.83 78.36 21.64 94.65 86.05 13.95 93.17 57.38 42.62 90.71

CR
(row 6th ) 86.45 13.55 95.89 61.93 38.07 90.96 24.77 75.23 82.77 85.9 14.1 95.43
(row 7th ) 73.13 29.87 97.17 62.42 37.58 96.49 57.2 42.8 96.12 79.97 20.03 97.44
(row 8th ) 88.99 11.01 97.91 49.76 50.24 95.22 23.36 76.64 92.85 84.1 15.9 96.57

US
ages, Tables 2 and 3 are provided that contain the DA, TP, and FP metrics for
the images in Figures 10 and 11 respectively. Analysing the results in Table 2
AN
indicates that, in terms of TP and FP, the proposed method always performs
better than other methods except for the images in the 2nd and 7th rows on
which our method is the second-best performer after SVM. In terms of DA,
M

our method is the best performer except for the images in the 2nd and 7th
rows again. For the image in the 2nd row our method does the third-best after
ED

(Khatami et al., 2015a) as the best performer, and (Khatami et al., 2015b) as
the second-best performer, and for the image in the 7th row our method is the
second-best performer after SVM.
PT

The same trend of improvements is also observed in Table 3 for images of

higher colour complexity. In terms of TP and FP, except for the image in the
1st row of Figure 11, the proposed method is always the best performer. For
CE

the image in the first row, our method is the second best performer after SVM.
In terms of DA, for the images in the 2nd, 3rd, and 5th rows, (Khatami et al.,
AC

2015b) is the best performer, for the image in the 1st row, (Khatami et al.,
2015a) is the best performer, and for the image in the 4th row the proposed
method is the best performer.
The overall mean quantitative performances of all the methods on all the
images in our dataset are provided in Table 4. This table shows that in terms of

26
 ACCEPTED MANUSCRIPT

Table 3: Quantitative comparison of the proposed method, (Khatami et al., 2015b), (Khatami
et al., 2015a), and SVM (Cortes & Vapnik, 1995) for the images in Fig. 11

Proposed method (Khatami et al., 2015b) (Khatami et al., 2015a) (Cortes & Vapnik, 1995)
Fig. 11
TP% FP% DA% TP% FP% DA% TP% FP% DA% TP% FP% DA%

T
(row 1st ) 97.77 2.23 76.71 88.98 11.02 87.27 84.47 15.53 93.5 98.06 1.94 76.86
(row 2nd ) 98.95 1.05 96 91.06 8.94 98.48 88.75 11.25 98.38 78.63 21.37 97.52

IP
(row 3th ) 97.92 2.08 91.04 85.99 14.01 96.15 67.81 32.18 92.22 96.04 3.96 84.11
(row 4th ) 77.86 22.14 96.97 44.97 55.03 95.8 35.75 64.24 95.11 72.71 27.29 85.48
(row 5th ) 95.32 4.68 78.74 47.73 52.27 97.95 37.89 62.10 96.85 86.5 13.5 82.93

CR
TP and FP, the proposed method is performing significantly better than other

US
methods. Therefore, the proposed method detects fire regions more accurately
as compared to other methods, even on images with colour complexity. In terms
of DA, the proposed method is the second-best performer after (Khatami et al.,
AN
2015b) with only 0.51% lower performance. Therefore, the highest TP rate is
the main strength of the proposed method. One important reason that con-
tributes to this is the non-linear contrast enhancement proposed in this paper.
M

Also, when compared to the algorithm proposed in (Khatami et al., 2015a), the
utilization of K-medoids instead of FCM is an advantage. K-medoids selects
ED

data samples (medoids) as cluster centres, while FCM considers a cluster by the
average value (centroid) of its associated data samples. In addition, K-medoids
clustering is effective in handling noise and outliers (Han et al., 2001). In other
PT

words, a medoid is appropriate to derive a representative tendency from its cen-

tral sample, even in skewed distributions. Indeed, a medoid is a data sample in
the data space, while a centroid may lie outside the data space.
CE

Table 4: The overall quantitative evaluation of the proposed method, (Khatami et al., 2015b),
(Khatami et al., 2015a), and SVM (Cortes & Vapnik, 1995) for the whole dataset
AC

Proposed method (Khatami et al., 2015b) (Khatami et al., 2015a) (Cortes & Vapnik, 1995)

TP% FP% DA% TP% FP% DA% TP% FP% DA% TP% FP% DA%

All Images 86.24 13.76 93.40 57.01 42.99 93.91 45.66 54.34 92.63 70.07 29.93 90.65

One the other hand, the weakness of the proposed method can be observed

27
 ACCEPTED MANUSCRIPT

when there is a high degree of colour similarity between different components of

an image. This causes non-fire regions get detected as fire, and decreasing the
detection accuracy.

T
5. Conclusions and Future Work

IP
Developing a robust fire detection system is a real necessity. A number of
successful approaches have been applied to address this need. They are different

CR
in response time and accuracy. Conventional clustering and classification algo-
rithms are not sufficient to solve real-world fire detection problems. A hybrid
intelligent algorithm was introduced to handle a two-class clustering problem

US
for detection of fire flames in colour images via a linear colour space conver-
sion. We used PSO to look for proper weights of the conversion matrix, and
AN
K-medoids as a fitness metric for the PSO procedure. Converting a contrast en-
hanced version of a sample image using the obtained conversion matrix produces
a well-differentiating colour space for fire flame pixels. The proposed method
M

is fast, and performs well when there is not a high degree of colour similarity
between fire and non-fire. One problem with the proposed approach is that it is
not effective on images with a high degree of colour similarity between fire and
ED

non-fire pixels. Therefore, further investigations are necessary to tackle these

problems. For further work, we will devise a non-linear conditional conversion
PT

formula which allows the elimination of more false positive detections. In ad-
dition, utilising other features such as texture or neighbourhood information
along with the colour features could possibly increase detection accuracy. On
CE

the other hand, decreasing or increasing the colour channels in the new colour
space may offer more flexibility to differentiate a specific target. Also, applying
the proposed method to other fields such as medical image processing is another
AC

interesting direction.

28
 ACCEPTED MANUSCRIPT

References

Celik, T. (2010). Fast and efficient method for fire detection using image pro-
cessing. ETRI journal , 32 , 881–890.

T
Celik, T., & Demirel, H. (2009). Fire detection in video sequences using a
generic color model. Fire Safety Journal , 44 , 147–158.

IP
Celik, T., Demirel, H., Ozkaramanli, H., & Uyguroglu, M. (2007). Fire de-

CR
tection using statistical color model in video sequences. Journal of Visual
Communication and Image Representation, 18 , 176–185.

Chakraborty, I., & Paul, T. K. (2010). A hybrid clustering algorithm for fire

US
detection in video and analysis with color based thresholding method. In
Advances in Computer Engineering (ACE), 2010 International Conference
AN
on (pp. 277–280). IEEE.

Chen, T.-H., Wu, P.-H., & Chiou, Y.-C. (2004). An early fire-detection method
based on image processing. In Image Processing, 2004. ICIP’04. 2004 Inter-
M

national Conference on (pp. 1707–1710). IEEE volume 3.

Cho, B.-H., Bae, J.-W., & Jung, S.-H. (2008). Image processing-based fire
ED

detection system using statistic color model. In Advanced Language Processing

and Web Information Technology, 2008. ALPIT’08. International Conference
on (pp. 245–250). IEEE.
PT

Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine learning,

20 , 273–297.
CE

Dai Duong, H., Nguyen, D. D., Ngo, L. T., & Tinh, D. T. (2011). On approach to
vision based fire detection based on type-2 fuzzy clustering. In Soft Computing
AC

and Pattern Recognition (SoCPaR), 2011 International Conference of (pp.

51–56). IEEE.

Eberhart, R. C., & Kennedy, J. (1995). A new optimizer using particle swarm
theory. In Proceedings of the sixth international symposium on micro machine
and human science (pp. 39–43). New York, NY volume 1.

29
 ACCEPTED MANUSCRIPT

Elmas, Ç., & Sönmez, Y. (2011). A data fusion framework with novel hybrid
algorithm for multi-agent decision support system for forest fire. Expert Sys-
tems with Applications, 38 , 9225–9236.

Fleury, A. d. T., Trigo, F. C., & Martins, F. P. R. (2013). A new approach based

T
on computer vision and non-linear kalman filtering to monitor the nebuliza-

IP
tion quality of oil flames. Expert Systems with Applications, 40 , 4760–4769.

CR
Gollner, M. J. (2016). Detection and suppression of fires: A cornerstone of fire
protection engineering. Fire Technology, 52 , 1193–1196.

Gonzalez, R. C., & Woods, R. E. (2006). Digital Image Processing (3rd Edition).

US
Upper Saddle River, NJ, USA: Prentice-Hall, Inc.

Habiboğlu, Y. H., Günay, O., & Çetin, A. E. (2012). Covariance matrix-based

AN
fire and flame detection method in video. Machine Vision and Applications,
23 , 1103–1113.

Han, J., Kamber, M., & Tung, K. (2001). Spatial clustering methods in data
M

mining: A survey. harvey j. miller and jiawei han (eds.), geographic data
mining and knowledge discovery.
ED

Ho, C.-C. (2013). Nighttime fire/smoke detection system based on a support

vector machine. Mathematical Problems in Engineering, 2013 .
PT

Kaufman, L., & Rousseeuw, P. J. (1990). Partitioning around medoids (program

pam). Finding groups in data: an introduction to cluster analysis, (pp. 68–
CE

125).

Khatami, A., Mirghasemi, S., Khosravi, A., & Nahavandi, S. (2015a). An

AC

efficient hybrid algorithm for fire flame detection. In 2015 International Joint
Conference on Neural Networks (IJCNN) (pp. 1–6). IEEE.

Khatami, A., Mirghasemi, S., Khosravi, A., & Nahavandi, S. (2015b). A new
color space based on k-medoids clustering for fire detection. In Systems, Man,

30
 ACCEPTED MANUSCRIPT

and Cybernetics (SMC), 2015 IEEE International Conference on (pp. 2755–

2760). IEEE.

Ko, B. C., Cheong, K.-H., & Nam, J.-Y. (2009). Fire detection based on vision
sensor and support vector machines. Fire Safety Journal , 44 , 322–329.

T
Otsu, N. (1975). A threshold selection method from gray-level histograms.

IP
Automatica, 11 , 23–27.

CR
Prema, C. E., Vinsley, S., & Jegadeesh, J. S. (2014). Efficient colour based
fire pixel classification using image processing. In Applied Mechanics and
Materials (pp. 52–57). Trans Tech Publ volume 626.

US
Rong, J., Zhou, D., Yao, W., Gao, W., Chen, J., & Wang, J. (2013). Fire flame
detection based on gica and target tracking. Optics & Laser Technology, 47 ,
AN
283–291.

Shen, P., Zhang, L., Song, J., Xu, H., Zhou, L., Wei, W., & Zhang, W. (2013).
Dm642-based fire detection in video sequences using statistical color model.
M

International Journal of Digital Content Technology and its Applications, 7 ,

669.
ED

Töreyin, B. U., Dedeoğlu, Y., Güdükbay, U., & Cetin, A. E. (2006). Computer
vision based method for real-time fire and flame detection. Pattern recognition
letters, 27 , 49–58.
PT

Truong, T. X., & Kim, J.-M. (2012). Fire flame detection in video sequences
using multi-stage pattern recognition techniques. Engineering Applications of
CE

Artificial Intelligence, 25 , 1365–1372.

Wang, Y., Yu, C., Tu, R., & Zhang, Y. (2011). Fire detection model in tibet
AC

based on grey-fuzzy neural network algorithm. Expert Systems with Applica-

tions, 38 , 9580–9586.

Xuan Truong, T., & Kim, J.-M. (2012). Fire flame detection in video sequences
using multi-stage pattern recognition techniques. Engineering Applications of
Artificial Intelligence, 25 , 1365–1372.

31
 ACCEPTED MANUSCRIPT

Yu, L., Wang, N., & Meng, X. (2005). Real-time forest fire detection with
wireless sensor networks. In Proceedings. 2005 International Conference on
Wireless Communications, Networking and Mobile Computing, 2005. (pp.
1214–1217). IEEE volume 2.

T
Zhao, Y., Tang, G., & Xu, M. (2015). Hierarchical detection of wildfire flame

IP
video from pixel level to semantic level. Expert Systems with Applications,
42 , 4097–4104.

CR
US
AN
M
ED
PT
CE
AC

32