Keywords

1 Introduction

According to [1] abnormal or unusual behavior are somehow interesting that catch the attention of human observers, and often quite easy to identify. In recent years, surveillance systems have been improved, the main reasons are the technological advances and the increasing availability of monitoring cameras in different environments [2,3,4]. The challenge of abnormal or unusual behavior detection for surveillance camera operators is that it requires a lot of attention without stop and is an exhausting process because the abnormal events occur with a low probability, making that the major part of the effort be wasted watching regular videos [1, 5]. Abnormal crowd behavior detection is a topic of interest in multiple fields as: computer vision, real-time applications [6,7,8] crowd behavior analysis, and others; capturing the attention of several research groups [9].

In the field of computer graphics [10], computer simulations [11, 12], and virtual reality [13]; models of people behavior have been used to simulate crowds in different scenarios [14, 15]. According to [16], the behavior of people in crowds could be different depending on situation; it makes more difficult to represent the dynamic models of a crowd.

Our approach is focused on fast abnormal events detection based on analysis of inter-frame pixel motion. We obtain a model of pixel density behavior in human crowds using statistical concepts, and replicate the abnormal crowd behavior in computer simulations.

This paper is organized as follows: Sect. 2 describes the related work on abnormal crowd behavior detection. Next, the statistical modelling and our algorithm are presented in the Sect. 3. In Sect. 4, we present the experimental results followed by the summary. Finally conclusions and future works are presented in Sect. 5.

2 Related Works

In the literature, several research groups have proposed methods to deal with the problem of abnormal human behavior detection in different contexts [1] like suspicious events [17], irregular behavior [18], abnormal behavior [19, 20], and others. ed to. Multiple authors have divided all these methods in two main approaches: model based detection and particle advection based detection. The choose of the model will depend of the density of the crowd to be analyzed [21].

Model based methods analyze individuals behavior using detection and segmentation, works like [22, 23] analyze the motion patterns using people tracking. In [24, 25] the extracted trajectories were used to represent normal patterns, atypical values were considered abnormal.

Particle advection based models [26,27,28] represents a holistic vision of a crowd. Common approaches in this field include: optical flow [14, 27, 28], gradients [29], spatio-temporal features [30], and others.

Our method for anomaly detection uses a particle advection based algorithm; we analyze pixel densities with statistical modelling to identify abrupt movements in the image and detect abnormal behavior.

3 Our Approach

For abnormal crowd behavior detection, we divide our method in three stages: Motion detection and motion analysis, statistical modelling of normal behavior, and abnormal crowd behavior detection and simulation.

3.1 Motion Detection and Motion Analysis

For motion detection, we use algorithm based on image differences in consecutive frames. First, we three consecutive frames are read: previous frame (or first frame), current frame and next frame (the algorithm could have one frame of delay). Then, we applicate RGB to Gray transformation to each frame for color channels simplification. We operate between images as color matrices obtaining the absolute difference between consecutive frames using the Eqs. 1 and 2:

$$ absdiff_{1} = \left| {Current\,Frame - Previous\,Frame} \right| $$
(1)
$$ absdiff_{2} = \left| {Next\,Frame - Current\,Frame} \right| $$
(2)

We apply the logical operation AND to these differences, finding pixels that have moved in current frame. This algorithm is graphically presented in Fig. 1.

Fig. 1.
figure 1

Motion detection algorithm

Full size image

With moved pixels, adjacent pixels are grouped to get little contours. Background subtraction approaches like [31] proposes that with the increase of the speed in the motion, increases the density of these grouped pixels. We group this moved pixels inside bounding boxes as show the Fig. 2.

Fig. 2.
figure 2

Motion pixel detection

Full size image

The Fig. 2 shows that the algorithm generates bounding boxes around moved pixels; the computational advantage of this approach is to reduce the image analysis to simple matrix operations.

3.2 Statistical Modelling

Normal Behavior Modelling

In the literature, several datasets for abnormal behavior detection have been created. For pixel density modelling, we get the average area of bounding boxes by frame during normal crowd behavior scenes. We use 3 video datasets focused on abnormal crowd behavior detection:

  • UMN dataset [28]: Abnormal behavior consist in panic situations resulting in evacuation or scape.

  • UCSD ped1 dataset [24]: Abnormal behavior represent different objects like bikes or cars with different speed and direction in comparison with common pedestrians.

  • Avenue Dataset [5]: Abnormal behavior are individuals running or moving with abnormal speeds respect to other pedestrians.

In order to have small variation in data, we set all videos to the same size (320 × 240 pixels). We tabulate the data for each frame. In total, 12154 frames were analyzed. The results of histograms are presented in Fig. 3.

Fig. 3.
figure 3

Histograms of pixel density

Full size image

Additionally we compute the mean, standard deviation and coefficient of variation given by the Eqs. 3, 4 and 5 respectively:

$$ \upmu = \frac{1}{N} \mathop \sum \limits_{i = 1}^{N} x_{i} $$
(3)
$$ \upsigma = \sqrt {\frac{1}{N}\mathop \sum \limits_{i = 1}^{N} \left( {x_{i} - \mu } \right)^{2} } $$
(4)
$$ {\text{CV}} = \frac{\sigma }{\mu } $$
(5)

In the Eqs. 3 and 4, \( N \) represents the total number of data, and \( x_{i} \) represents each single data (\( i = 1,2,3, \ldots N \)). As we can see in Fig. 3, all datasets present similar trends. Data of mean, standard deviation and coefficient of variability are tabulated in Table 1.

Table 1. Statistical values of pixel density
Full size table

According to standard deviation and coefficient of variation, Avenue dataset [5] is the most unstable for the application of this method.

Abnormal Crowd Detection

For abnormal crowd behavior we compute the atypical values in pixel densities. When an atypical value is detected, we will assume that crowd experiments an abnormal behavior. For this, we calculate the upper limit for atypical values given by the equation:

$$ Ul = Q_{3} + 3 IQR $$
(6)

Where \( Q_{3} \) represent the third quartile and \( IQR \) is the interquartile range. We only take values over the upper limit because there are not atypical values under lower limit. The interquartile range (IQR) is defined as:

$$ IQR = Q_{3} - Q_{1} $$
(7)

Where \( Q_{1} \) is the first quartile.

The value of upper limit for extreme atypical values in each dataset is presented in Table 2.

Table 2. Upper limits for different datasets
Full size table

The Table 2 shows that UMN dataset have the maximum upper limit, so we will take this value to implement the algorithm.

3.3 Abnormal Behavior Detection and Simulation Algorithm

With the atypical values calculated, we determinate that abnormal crowd behavior occurs when different pixels exceed upper limit density.

Simulation

We obtain the center points of bounding boxes during abnormal crowd in function of time that they appear. In video screens the position \( y = 0 \) begins at up-left corner, for this reason, we must subtract 240 minus “y” positions to match it with Cartesian coordinates. We present graphical results in Fig. 4.

Fig. 4.
figure 4

Simulation of abnormal crowd. (a) Video detection (b) Simulation points

Full size image

4 Results and Discussion

4.1 Execution Time

To evaluate the execution time we run the program in two computers with the following characteristics:

  • PC1: Processor Intel (R) Core (TM) i3, 2.4 GHz, 4.0 GB of RAM memory

  • PC2: Processor Intel (R) Core (TM) i7, 2.8 GHz, 7.7 GB of RAM memory

The results of evaluation are presented in Table 3.

Table 3. Execution time of the algorithm in datasets
Full size table

Table 3 shows that in computers with low and high features the algorithm works fast, thus the algorithm can be used in real time applications.

4.2 Algorithm Performance

For algorithm evaluation, we count the number of bounding boxes that present atypical values, defining a threshold of counted atypical values to determinate an abnormal behavior. In each threshold we tabulate the true positive and false positive rates. We compare our algorithm based on statistical pixel density model (SPDM) in two datasets: UMN [28] and UCSD ped1 [24] according to their respective ground–truth annotations. In the first dataset we evaluate our approach and social-force model presented in [28]. In the second dataset we compare our method with: MPPCA + SF [24], Adam [32], y subspace [33]. Results are presented in Fig. 5.

Fig. 5.
figure 5

ROC curves for abnormal crowd behavior datasets. (a) UMN dataset. (b) UCSD ped1 dataset

Full size image

In Fig. 5, our algorithm presents better performance that methods presented in the state of art for frame level analysis in two datasets. In UCSD ped1 [24], many approaches determinate the performance at pixel level, however this paper not deal with this task.

5 Conclusions and Future Work

Despite the videos for statistical modeling are different, they present similar features during density pixel estimation; it allows generating a global model that works in different applications.

Due to the low computational cost of image processing, the proposed method for motion detection works for real time applications as show the Table 3.

In our method, we use abrupt changes in pixel density to detect anomalies. This proposal for abnormal crowd behavior detection is robust compared with approaches in the literature.

In the future, we will improve the detection including pixel level analysis; however it is necessary to include more variables in the process. This will allow improving simulation and avoidance systems like [34,35,36].