Learning Abnormal Behavioural

for Surveillance Systems

By Ahmed Ibrahim
23/2/2011
 Outline
• Introduction
• Key Challenges
• Proposed approaches
• Preliminary experiments
• Discussion and future directions
 What is Abnormal Behaviour ?
• Abnormal Behaviour is a pattern in the data
that does not conform to the expected normal
behaviour.
• Also referred to as outliers, exceptions,
suspicion, surprise, etc. F2
o
1 N 1
O
• Example: set of data points 3

Feature space.
o2
N2

F1
 Video Surveillance Example

A car on pedestrian roadway

A piece of luggage left in a check in-area

Source: Performance Evaluation of Tracking and Surveillance dataset (PETS)

 Key Challenges
• Defining a representative normal behaviour is
challenging.
• The boundary between normal and outlying
behaviour is often not precise.
• The exact notion of an outlier is different for
different video surveillance applications.
• Availability of labelled data for
training/validation.
• Data always contain noise.
• Normal behaviour keeps evolving
 Abnormal Behavior Detection
Framework

Source: PETS

Motion Detection Object Tracking Behavior Behavior Type

Understanding

• Background • Model • Classification • Label

subtraction • Region (supervised) • Score
• Temporal • Active contour • Clustering
differencing • Feature based (unsupervised)
• Optical flow
 Unsupervised Behavior Modeling
•The following trajectory
has been generating by:
•Applying principal
component analysis
on the video stream;
•Selecting the first
three components
•Every point on the
trajectory represents a
frame from the video
stream
 Learning Outliers

Statistically
Resampling

Gathering Visually
Real Data Resampling

Behavior
Model
Proposed Statistical Approach
Statistical Resampling Example
Subspace
trajectory of
waking pedestrian
in outdoor

Irregular segments

PC: for principal components

Proposed Visual Approach
 Visual Resampling Example
Time delay
Subspace
trajectory of
waking pedestrian

Subspace
trajectory of
synthetic
pedestrian

PC: for principal components Similar segments

 Behavior Model Output
• Label : each test instance is given a normal or
anomaly label
• Score: each test instance is assigned an
anomaly score
• Allows the output to be ranked
• Requires an additional threshold parameter
 Research Plan
• To test the feasibility of statistical resampling:
– The Ionosphere dataset from UCI machine learning
Repository will be used.
– This dataset are radar signals sent into the ionosphere and
the class value indicates whether or not the signal
returned information “Good” or “Bad” on the structure of
the ionosphere.
• To test the feasibility of visual resampling: A set of
animated videos with real backgrounds will be
generated for the following events:
– Walk, Run, Jump, Gallop sideways, Bend , One-hand wave,
Two-hands wave, Jump in place, Jumping Jack, Skip.
 References
• [1] Weiming Hu; Tieniu Tan; Liang Wang; S. Maybank; , "A survey on visual
surveillance of object motion and behaviors,"Systems, Man, and Cybernetics, Part
C: Applications and Reviews, IEEE Transactions, 2004
• [2] Monekosso, Remagnino, ”Synthetic Training Data Generation for Activity
Monitoring and Behavior Analysis” , Proceedings of the European Conference on
Ambient Intelligence, 2009.
• [3] Garg, Aggarwal, Sofat, ” Vision Based Hand Gesture Recognition”, 2009.
• [4] Poppe, “A survey on vision-based human action recognition”, Image and Vision
Computing Journal, 2010.
• [5] Mitra, Acharya, ”Gesture Recognition: A Survey”, IEEE transactions on systems,
man, and cybernetics, 2007.
• [6] Wang, Suter, “Recognizing Human Activities from Silhouettes: Motion Subspace
and Factorial Discriminative Graphical Model”, IEEE Conference on In Computer
Vision and Pattern Recognition, 2007.
• [7] Hua, “Probabilistic Variational Methods for Vision based Complex Motion
Analysis”, PhD dissertation, Electrical and Computer Engineering, 2006.
• [8] Incertis, Garcia-Bermejo, Casanova, "Hand Gesture Recognition for Deaf People
Interfacing", 18th International Conference on Pattern Recognition, 2006.