CONTENT BASED FACE

RECOGNITION
Ankur Jain 01D05007
Pranshu Sharma 01005026
Prashant Baronia 01D05005
Swapnil Zarekar 01D05001
Under the guidance
of
Prof. Pushpak Bhattacharya
Introduction

Problem Statement :
Given an image, to identify it as a face
and/or extract face images from it.
To retrieve the similar images (based on a
heuristic) from the given database of face
images.




Why face recognition ?
Various potential applications, such as

person identification.
human-computer interaction.
security systems.

Faces are complex, multidimensional
and meaningful visual stimuli.
Face Recognition is difficult.
Face Images are similar in overall
configuration.
Difference From Image Recognition
Approach
Similar to Content Based Image Retrieval
(CBIR).
Neural Networks and Self Organizing Maps
(SOMs).
Principal Component Analysis (PCA).
Relevance feed back.


Stages of Face Recognition
(1) face location detection
(2) feature extraction
(3) facial image classification

Approaches of Feature Extraction
(1) local feature : eyes, nose, mouth information
easily affected by irrelevant information .
(2) global feature :
extract feature from whole image .
Face Recognition Using
Eigenfaces
Face Images are projected into a feature
space (Face Space) that best encodes the
variation among known face images.
The face space is defined by the
eigenfaces, which are the eigenvectors of
the set of faces.

Eigen Space and Eigen Faces
Initialization :
Acquire the training set and calculate eigenfaces
(using PCA projections) which define eigenspace.
When a new face is encountered, calculate its weight.
Determine if the image is face.
If yes, classify the weight pattern as known or
unknown.
(Learning) If the same unknown face is seen several
times incorporate it into known faces.

Steps In Face Recognition
PCA
Main assumption of PCA
approach:
Face space forms a cluster in image space.
PCA gives suitable representation.

Eigenfaces (1)
Calculation of Eigenfaces
(1) Calculate average face : v.
(2) Collect difference between training images and
average face in matrix A (M by N), where M is the
number of pixels and N is the number of images.



(3) The eigenvectors of covariance matrix C (M by M)
give the eigenfaces.
M is usually big, so this process would be time consuming.


What to do?
T
AA C
Eigenfaces (2)
Calculation of Eigenvectors of C
If the number of data points is smaller than the dimension
(N<M), then there will be only N-1 meaningful eigenvectors.

Instead of directly calculating the eigenvectors of C, we can
calculate the eigenvalues and the corresponding
eigenvectors of a much smaller matrix L (N by N).


if
i
are the eigenvectors of L then A
i
are the eigenvectors
for C.
The eigenvectors are in the descent order of the corresponding
eigenvalues.

A A L
T

Eigenfaces (3)
Representation of Face Images using
Eigenfaces
The training face images and new face images can be
represented as linear combination of the eigenfaces.
When we have a face image u :



Since the eigenvectors are orthogonal :

i
i i
a u
i
T
i
u a
Eigenfaces (4)
Experiment and Results
Data used here are from the ORL database of faces.
Facial images of 16 persons each with 10 views are used.
- Training set contains 167 images.
- Test set contains 163 images.

First three eigenfaces :
Classification Using Nearest Neighbor
Save average coefficients for each person. Classify new
face as the person with the closest average.
Recognition accuracy increases with number of
eigenfaces till 15.
Later eigenfaces do not help much with recognition.


Best recognition rates
Training set 99%
Test set 89%
0.4
0.6
0.8
1
0 50 100 150
number of eigenfaces
a
c
c
u
r
a
c
y
validation set training set
Neural Networks
and
TS-SOM
What are Neural Networks ?
Individual units to simulate Neurons
Parallel Processing
Many inputs and single output
Organization/structure of the TLUs is
important
What is SOM ?
TS-SOM :- Tree structure self-organizing
maps
Competitive learning ANN
Each unit of map receives identical inputs
Units compete for selection
Modification of selected node and its
neighbors
Training of SOM
Randomly initialized
Selection based on some query parameter
On selection a node and its neighbors are
modified
Degree of modification reduces with each
iteration
Example of a two-dimensional TS-SOM structure of 3 levels
Algorithm
Calculate weight vector for first level.
Initialize weight vectors of other levels.
Calculate centroid associated to each
node as mean of closest training samples.
Iterate to the next level.


Relevance Feedback

System content based retrieval.
Point of human intervention
User analysis of system output.
User selects most relevant
Query iterated if output not satisfactory



Interaction Between User & System
1. A random set of faces is presented to the
user.
2. User interactive selection of faces.
3. System content-based face retrieval.
4. User analysis of retrieved faces.
Requested face was found -> Exit
Similar faces were found. -> Go to 2
No similar faces were found.
User tired -> Exit
User not tired (re initialization -> Go to 1


Comparison of the Two Approaches
Training time
Nearest neighbor is much faster.
Storage
About the same.
Classification time
Nearest neighbor is slightly slower.
Accuracy
Neural network is able to achieve the same accuracy
using 5 eigenfaces with nearest neighbor using 15, and a
higher accuracy when using 15.
Neural network models the problem better,
but takes more training time.
Future Work
Face Detection in motion pictures.
Detailed study of the proposed system
assuming PCA assumptions not to be true.
Investigate whether eigenfaces is a good
solution for this problem by comparing with
other feature extraction techniques such as
DCT

References
Navarrete P. and Ruiz-del-Solar J. (2002), Interactive Face
Retrieval using Self-Organizing Maps, 2002 Int. Joint Conf. on
Neural Networks IJCNN 2002, May 12-17, Honolulu, USA.
A tutorial on Principal Components Analysis, By Lindsay I
Smith.
Eigenfaces for Recognition, Turk, M. and Pentland A., (1991)
Journal of Cognitive Neuroscience, Vol. 3, No. 1, pp. 71-86.
Ruiz-del-Solar, J., and Navarrete, P. (2002). Towards a
Generalized Eigenspace-based Face Recognition Framework,
4th Int. Workshop on Statistical Techniques in Pattern
Recognition, August 6-9, Windsor, Canada.
Simulating Neural Networks by James A. Freeman.
Artificial Intelligence by Neil J. Nielsson.