The Eigenfaces method

Face Recognition and Biometric Systems

 Plan of the lecture

Principal Components Analysis

„ main
i idea
id
Feature extraction byy PCA
„ face recognition
Eigenfaces
„ training
„ feature extraction

Face Recognition and Biometric Systems

 Literature

M.A.Turk, A.P.Pentland
Face Recognition Using Eigenfaces

Face Recognition and Biometric Systems

Phases of recognition

Detection Normalisation

Feature vectors Feature

comparison extraction

Face Recognition and Biometric Systems

Example...
Example

Face Recognition and Biometric Systems

Example...
Example

Face Recognition and Biometric Systems

Example...
Example

Face Recognition and Biometric Systems

 PCA – main issues
Dimensionality
„ input space
„ data
Coordinates
C di t system t suited
it d to
t a sett
Dimensionalityy reduction
„ reduction error
Principal Components Analysis (PCA)
„ statistical method
Face Recognition and Biometric Systems
 PCA – main issues
Input data set
„ vectors / points
„ input space
S
Space orthonormal
th l basis
b i
„ each vector from the input space can be
expressed as a linear combination of basis
vectors
„ basis vector defines a dimension

Face Recognition and Biometric Systems

 PCA – main issues

Dimensions are sorted

„ each basis vector assigned with a weight
Main directions of variance in the
input set
„ each direction generates one dimension
Dimension importance proportional
to variance
Face Recognition and Biometric Systems
 Input face space
Normalised image
„ described by pixel values
Image as a point in a space
„ e.g.
g 64x75 pixels
p – 4800 dimensions
Information excess
„ too many dimensions
Elimination of redundant information
„ relevant information must remain
„ feature extraction
Face Recognition and Biometric Systems
 PCA for feature extraction
Set of normalised images
„ input vectors (a.k.a. training set)
PCA:
„ finds new orthogonal basis
„ number of dimensions can be reduced
„ creates „face space”
Feature extraction by PCA
„ find a corresponding point
in a new space
p
„ works for any face image (not only from training set)
Eigenfaces
g method – PCA for face images
g
Face Recognition and Biometric Systems
 The Eigenfaces method
Training
1. create covariance matrix for the training set
2
2. calculate eigenvalues and eigenvectors
Š eigenvectors are the orthonormal basis and define
the face space
Š eigenvalues are associated with eigenvectors
Š eigenvalues proportional to variance
3
3. select eigenvectors
Feature extraction
„ face image mapped to the new space
(coordinates must be found)
„ anyy face image
g mayy be processed
p
Face Recognition and Biometric Systems
 Eigenfaces: training
Input vectors: u1,...,uM
„ u – N-dimensional vector
„ M – number of vectors in the set
1 M
Average vector: μ = ∑ ui
M i =1
Covariance matrix:
1 M Τ 1
C = ∑ (ui − μ)(ui − μ) or C = A ⋅ AT

M i =1 M
Face Recognition and Biometric Systems
 Training – example

Training set (M=4, N=3):

[1, 0, 2]
[[0,, 3,, 1]]
[4, 1, 2]
[3
[3, 0
0, -1]
Average vector, covariance matrix

Face Recognition and Biometric Systems

 Eigenfaces: training
Characteristic equation:
det((C − λ ⋅ I) = 0
„ eigenvalues (λ )

Eigenvectors (v)
( )
„ one for each eigenvalue
g
C⋅ v = λ ⋅ v
Jacobi method (numerical method)
„ OpenCV
p - cvEigenVV
g function
Face Recognition and Biometric Systems
 Training – optimisation
Problem: large size of covariance
matrix (NxN), e.g. 4800x4800
Trick: A⋅ A ⋅ v = λ ⋅ v
T

A ⋅ A ⋅ v' = λ '⋅v'
T

A ⋅ A ⋅ (A ⋅ v' ) = λ'⋅(A ⋅ v' )

T

λ = λ ' v = A ⋅ v'
Av’ – desired eigenvectors
Face Recognition and Biometric Systems
 Eigenfaces: training
Ei
Eigenvectors
t – properties
ti
„ dimensionality equal to input vectors
„ orthonormal
th l (length
(l th = 1)
„ sorted by corresponding eigenvalues
„ may be scaled to pixel value range
Eigenfaces – eigenvectors transformed
to images
„ example
„ dimensionality reduction
New space, less dimensions
Face Recognition and Biometric Systems
 Eigenfaces: training

C00 ... C0n

... ... ...

Cn0 ... Cnn

Normalised Covariance Eigenfaces

images matrix

Face Recognition and Biometric Systems

 Eigenfaces: feature extraction
F t
Feature extraction
t ti iinput:
t
„ set of eigenvectors and eigenvalues
(delivered by training)
„ normalised image
Projection: x' = ψ ⋅ x
T

„ ψ - matrix
t i with
ith eigenvectors
i t
„ x – normalised image after average face
subtraction
„ x’ – transformed vector
Face Recognition and Biometric Systems
 Feature extraction - example

2-dimensional space:
„ eigenvectors:
2 2 2 2
[ ; ] [[− ; ]
2 2 2 2
Vectors projection:
„ [3; 1], [-2; -2], [10, 9]
Di
Dimensionality
i lit reduction
d ti

Face Recognition and Biometric Systems

 Eigenfaces: feature extraction

ψ matrix can be cut to reduce dimensions

ψ ψ’ ψ’’

Feature vector element is a scalar product:

wi = vi ⋅ x
T
w = ψ' ⋅x T

Feature vector – cut projected vector x’

Face Recognition and Biometric Systems

 Eigenfaces: feature extraction
K1

K2

K3
...
...
Scalar products between
normalised image and Feature vector
eigenvectors
Face Recognition and Biometric Systems
 Back projection: face image
Feature vector – face description
„ information reduction
Back projection: face image recovered
from feature vector
„ reduced information are lost
P j ti error:
Projection
„ depends on similarity to the training set
„ 2D example
„ face images
g
Face Recognition and Biometric Systems
Face Recognition and Biometric Systems
Face Recognition and Biometric Systems
 Back projection: detection

Back projection of images:

„ face -> slightly modified face image
„ flower -> image
g similar to a face
Back projection error is higher for
non face images
non-face
Can be used as a verifier
„ threshold of accepted projection error

Face Recognition and Biometric Systems

 Summary
Eigenfaces – a basic face recognition
method
„ many derived methods
Training and feature extraction
Holistic approach
High speed
Average / low effectiveness
„ may be improved
Face Recognition and Biometric Systems
Thank you for your attention!

Face Recognition and Biometric Systems