You are on page 1of 66

# Image Transformation

Full Disclosure

## Szeliski Sec 2.1.2

Alexei Efros, UC Berkeley
Dr. Sohaib / Aamer Zaheer (LUMS)
Some slides from Steve Seitz

## Whats on the menu today?

Spatial and Geometric Transformation
What Can we do with it?
Registration anyone?

## Parametric (global) transformations

Translation, Scaling, Rotation, Shear

Warping
inverse of Transformation
Interpolation
Linear interpolation
Bilinear interpolation

## Some more in depth look

Image Transformations
image filtering: change range of image
g(x) = T(f(x))
f

T
x

## image warping: change domain of image

g(x) = f(T(x))
f

T
x

x
Source: Efros

Image Transformations
image filtering: change range of image
g(x) = T(f(x))
f

g(x) = f(T(x))
f

Source: Efros

## Parametric (global) warping

Examples of parametric warps:

translation

affine

rotation

perspective

aspect

cylindrical
Source: Efros

2D Transformation
Definition: A mapping from one 2D coordinate
system to another
Also called
spatial transformation,
geometric transformation,
warp

## Image Registration: Process of transforming

two images so that same features overlap

Reference
Image

Mission
Images

## Source: Dr. Sohaib / Aamer Zaheer

Panoramas
Multiple Images Stitched Together

Applications
of 2D Image
Registration

## Source: Dr. Sohaib / Aamer Zaheer

Panoramas
Multiple Images Stitched Together

Applications
of 2D Image
Registration

Image by Sergey Semenov (http://www.sergesemenov.com/) - Winner of Epson International Photographic Pano Award 2013
http://www.dailymail.co.uk/sciencetech/article-2260276/New-York-youve-seen-Incredible-interactive-panorama-lets-zoom.html

TRANSFORMATIONS

T
p = (x,y)

p = (x,y)

## Transformation T is a coordinate-changing machine:

p = T(p)
What does it mean that T is global?
Is the same for any point p
can be described by just a few numbers (parameters)

## Lets represent a linear T as a matrix:

p = Mp

x'
x
y ' = M y

Source: Efros

Scaling
Scaling a coordinate means multiplying each of its components by
a scalar
Uniform scaling means this scalar is the same for all components:

Source: Efros

Scaling
Non-uniform scaling: different scalars per component:

X 2,
Y 0.5

Source: Efros

Scaling

Scaling operation:

x ' = ax
y ' = by

## Or, in matrix form:

x ' a 0 x
y ' = 0 b y

scaling matrix S
Whats inverse of S?

Source: Efros

2-D Rotation

(x, y)
(x, y)

x = x cos(q) - y sin(q)
y = x sin(q) + y cos(q)

Source: Efros

2-D Rotation

(x, y)
(x, y)

x = r cos (f)
y = r sin (f)
x = r cos (f + q)
y = r sin (f + q)
Trig Identity
x = r cos(f) cos(q) r sin(f) sin(q)
y = r sin(f) cos(q) + r cos(f) sin(q)
Substitute
x = x cos(q) - y sin(q)
y = x sin(q) + y cos(q)
Source: Efros

2-D Rotation
This is easy to capture in matrix form:

## x ' cosq sin q x

y ' = sin q cosq y

R
Even though sin(q) and cos(q) are nonlinear functions of q,
x is a linear combination of x and y
y is a linear combination of x and y

## What is the inverse transformation?

Rotation by q
For rotation matrices

R 1 = R T

Source: Efros

2x2 Matrices
What types of transformations can be
represented with a 2x2 matrix?

2D Identity?

x' = x
y' = y

x' = 1 0 x
y ' 0 1 y

## 2D Scale around (0,0)?

x' = s x * x
y' = s y * y

x ' s x
y ' = 0

0 x
s y y
Source: Efros

2x2 Matrices
What types of transformations can be
represented with a 2x2 matrix?

## 2D Rotate around (0,0)?

x' = cos Q * x sin Q * y
y ' = sin Q * x cos Q * y

## x ' cos Q sin Q x

y ' = sin Q cos Q y

2D Shear?
x ' = x shx * y
y ' = shy * x y

x ' 1
y ' = sh
y

shx x
1 y
Source: Efros

2x2 Matrices
What types of transformations can be
represented with a 2x2 matrix?

x' = x
y' = y

x ' = 1 0 x
y ' 0 1 y

## 2D Mirror over (0,0)?

x' = x
y' = y

x' = 1 0 x
y ' 0 1 y
Source: Efros

2x2 Matrices
What types of transformations can be
represented with a 2x2 matrix?

2D Translation?
x' = x t x
y' = y t y

NO!

## Only linear 2D transformations

can be represented with a 2x2 matrix

Source: Efros

## All 2D Linear Transformations

Linear transformations are combinations of

Scale,
Rotation,
Shear, and
Mirror

x ' a b x
y ' = c d y

## Origin maps to origin

Lines map to lines
Parallel lines remain parallel
Ratios are preserved
Closed under composition

x' = a b e
y ' c d g

f i
h k

j x
l y
Source: Efros

j =(0,1)

v =(vx,vy)
p

u=(ux,uy)
i =(1,0)
q=4i+3j = (4,3)

p=4u+3v

Source: Efros

v =(vx,vy)

j =(0,1)

puv

pij

u=(ux,uy)
i =(1,0)
puv = (4,3)
px=4ux+3vx
py=4uy+3vy

pij = 4u+3v

ux
pij =
u y

v x 4 u x
=
v y 3 u y

vx
puv
vy

Source: Efros

## Whats the inverse transform?

j =(0,1)

v =(vx,vy)
pij

puv

u=(ux,uy)
i =(1,0)
pij = (5,4) = pxu + pyv

puv =

ux

u y

-1

vx

vy

5 u x
= u
4 y

puv = (px,py) = ?
-1

vx
pij
vy

## How can we change from any basis to any basis?

What if the basis are orthogonal?
Source: Efros

j =(0,1)

v =(vx,vy)
pij

puv

u=(ux,uy)
i =(1,0)
pij = (5,4)

ux
uv
p =
v y

## puv = (upij, vpij)

ux

vy

5 u x
= v
4 x

uy
ij
p

vy
Source: Efros

Homogeneous Coordinates
Q: How can we represent translation as a 3x3
matrix?

x' = x t x
y' = y t y

Source: Efros

Homogeneous Coordinates
Homogeneous coordinates
represent coordinates in 2
dimensions with a 3-vector

x
x homogeneou
s coords
y y

Source: Efros

Homogeneous Coordinates
Add a 3rd coordinate to every 2D point
(x, y, w) represents a point at location (x/w, y/w)
(x, y, 0) represents a point at infinity
(0, 0, 0) is not allowed
y
2

## (2,1,1) or (4,2,2) or (6,3,3)

Convenient
coordinate system to
represent many
useful
transformations

Source: Efros

Homogeneous Coordinates
Q: How can we represent translation as a 3x3
matrix?

x' = x t x
y' = y t y
A: Using the rightmost column:

1 0 t x

Translation = 0 1 t y
0 0 1

Source: Efros

Translation
Example of translation

Homogeneous Coordinates
x ' 1 0 t x x x t x
y ' = 0 1 t y = y t
y
y

1 0 0 1 1 1

tx = 2
ty = 1
Source: Efros

Basic 2D Transformations
Basic 2D transformations as 3x3 matrices

x ' s x
y ' = 0

1 0

x ' 1 0 t x x
y ' = 0 1 t y
y

1 0 0 1 1

Translate

x' cos Q
y ' = sin Q

1 0

sin Q
cos Q
0

Rotate

0
sy
0

0 x
0 y
1 1

Scale
0 x
0 y
1 1

x ' 1
y ' = sh
y
1 0

shx
1
0

0 x
0 y
1 1

Shear
Source: Efros

Matrix Composition
Transformations can be combined by
matrix multiplication

## x' 1 0 tx cos Q sin Q 0 sx 0 0 x

y ' = 0 1 ty sin Q cos Q 0 0 sy 0 y

w' 0 0 1 0

0
1
0 0 1 w

p =

T(tx,ty)

R(Q)

S(sx,sy)

Source: Efros

Affine Transformations
x' a b
Affine transformations are combinations of
y' = d e
Linear transformations, and
w 0 0

Translations

c x
f y
1 w

## Origin does not necessarily map to origin

Lines map to lines
Parallel lines remain parallel
Ratios are preserved
Closed under composition
Models change of basis

## Will the last coordinate w always be 1?

Source: Efros

Projective Transformations
Projective transformations
Affine transformations, and
Projective warps

x' a
y' = d
w' g

b
e
h

c x
f y
i w

## Origin does not necessarily map to origin

Lines map to lines
Parallel lines do not necessarily remain parallel
Ratios are not preserved
Closed under composition
Models change of basis

Source: Efros

2D image transformations

## These transformations are a nested set of groups

Closed under composition and inverse is a member
Source: Efros

Hierarchy of 2D Transformations

Length Ratios
Length Ratios
Length Cross-Ratios

2D Affine Warping

## Source: Dr. Sohaib/ Aamer Zaheer

Warping
Inputs:
Image X
Affine Transformation A = [a1 a2 b1 a3 a4 b2]T

Output:
Generate X such that X = AX

Obvious Process:
For each pixel in X
Apply transformation
At that location in X, put the same color as at the original
location in X

Problems?

## Source: Dr. Sohaib/ Aamer Zaheer

Warping
This will leave holes
Because every pixel does not map to an integer
location!

Reverse Transformation
For each integer location in X
Apply inverse mapping
Problem?

## Will not result in answers at integer locations, in

general
Bilinearly interpolate from 4 neighbors

## Source: Dr. Sohaib/ Aamer Zaheer

Interpolation
In 1D

3
Use y=mx+c

m = 1, c = -2

Substitute
x=4.3, => y = 2.3

4
4.3 ?

## Source: Dr. Sohaib/ Aamer Zaheer

2D Bilinear Interpolation
Four nearest points of
(x, y)

x, y

x, y

x, y , x, y , x, y , x, y

x, y

where

x = int( x)
y = int( y )

x, y

x, y

x = x 1
y = y 1
Source: Dr. Sohaib/ Aamer Zaheer

Bilinear Interpolation
f ' ( x , y ) = x y f ( x, y ) x y f ( x , y ) x y f ( x , y ) x y f ( x, y )

x = x x
y = y y

x, y

x, y

x = x x

y = y y
y

x, y

x, y
Source: Dr. Sohaib/ Aamer Zaheer

## A LITTLE MORE IN DEPTH

COVERAGE

Groups
A group is a set G together with an operation that
combines two elements of a and b to form another
element a b. To form a group, (G, ) must satisfy
the following axioms
Closure
For all a, b in G, a b is also in G

Associativity
For all a, b, c in G, (a b) c = a (b c)

Identity
There exists an element e in G s.t. a e = e a = a

Inverse
For each a in G, there exists an element b in G s.t. a b = e

Translation

Similarity

## Each higher group completely contains the lower group

Affine Group
General 2 x 3 linear transform

## Contains rotation, scaling, shear,

translation and any
combination thereof
Preserves Parallel lines [Proof?]

## Ref: Steve Mann & Rosalind W. Picard, Video Orbits of the

Projective Group: A simple approach to featureless estimation
of parameters, IEEE Trans. on Image Processing, Vol. 6, No.
9, September 1997

Order of Transformations
Rotation/Scaling/Shear, followed by Translation

1 0 b1 a1
0 1 b a
2 3

0 0 1 0

a2
a4
0

0 a1
0 = a3
1 0

a2
a4
0

b1
b2
1

a1
a
3
0

a2
a4
0

0 1 0 b1 a1
0 0 1 b2 = a3
1 0 0 1 0

a2
a4
0

a1b1 a2b2
a3b1 a4b2

## Projective Group (Homography)

3 x 3 transform defined in
Homogeneous coordinates

## Simulates out of plane rotations

Preserves straight lines
Physical Interpretation:
Plane + Camera

## Examples of Projective Transformations

http://wearcam.org/orbits/gallery.html

Hierarchy of 2D Transformations

Length Ratios
Length Ratios
Length Cross-Ratios

## Pseudo Perspective Model

Linear approximation of projective model

## Small motion approximation of full homography

Does not preserve straight lines
Obtained through Taylor Expansion

Bilinear Model
Linear approximation of projective
model

## 8 parameters, so perfectly maps four

points
Does not preserve straight lines
Preserves relative distances (nonchirping model
Obtained through Taylor Expansion

Displacement Models

## Ref: Steve Mann & Rosalind W. Picard, Video Orbits of

the Projective Group: A simple approach to featureless
estimation of parameters, IEEE Trans. on Image
Processing, Vol. 6, No. 9, September 1997