Seminar

A Digital Image Stabilization Method Based

on the HilbertHuang Transform
Konstantinos Ioannidis and Ioannis Andreadis
IEEE TRANSACTIONS on Instrumentation And Measurement,
VOL. 61, NO. 9, SEPTEMBER 2012

Presented by
Shibudas K Subhashdas
1/22

Introduction
Introduction
Proposed method in a Nutshell
Local Motion Vector (LMV)

An innovative technique for digital image stabilization

(DIS) based on the Hilbert Huang transform (HHT)

Estimation
Hilbert Huang Transform( HHT)
Empirical Mode
Decomposition (EMD)
process
Hilbert Transform
Detect the unwanted Camera
Motion
Experimental Results
Conclusion

2/22

Proposed method in a Nutshell

Introduction
Proposed method in a Nutshell
Local Motion Vector (LMV)
Estimation
Hilbert Huang Transform( HHT)
Empirical Mode

Local Motion Vector (LMV) Estimation

Hilbert Huang Transform ( HHT)
Empirical Mode Decomposition (EMD) process
Hilbert Transform

Decomposition (EMD)
process

Detect the unwanted Camera Motion

Hilbert Transform
Detect the unwanted Camera
Motion
Experimental Results
Conclusion

3/22

Local Motion Vector (LMV) Estimation

Introduction
Proposed method in a Nutshell
Local Motion Vector (LMV)

LMV represents the offset of specific image regions

between two consecutive images (displacement vector)

Estimation
Hilbert Huang Transform( HHT)
Empirical Mode
Decomposition (EMD)
process

LMV includes intentional and unwanted motion of the

camera
Block based motion estimation used in LMV

Hilbert Transform
Detect the unwanted Camera
Motion
Experimental Results
Conclusion

4/22

Local Motion Vector (LMV) Estimation

Introduction
Proposed method in a Nutshell
Local Motion Vector (LMV)

The criteria for selecting the best match is sum of absolute

difference (SAD)

Estimation
Hilbert Huang Transform( HHT)
Empirical Mode
Decomposition (EMD)

1
SAD(i, j ) 2
N

N 1 N 1

[s(n1, n2, k ) s(n1 i, n2 j, k 1)]

n1 0 n 2 0

process

Previous Frame

Hilbert Transform

Current Frame

Detect the unwanted Camera

Motion
Experimental Results
Conclusion

S(1,1)
S(0,0)
S(1,0)
S(0,1)
5/22

Local Motion Vector (LMV) Estimation

Introduction
Proposed method in a Nutshell

Minimum SAD value defines the displacement vector

Local Motion Vector (LMV)

Estimation
Hilbert Huang Transform( HHT)
Empirical Mode
Decomposition (EMD)

[d1, d 2] arg min[ SAD(i, j )]

Full search (FS) algorithm is used in this method

process
Hilbert Transform
Detect the unwanted Camera
Motion
Experimental Results
Conclusion

6/22

Local Motion Vector (LMV) Estimation

Introduction
Proposed method in a Nutshell
Local Motion Vector (LMV)

Comparison of Full search (FS) algorithm with other

search method

Estimation
Hilbert Huang Transform( HHT)
Empirical Mode
Decomposition (EMD)
process
Hilbert Transform

three step search algorithm

Detect the unwanted Camera

Motion
Experimental Results
Conclusion

Four step search algorithm

Diamond search algorithm

7/22

Local Motion Vector (LMV) Estimation

Introduction
Proposed method in a Nutshell

Using LMV estimation - x(t) (displacement vector)

Local Motion Vector (LMV)

Estimation
Hilbert Huang Transform( HHT)
Empirical Mode
Decomposition (EMD)

1
SAD(i, j ) 2
N

N 1 N 1

[s(n1, n2, k ) s(n1 i, n2 j, k 1)]

n1 0 n 2 0

[d1, d 2] arg min[ SAD(i, j )]

process
Hilbert Transform
Detect the unwanted Camera
Motion
Experimental Results
Conclusion

8/22

Hilbert Huang Transform ( HHT)

Introduction
Proposed method in a Nutshell

Empirical Mode Decomposition (EMD) process

Local Motion Vector (LMV)

Estimation
Hilbert Huang Transform( HHT)
Empirical Mode
Decomposition (EMD)
process
Hilbert Transform
Detect the unwanted Camera
Motion
Experimental Results
Conclusion

9/22

Hilbert Huang Transform ( HHT)

Introduction
Proposed method in a Nutshell

Empirical Mode Decomposition (EMD) process

Local Motion Vector (LMV)

Estimation
Hilbert Huang Transform( HHT)

m11 (t )

U (t ) L(t )
2

Empirical Mode
Decomposition (EMD)
process
Hilbert Transform
Detect the unwanted Camera
Motion
Experimental Results
Conclusion

10/22

Hilbert Huang Transform ( HHT)

Introduction
Proposed method in a Nutshell

Empirical Mode Decomposition (EMD) process

Shifting Process

h1 (t ) x(t ) m1 (t )

Local Motion Vector (LMV)

Estimation

h11 (t ) h1 (t ) m11 (t )

Hilbert Huang Transform( HHT)

Empirical Mode

h12 (t ) h11 (t ) m12 (t )

Decomposition (EMD)
process
Hilbert Transform

h1k (t ) h1( k 1) (t ) m1k (t )

Detect the unwanted Camera

Motion
Experimental Results
Conclusion

This will continue until value sum of difference (SD)

reaches between 0.2 and 0.3
T

SD

| h
t 0

k 1
T

(t ) hk (t ) |2

h
t 0

2
k 1

(t )
11/22

Hilbert Huang Transform ( HHT)

Introduction
Proposed method in a Nutshell
Local Motion Vector (LMV)
Estimation
Hilbert Huang Transform( HHT)
Empirical Mode
Decomposition (EMD)

Empirical Mode Decomposition (EMD) process

If 0.2<= SD <=0.3
c1= h1k be the first IMF (Intrinsic Mode function)
c1 (t) is removed from the image to get residue

r1 (t ) x(t ) c1 (t )

process
Hilbert Transform

r1 is the residue

Detect the unwanted Camera

Motion
Experimental Results
Conclusion

Residue is treated as the new data and subjected to

same shifting process to give c2(t)

r2 (t ) r1 (t ) c2 (t )
rw (t ) rw1 (t ) cw (t )

Cw(t) is the wth

IMF
12/22

Hilbert Huang Transform ( HHT)

Introduction
Proposed method in a Nutshell
Local Motion Vector (LMV)
Estimation
Hilbert Huang Transform( HHT)

Empirical Mode Decomposition (EMD) process

When the residue rw becomes a monotonic function
from which no IMF can be extracted.

Empirical Mode
Decomposition (EMD)
process
Hilbert Transform
Detect the unwanted Camera
Motion
Experimental Results
Conclusion

13/22

Hilbert Huang Transform ( HHT)

Introduction
Proposed method in a Nutshell
Local Motion Vector (LMV)
Estimation
Hilbert Huang Transform( HHT)
Empirical Mode
Decomposition (EMD)
process
Hilbert Transform
Detect the unwanted Camera
Motion
Experimental Results
Conclusion

Empirical Mode Decomposition (EMD) process

Sum of the IMFs and the residue recovers the original
signal.
w

x(t ) c j rw
j 1

w 1

w 1

j 1

j 1

x(t ) c j cw rw c j rw1
w2

w 1

j 1

j 1

c j cw1 rw1 c j rw 2
1

c j c2 r2 c1 r1 x (t )
j 1

14/22

Hilbert Huang Transform ( HHT)

Introduction
Proposed method in a Nutshell
Local Motion Vector (LMV)
Estimation
Hilbert Huang Transform( HHT)
Empirical Mode
Decomposition (EMD)
process

Hilbert Transform
It is used to compute instantaneous frequencies and
amplitudes

1
x( )
H ( x(t )) y (t ) P
d
t

Where P denotes Cauchy principal value

Hilbert Transform
Detect the unwanted Camera
Motion
Experimental Results
Conclusion

z(t) analytical signal is given by

z (t ) x(t ) iy (t ) (t )ei ( t )
Where (t) Instantaneous Amplitude

(t) Instantaneous Phase

15/22

Hilbert Huang Transform ( HHT)

Introduction
Proposed method in a Nutshell
Local Motion Vector (LMV)
Estimation
Hilbert Huang Transform( HHT)
Empirical Mode
Decomposition (EMD)

Hilbert Transform
Instantaneous Amplitude
(t ) x 2 (t ) y 2 (t )

Instantaneous Phase

process
Hilbert Transform
Detect the unwanted Camera

y (t )

x(t )

(t ) tan 1

Instantaneous Frequency

Motion
Experimental Results
Conclusion

f (t )

1 d (t )
2 dt

16/22

Hilbert Huang Transform ( HHT)

Introduction
Proposed method in a Nutshell
Local Motion Vector (LMV)
Estimation
Hilbert Huang Transform( HHT)

Detect the unwanted Camera Motion

EMD process divides the initial signal into finite
number of sub signals based on their frequencies

Empirical Mode
Decomposition (EMD)
process
Hilbert Transform
Detect the unwanted Camera
Motion
Experimental Results
Conclusion

17/22

Hilbert Huang Transform ( HHT)

Introduction
Proposed method in a Nutshell
Local Motion Vector (LMV)
Estimation
Hilbert Huang Transform( HHT)
Empirical Mode
Decomposition (EMD)
process

Detect the unwanted Camera Motion

The power of each IMF is proportional to the
amplitude of its sample.
K

Pi i 2 (t )
t 0

Hilbert Transform
Detect the unwanted Camera
Motion

Where i amplitude of a IMFs

i = 1,2 .. w+1

Experimental Results
Conclusion

t frame number

18/22

Hilbert Huang Transform ( HHT)

Introduction
Proposed method in a Nutshell
Local Motion Vector (LMV)
Estimation
Hilbert Huang Transform( HHT)
Empirical Mode
Decomposition (EMD)
process
Hilbert Transform
Detect the unwanted Camera
Motion

Detect the unwanted Camera Motion

IMF with lower indices include high frequency
component (jitter)
From lower to higher IMFs , the energy content is
reduced
After a certain IMF , a sharp increase of the energy
occur due to intentional camera motion

Experimental Results
Conclusion

19/22

Hilbert Huang Transform ( HHT)

Introduction
Proposed method in a Nutshell
Local Motion Vector (LMV)
Estimation
Hilbert Huang Transform( HHT)

Detect the unwanted Camera Motion

IMF with higher index and lower energy content to
the last IMF which includes jitter components

Empirical Mode
Decomposition (EMD)
process
Hilbert Transform
Detect the unwanted Camera
Motion
Experimental Results
Conclusion

20/22

Hilbert Huang Transform ( HHT)

Introduction
Proposed method in a Nutshell
Local Motion Vector (LMV)
Estimation
Hilbert Huang Transform( HHT)
Empirical Mode
Decomposition (EMD)
process
Hilbert Transform

Detect the unwanted Camera Motion

Jitter and intentional camera motion
d

X J (t ) ci (t )
i 1

where

X G (t ) ci (t ) rw
i d

d arg min[ Pi ]

Detect the unwanted Camera

Motion
Experimental Results
Conclusion

21/22

Experimental Results
Introduction
Proposed method in a Nutshell
Local Motion Vector (LMV)
Estimation
Hilbert Huang Transform( HHT)
Empirical Mode
Decomposition (EMD)
process
Hilbert Transform

erms

(x
n 1

xn ) 2 ( yn yn ) 2

Detect the unwanted Camera

Motion
Experimental Results
Conclusion

Where N is the number of frames

( xn , Optimal
yn )

camera motion

Resulting camera motion

( xn , yn )

22/22

Conclusion
Introduction
Proposed method in a Nutshell
Local Motion Vector (LMV)
Estimation
Hilbert Huang Transform( HHT)
Empirical Mode

DIS method based on HHT has been presented

Jitter signal is defined based on its two principle feature
High frequency
Low energy

Decomposition (EMD)
process
Hilbert Transform
Detect the unwanted Camera

Experimental results have proven that the proposed

method can successfully decompose two camera motions

Motion
Experimental Results
Conclusion

23/22