Вы находитесь на странице: 1из 181

# Fundamentals of Image and Video

Processing
by
Dr. S. D. Ruikar
Associate Professor,
Department of Electronics Engineering
Walchand College of Engineering,
Sangli
MODULE 2: Image
Enhancement and Restoration
Syllabus
Spatial Domain: Gray level transformation,
Histogram processing , Arithmetic / Logic
operations, Spatial filtering, smoothing filters,
sharpening filters
Frequency Domain: Fourier transforms,
smoothing frequency domain filters, sharpening
filters, Homographic filtering
Image degradation model, Noise modeling,
Mean filter, Order statistics filter, Inverse filter,
Weiner filter, Blind image restoration
IMAGE QUALITY AND NEED FOR
IMAGE ENHANCEMENT
Enhancement Vs restoration
Spatial Domain vs. Transform Domain

Spatial domain
image plane itself, directly process the intensity values of
the image plane

Transform domain
process the transform coefficients, not directly process
the intensity values of the image plane
Spatial Domain Process

g ( x, y ) T [ f ( x, y )])
f ( x, y ) : input image
g ( x, y ) : output image
T : an operator on f defined over
a neighborhood of point ( x, y )
Spatial Domain Process
Spatial Domain Process
Intensity transformation function
s T (r )
Some Basic Intensity Transformation
Functions
Image Negatives

Image negatives
s L 1 r
Example: Image Negatives

Small
lesion
Log Transformations
Log Transformations
s c log(1 r )
Example: Log Transformations
Power-Law (Gamma) Transformations

s cr
Example: Gamma Transformations
Example: Gamma Transformations
Cathode ray tube
(CRT) devices have an
intensity-to-voltage
response that is a
power function, with
exponents varying
from approximately
1.8 to 2.5

sr 1/ 2.5
Example: Gamma Transformations

18
Example: Gamma Transformations

19
Piecewise-Linear Transformations

Contrast Stretching
Expands the range of intensity levels in an image so that it spans
the full intensity range of the recording medium or display device.

Intensity-level Slicing
Highlighting a specific range of intensities in an image often is of
interest.
Highlight the major
blood vessels and
study the shape of the
flow of the contrast
medium (to detect
blockages, etc.)

## Measuring the actual

flow of the contrast
medium as a function of
time in a series of
images
7/1/2017 22
Bit-plane Slicing
Bit-plane Slicing
Bit-plane Slicing
Image Quality Factors
Contrast
Histogram Processing

Histogram Equalization

Histogram Matching

## Using Histogram Statistics for Image Enhancement

Histogram Processing

Histogram h(rk ) nk
rk is the k th intensity value
nk is the number of pixels in the image with intensity rk

nk
Normalized histogram p(rk )
MN
nk : the number of pixels in the image of
size M N with intensity rk
Histogram Equalization
The intensity levels in an image may be viewed as
random variables in the interval [0, L-1].
Let pr (r ) and ps ( s) denote the probability density
function (PDF) of random variables r and s.
Histogram Equalization

s T (r ) 0 r L 1

## a. T(r) is a strictly monotonically increasing function

in the interval 0 r L -1;
b. 0 T (r ) L -1 for 0 r L -1.
Histogram Equalization

s T (r ) 0 r L 1

## a. T(r) is a strictly monotonically increasing function

in the interval 0 r L -1;
b. 0 T (r ) L -1 for 0 r L -1.

## T (r ) is continuous and differentiable.

ps (s)ds pr (r )dr
Example

## Suppose that the (continuous) intensity values

in an image have the PDF

2r
, for 0 r L-1
pr (r ) ( L 1) 2

0,
otherwise

## Find the transformation function for equalizing

the image histogram.
Example: Histogram Equalization
Suppose that a 3-bit image (L=8) of size 64 64 pixels (MN = 4096)
has the intensity distribution shown in following table.
Get the histogram equalization transformation function and give the
ps(sk) for each sk.
Example: Histogram Equalization

0
s0 T (r0 ) 7 pr (rj ) 7 0.19 1.33 1
j 0
1
s1 T (r1 ) 7 pr (rj ) 7 (0.19 0.25) 3.08 3
j 0
s2 4.55 5 s3 5.67 6
s4 6.23 6 s5 6.65 7
s6 6.86 7 s7 7.00 7
Example: Histogram Equalization
Question
Is histogram equalization always good?

No
Histogram Matching
Histogram matching (histogram specification)
generate a processed image that has a specified histogram
Let pr (r ) and pz ( z ) denote the continous probability
density functions of the variables r and z. pz ( z ) is the
specified probability density function.
Let s be the random variable with the probability
r
s T ( r ) ( L 1) pr ( w) dw
0

## Define a random variable z with the probability

z
G ( z ) ( L 1) pz (t ) dt s
0
Histogram Matching
r
s T (r ) ( L 1) pr ( w)dw
0
z
G ( z ) ( L 1) pz (t )dt s
0

1
z G ( s) G 1
T (r)
Histogram Matching: Procedure

Obtain pr(r) from the input image and then obtain the values of s
r
s ( L 1) pr (w)dw
0

Use the specified PDF and obtain the transformation function G(z)
z
G( z ) ( L 1) pz (t )dt s
0

Mapping from s to z

z G 1 (s)
Example: Histogram Matching
Example: Histogram Matching
Example: Histogram Matching
Local Histogram Processing

pixel

## At each location, the histogram of the points in the

neighborhood is computed. Either histogram equalization or
histogram specification transformation function is obtained

## Move to the next location and repeat the procedure

Local Histogram Processing: Example
Using Histogram Statistics for Image
Enhancement
Average Intensity L 1 M 1 N 1
m ri p(ri )
1

MN
f ( x, y)
x 0 y 0
i 0
L 1
un (r ) (ri m) n p(ri )
i 0

Variance L 1 M 1 N 1
u2 (r ) (ri
1
m) p(ri ) f ( x, y) m
2 2 2

i 0
MN x 0 y 0
Using Histogram Statistics for Image
Enhancement

## Local average intensity

L 1
msxy ri psxy (ri )
i 0

## sxy denotes a neighborhood

Local variance
L 1
2
sxy (ri msxy ) psxy (ri )
2

i 0
Using Histogram Statistics for Image
Enhancement: Example

## E f ( x, y ), if msxy k0 mG and k1 G sxy k 2 G

g ( x, y )
f ( x, y ), otherwise

## mG : global mean; G : global standard deviation

k0 0.4; k1 0.02; k2 0.4; E 4
Image Quality Assessment tool
Histograms
Image Quality Metrics
Image Quality Metrics
Image Enhancement
Point Operations
Non Linear Equations
Square
Square root
Logarithm
Power
Contrast Stretching
Contrast Stretching
Range Normalization
Clipping & Thresholding
Multiple Thresholding
Intensity Slicing
Bitplane Slicing
Histogram Stretching
Histogram Stretching
Histogram Sliding
Histogram Equalization
Histogram Specification
Spatial Filtering

## A spatial filter consists of (a) a neighborhood, and (b) a

predefined operation

## Linear spatial filtering of an image of size MxN with a filter of

size mxn is given by the expression

a b
g ( x, y ) w(s, t ) f ( x s, y t )
s a t b
Spatial Filtering
Spatial Correlation

## The correlation of a filter w( x, y ) of size m n

with an image f ( x, y ), denoted as w( x, y ) f ( x, y )

a b
w( x, y ) f ( x, y ) w(s, t ) f ( x s, y t )
s a t b
Spatial Convolution

## The convolution of a filter w( x, y ) of size m n

with an image f ( x, y ), denoted as w( x, y ) f ( x, y )

a b
w( x, y ) f ( x, y ) w(s, t ) f ( x s, y t )
s a t b
Smoothing Spatial Filters

reduction

## Blurring is used in removal of small details and bridging of

small gaps in lines or curves

## Smoothing spatial filters include linear filters and nonlinear

filters.
Spatial Smoothing Linear Filters

## The general implementation for filtering an M N image

with a weighted averaging filter of size m n is given
a b

w(s, t ) f ( x s, y t )
g ( x, y ) s a t b
a b

w(s, t )
s a t b

where m 2a 1, n 2b 1.
Two Smoothing Averaging Filter Masks
Example: Gross Representation of Objects
Order-statistic (Nonlinear) Filters

Nonlinear

## Replacing the value of the center pixel with the value

determined by the ranking result

## E.g., median filter, max filter, min filter

Median Filter
Median Filter
Median Filter
Median Filter
Median Filter
Example: Use of Median Filtering for Noise
Reduction
Sharpening Spatial Filters

Foundation

Laplacian Operator

## Using First-Order Derivatives for Nonlinear Image

Sharpening Spatial Filters: Foundation

## The first-order derivative of a one-dimensional function f(x)

is the difference

f
f ( x 1) f ( x)
x

## The second-order derivative of f(x) as the difference

2 f
f ( x 1) f ( x 1) 2 f ( x)
x 2
Sharpening Spatial Filters: Laplace Operator

## The second-order isotropic derivative operator is the

Laplacian for a function (image) f(x,y)

2
f 2
f
f 2 2
2

x y
2 f
f ( x 1, y ) f ( x 1, y ) 2 f ( x, y )
x 2

2 f
f ( x, y 1) f ( x, y 1) 2 f ( x, y )
y 2

2 f f ( x 1, y ) f ( x 1, y ) f ( x, y 1) f ( x, y 1)
- 4 f ( x, y )
Sharpening Spatial Filters: Laplace Operator
Sharpening Spatial Filters: Laplace Operator

## Image sharpening in the way of using the Laplacian:

g ( x, y ) f ( x, y ) c 2 f ( x, y )
where,
f ( x, y ) is input image,
g ( x, y ) is sharpenend images,
c -1 if 2 f ( x, y ) corresponding to Fig. 3.37(a) or (b)
and c 1 if either of the other two filters is used.
Unsharp Masking and Highboost Filtering

Sharpen images consists of subtracting an unsharp (smoothed)
version of an image from the original image
e.g., printing and publishing industry

Steps
1. Blur the original image

## 3. Add the mask to the original

Unsharp Masking and Highboost Filtering

## Let f ( x, y ) denote the blurred image, unsharp masking is

g mask ( x, y ) f ( x, y ) f ( x, y )
Then add a weighted portion of the mask back to the original
g ( x, y ) f ( x, y ) k * g mask ( x, y ) k 0

## when k 1, the process is referred to as highboost filtering.

Unsharp Masking and Highboost Filtering: Example
Image Sharpening based on First-Order Derivatives

is defined as
f
g x x
f grad( f )
g y f
y

## The magnitude of vector f , denoted as M ( x, y )

Gradient Image M ( x, y ) mag(f ) g x 2 g y 2
Image Sharpening based on First-Order Derivatives

## The magnitude of vector f , denoted as M ( x, y )

M ( x, y ) mag(f ) g x 2 g y 2

M ( x, y ) | g x | | g y |

z1 z2 z3
M ( x, y) | z8 z5 | | z6 z5 |
z4 z5 z6
z7 z8 z9
Image Sharpening based on First-Order Derivatives

## Roberts Cross-gradient Operators

M ( x, y ) | z9 z5 | | z8 z6 |

Sobel Operators
M ( x, y) | ( z7 2 z8 z9 ) ( z1 2 z2 z3 ) |
z1 z2 z3 | ( z3 2 z6 z9 ) ( z1 2 z4 z7 ) |
z4 z5 z6
z7 z8 z9
Image Sharpening based on First-Order Derivatives
Example
Example:

Combining
Spatial
Enhancement
Methods

Goal:

Enhance the
image by
sharpening it
and by bringing
out more of the
skeletal detail
Example:

Combining
Spatial
Enhancement
Methods

Goal:

Enhance the
image by
sharpening it
and by bringing
out more of the
skeletal detail
Example
Spatial Filtering Techniques
Image Smoothing Filter
Variable Weights
Gaussian Filters
Directional Smoothing
Conservative smoothing
Sharpening Filters
High-Boost Filter
Frequency Domain Filtering
Image Smoothing Frequency
Domain 1D.
2D Case
Butterworth Filter
Band-Pass filter
Homomorphic Filtering
What is Image Restoration?
Image restoration attempts to restore
images that have been degraded
Identify the degradation process and attempt
to reverse it
Similar to image enhancement, but more
objective
Noise and Images
The sources of noise in digital
images arise during image
acquisition (digitization) and
transmission
Imaging sensors can be
affected by ambient
conditions
Interference can be added
to an image during
transmission
Modeling Process
An Image Degradation Model
Two types of degradation
Spatial domain restoration (denoising) techniques are preferred
Image blur
Frequency domain methods are preferred
We model the degradation process by a degradation function h(x,y),
an additive noise term, (x,y), as g(x,y)=h(x,y)*f(x,y)+ (x,y)
f(x,y) is the (input) image free from any degradation
g(x,y) is the degraded image
* is the convolution operator
The goal is to obtain an estimate of f(x,y) according to the knowledge
In frequency domain: G(u,v)=H(u,v)F(u,v)+N(u,v)
Three cases are considered in this Chapter
g(x,y)=f(x,y)+ (x,y)
g(x,y)=h(x,y)*f(x,y)
g(x,y)=h(x,y)*f(x,y)+ (x,y)
A Model of the Image
Noise Model
We first consider the degradation due to noise only
h is an impulse for now ( H is a constant)
White noise
Autocorrelation function is an impulse function multiplied by a
constant
N 1 M 1
a( x, y ) ( s, t ) ( s x, t y ) N 0 ( x, y )
t 0 s 0
It means there is no correlation between any two pixels in the noise
image
There is no way to predict the next noise value
The spectrum of the autocorrelation function is a constant (white)
Gaussian Noise
Noise (image) can be classified according the
distribution of the values of pixels (of the noise image) or
its (normalized) histogram
Gaussian noise is characterized by two parameters,
(mean) and 2 (variance), by

1 ( z ) 2 / 2 2
p( z ) e
2
70% values of z fall in the range [(-),(+)]
95% values of z fall in the range [(-2),(+2)]
Gaussian Noise
Noise Modeling
Other Noise Models
Rayleigh noise
2
( z a )e ( z a ) / b for z a
2

p( z ) b

0 for z a
The mean and variance of this
density are given by
b( 4 )
a b / 4 and 2
4
a and b can be obtained through
mean and variance
Other Noise Models
Erlang (Gamma) noise
a b z b 1 az
e for z 0
p( z ) (b 1)!
0 for z 0
The mean and variance of this
density are given by
b
b / a and 2
a2
a and b can be obtained
through mean and variance
Other Noise Models

Exponential noise
ae az for z 0
p( z )
0 for z 0
The mean and variance
of this density are given
by 1
1 / a and
2

a2
Special case pf Erlang
PDF with b=1
Other Noise Models
Uniform noise
1
if a z b
p( z ) b a
0 otherwise
The mean and variance
of this density are given
by
(b a ) 2
(a b) / 2 and 2
12
Other Noise Models
Impulse (salt-and-pepper)
noise Pa for z a
p( z ) P for z b
b
0 otherwise

If either Pa or Pb is zero,
the impulse noise is called
unipolar
a and b usually are
extreme values because
impulse corruption is
usually large compared
with the strength of the
image signal
It is the only type of noise
that can be distinguished
from others visually
A Sample Image
Effect of Adding Noise to Sample Image
Effect of Adding Noise to Sample Image
Periodic Noise
Arises typically from electrical
or electromechanical
interference during image
acquisition
It can be observed by visual
inspection both in the spatial
domain and frequency
domain
The only spatially dependent
noise will be considered
Periodic Noise
Estimation of Noise Parameters

Periodic noise
Parameters can be estimated by inspection of the
spectrum
Noise PDFs
From sensor specifications
If imaging sensors are available, capture a set of
images of plain environments
If only noisy images are available, parameters of
the PDF involved can be estimated from small
patches of constant regions of the noisy images
Blur
Estimation of Noise Parameters

## In most cases, only mean and variance are to

be estimated
Others can be obtained from the estimated mean
and variance
Assume a sub-image with plain scene is
available and is denoted by S
1 z ( xi , yi )
N S ( x , y )S
i i

0 2 1 ( z ( xi , yi ) ) 2 12 1 i i
( z ( x , y )
) 2

N S ( x , y )S
i i
NS ( xi , yi )S
2 1
2 i i
(
N S 1 ( x , y )S
z ( x , y )
) 2

i i
Restoration in the presence of
Noise
Order Statistic Filter
Inverse Filter
Inverse Filtering Image

Filtering with H
cut off outside a
Full inverse
Filtering

## Filtering with H cut

off outside a radius
of 70

Filtering with H
cut off outside a
Wiener Filter

## The Wiener filter was proposed by Norbert

Wiener in 1940.
It was published in 1949
Its purpose is to reduce the amount of a
noise in a signal.
This is done by comparing the received
signal with a estimation of a desired
noiseless signal.
Wiener filter is not an adaptive filter as it
assumes input to be stationery.
Wiener Filter
It takes a statistical approach to solve its goal
Goal of the filter is to remove the noise from a signal
Before implementation of the filter it is assumed thatthe
user knows the spectral properties of the original signal and
noise.
Spectral properties like the power functions for both the
original signal and noise.
And the resultant signal required is as close to the original
signal.
Signal and noise are both linear stochastic processes with
known spectral properties.
The aim of the process is to have minimum mean square
error
That is, the difference between the original signal and the
new signal should be as less as possible.
Wiener Filter

## Considering we need to design a wiener filter in

frequency domain as W(u,v)
Restored image will be given as;

Xn(u,v) = W(u,v).Y(u,v)

## Where Y(u,v) is the received signal and Xn(u,v) is

the restored image
Wiener Filter

## We choose W(k,l) to minimize:

= E {(f f)2 }
Obtained from [1]

## Where the equation represents

the mean square error.
The wiener filter can be
represented by the equation:
Wiener Filter
H *
(u , v ) S (u , v )
F (u, v) G (u, v)
f

## S f (u, v) H (u, v) S (u, v)

2

H * (u, v)
G (u, v)
H (u, v) S (u, v) / S f (u, v)
2

1 H (u, v)
2

G (u, v)
H (u, v) H (u, v) S (u, v) / S f (u, v)
2

## H(u,v) = degradation function

|H(u,v)|^2 = H*(u,v)H(u,v)
H*(u,v) = complex conjugate of H(u,v)
Sn(u,v) = |N(u,v)|^2 power spectrum of noise
Sf(u,v) = |F(u,v)|^2 power spectrum of undegraded image
. G(u,v) is the transform of the degraded image.
Minimum Mean Square Error Filtering
(Wiener Filtering)
1 H (u , v )
2

F (u, v) G(u, v)
H (u, v) H (u, v) S (u, v) / S f (u, v)
2

Constant Unknown

1 H ( u , v )
2

F (u, v) G(u, v)
H (u, v) H (u, v) K
2

Wiener Filtering

## Radially limited Wiener filtering

Full inverse
filtering inverse filtering (K was chosen interactively)

## Dr. S D Ruikar 177

Inverse filtering Wiener filtering
Wiener Filtering

Reduce
d noise
variance

## Dr. S D Ruikar 178

Wiener Filter Results
Blind Image Restoration
Thank you