Вы находитесь на странице: 1из 181

Fundamentals of Image and Video

Processing
by
Dr. S. D. Ruikar
Associate Professor,
Department of Electronics Engineering
Walchand College of Engineering,
Sangli
MODULE 2: Image
Enhancement and Restoration
Syllabus
Spatial Domain: Gray level transformation,
Histogram processing , Arithmetic / Logic
operations, Spatial filtering, smoothing filters,
sharpening filters
Frequency Domain: Fourier transforms,
smoothing frequency domain filters, sharpening
filters, Homographic filtering
Image degradation model, Noise modeling,
Mean filter, Order statistics filter, Inverse filter,
Weiner filter, Blind image restoration
IMAGE QUALITY AND NEED FOR
IMAGE ENHANCEMENT
Enhancement Vs restoration
Spatial Domain vs. Transform Domain

Spatial domain
image plane itself, directly process the intensity values of
the image plane

Transform domain
process the transform coefficients, not directly process
the intensity values of the image plane
Spatial Domain Process

g ( x, y ) T [ f ( x, y )])
f ( x, y ) : input image
g ( x, y ) : output image
T : an operator on f defined over
a neighborhood of point ( x, y )
Spatial Domain Process
Spatial Domain Process
Intensity transformation function
s T (r )
Some Basic Intensity Transformation
Functions
Image Negatives

Image negatives
s L 1 r
Example: Image Negatives

Small
lesion
Log Transformations
Log Transformations
s c log(1 r )
Example: Log Transformations
Power-Law (Gamma) Transformations


s cr
Example: Gamma Transformations
Example: Gamma Transformations
Cathode ray tube
(CRT) devices have an
intensity-to-voltage
response that is a
power function, with
exponents varying
from approximately
1.8 to 2.5

sr 1/ 2.5
Example: Gamma Transformations

18
Example: Gamma Transformations

19
Piecewise-Linear Transformations

Contrast Stretching
Expands the range of intensity levels in an image so that it spans
the full intensity range of the recording medium or display device.

Intensity-level Slicing
Highlighting a specific range of intensities in an image often is of
interest.
Highlight the major
blood vessels and
study the shape of the
flow of the contrast
medium (to detect
blockages, etc.)

Measuring the actual


flow of the contrast
medium as a function of
time in a series of
images
7/1/2017 22
Bit-plane Slicing
Bit-plane Slicing
Bit-plane Slicing
Image Quality Factors
Contrast
Histogram Processing

Histogram Equalization

Histogram Matching

Local Histogram Processing

Using Histogram Statistics for Image Enhancement


Histogram Processing

Histogram h(rk ) nk
rk is the k th intensity value
nk is the number of pixels in the image with intensity rk

nk
Normalized histogram p(rk )
MN
nk : the number of pixels in the image of
size M N with intensity rk
Histogram Equalization
The intensity levels in an image may be viewed as
random variables in the interval [0, L-1].
Let pr (r ) and ps ( s) denote the probability density
function (PDF) of random variables r and s.
Histogram Equalization

s T (r ) 0 r L 1

a. T(r) is a strictly monotonically increasing function


in the interval 0 r L -1;
b. 0 T (r ) L -1 for 0 r L -1.
Histogram Equalization

s T (r ) 0 r L 1

a. T(r) is a strictly monotonically increasing function


in the interval 0 r L -1;
b. 0 T (r ) L -1 for 0 r L -1.

T (r ) is continuous and differentiable.

ps (s)ds pr (r )dr
Example

Suppose that the (continuous) intensity values


in an image have the PDF

2r
, for 0 r L-1
pr (r ) ( L 1) 2

0,
otherwise

Find the transformation function for equalizing


the image histogram.
Example: Histogram Equalization
Suppose that a 3-bit image (L=8) of size 64 64 pixels (MN = 4096)
has the intensity distribution shown in following table.
Get the histogram equalization transformation function and give the
ps(sk) for each sk.
Example: Histogram Equalization

0
s0 T (r0 ) 7 pr (rj ) 7 0.19 1.33 1
j 0
1
s1 T (r1 ) 7 pr (rj ) 7 (0.19 0.25) 3.08 3
j 0
s2 4.55 5 s3 5.67 6
s4 6.23 6 s5 6.65 7
s6 6.86 7 s7 7.00 7
Example: Histogram Equalization
Question
Is histogram equalization always good?

No
Histogram Matching
Histogram matching (histogram specification)
generate a processed image that has a specified histogram
Let pr (r ) and pz ( z ) denote the continous probability
density functions of the variables r and z. pz ( z ) is the
specified probability density function.
Let s be the random variable with the probability
r
s T ( r ) ( L 1) pr ( w) dw
0

Define a random variable z with the probability


z
G ( z ) ( L 1) pz (t ) dt s
0
Histogram Matching
r
s T (r ) ( L 1) pr ( w)dw
0
z
G ( z ) ( L 1) pz (t )dt s
0

1
z G ( s) G 1
T (r)
Histogram Matching: Procedure

Obtain pr(r) from the input image and then obtain the values of s
r
s ( L 1) pr (w)dw
0

Use the specified PDF and obtain the transformation function G(z)
z
G( z ) ( L 1) pz (t )dt s
0

Mapping from s to z

z G 1 (s)
Example: Histogram Matching
Example: Histogram Matching
Example: Histogram Matching
Local Histogram Processing

Define a neighborhood and move its center from pixel to


pixel

At each location, the histogram of the points in the


neighborhood is computed. Either histogram equalization or
histogram specification transformation function is obtained

Map the intensity of the pixel centered in the neighborhood

Move to the next location and repeat the procedure


Local Histogram Processing: Example
Using Histogram Statistics for Image
Enhancement
Average Intensity L 1 M 1 N 1
m ri p(ri )
1

MN
f ( x, y)
x 0 y 0
i 0
L 1
un (r ) (ri m) n p(ri )
i 0

Variance L 1 M 1 N 1
u2 (r ) (ri
1
m) p(ri ) f ( x, y) m
2 2 2

i 0
MN x 0 y 0
Using Histogram Statistics for Image
Enhancement

Local average intensity


L 1
msxy ri psxy (ri )
i 0

sxy denotes a neighborhood

Local variance
L 1
2
sxy (ri msxy ) psxy (ri )
2

i 0
Using Histogram Statistics for Image
Enhancement: Example

E f ( x, y ), if msxy k0 mG and k1 G sxy k 2 G


g ( x, y )
f ( x, y ), otherwise

mG : global mean; G : global standard deviation


k0 0.4; k1 0.02; k2 0.4; E 4
Image Quality Assessment tool
Histograms
Advantages of Histogram
Advantages of Histogram
Image Quality Metrics
Image Quality Metrics
Image Enhancement
Point Operations
Non Linear Equations
Square
Square root
Logarithm
Power
Contrast Stretching
Contrast Stretching
Range Normalization
Clipping & Thresholding
Multiple Thresholding
Intensity Slicing
Bitplane Slicing
Histogram Stretching
Histogram Stretching
Histogram Sliding
Histogram Equalization
Histogram Specification
Spatial Filtering

A spatial filter consists of (a) a neighborhood, and (b) a


predefined operation

Linear spatial filtering of an image of size MxN with a filter of


size mxn is given by the expression

a b
g ( x, y ) w(s, t ) f ( x s, y t )
s a t b
Spatial Filtering
Spatial Correlation

The correlation of a filter w( x, y ) of size m n


with an image f ( x, y ), denoted as w( x, y ) f ( x, y )

a b
w( x, y ) f ( x, y ) w(s, t ) f ( x s, y t )
s a t b
Spatial Convolution

The convolution of a filter w( x, y ) of size m n


with an image f ( x, y ), denoted as w( x, y ) f ( x, y )

a b
w( x, y ) f ( x, y ) w(s, t ) f ( x s, y t )
s a t b
Smoothing Spatial Filters

Smoothing filters are used for blurring and for noise


reduction

Blurring is used in removal of small details and bridging of


small gaps in lines or curves

Smoothing spatial filters include linear filters and nonlinear


filters.
Spatial Smoothing Linear Filters

The general implementation for filtering an M N image


with a weighted averaging filter of size m n is given
a b

w(s, t ) f ( x s, y t )
g ( x, y ) s a t b
a b

w(s, t )
s a t b

where m 2a 1, n 2b 1.
Two Smoothing Averaging Filter Masks
Example: Gross Representation of Objects
Order-statistic (Nonlinear) Filters

Nonlinear

Based on ordering (ranking) the pixels contained in the


filter mask

Replacing the value of the center pixel with the value


determined by the ranking result

E.g., median filter, max filter, min filter


Median Filter
Median Filter
Median Filter
Median Filter
Median Filter
Example: Use of Median Filtering for Noise
Reduction
Sharpening Spatial Filters

Foundation

Laplacian Operator

Unsharp Masking and Highboost Filtering

Using First-Order Derivatives for Nonlinear Image


Sharpening The Gradient
Sharpening Spatial Filters: Foundation

The first-order derivative of a one-dimensional function f(x)


is the difference

f
f ( x 1) f ( x)
x

The second-order derivative of f(x) as the difference

2 f
f ( x 1) f ( x 1) 2 f ( x)
x 2
Sharpening Spatial Filters: Laplace Operator

The second-order isotropic derivative operator is the


Laplacian for a function (image) f(x,y)

2
f 2
f
f 2 2
2

x y
2 f
f ( x 1, y ) f ( x 1, y ) 2 f ( x, y )
x 2

2 f
f ( x, y 1) f ( x, y 1) 2 f ( x, y )
y 2

2 f f ( x 1, y ) f ( x 1, y ) f ( x, y 1) f ( x, y 1)
- 4 f ( x, y )
Sharpening Spatial Filters: Laplace Operator
Sharpening Spatial Filters: Laplace Operator

Image sharpening in the way of using the Laplacian:

g ( x, y ) f ( x, y ) c 2 f ( x, y )
where,
f ( x, y ) is input image,
g ( x, y ) is sharpenend images,
c -1 if 2 f ( x, y ) corresponding to Fig. 3.37(a) or (b)
and c 1 if either of the other two filters is used.
Unsharp Masking and Highboost Filtering

Unsharp masking
Sharpen images consists of subtracting an unsharp (smoothed)
version of an image from the original image
e.g., printing and publishing industry

Steps
1. Blur the original image

2. Subtract the blurred image from the original

3. Add the mask to the original


Unsharp Masking and Highboost Filtering

Let f ( x, y ) denote the blurred image, unsharp masking is


g mask ( x, y ) f ( x, y ) f ( x, y )
Then add a weighted portion of the mask back to the original
g ( x, y ) f ( x, y ) k * g mask ( x, y ) k 0

when k 1, the process is referred to as highboost filtering.


Unsharp Masking: Demo
Unsharp Masking and Highboost Filtering: Example
Image Sharpening based on First-Order Derivatives

For function f ( x, y ), the gradient of f at coordinates ( x, y )


is defined as
f
g x x
f grad( f )
g y f
y

The magnitude of vector f , denoted as M ( x, y )


Gradient Image M ( x, y ) mag(f ) g x 2 g y 2
Image Sharpening based on First-Order Derivatives

The magnitude of vector f , denoted as M ( x, y )


M ( x, y ) mag(f ) g x 2 g y 2

M ( x, y ) | g x | | g y |

z1 z2 z3
M ( x, y) | z8 z5 | | z6 z5 |
z4 z5 z6
z7 z8 z9
Image Sharpening based on First-Order Derivatives

Roberts Cross-gradient Operators


M ( x, y ) | z9 z5 | | z8 z6 |

Sobel Operators
M ( x, y) | ( z7 2 z8 z9 ) ( z1 2 z2 z3 ) |
z1 z2 z3 | ( z3 2 z6 z9 ) ( z1 2 z4 z7 ) |
z4 z5 z6
z7 z8 z9
Image Sharpening based on First-Order Derivatives
Example
Example:

Combining
Spatial
Enhancement
Methods

Goal:

Enhance the
image by
sharpening it
and by bringing
out more of the
skeletal detail
Example:

Combining
Spatial
Enhancement
Methods

Goal:

Enhance the
image by
sharpening it
and by bringing
out more of the
skeletal detail
Example
Spatial Filtering Techniques
Image Smoothing Filter
Variable Weights
Gaussian Filters
Directional Smoothing
Conservative smoothing
Sharpening Filters
High-Boost Filter
Frequency Domain Filtering
Image Smoothing Frequency
Domain 1D.
2D Case
Butterworth Filter
Band-Pass filter
Homomorphic Filtering
What is Image Restoration?
Image restoration attempts to restore
images that have been degraded
Identify the degradation process and attempt
to reverse it
Similar to image enhancement, but more
objective
Noise and Images
The sources of noise in digital
images arise during image
acquisition (digitization) and
transmission
Imaging sensors can be
affected by ambient
conditions
Interference can be added
to an image during
transmission
Image Degradation model
Modeling Process
Image Degradations
An Image Degradation Model
Two types of degradation
Additive noise
Spatial domain restoration (denoising) techniques are preferred
Image blur
Frequency domain methods are preferred
We model the degradation process by a degradation function h(x,y),
an additive noise term, (x,y), as g(x,y)=h(x,y)*f(x,y)+ (x,y)
f(x,y) is the (input) image free from any degradation
g(x,y) is the degraded image
* is the convolution operator
The goal is to obtain an estimate of f(x,y) according to the knowledge
about the degradation function h and the additive noise
In frequency domain: G(u,v)=H(u,v)F(u,v)+N(u,v)
Three cases are considered in this Chapter
g(x,y)=f(x,y)+ (x,y)
g(x,y)=h(x,y)*f(x,y)
g(x,y)=h(x,y)*f(x,y)+ (x,y)
A Model of the Image
Degradation/Restoration Process
Noise Model
We first consider the degradation due to noise only
h is an impulse for now ( H is a constant)
White noise
Autocorrelation function is an impulse function multiplied by a
constant
N 1 M 1
a( x, y ) ( s, t ) ( s x, t y ) N 0 ( x, y )
t 0 s 0
It means there is no correlation between any two pixels in the noise
image
There is no way to predict the next noise value
The spectrum of the autocorrelation function is a constant (white)
Gaussian Noise
Noise (image) can be classified according the
distribution of the values of pixels (of the noise image) or
its (normalized) histogram
Gaussian noise is characterized by two parameters,
(mean) and 2 (variance), by

1 ( z ) 2 / 2 2
p( z ) e
2
70% values of z fall in the range [(-),(+)]
95% values of z fall in the range [(-2),(+2)]
Gaussian Noise
Noise Modeling
Other Noise Models
Rayleigh noise
2
( z a )e ( z a ) / b for z a
2

p( z ) b

0 for z a
The mean and variance of this
density are given by
b( 4 )
a b / 4 and 2
4
a and b can be obtained through
mean and variance
Other Noise Models
Erlang (Gamma) noise
a b z b 1 az
e for z 0
p( z ) (b 1)!
0 for z 0
The mean and variance of this
density are given by
b
b / a and 2
a2
a and b can be obtained
through mean and variance
Other Noise Models

Exponential noise
ae az for z 0
p( z )
0 for z 0
The mean and variance
of this density are given
by 1
1 / a and
2

a2
Special case pf Erlang
PDF with b=1
Other Noise Models
Uniform noise
1
if a z b
p( z ) b a
0 otherwise
The mean and variance
of this density are given
by
(b a ) 2
(a b) / 2 and 2
12
Other Noise Models
Impulse (salt-and-pepper)
noise Pa for z a
p( z ) P for z b
b
0 otherwise

If either Pa or Pb is zero,
the impulse noise is called
unipolar
a and b usually are
extreme values because
impulse corruption is
usually large compared
with the strength of the
image signal
It is the only type of noise
that can be distinguished
from others visually
A Sample Image
Effect of Adding Noise to Sample Image
Effect of Adding Noise to Sample Image
Periodic Noise
Arises typically from electrical
or electromechanical
interference during image
acquisition
It can be observed by visual
inspection both in the spatial
domain and frequency
domain
The only spatially dependent
noise will be considered
Periodic Noise
Estimation of Noise Parameters

Periodic noise
Parameters can be estimated by inspection of the
spectrum
Noise PDFs
From sensor specifications
If imaging sensors are available, capture a set of
images of plain environments
If only noisy images are available, parameters of
the PDF involved can be estimated from small
patches of constant regions of the noisy images
Blur
Estimation of Noise Parameters

In most cases, only mean and variance are to


be estimated
Others can be obtained from the estimated mean
and variance
Assume a sub-image with plain scene is
available and is denoted by S
1 z ( xi , yi )
N S ( x , y )S
i i

0 2 1 ( z ( xi , yi ) ) 2 12 1 i i
( z ( x , y )
) 2

N S ( x , y )S
i i
NS ( xi , yi )S
2 1
2 i i
(
N S 1 ( x , y )S
z ( x , y )
) 2

i i
Restoration in the presence of
Noise
Order Statistic Filter
Inverse Filter
Degraded
Inverse Filtering Image

Filtering with H
cut off outside a
radius of 40
Full inverse
Filtering

Filtering with H cut


off outside a radius
of 70

Filtering with H
cut off outside a
radius of 85
Wiener Filter

The Wiener filter was proposed by Norbert


Wiener in 1940.
It was published in 1949
Its purpose is to reduce the amount of a
noise in a signal.
This is done by comparing the received
signal with a estimation of a desired
noiseless signal.
Wiener filter is not an adaptive filter as it
assumes input to be stationery.
Wiener Filter
It takes a statistical approach to solve its goal
Goal of the filter is to remove the noise from a signal
Before implementation of the filter it is assumed thatthe
user knows the spectral properties of the original signal and
noise.
Spectral properties like the power functions for both the
original signal and noise.
And the resultant signal required is as close to the original
signal.
Signal and noise are both linear stochastic processes with
known spectral properties.
The aim of the process is to have minimum mean square
error
That is, the difference between the original signal and the
new signal should be as less as possible.
Wiener Filter

Considering we need to design a wiener filter in


frequency domain as W(u,v)
Restored image will be given as;

Xn(u,v) = W(u,v).Y(u,v)

Where Y(u,v) is the received signal and Xn(u,v) is


the restored image
Wiener Filter

We choose W(k,l) to minimize:

= E {(f f)2 }
Obtained from [1]

Where the equation represents


the mean square error.
The wiener filter can be
represented by the equation:
Wiener Filter
H *
(u , v ) S (u , v )
F (u, v) G (u, v)
f

S f (u, v) H (u, v) S (u, v)


2

H * (u, v)
G (u, v)
H (u, v) S (u, v) / S f (u, v)
2

1 H (u, v)
2

G (u, v)
H (u, v) H (u, v) S (u, v) / S f (u, v)
2

H(u,v) = degradation function


|H(u,v)|^2 = H*(u,v)H(u,v)
H*(u,v) = complex conjugate of H(u,v)
Sn(u,v) = |N(u,v)|^2 power spectrum of noise
Sf(u,v) = |F(u,v)|^2 power spectrum of undegraded image
. G(u,v) is the transform of the degraded image.
Minimum Mean Square Error Filtering
(Wiener Filtering)
1 H (u , v )
2

F (u, v) G(u, v)
H (u, v) H (u, v) S (u, v) / S f (u, v)
2

Constant Unknown

1 H ( u , v )
2

F (u, v) G(u, v)
H (u, v) H (u, v) K
2

Dr. S D Ruikar 176


Wiener Filtering

Radially limited Wiener filtering


Full inverse
filtering inverse filtering (K was chosen interactively)

Dr. S D Ruikar 177


Inverse filtering Wiener filtering
Wiener Filtering

Reduce
d noise
variance

Dr. S D Ruikar 178


Wiener Filter Results
Blind Image Restoration
Thank you