Image is NOT Perfect Sometimes

EE465: Introduction to Digital Image

Processing 1
Image Enhancement for Visually
Impaired Patients

http://www.eri.harvard.edu/faculty/peli/proje
cts/enhancement.html
EE465: Introduction to Digital Image
Processing 2
Image Enhancement

 Introduction
 Spatial domain techniques
 Point operations
 Histogram equalization and matching
 Applications of histogram-based
enhancement
 Frequency domain techniques
 Unsharp masking
 Homomorphic filtering*

EE465: Introduction to Digital Image

Processing 3
Recall:
 There is no boundary of imagination in the
virtual world
 In addition to geometric transformation
(warping) techniques, we can also
photometrically transform images
 Ad-hoc tools: point operations
 Systematic tools: histogram-based methods
 Applications: repair under-exposed or over-
exposed photos, increase the contrast of iris
images to facilitate recognition, enhance
microarray images to facilitate segmentation.

EE465: Introduction to Digital Image

Processing 4
Point Operations Overview
Point operations are zero-memory operations where
a given gray level x[0,L] is mapped to another
gray level y[0,L] according to a transformation
y  f (x)
y
L

x
L
L=255: for grayscale images

EE465: Introduction to Digital Image

Processing 5
Lazy Man Operation

yx
y
L

x
L
No influence on visual quality at all

EE465: Introduction to Digital Image

Processing 6
Digital Negative
L

y  Lx

0 L x

EE465: Introduction to Digital Image

Processing 7
Contrast Stretching

 x 0 xa
 yb
y    ( x  a )  ya a xb
  ( x  b)  y ya
 b b xL x
0 a b L

a  50, b  150,  0.2,   2,   1, ya  30, yb  200

EE465: Introduction to Digital Image

Processing 8
Clipping

 0 0 xa

y    ( x  a) a  x  b
  (b  a) b  x  L

a b x
0 L

a  50, b  150,   2
EE465: Introduction to Digital Image
Processing 9
Range Compression

y  c log 10 (1  x)

0 L x

c=100
EE465: Introduction to Digital Image
Processing 10
Summary of Point Operation

 So far, we have discussed various forms of

mapping function f(x) that leads to different
enhancement results
 MATLAB function >imadjust
 The natural question is: How to select an
appropriate f(x) for an arbitrary image?
 One systematic solution is based on the
histogram information of an image
 Histogram equalization and specification
EE465: Introduction to Digital Image
Processing 11
Histogram based Enhancement
Histogram of an image represents the relative frequency
of occurrence of various gray levels in the image
3000

2500

2000

1500

1000

500

0
0 50 100 150 200

MATLAB function >imhist(x)

EE465: Introduction to Digital Image

Processing 12
Why Histogram?
4
x 10
4

3.5

2.5

1.5

0.5

0 50 100 150 200 250

It is a baby in the cradle!

Histogram information reveals that image is under-exposed

EE465: Introduction to Digital Image

Processing 13
Another Example

7000

6000

5000

4000

3000

2000

1000

0 50 100 150 200 250

Over-exposed image

EE465: Introduction to Digital Image

Processing 14
How to Adjust the Image?

 Histogram equalization
 Basic idea: find a map f(x) such that the histogram
of the modified (equalized) image is flat (uniform).
 Key motivation: cumulative probability function
(cdf) of a random variable approximates a uniform
distribution
x
Suppose h(t) is the histogram (pdf) s( x)   h(t )
t 0

EE465: Introduction to Digital Image

Processing 15
Histogram Equalization
x
y  L   h(t )
Uniform
t 0
Quantization
L
x Note:  h(t )  1
y s   h(t ) t 0
t 0
cumulative probability function
L 1

x
0 L
http://en.wikipedia.org/wiki/Inverse_transform_sampling
16
EE465: Introduction to Digital Image
Processing
MATLAB Implementation
function y=hist_eq(x)

[M,N]=size(x); Calculate the histogram

for i=1:256 of the input image
h(i)=sum(sum(x= =i-1));
End

y=x;s=sum(h);
for i=1:256
I=find(x= =i-1); Perform histogram
equalization
y(I)=sum(h(1:i))/s*255;
end

EE465: Introduction to Digital Image

Processing 17
 Image Example

before after

EE465: Introduction to Digital Image

Processing 18
 Histogram Comparison
3000 3000

2500 2500

2000 2000

1500 1500

1000
1000

500
500

0
0 50 100 150 200 0
0 50 100 150 200 250 300

before equalization after equalization

EE465: Introduction to Digital Image

Processing 19
Adaptive Histogram Equalization

http://en.wikipedia.org/wiki/Adaptive_histogram_equalization

EE465: Introduction to Digital Image

Processing 20
Histogram Specification/Matching
Given a target image B, how to modify a given image A such that
the histogram of the modified A can match that of target image B?

histogram1 histogram2

S-1*T

T S

?
EE565 Advanced Image Processing
Copyright Xin Li 2008 21
Application (I): Digital Photography

EE465: Introduction to Digital Image

Processing 22
Application (II): Iris Recognition

before after

EE465: Introduction to Digital Image

Processing 23
Application (III): Microarray Techniques

before after

EE465: Introduction to Digital Image

Processing 24
Application (IV)

EE465: Introduction to Digital Image

Processing 25
Application (V): Entertainment

Histogram-equalized
LDR video
Adaptive histogram
-equalized1
1EricP. Bennett and Leonard McMillan. “Video enhancement
using per-pixel virtual exposures,” In ACM SIGGRAPH 2005

EE465: Introduction to Digital Image

Processing 26
Image Enhancement

 Introduction
 Spatial domain techniques
 Point operations
 Histogram equalization and matching
 Applications of histogram-based
enhancement
 Frequency domain techniques
 Unsharp masking
 Homomorphic filtering*

EE465: Introduction to Digital Image

Processing 27
Frequency-Domain Techniques (I):
Unsharp Masking
y (m, n)  x(m, n)  g (m, n),   0
g(m,n) is a high-pass filtered version of x(m,n)

• Example (Laplacian operator)

1
g (m, n)  x(m, n)  [ x(m  1, n)  x(m  1, n) 
4
x(m, n  1)  x(m, n  1)]

EE465: Introduction to Digital Image

Processing 28
MATLAB Implementation

% Implementation of Unsharp masking

function y=unsharp_masking(x,lambda)

% Laplacian operation
h=[0 -1 0;-1 4 -1;0 -1 0]/4;
dx=filter2(h,x);
y=x+lambda*dx;

EE465: Introduction to Digital Image

Processing 29
1D Example
250
250

200
200

150
150

100
100

50 50

0 0
0 50 100 150 200 250 0 50 100 150 200 250

x(n) xlp(n)
220
8

200
6

4 180

2 160

0
140

-2
120

-4
100
-6

80
-8 0 50 100 150 200 250 300
0 50 100 150 200 250 300

g(n)=x(n)-xlp(n) y ( n )  x ( n )  g ( n )

EE465: Introduction to Digital Image

Processing 30
2D Example

MATLAB command >roidemo

EE465: Introduction to Digital Image
Processing 31
Frequency-Domain Techniques (II):
Homomorphic filtering
Basic idea: f ( x, y )  i ( x, y ) r ( x, y )

Illumination reflectance
(low freq.) (high freq.)

ln f ( x, y )  ln i ( x, y )  ln r ( x, y )

freq. domain enhancement

EE465: Introduction to Digital Image

Processing 32
Image Example

before after

EE465: Introduction to Digital Image

Processing 33
Summary of Nonlinear Image
Enhancement
 Understand how image degradation occurs first
 Play detective: look at histogram distribution, noise
statistics, frequency-domain coefficients…
 Model image degradation mathematically and try inverse-
engineering
 Visual quality is often the simplest way of evaluating
the effectiveness, but it will be more desirable to
measure the performance at a system level
 Iris recognition: ROC curve of overall system
 Microarray: ground-truth of microarray image segmentation
result provided by biologists

EE465: Introduction to Digital Image

Processing 34