Feature extraction

Scheme and tools in computer vision

z Recognition of features is, in general, a complex procedure requiring a

variety of steps that successively transform the iconic data to
recognition information.

z Handling unconstrained environments is still very challenging

problem.
Visual Appearance-base Feature Extraction (Vision)
Image Processing Scheme Tools in Computer Vision
Computer Vision

edge
conditioning thresholding detection filtering
image

simplified image

connected
labeling/grouping component correlation
labeling
environment Hough
groups of pixel transfor-
mation

extracting disparity
cognition / action

properties

matching model
Feature Extraction (Vision): Tools
z Conditioning
¾ Suppresses noise
¾ Background normalization by suppressing uninteresting systematic or patterned
variations
¾ Done by:
‹ gray-scale modification (e.g. thresholding)
‹ (low pass) filtering
z Labeling
¾ Determination of the spatial arrangement of the events, i.e. searching for a
structure
z Grouping
¾ Identification of the events by collecting together pixel participating in the same
kind of event
z Extracting
¾ Compute a list of properties for each group
z Matching (see chapter 5)
Filtering and Edge Detection
z Gaussian Smoothing
¾ Removes high-frequency noise
¾ Convolution of intensity image I with G:

with:

z Edges
¾ Locations where the brightness undergoes a sharp change,
¾ Differentiate one or two times the image
¾ Look for places where the magnitude of the derivative is large.
¾ Noise, thus first filtering/smoothing required before edge detection
Edge Detection
z Ultimate goal of edge detection
¾ an idealized line drawing.
z Edge contours in the image correspond to important scene contours.
Optimal Edge Detection: Canny

z The processing steps

¾ Convolution of image with the Gaussian function G
¾ Finding maxima in the derivative
z Canny combines both in one operation

(a) A Gaussian function. (b) The first derivative of a Gaussian function.

Optimal Edge Detection: Canny 1D example

(a) Intensity 1-D profile of an ideal step edge.

(b) Intensity profile I(x) of a real edge.
(c) Its derivative I’(x).
(d) The result of the convolution R(x) = G’ ⊗ I, where G’ is the first
derivative of a Gaussian function.
Optimal Edge Detection: Canny
z 1-D edge detector can be defined with the following steps:
1. Convolute the image I with G’ to obtain R.
2. Find the absolute value of R.
3. Mark those peaks |R| that are above some predefined threshold T. The
threshold is chosen to eliminate spurious peaks due to noise.
z 2D → Two dimensional Gaussian function
Optimal Edge Detection: Canny Example
a) Example of Canny edge detection
b) After nonmaxima suppression (reduce the thickness of all edges to a
single pixel)
Nonmaxima Suppression

z Output of an edge detector is usually a

b/w image where the pixels with
gradient magnitude above a predefined
threshold are black and all the others
are white

z Nonmaxima suppression generates

contours described with only one pixel
thinness by determining the local
maximum.
Gradient Edge Detectors
z Roberts

z Prewitt

z Sobel
Example

a) Raw image

b) Filtered
(Sobel)

c) Thresholding

d) Nonmaxima
suppression

Fig.4.46
Comparison of Edge Detection Methods

z Average time required to compute the edge figure of a 780 x 560 pixels image.
z The times required to compute an edge image are proportional with the accuracy of
the resulting edge images
Dynamic Thresholding
z Changing illumination
¾ Constant threshold level in edge detection is not suitable
z Dynamically adapt the threshold level
¾ consider only the n pixels with the highest gradient magnitude for further
calculation steps
¾ The gradient magnitude of the point where n is reached (counted backward from
highest value) is used as the temporary threshold value

(a) Number of pixels with a specific gradient magnitude in the image of Fig. 4.46(b).
(b) Same as (a), but with logarithmic scale
Hough Transform: Straight Edge Extraction
z All points p on a straight-line edge must satisfy yp = m1 xp + b1 .
z Each point (xp, yp) that is part of this line constraints the parameter m1

and b1.
z The Hough transform finds the line (line-parameters m, b) that get most

“votes” from the edge pixels in the image.

z This is realized by four steps

1. Create a 2D array A [m,b] with axes that tessellate the values of m and b.
2. Initialize the array A to zero: A[m b]=0 for all values of m, b.
3. For each edge pixel (xp, yp) in the image, loop over all values of m and b:
if yp = m1 xp + b1 then A[m,b]+=1
4. Search cells in A with largest value. They correspond to extracted straight-
line edge in the image.
Grouping, Clustering: Assigning Features to Features

pixels feature object

z Connected Component Labeling

1 2 1 1
1 1 1 1
1 1 1=2 1 1
1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 3=4 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1
1 1 1 1 3 4 1 1 1 1 3 3
1 1 1 1 33 1 1 1 1 33
1 1 33 3 1 1 3 3 3
1 1 3 3 3 3 3 3 3 1 1 3 3 3 3 3 3 3
1 1 3 3 3 3 3 3 1 1 3 3 3 3 3 3
3 3 3 3 3 3 3 3 3 3 3 3
3 3 3 3
3 3 3 3
Floor Plane Extraction
z Vision based identification of traversable
z Assumptions

¾ Obstacles differ in appearance from the ground

¾ The ground is flat and its angle to the camera is known
¾ There are no overhanging obstacles
z Floor plane extraction in artificial environments
¾ Shakey, the first autonomous mobile robot developed from 1977 to 1972 at
SRI, use vision based floor plane extraction
¾ Textureless white floor tiles, with walls painted with a high-contrast strip
of black paint and the edges of all simple polygonal obstacles were also
painted black
Adaptive Floor Plane Extraction
z The parameters defining the
expected appearance of the floor
are allowed to vary over time
z Use edge detection and color

detection jointly
z The processing steps

¾ As pre-processing, smooth If using

a Gaussian smoothing operator
¾ Initialize a histogram array H with
n intensity values:
for
¾ For every pixel (x,y) in If increment
the histogram:
Whole-Image Features
z OmniCam: a CCD
camera mounted
toward a parabolic
mirror
z Compact

representations of
the entire local
region
z Extract one or

more features
from the image
that are correlated
well with the
robot’s position
Direct Extraction: image histograms
• When combined with the catadioptric camera’s field of view
invariance, we can create a system that is invariant to robot rotation
and insensitive to small-scale robot translation.

• A color camera’s output image contains information along multiple

bands: R, G, and B values as well as hue, saturation, and luminance
values

• The simplest histogram-based extraction strategy is to build separate

1D histograms charactering each band. We use Gi to refer to an array
storing the values in band i for all pixels in G.
Image Histograms
z The processing steps
¾ As pre-processing, smooth using a Gaussian smoothing operator
¾ Initialize with n levels:
¾ For every pixel (x,y) in increment the histogram:

Jeffrey divergence:
2hi 2k i
d ( H , K ) = ∑ (hi log + k i log )
i hi + k i hi + k i
Using measure such as Jeffrey
divergence, mobile robots have
used whole-image histogram
features to identify their position in
real-time against a database of
previously recorded images of
locations in their environment.
Image Fingerprint Extraction
z Highly distinctive combination of simple features v: vertical edge
A,B,C,..,P: hue bins

String from current location

Strings from the database

Homework 5 (Due two weeks from now)
z Use MATLAB to do feature detection on the image I posted on
pipeline. You are asked to use Canny detection and Sobel detection.
Compare the results of two algorithms. You need submit the following
items
¾ Report including introduction of two detection algorithms, the
procedures and final results.
¾ Matlab file for each algorithm so that I can run them on my PC.