Академический Документы
Профессиональный Документы
Культура Документы
Bobick Segmentation
Segmentation
Administrivia
PS 4: Out but I was a bit late so due date pushed back to
Oct 29.
Why segmentation?
CS 4495 Computer Vision A. Bobick Segmentation
X. Ren and J. Malik. Learning a classification model for segmentation. ICCV 2003.
Slide by Svetlana Lazebnik
CS 4495 Computer Vision A. Bobick Segmentation
D. Tsai, M. Flagg, and J. M. Rehg. Motion Coherent Tracking with Multi-label MRF optimization.
BMVC 2010.
CS 4495 Computer Vision A. Bobick Segmentation
J. Strom, A. Richardson, E. Olson. Graph-based Segmentation for Colored 3D Laser Point Clouds.
IROS 2010.
pixel count
black pixels
gray
1 2 pixels
input image
intensity
Kristen Grauman
CS 4495 Computer Vision A. Bobick Segmentation
Noisy Images
pixel count
input image
intensity
pixel count
input image
Kristen Grauman intensity
CS 4495 Computer Vision A. Bobick Segmentation
Noisy Images
pixel count
input image
intensity
Kristen Grauman
CS 4495 Computer Vision A. Bobick Segmentation
Clustering
0 190 255
intensity
3
1 2
Kristen Grauman
CS 4495 Computer Vision A. Bobick Segmentation
Clustering
With this objective, it is a chicken and egg problem:
Q: how to determine which points to associate with each
cluster center, ci?
A: for each point p, choose closest ci
Kristen Grauman
CS 4495 Computer Vision A. Bobick Segmentation
Andrew Moore
CS 4495 Computer Vision A. Bobick Segmentation
Andrew Moore
CS 4495 Computer Vision A. Bobick Segmentation
Andrew Moore
CS 4495 Computer Vision A. Bobick Segmentation
Andrew Moore
CS 4495 Computer Vision A. Bobick Segmentation
Andrew Moore
CS 4495 Computer Vision A. Bobick Segmentation
Segmentation as clustering
Depending on what we choose as the feature space, we
can group pixels in different ways.
Number of Clusters
K=2
K=3
Segmentation as clustering
Depending on what we choose as the feature space, we
can group pixels in different ways.
R=255
Grouping pixels based G=200
on color similarity B=250
B R=245
G G=220
B=248
R=15 R=3
G=189 G=12
R B=2 B=2
Feature space: color value (3-d) Kristen Grauman
CS 4495 Computer Vision A. Bobick Segmentation
Segmentation as clustering
K-means clustering based on intensity or color is
essentially vector quantization of the image attributes
Segmentation as clustering
Depending on what we choose as the feature space, we
can group pixels in different ways.
Intensity
Segmentation as clustering
Source: K. Grauman
CS 4495 Computer Vision A. Bobick Segmentation
Segmentation as clustering
Clustering based on (r,g,b,x,y) values enforces more
spatial coherence
Segmentation as clustering
Color, brightness, position alone are not enough to
distinguish all regions
CS 4495 Computer Vision A. Bobick Segmentation
Segmentation as clustering
Depending on what we choose as the feature space, we
can group pixels in different ways.
F2
Filter bank
of 24 filters
F24
mean mean
d/dx d/dy
value value
Win. #1 4 10
Win.#2 18 7
Win.#9 20 20
Dimension 1 (mean d/dx value)
Count
Image Texton map
Texton index
Count
Count
Texton index Texton index
Malik, Belongie, Leung and Shi. IJCV 2001. Adapted from Lana Lazebnik
CS 4495 Computer Vision A. Bobick Segmentation
Kristen Grauman
CS 4495 Computer Vision A. Bobick Segmentation
Make it better
K-means heavily sensitive to initial conditions and
(typically) need to know K in advance.
Feature space
image (L*u*v* color values)
CS 4495 Computer Vision A. Bobick Segmentation
CIE xy
chromaticity
diagram, 1931
CS 4495 Computer Vision A. Bobick Segmentation
X
y=
X+Y+Z
Y
y=
X+Y+Z
CIE xy
chromaticity
diagram, 1931
CS 4495 Computer Vision A. Bobick Segmentation
Cylindrical view
Think of chroma
(here a*, b*) defining
a planar disc at each
luminance level (L)
CS 4495 Computer Vision A. Bobick Segmentation
My favorite
CS 4495 Computer Vision A. Bobick Segmentation
Feature space
image (L*u*v* color values)
CS 4495 Computer Vision A. Bobick Segmentation
Center of
mass
Mean Shift
vector
Center of
mass
Mean Shift
vector
Center of
mass
Mean Shift
vector
Center of
mass
Mean Shift
vector
Center of
mass
Mean Shift
vector
Center of
mass
Mean Shift
vector
Center of
mass
http://www.caip.rutgers.edu/~comanici/MSPAMI/msPamiResults.html
CS 4495 Computer Vision A. Bobick Segmentation
Mean shift
Pros:
Does not assume shape on clusters
One parameter choice (window size)
Generic technique
Find multiple modes
Cons:
Selection of window size
Does not scale well with dimension of feature space
Kristen Grauman
CS 4495 Computer Vision A. Bobick Segmentation
Images as graphs
q
wpq
w
p
Fully-connected graph
node (vertex) for every pixel
link between every pair of pixels, p,q
affinity weight wpq for each link (edge)
wpq measures similarity
similarity is inversely proportional to difference (in color and position)
Measuring affinity
One possibility:
Kristen Grauman
CS 4495 Computer Vision A. Bobick Segmentation
j
wij
i
A B C
Source: S. Seitz
CS 4495 Computer Vision A. Bobick Segmentation
Graph cut
B
A
cut ( A, B) = w
p A, qB
p ,q
Source: S. Seitz
CS 4495 Computer Vision A. Bobick Segmentation
B
A
cut ( A, B) = w
p A, qB
p ,q
Minimum cut
Problem with minimum cut:
Weight of cut proportional to number of edges in the cut;
tends to produce small, isolated components.
B
A
Normalized Cut
fix bias of Min Cut by normalizing for size of segments:
cut ( A, B) cut ( A, B )
+
assoc( A, V ) assoc ( B,V )
assoc(A,V) = sum of weights of all edges that touch A
Ncut value small when we get two clusters with many edges
with high weights, and few edges of low weight between them
Approximate solution for minimizing the Ncut value :
generalized eigenvalue problem.
J. Shi and J. Malik, Normalized Cuts and Image Segmentation, CVPR, 1997 Source: Steve Seitz
CS 4495 Computer Vision A. Bobick Segmentation
Example results
CS 4495 Computer Vision A. Bobick Segmentation
http://www.cs.berkeley.edu/~fowlkes/BSE/
CS 4495 Computer Vision A. Bobick Segmentation
Cons:
Time complexity can be high
Dense, highly connected graphs many affinity computations
Solving eigenvalue problem
Preference for balanced partitions
Kristen Grauman
CS 4495 Computer Vision A. Bobick Segmentation
The end
CS 4495 Computer Vision A. Bobick Segmentation
Figures courtesy of
McAdam ellipses D. Forsyth
CS 4495 Computer Vision A. Bobick Segmentation