Today's Lecture

● Segmentation
– Threshold
● Histogram analysis
● Local thresholds
● Hysteresis threshold
– Split and Merge
● Region merging
● Region splitting
– Watershed
● As a region growing algorithm
● Region merging
● Seeds

● Book: sections 6.1, 6.3 and 13.7

 Segmentation
● Purpose:
– Detect objects
– Find which pixels belong to an object
● Methods:
– Region Based (today's lecture)
● Threshold
● Split and Merge
● Watershed
● …
– Edge Based (“Image Understanding” lectures)
● Border tracing
● Snakes
● Active shape models
● …
Segmentation
 Threshold
● Simplest form of segmentation
● Associates each pixel to object or background based on
the pixel's grey value
– Static or global threshold: same threshold for all pixels

g  x = {
1 if f  x T
0 otherwise
– Adaptive or local threshold: threshold depends on local
neighbourhood

g  x = {
1 if f  x T  x 
0 otherwise

– Hysteresis threshold: combine results of two thresholds

 Finding a Threshold Level
● For a global threshold, all relevant information is in the
image's histogram
– Because no neighbourhood information is used, only each
individual pixel's grey value
– Histogram shows distribution of grey values
– Object and background often have separate peaks
Bimodal Histograms

Bayes minimum error Local minimum

(theoretically optimal) between peaks
 Bimodal Histograms
● k-means clustering (“isodata” method)
– Ridler and Calvard (1978)
– Assumes the two modes are of similar width and height
– Iterative method, depends on initialisation
● Minimizing intra-class variance (“Otsu” method)
– Otsu (1979)
– Equivalent to maximizing inter-class variance (easy to compute)
2 2
 b t =P 1 t P 2 t  [ 1 t−2 t  ]

● Minimizing error
– Kittler and Illingworth (1986)
– Assumes 2 Normal distributions
J t =12 [ P 1 t log 1 t P2 t  log 2 t  ] −2 [ P 1 t log P1 t P 2 t log P 2 t ]
 Improving the Histogram

histogram of pixels
with strong gradient
Foreground peak magnitude only
much smaller than
background peak
 Unimodal Histograms

Background peak

Foreground doesn't
have a peak!
 Unimodal Histograms

Fitting a Gaussian to the

background peak

2σ

Chord method, a.k.a.:

● skewed bi-modality

● maximum distance to triangle

Zack, Rogers and Latt (1977)

Rosin (2001)
 Finding a Threshold Level
Other methods used besides histogram analysis:
● Manual determination on a training set of images
– Threshold becomes a “magic number”
– Results useless if imaging circumstances change
– E.g. in CT the grey-value is an absolute measure
● Using a priori knowledge:
– Volume: if it is known that 25% of the image is foreground,
choose a threshold value so that 25% of the pixels are
above it
– Shape: if round objects are expected, choose a threshold
value that maximizes some roundness measure of the result
– …
Multi-Channel Threshold

green

red
Multi-Channel Threshold

green

red
 Adaptive Threshold
● Simplest form: compute T(x) by filtering:

g  x = {
1 if f  x f  x ⊗h x 
0 otherwise
● More complex form:
– Divide image into smaller regions
– Calculate histogram-based static threshold within each
region separately
– This might result in artefacts at the region boundaries
● Most complex form:
– Calculate histogram-based threshold within neighbourhood
of each pixel
– This is a complex form of filtering
Simplest Adaptive Threshold
Using a threshold function based on filtering.
In this case, filter is:
threshold = closing(img,25)-30

global threshold local threshold

Simplest Adaptive Threshold
Computing a threshold in small regions in the
image (in this case, isodata threshold).
Problems when:
● a region has uniform intensity

● the regions are too big

32x32 px regions 128x128 px regions

 Hysteresis Threshold
High threshold
(used e.g. in Canny’s
edge detector)

Regions in “low” that

have some pixels set
in “high”
Low threshold
 Split and Merge
● Divides up image into regions of uniform grey value,
using a homogeneity measure H (e.g. variance)
● Looks for largest regions that satisfy homogeneity
measure
● Split:
– Start with whole image as one region, recursively split
regions that do not satisfy the homogeneity criterion
● Merge:
– Start with each pixel being a region, merge two neighbouring
regions if the resulting region still satisfies the homogeneity
criterion
● These two methods usually do not lead to same
segmentation!
 Split and Merge
● Start with some intermediate-sized regions
(checkerboard)
● Recursively split regions that do not satisfy
homogeneity criterion
● Recursively merge regions until no more merging is
possible
● It is also possible to do one pass merging, one pass
splitting, one pass merging, …
● Remove small regions by merging with the most similar
neighbour
Split and Merge
 Alternatives in Split and Merge
● Homogeneity criterion can be:
– Maximum allowed variance within the region (σ2)
– Maximum allowed range within the region (max-min)
– Integral of gradient magnitude within the region
● (doesn't work if regions are too small)
– Uniformity of some texture measure
● (doesn't work if regions are too small)
– …
● Instead of homogeneity, you can use:
– Region shape (e.g. roundness measure)
● (only works for merging, not splitting)
– …
● Merging often used as second step to watershed
 The Watershed Segmentation

US: watershed, divide

GB: basin, watershed
The Watershed Segmentation
The Watershed Segmentation
watersheds

local minima
 Algorithm 1

new label

“water level”
Algorithm 1

same label
 Algorithm 1

new label

same label
Algorithm 1

Two regions touch:

Place watershed line
 Algorithm 1
Two regions touch:
Place watershed line
 Algorithm 1
● Sorted list of all pixels in image, lowest grey value first
– (i.e. priority queue; use QuickSort)
● Take pixel from queue, examine neighbours:
– 0 labels: pixel is local minimum, assign new label
– 1 label: part of same catchment basin, assign same label
– 2+ labels: watershed line, assign “watershed” label
● Finished after processing all pixels in queue
● Pros/cons:
– Independent of image dimensionality & sampling grid
– Queue needs additional of memory
– After making queue, input image is no longer needed
– Flat areas (plateaus) not handled correctly
– But it's easy to build region merging into the algorithm
Algorithm 2

local minima
Algorithm 2

grow region
 Algorithm 2

grow regions
Algorithm 2

Two regions touch:

Place watershed line
 Algorithm 2
● Start with all local minima identified and labeled (seeds)
● Add all neighbours of local minima to a priority queue
● Process pixels in queue:
– Take pixel from queue, examine neighbours:
● 1 label: part of same catchment basin, assign same label
● 2+ labels: watershed line, assign “watershed” label
– Enqueue unlabeled neighbours
● Finished when queue is empty
● Pros/cons:
– Independent of image dimensionality & sampling grid
– Can edit initial seed image
– Detecting local minima can be expensive
– Use stable priority queue to handle watershed lines across
flat areas (plateaus)
 Oversegmentation
● Noise creates many local minima:
– Smoothing filter is a must!
● Even without noise regions tend to have more than one
local minimum
● Region merging:
– Built in: when finding a watershed pixel, decide whether to
merge the two regions or keep the watershed:
● Size, depth
– As a 2nd step: use the “merge” part of Split and Merge:
● Homogeneity, shape, texture, edge strength, size, depth
– As preprocessing to Algorithm 2: remove local minima:
● Position, density, grey-value, manual selection
● It's even possible to create a seed image by hand, independent
of local minima
 Built-in Region Merging

merging based on: Two regions touch:

● size Place watershed line ???
● depth

size
depth

Equivalence table:
Oversegmentation
No smoothing

More smoothing:
Opening, d = 5

Little smoothing:
Gaussian, σ = 1 Merging: Depth <= 50
Applying the Seeded Watershed

Local minima of Watershed based on

blurred image seeds, after pruning
Applying the Seeded Watershed

Using original grey values Using blurred image

Applying the Watershed
● To separate uniform regions
with different grey value:
(we're looking at the background as
another object here)
– Apply gradient magnitude
– Watershed lines will go along
ridges of gradient magnitude
– Proper merging of regions is
required unless each region is
perfectly flat (unlikely!)
● Useful for example when
background is not uniform but
boundaries are clearly visible
Applying the Watershed
Applying the Seeded Watershed
 Watershed on the DT
● The Distance Transform (DT) returns, for each pixel, the
distance to the background. Constrictions cause an
object to obtain 2 local maxima
● The watershed is applied to the inverse DT

Input The distance Watershed of -DT

merged objects transform
 Watershed on the DT
● Distance to boundary does not yield expected object
boundary: we’re using the wrong definition for that!
● We’re expecting boundary to be equidistant to object
centres – we can do that too...

Poorly split objects The distance Watershed of DT

transform
 Another Thought
● Could we apply the watershed on a histogram to find a
good threshold value?
 Summary of Today’s Lecture
● Threshold
– Histogram analysis
– Local thresholds
– Hysteresis threshold
● Split and Merge
– Region merging
– Region splitting
● Watershed
– As a region growing algorithm
– Region merging
– Seeds