Comments on the resampling step in particle

filtering
Lennart Svensson
March 16, 2012
Particle filters (PFs) are known to degenerate over time [1], which means that
most of the particles have negligible weights ( 0). Resampling is a technique
used in all PFs to delete these particles and instead create multiple copies of
the particles with larger weights (the number of copies of a particle should
essentially be in proportion to its weight).
(1)
(N )
(1)
(N )
Suppose that we have a set of particles, xk , . . . , xk , and weights, wk , . . . , wk .
The traditional resampling method [1, 2] works like follows:
(i)

(i)

k = xk for i = 1, . . . , N .
Set x
For i = 1, . . . , N , generate j(i) such that
(r)

Pr{j(i) = r} = wk .

(1)

For i = 1, . . . , N , set
(i)

(j(i))

k
xk = x
1
(i)
wk = .
N

(2)
(3)

It is useful to note that a vector of j(i)-variables can be generated conveniently

in Matlab using the command line
[, j] = histc(rand(N,1), [0 cumsum(w)]);
(i)
(i)
and that the first part of the algorithm (
xk = xk ) is not needed in the implementation.
Alternative algorithms to perform resampling are discussed and analysed
in [3]. One algorithm that compares favourably with the other algorithms is
systematic sampling, which generates the j(i)-variables according to a different
scheme:
Generate u unif[0, 1].
For i = 1, . . . , N , find r such that (i1+u)/N
and set j(i) = r.
1

 Pr1

n=1

(n)

wk ,

Pr

n=1

(n)

wk

Though this may look more complicated it is actually a slightly faster algorithm,
which can also be implemented conveniently in Matlab:
[, j] = histc(rand(1)/N+0:1/N:(N-1)/N, [0 cumsum(w)]);
However, since the complexity of both algorithms are O(N ) (and really fast!),
the benefits with systematic resampling are instead mainly a closer match between the number of produced copies of a particle and its weight. Having said
that, both algorithms work very well in practice and are asymptotically optimal,
i.e., that the additional approximations imposed by the resampling step goes to
0 as N .
To understand why resampling inevitably leads to an approximation for
most settings with small N , you may consider a simple example. Suppose
(1)
that you only have two particles (i.e., N = 2), with weights wk = 0.2 and
(2)
wk = 0.8. After resampling we will have two particles with weight 0.5 which
cannot represent the previous distribution. The best alternative may be to
create two copies of the second particle though that means that (what used to
be) the first particle disappears in the resampling step.

References
[1] A. Doucet, S. Godsill, and C. Andrieu. On sequential monte carlo sampling
methods for bayesian filtering. Statistics and Computing, 10(3):197208,
2000.
[2] B. Ristic, S. Arulampalam, and N. Gordon. Beyond the Kalman filter :
particle filters for tracking applications. Norwood, MA: Artech House, 2004.
[3] Jeroen D. Hol, Thomas B. Schon, and Fredrik Gustafsson. On resampling
algorithms for particle filters. In Nonlinear Statistical Signal Processing
Workshop, Cambridge, United Kingdom, 2006.