Robust Subband Video Coding with Leaky Prediction

Arild Fuldseth Tor A. Ramstad

Department of Telecommunications,
Norwegian University of Science and Technology (NTNU)
N-7034 Trondheim, Norway
Telephone: +47 73 59 26 50. Fax: +47 73 59 26 40.
email: ffuldseth,torg@tele.unit.no.

ABSTRACT the cost of less prediction gain. Thus, for a given

channel-to-noise ratio (CSNR), there is an opti-
The use of leaky prediction in a video trans- mal value of the interframe prediction coecient
mission system designed for noisy channels is  corresponding to the best possible trade-o be-
investigated. For the proposed system, graceful tween the amount of error propagation and the
degradation is achieved by nding good index
maps between the quantized source coder pa-
rameters and the amplitude levels of a QAM
signal constellation. The contribution of this
paper, however, is to investigate the use of
leaky prediction to further increase the robust-
ness to channel noise. It is shown that by us-
ing a temporal prediction coecient slightly less
than one, large improvements can be achieved
with only a small degradation for the noiseless
case. The performance of the proposed system
is comparable to a reference system based on
the H.263 video coder for high CSNR values,
and degrades far more gracefully for low CSNR
values.
prediction gain. In this paper, approximate opti-
mal values of  for various CSNR values are found
experimentally. Furthermore, for the cases where
the CSNR is not known at the encoder side, the ef-
fects of channel mismatch are addressed. Finally,
the proposed system is compared to a traditional
video communication system based on the H.263
video coder.
2. SYSTEM DESCRIPTIONS
2.1. Reference System
The reference system which is based on the H.263
video coder and QAM is illustrated in Figure 1.
The H.263 coder was implemented according to [6]
without any of the optional coding techniques, and
without the chrominance information. In addition
1. INTRODUCTION
Recently, there has been much interest in
combined source coding and channel cod-
ing/modulation for transmission over noisy chan-
nels. In [1], the index assignment problem was
addressed by using simulated annealing and as-
suming a binary symmetric channel (BSC). The
idea behind this method was to assign binary code-
words to the quantizer indices such as to minimize
the eects of channel errors. In [2, 3, 4], the simu-
to the specications given in [6], resynchroniza-
tion based on the group of block (GOB) header as
well as error detection and error concealment tech-
niques are used in the decoder. Error detection is
based on detection of illegal codewords, while er-
ror concealment is performed by repeating infor-
mation from the previous GOB, or from the pre-
vious frame whenever an error is detected. Fur-
thermore, in order to reduce the propagation of
errors through subsequent frames, NI intrablocks
lated annealing method was modied to optimize
the index assignment (index map) with multilevel
modulation instead of a BSC, and with the trans-
mitted power as an additional constraint. In par-
ticular, the method was used to design a robust
system for still image transmission by optimizing
the index maps from the source coder parame-
ter space to the amplitude levels of a quadrature
amplitude modulation (QAM) signal constellation
are introduced systematically in each frame at the
encoder side. Note that the eects of introducing
intrablocks systematically are similar to the eects
of leaky prediction, since the error propagation for
noisy channels is reduced at the cost of less cod-
ing eciency for a noiseless channel. The output
bit stream from the H.263 coder is mapped to a
16-QAM signal constellation by traditional Gray
coding. Note that since the H.263 coder uses vari-
able length coding, it is not possible to take the
(e.g. 64-QAM). Furthermore, in [5] the techniques
in [2, 3, 4] were extended to video coding. In
this work, we address the problem of channel error
propagation due to the temporal prediction feed-
back loop of the video coder structure. In par-
ticular, the robustness of the video transmission
system of [5] is increased further by using leaky
interframe prediction. This reduces the propa-
gation of channel errors to subsequent frames at
signicance or the meaning of the individual bits
into account when choosing the mapping.
2.2. Proposed System
The proposed system, illustrated in Figure 2, uses
subband decomposition and temporal prediction
with motion compensation as in H.263. The sub-
band signals are coded by subdividing each sub-
band into blocks of 4  4 samples. Each block is
 Video frames Bit stream QAM symbols
- H.263
encoder
- coding
Gray -
Figure 1. Reference system, (transmitter only).
Block classication
Quantizer scale factors
-- MAP -
Video - VQ3 - MAP3 - QAM
symbols
-
frames i- lterbank
Analysis - Classi-
6 cation - VQ2 - MAP2 -
- VQ1 - MAP1 -
???
- Class 0 Synthesis
lterbank

- ?i

...
.......
....
....
....
 compensation
...

..
Motion  Frame
delay
Motion
vectors
6
- estimation 
Motion

Figure 2. Proposed system, (transmitter only). The normalization and renormalization operations are assumed
to be included in VQ1-VQ3.
classied into one of four classes according to its
mean squared value, and the subband samples be-
longing to class 1-3 are normalized using the corre-
sponding class mean squared value. Next, the nor-
malized subband samples are quantized using vec-
tor quantizers (VQs) of three dierent rates (mea-
sured in bits per source sample), with the high-
est rate for the class corresponding to the high-
est mean squared value. The blocks having the
smallest mean squared value (class 0) are neither
quantized nor transmitted.
Next, the VQ output indices are mapped to the
amplitude levels of the QAM channel symbols by
the index maps. The index maps were designed to
have the following two properties: First, in order
to minimize the eects of channel errors, points
that are close in the QAM signal constellation
(channel space) should correspond to points (code-
book vectors) that are close in the source space.
Second, in order to maximize the minimum dis-
classes are listed in Table 1. The QAM signal con-
stellations are chosen with an odd number of am-
plitudes in each dimension. This allows for power
ecient transmission by assigning the codebook
vector having the highest probability to a chan-
nel symbol with zero amplitude. As an example,
consider class 2. The subband samples belong-
ing to this class are divided into vectors of length
L = 3, and quantized using a VQ codebook of size
N = 81. The codebook indices are then mapped
to an 81 QAM signal constellation (K = 2) by the
corresponding index map.
class L N K
1 2 9 1
2 3 81 2
3 1 15 1
Table 1. Source vector dimension (L), codebook
size (N ), and channel space dimension (K ) for each
tance, dmin , of the signal constellation for a xed class. The case K = 1 corresponds to the real or
transmitted power, the most probable codebook imaginary axis of a QAM constellation.
vectors should be mapped to the QAM points hav-
ing the smallest amplitude. In this work, simu- The side information parameters consisting of
lated annealing was used to design the index maps
as described in [5].
For the proposed system, the choices of source
vector dimension (L), codebook size (N ), and
channel space dimension (K ) for each of the three
the motion vectors, the block classication table,
and the quantizer scale factors are particularly
sensitive to channel noise. Also, robust mappings
are dicult to design in this case, since the neigh-
bor properties for these parameters are not well
dened. In order to reduce the error rate for the In the rst experiment, the reference system
side information, a sparse subset of an 81-QAM was simulated with a quantizer step size of 12
signal constellation is used. This reduces the error (QP = 6) and with various values of NI (the num-
rate signicantly for a given noise level. In ad-
dition, power ecient transmission is achieved by
mapping the most probable values of the side in-
formation parameters to the QAM symbols having
the smallest amplitudes.
3. LEAKY PREDICTION
ber of intra blocks per frame), resulting in vari-
ous bit rates and channel symbol rates. The av-
erage number of bits per pixel (bpp) and QAM
symbols per pixel (spp) for each value of NI are
shown in Table 2. In Figure 3, the resulting peak
signal-to-noise ratio (PSNR) of the reconstructed
sequence is evaluated for various values of NI and
As mentioned above, the optimal value of the leaky
prediction coecient  is a result of a trade-o be-
tween limiting the channel noise propagation and
maximizing the interframe prediction gain. In [7],
a similar optimization problem for a DPCM sys-
tem with scalar quantization was studied. In the
case of a 1st order predictor, it was shown that for
a temporal correlation coecient , the normal-
ized reconstruction error can be expressed as
 = [q + m + n =(2 ? 1)][1 + 2 ? 2] (1)
where q and n correspond to the quantization
and the channel noise respectively, and m is a
mutual term.
the CSNR. As can be seen from the gure, the
robustness to channel noise increases slightly with
an increased value of NI .
NI bpp spp
0 0.338 0.085
1 0.348 0.087
3 0.367 0.092
9 0.425 0.106
Table 2.Average bpp and spp for various values of NI
Next, the proposed system was simulated with
various values of  at a channel rate of 0.092 spp.
The results are illustrated in Figure 4. From the
Thus, the optimal value of  must satisfy the gure, we observe that the robustness to channel
following fth order equation
(1 ? 2 )2( ? )(q + m ) ? (2 ? 2 + )n = 0 (2)
Assuming that the quantizer as well as the source
and channel statistics are known, the optimal
value of the prediction coecient, opt can be
found numerically. A special case is for very noisy
channels i.e. n  q + m . In this case we have
[7]
q could be as high as 0.99 or higher. The resulting
opt ' (1 ? 1 ? 2)= (3)
A similar theoretical analysis can be applied to
noise can be improved signicantly by reducing 
from 1.0 to e.g. 0.99. On the other hand, the cor-
responding performance loss for high CSNR val-
ues is small. Furthermore, optimal values of  can
be estimated as follows. At CSNR values above
21 dB,  should be above 0.99, approaching 1.0
as the CSNR increases. As the CSNR is reduced
from 21 dB to 16 dB, opt decreases to about 0.90.
Note that opt does not approach zero as CSNR
decreases to 0. This can be seen from equation 3.
Typically, the temporal correlation coecient 
value of opt is in the order of 0.9 (equation 3). Al-
though equation 3 does not apply directly to the
problem at hand, it indicates an optimal value of
the proposed system, but some modications are  that is much closer to 1.0 than to 0.0 as CSNR
needed to take the subband decomposition and the approaches zero.
classication scheme into account. This will be a
topic for further research. In this paper the opti-
mal values of  were estimated experimentally.
4. EXPERIMENTS
All our experiments were performed by coding
400 luminance frames of the QCIF Foreman se-
quence, and by simulating transmission over an
additive white Gaussian noise (AWGN) channel
with maximum likelihood detection on the receiver
Finally, the proposed system was compared to
the reference system. The reference system was
simulated with NI = 3 resulting in a QAM symbol
rate of 0.092 spp. The proposed system was sim-
ulated at the same spp and with  = 0:99. The
equivalent bit rate in terms of bpp for the pro-
posed system is higher than for the reference sys-
tem since the power ecient index maps allow for
larger signal constellations (e.g. 81 QAM vs. 16
QAM). This is necessary in order to compensate
side. For the reference system, the simulations
were conducted by assuming error free transmis-
sion of the picture header, ensuring correct re-
synchronization of the bit pattern at the begin-
ning of each frame. Furthermore, the H.263 coder
was implemented without rate control, using the
average channel rate as a reference. All our ex-
periments were conducted by averaging over 20
separate simulations of the entire sequence using
dierent random noise sequences.
for the less ecient xed length coding strategy
of the proposed system. Note however, that since
the performances of the two systems are compared
for the same channel symbol rate and CSNR, the
equivalent bpp is irrelevant. The results are illus-
trated in Figure 5. The performances of the two
systems are quite close at CSNR values down to
about 20 dB. Below this point, the reference sys-
tem degrades abruptly, while the proposed system
degrades gracefully.
 35 35

30 30

PSNR (dB)
PSNR (dB)

25
25

20
20 12 14 16 18 20 22 24
12 14 16 18 20 22 24 CSNR (dB)
CSNR (dB)

Figure 3. PSNR vs. CSNR for various values of NI , Figure 5. Right: PSNR vs. CSNR, reference system,
NI = 0 (||), NI = 1 ({ { {), NI = 3 ({
{
{), NI = 3 (||), proposed system,  = 0:99 (- - -).
NI = 9 (




). 6. ACKNOWLEDGEMENT
The authors want to thank Gisle Bjntegaard at
35
Telenor Research for suggesting the error detection
and concealment techniques for the H.263 video
coder, making it a realistic and challenging refer-
ence.
30
REFERENCES
[1] N. Farvardin, \A study of vector quantization
PSNR (dB)

for noisy channels," IEEE Trans. Inform. The-

ory, vol. 36, pp. 799{809, July 1990.
25
[2] J. M. Lervik, A. Fuldseth, and T. A. Ram-
stad, \Combined image subband coding and
multilevel modulation for communication over
power- and bandwidth limited channels," in
Proc. Workshop on Visual Signal Processing
and Communications, (New Brunswick, NJ,
20
12 14 16 18 20 22 24
CSNR (dB)

Figure 4. PSNR vs. CSNR for various values of  USA), pp. 173{178, IEEE, Sept. 1994.
(From top to bottom at CSNR = 24 dB:  = 1.0,
0.99, 0.98, 0.95, 0.9,0.85, 0.80).
5. CONCLUSIONS
For the proposed video transmission system we
have demonstrated how the robustness to chan-
nel noise can be improved by using leaky inter-
frame prediction matched to the channel statistics.
For the cases where the channel statistics is not
known on the encoder side, the robustness can be
[3] A. Fuldseth and J. M. Lervik, \Combined
source and channel coding for channels with a
power constraint and multilevel signaling," in
Proc. ITG Conference, (Munchen, Germany),
pp. 429{436, ITG, Oct. 1994.
[4] J. M. Lervik, A. Grvlen, and T. A. Ram-
stad, \Robust digital signal compression and
modulation exploiting the advantages of ana-
log communication," in Proc. IEEE GLOBE-
COM, (Singapore), pp. 1044{1048, IEEE, Nov.
1995.
[5] A. Fuldseth and T. A. Ramstad, \Combined
increased signicantly at the cost of a small qual-
ity reduction for noiseless channels. Finally, the
proposed system compares favorably to a refer-
ence system based on the H.263 video coder due to
graceful degradation for poor channel conditions.
The inherent robustness to channel noise makes
the proposed system particularly useful for video
transmission over time varying wireless channels
such as in mobile and broadcasting applications.
The ideas behind the proposed system might also
be of interest for the functionalities of the new
MPEG4 video coding standard for applications re-
quiring error resilience.
video coding and multilevel modulation," in
Proc. Int. Conf. on Image Processing, (Lau-
sanne, Switzerland), Sept. 1996. to appear.
[6] Telecommunication Standardization Sector,
Study Group 15, Working Party 15/1, Expert's
Group on Very Low Bitrate Videophone, Video
Codec Test Model, TMN5, Jan. 1995. Source:
Telenor Research.
[7] K. Y. Chang and R. W. Donaldson, \Analysis,
optimization, and sensitivity study of dier-
ential PCM systems operating on noisy com-
munication channels," IEEE Trans. Commun.,
vol. COM{20, pp. 338{350, June 1972.