1

Robust Signal Compression using Joint

Fixed- and Variable-length Coding

A. Aydn Alatan and John W. Woods, Fellow, IEEE

Dr. A. A. Alatan is with the ECE Department and the New Jersey Center for Multimedia Research at NJIT, Newark,
NJ, 07102. Prof. J. W. Woods is with the ECSE Department and the Center for Image Processing Research at Rensselaer
Polytechnic Institute, Troy NY 12180-3590.
This work was partially supported by U.S. Army Research Oce under grant # DAAH04-96-1-0380

SUBMITTED TO IEEE INT. SYMP. ON INFORMATION THEORY 2000

Abstract
In our method, a transmitted signal is decomposed into two parts which are then encoded using xed- and
variable-length coding, respectively. After initial quantization of the signal by a xed-length coded Lloyd-Max
quantizer, the saturated oset or residue is then further quantized by a variable-length entropy-constrained scalar
quantizer(ECSQ). Compared to conventional variable-length codewords with synchronization-symbols or xedlength coding strategies, the proposed method enjoys a better distortion-rate performance on some particular
channels. The reason for this good performance is thought to be twofold: appropriate signal decomposition
leads to minimum distortion-rate ineciency for the xed-length coded part and this part leads to more graceful
degradation compared to variable-length-only coded sources. Such a decomposition is also attractive for joint
source and channel coding, since this approach lets any signal to be decomposed into two subsignals with dierent
error immunities.

Keywords
xed- and variable-length, xed-length codes, variable-length codes, synchronization, robust transmission,
joint source-channel coding, forward error correction
I. Introduction

Shannon's research [?] showed the asymptotic optimality of separation of source and channel coding
stages in data communications. However, considering nite data length, decoder complexity, and delay
constraints, solving image compression and transmission problems independent of each other can be
suboptimal.
The entropy of a source can be approached by assigning codewords with dierent lengths according to
their probabilities. Such variable-length coded data has proved to be very ecient in removing redundancy
within symbols. However, even a single bit error usually leads to catastrophe for the resulting fragile
bit-streams. On the other hand, xed-length coded data enjoys complete immunity to error propagation,
whereas the compression performance is usually either insucient for scalars or else, in the case of vectors,
too computationally demanding.
Here, we present a novel approach utilizing xed- and variable-length codewords jointly to their best
advantage. The aim is to minimize the error propagation problem due to variable-length coded data while
losing from compression eciency as small as possible.
II. Error resilience over noisy channels

Fixed-length coding of a scalar or vector quantizer indices does not let errors propagate between symbols,
hence it is more resilient to bit errors. Although xed-length coded bit-streams suer from diminished
compression eciency, a decent channel performance still can be obtained by designing a quantizer which
takes into account channel errors [?].
On the other hand, variable-length coded bit-streams can be made less susceptible to errors by applying
several known methods. In many coding and transmission standards, in order to stop error propagation,
synchronization words are added at the cost of some overhead on the total bit-rate [?]. Forward error

SUBMITTED TO IEEE INT. SYMP. ON INFORMATION THEORY 2000

correction codes can also be used in any noisy channel, but results deteriorate quickly when the channel
condition worsens. Conversion from variable- to xed-length codes is another known solution, but it is
possible to add or insert source symbols erroneously by this method and this situation can not be accepted
for images and video which consist of data in strict locations [?].
There are some hybrid methods which use both variable and xed-length codes. In one of these approaches, xed- or variable-length codes are selected according to their compression performance on each
block [?]. However, rather than better error resilience, their main motivation was only better compression
by selecting an appropriate codeword type.
Among all the methods mentioned above, the most widely accepted approach for error resilient coding
is to use synchronization words at the end of variable-length coded symbol groups, usually called a
'slice', consisting of xed number of symbols. This is the adopted approach in current image and video
transmission standards.
In [?], a comprehensive comparison between variable- and xed-length coding schemes in noisy channels
is presented. According to the authors, the variable-length coding has a clear edge over xed-length
for noiseless channels and channels with less noise, as expected. As soon as the noise level increases,
xed-length coding handles the errors better, leading to a superior performance over the variable-length
counterpart.
In the next section, a new method is proposed to improve the performance of the conventional approaches, like xed- or variable-length coding. It should be noted that the proposed approach is independent of the methods which are utilized within its own xed- and variable-length coding stages. In other
words, any state-of-the-art xed- or variable-length coding method can be inserted into corresponding
places of the algorithm to improve its performance.
III. Joint Fixed- and Variable-length Coding

A xed-length coding strategy is optimal for a uniformly distributed ( at) source probability density
function (pdf). For a zero-mean memoryless Gaussian source, it is possible to nd a somewhat at region
centered around the origin of the pdf. This atter region is comparatively larger for a Gaussian random
variable, and innitesimal for a Laplacian source. These uniform regions can be encoded using xed-length
codewords while slightly losing from compression eciency. In the mean time, it is still possible to encode
the tail of this distribution (outside this somewhat at region) by a variable-length coding scheme.
In our approach, the overload distortion of the xed-length coded quantizer is eliminated by a residual
quantizer whose output is entropy coded. A typical system is shown in Figure ??.
In this system, when a signal value, x, is found to have been saturated by the xed-length coded
quantizer, Qf ,

is subtracted from its xed-length quantized version, x^f and the switch between xed

and variable-length quantizers is closed. The dierence is then passed to the residual stage, where this
saturation oset, xs is quantized by variable-length quantizer, Qv , and the quantized residue x^v is encoded

SUBMITTED TO IEEE INT. SYMP. ON INFORMATION THEORY 2000

Qv
Variable-rate Quantizer
SAT.
REG.

SAT.
REG.

^
Xf

Xs
^
X
v

saturated

Fixed-rate Quantizer

Qf

^
Xf

C
H
A
N
N
E
L

+
~
Xv+

saturated

~
X
~
Xf

Fig. 1. A block diagram of the proposed method

using a variable-length code. After the transmission of x^f and x^v , the receiver checks whether a signal
value is saturated or not, using the received xed-length coded part,~xf , and if so, it will decode and add
the received quantized dierence, x~v , on top of x~f .
Every xed-length coded source symbol is expected to have strong immunity to error propagation as
well as some compression eciency, whereas the variable-length coded part handles the saturation eects.
During data transmission, if it is not possible to send xed and variable-length streams on dierent
channels, one possible solution is to multiplex these streams together, so that the xed-length coded
stream is sent rst, followed by the variable-length stream.
There are two important parameters for the system in Figure ??:

parameter

which gives the width

of the reserved non-saturating region of the signal pdf and Rfixed which denotes the bit rate reserved for
the xed-length quantizer, Qf , from the overall total rate,

Rtotal

. For a given channel and

Rtotal

value,

our design goal is to nd the optimal (S; Rfixed ) pair so that the distortion of the reconstructed signal
at the receiver is minimized. Since, in noisy channels, an analytical distortion analysis for variable-length
coded data is not possible, the "best" (S; Rfixed ) pair, rather than the optimal, is found by the using
operational rate-distortion characteristics obtained via simulations.
IV. Simulations
A. Description of the Simulated System

The simulation setup is shown in Figure ??. The xed-length quantizer is a derivative of Lloyd-Max
quantizer (LMQ) [?]. Since

is a design parameter, the outermost two input thresholds of the LMQ

is xed, while the rest of the thresholds are determined using the standard algorithm by the help of a
training data set. It should be noted that this quantizer has one more degree of freedom (S ) compared
to conventional LMQ. The LMQ indices are then encoded using a scalar pseudo-Gray code [?] for better

SUBMITTED TO IEEE INT. SYMP. ON INFORMATION THEORY 2000

noise performance.
The variable-length coded or saturated component is quantized using an ECSQ [?], whose output indices
are entropy coded using Human coding. It is assumed that the Human tables are available at both
transmitter and receiver sides. This variable-length bit-stream is protected from error propagation by
carving it up into slices and adding end-of-slice (EOS) markers. These markers prevent the propagation
of possible errors between slices. Bit-stung is also used to prevent occurance of dummy EOS markers
within the bit-stream. The slices contain a predetermined number of symbols, Ns , and if the reconstructed
symbol number is not equal to Ns , all the data in that slice is assumed to be lost and replaced with the
mean of the source signal.
In order to be fair, the compared variable- and xed-length methods are selected from the same simulated
system. One of the compared methods is a variable-length-only scheme whose symbols are encoded using
ECSQ followed by a Human coding. The resulting bit-stream is also divided into a number of slices with
EOS markers. The other compared method is a LMQ-based xed-length-only scheme with pseudo-Gray
coding.
The simulated channel is taken to be a simple binary symmetric channel (BSC) and simulations are
conducted for dierent bit error-rates (BER) over this channel.
B. Results

The performance of conventional and proposed methods are compared for dierent channel characteristics and a set of rates for

Rtotal

. For each case, the best (S; Rfixed ) pair is determined via simulation.

Dierent slice lengths are used to observe eects of error propagation versus EOS marker overhead.
Tables ??, ?? and ?? show the signal-to-noise ratio (SNR) of 10; 000 reconstructed Gaussian signals
after 100 channel iterations for three of the methods (xed-only, variable-only and xed + variable) for
slice lengths,

Ns

equal to 10, 100 and 1000, samples, respectively. The EOS marker length is selected as

16 bits. The LMQ and ECSQ are also trained using another set of 10; 000 symbols.
Experimentally, for the proposed approach, the "best"
Rtotal

Rfixed

values usually occurred at

Rfixed

1 b=p with the S region size increasing almost linearly with both Rtotal and channel BER. Hence,

for a practical system, given the channel BER and total bit-budget, the (S; Rfixed ) parameter pair of a
system can be empirically determined. Simulations also reveal that the best selection of

Ns

is usually

equal to 100 symbols.

As it can be observed from Figure ??, for the noise-free case, as expected, the variable-length-only
strategy gives the minimum distortion. However, as soon as the channel becomes noisy, xed-length
coding scheme degrades more gracefully, hence surpasses the rate-distortion performance of the variablelength coding. For the channel with

BE R

= 10 5 , which can be entitled as a \performance transition

channel" between xed- and variable-length schemes, joint xed- and variable-length coding clearly obtains
the best rate-distortion performance among all three methods.

SUBMITTED TO IEEE INT. SYMP. ON INFORMATION THEORY 2000

Gaussian memoryless source

Rtotal=3 b/p

Gaussian memoryless source

16

Rtotal=4 b/p

25

14
20
12

15
SNR (dB)

SNR (dB)

10

10

4
5

Fixedlengthonly
Variablelengthonly
Fixed and Variablelength

0
no error

Fixedlengthonly
Variablelengthonly
Fixed and Variablelength
4

0
no error

log10(BER)

(a)
Gaussian memoryless source

log10(BER)

(b)
Rtotal=5 b/p

Gaussian memoryless source

30

Rtotal=6 b/p

35

30

25

25

SNR (dB)

SNR (dB)

20

15

20

15
10
10

Fixedlengthonly
Variablelengthonly
Fixed and Variablelength

0
no error

0
no error

Fixedlengthonly
Variablelengthonly
Fixed and Variablelength
5

log10(BER)

log10(BER)

(c)

(d)

Fig. 2. Noisy channel performance of Gaussian memoryless source for xed-length-only, variable-length-only and joint xed
and variable-length coding for bit-rates (a) 3, (b) 4, (c) 5 and (d) 6 b/p.
V. Conclusion

A new approach to robust data compression is presented. Separating a signal value into two (saturating
and non-saturating) parts and encoding these parts appropriately, is shown to be eective compared to
the well-known variable- and xed-length only strategies at some channels. By appropriately changing the
(S; Rfixed ) parameters, one also can either obtain xed-length-only (Rfixed = Rtotal ) or variable-lengthonly (S = 0; Rfixed = 0) scenarios as limiting cases of the proposed system.
The good performance of this method is thought due to two main factors: we utilize xed-length codes
only when there is a small loss in eciency and xed-length coded part degrades gracefully in noisy
channels.

SUBMITTED TO IEEE INT. SYMP. ON INFORMATION THEORY 2000

TABLE I
For a slice length,

Ns = 10, SN R

values of the reconstructed symbols for a memoryless Gaussian source at

various

Rtotal

and channel

BER

values.

BER
SN R (dB )
S
3

10
F

F+V

13.8

7.8

4.5

2.0
13.4

Rtotal

18.0
4.5

3.0

[b=s]

20.4

18.0

5.0

4.5

21.3

20.8

5.0

5.0

10
V

F+V

14.5

8.4

3.5

2.5

0.3

19.9

15.8

4.5

2.5

0.4

24.5

21.9

5.0

3.5

0.4

28.2

26.6

6.0

4.0

0.1

10
V

F+V

14.5

9.9

3.5

2.5

0.3

20.2

18.5

5.0

2.5

0.8

25.2

23.6

5.0

3.0

1.1

30.3

31.1

6.0

3.0

0.3

Noise-free
V

F+V

14.5

10.2

5.7

3.5

2.5

6.8

20.2

17.7

4.5

3.0

13.1

25.2

25.4

5.0

2.5

15.1

30.4

31.2

6.0

3.0

3.6

10.6
18.6
25.1

The most attractive property of this approach is dividing any source into subsources with dierent error
immunities. Hence, unequal channel error protection of these subsources will be possible by appropriate
protection.
References

[1] C. E. Shannon, \A Mathematical Theory of Communication", Bell System Technical Journal, vol. 27, pp. 379{423
623{656, July-Oct 1948.
[2] N. Farvardin and V. Vaishampayan, \Optimal Quantizer Design for Noisy Channels : An Approach to Combined
Soruce-Channel Coding", IEEE Trans. on Information Theory, vol. 33, no. 6, pp. 827{838, November 1987.
[3] M. Y. Cheung and J. Vaisey, \A Comparison of Scalar Quantization Strategies for Noisy Channel Data Transmission",
IEEE Trans. on Communications, vol. 43, no. 2/3/4, pp. 738{742, Feb/Mar/Apr 1995.
[4] T. J. Ferguson and J. H. Rabinowitz, \Self-synchronizing Human Codes", IEEE Trans. on Information Theory, vol.
30, no. 4, pp. 687{693, July 1984.
[5] P. Nasiopoulos and R. K. Ward, \A Hybrid Coding Method for Digital HDTV", IEEE Trans. on Consumer Electronics,
vol. 41, no. 4, pp. 1080{1087, 1995.
[6] A. Gersho and R. M. Gray, Vector Quantization and Signal Compression, Kluwer Academic Press, 1992.
[7] K. Zeger and A. Gersho, \Pseudo-Gray Coding", IEEE Trans. on Communications, vol. 38, no. 12, pp. 2147{2158,
December 1990.
[8] P. A. Chou, T. Lookabaugh and R. M. Gray, \Entropy Constrained Vector Quantization", IEEE Trans. on Acoustics,
Speech and Signal Processing, vol. 37, no. 1, pp. 31{42, Jan 1989.

SUBMITTED TO IEEE INT. SYMP. ON INFORMATION THEORY 2000

TABLE II
For a slice length,

Ns = 100, SN R

values of the reconstructed symbols for a memoryless Gaussian source at

various

Rtotal

and channel

BER

values.

BER
SN R (dB )
S

10
F

F+V

13.8

9.1

4.5

2.0

Rtotal

18.0

15.1

4.5

3.0

[b=s]

20.4

19.1

5.0

3.5

21.3

21.1

5.0

3.5

10
V

F+V

0.3

14.5

12.5

3.5

1.5

0.2

19.9

20.2

4.5

2.0

0.3

24.5

23.8

5.0

3.0

0.5

28.2

26.7

6.0

4.0

10
V

F+V

7.5

14.5

12.7

3.5

1.5

9.8

20.2

21.2

5.0

2.0

15.1

25.2

26.5

5.0

2.5

18.1

30.4

32.3

6.0

2.5

Noise-free
V

F+V

15.1

14.5

13.3

15.5

3.5

1.5

21.0

20.2

21.2

4.5

2.0

25.1

25.3

26.6

5.0

2.5

27.3

30.4

32.6

6.0

2.5

23.3
27.8
33.5

TABLE III
For a slice length,

Ns = 1000, SN R

values of the reconstructed symbols for a memoryless Gaussian source

at various

Rtotal

and channel

BER

values.

BER
SN R (dB )
S
3

10
F

F+V

13.8

8.9

4.5

2.0
14.3

Rtotal

18.0
4.5

3.5

[b=s]

20.4

18.4

5.0

4.5

21.3

20.7

5.0

5.0

10
V

F+V

14.5

12.5

3.5

1.5

0.8

19.9

18.0

4.5

1.5

0.5

24.5

22.6

5.0

3.0

0.8

28.2

26.6

6.0

3.5

0.2

10
V

F+V

14.5

14.4

3.5

1.5

7.8

20.2

20.8

5.0

2.0

10.1

25.2

26.4

5.0

2.5

12.5

30.3

30.4

6.0

3.0

9.0

Noise-free
V

F+V

14.5

14.6

16.0

3.5

1.5

19.1

20.2

21.1

4.5

2.0

23.3

25.3

26.4

5.0

2.5

27.2

30.4

32.7

6.0

2.5

14.8

23.3
27.8
33.5