Packet Loss and Compression Effects on Vocal Recognition
Pedro Mayorga O.1,2, Laurent Besacier3, Ana M. Hernández C.1,2
1
Instituto Tecnológico de Mexicali, 2Facultad de Ingeniería de la UABC, 3GEOD Team-CLIPS
Laboratory, UMR CNRS 5524, INPG-UJF
1,2
Mexicali B.C., México, 3Grenoble Cedex 9, France.
pedromayorga@hotmail.com, ahmxli@yahoo.com
Abstract
This paper deals with the effects of packet loss
and voice data compression on vocal recognition over
IP connections. One contribution of this paper
consists in diagnosing most precisely the problems due
to the compression and the packet losses for two
different recognition tasks: the automatic speech
recognition and speaker recognition. Another
contribution corresponds to the proposal for
recovering techniques in order to improve the
robustness of systems under significant packet losses
conditions. From the diagnosis, a more important
degradation due to the compression on the speaker
verification task was noted. The experimental results
show that the techniques of interleaving based on the
transmitter combined with the interpolation based on
the receiver, prove to be the most efficient.
Keywords –– Packet loss, robustness, VoIP.
1. Introduction
Now is possible access a distant vocal recognition
server in different ways, from mobile phones or from
other communication terminals. However, there are
several constraints with remote vocal recognition:
firstly, the lost packets and secondly, the degradation
of transmission due to the speech compression after
applying speech coders. Today the most popular
speech coders in voice transmission over IP (VoIP)
are: G.723.1, G.729, G.728, G.726/7 and G.711. This
set of coders is also used in video transmission and is
known as the standard H323. There are several
software packages for videoconferencing which can
also be used for voice transmission on the Internet, for
example Microsoft’s NetMeetingTM [1][2].
TM
Microsoft’s NetMeeting uses the H323 standard.
In interactive speech applications over the
Internet, the UDP transport protocol is used to carry
speech or audio signal packets, nevertheless a data
Proceedings of the Electronics, Robotics and Automotive Mechanics Conference (CERMA'06)
0-7695-2569-5/06 $20.00 © 2006
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on February 20, 2009 at 05:43 from IEEE Xplore. Restrictions apply.
packet can be lost and UDP does not have any error
recovery mechanism. In the literature, it has been
shown that packet loss reduces speech recognition
performance more dramatically than speech
compression [3][4][5]. On the other hand, for another
task like speaker verification, the effect of packet loss
is no very important, while noise and speech
compression effects are more crucial [6][7].
Different strategies can be proposed to access a
distant vocal recognition server [8][9]. In recent times,
one solution was proposed where speech spectral
analysis (the extraction of acoustic vectors) is
performed on the client terminal in order to transmit
the acoustic features to the distant vocal recognition
server. This approach allows avoiding speech
compression degradation, but does not to solve the
problem of packet losses over Internet.
2. Methodology
1) Databases used: We conducted a series of speech
recognition experiments with 120-recorded sentences
focused on reservation and tourist information task
(CSTAR120 database) used in CSTAR project [10].
In speaker verification task the XM2VTS [11]
database was used. In acquiring the XM2VTS
database, 295 volunteers from the University of Surrey
visited a recording studio four times at approximately
one month intervals. On each visit (session) two
recordings (shots) were made. The first shot consisted
of speech while the second consisted of rotating head
movements. This multimodal database is used by
many partners of COST Action 275 [12]. The work
described in this paper was made on its speech part
were the volunteers were asked to read three sentences
twice. The three sentences remained the same
throughout all four recording sessions and a total of
7080 speech files were made available on four CD-
ROMS. The audio, which had originally been stored in
mono, 16 bits, 32 kHz, PCM wave files, was
 downsampled to 8kHz. This is input sampling
frequency required in the speech codecs for VoIP.
2) Codecs used: H323 is a standard commonly used
for transmitting voice and video over IP networks. The
audio codecs used in this standard are G711, G722,
G723, G728 and G729. We conducted our
experiments with the codec which has the lowest bit
rate: G723.1 (6.4 and 5.3 kbit/s), and the one with the
highest bit rate: G711 (64 kbit/s).
3) In speech recognition experiments the French
speech recognition system RAPHAEL was used.
RAPHAEL uses JANUS-III toolkit from CMU [13].
The context dependent acoustic model (750 CD
codebooks, 16 gaussians each) is learned on a corpus,
which contains 12 hours of continuous speech of 72
speakers extracted from Bref80 database [14]. The
system uses 24-dimensional LDA features obtained
from 43-dimensional acoustic vectors (13MFCC,
13¨MFCC, 13¨¨MFCC, E, ¨E, ¨¨E, zero-crossing
parameter) and extracted every 10ms. The vocabulary
(5k words) contains nearly 5500 phonetic variants of
2900 distinct words; it is specific to the tourist
reservation and information domain. The trigram
language model that we used for our experimentation
was computed using Internet documents because it has
been shown that they give a very large amount of
training data for spoken language modeling [15].
Speaker verification experiments with the ELISA
system. The ELISA consortium groups several
laboratories working on speaker verification. One of
the main objectives of the consortium is to emphasize
assessment of performance. Particularly, the
consortium has developed a common speaker
verification system which has been used for
participants at various NIST speaker verification
evaluations campaigns [16].
The ELISA system is a complete frame work
designed for speaker verification. It is a GMM-based
system including audio parameterization as well as
score normalization techniques for speaker
verification.
For the purpose of our experiments, the Lausanne
protocol (configuration 2) is adopted. This has already
been defined for the XM2VTS database. There are 199
clients in the XM2VTS DB. The training of the client
models is carried out using full session 1 and full
session 2 of the clients part of XM2VTS [17]. 398
client test accesses are obtained using the full session
4(x2 shots) of the clients part. 111440 impostor
accesses are obtained using the impostor part of the
database (70 impostor x4 sessions x2 shots x199
clients = 111440 impostor accesses). The 25
evaluation impostors of XM2VTS are used to develop
a world model. The text-independent speaker
Proceedings of the Electronics, Robotics and Automotive Mechanics Conference (CERMA'06)
0-7695-2569-5/06 $20.00 © 2006
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on February 20, 2009 at 05:43 from IEEE Xplore. Restrictions apply.
verification experiments are conducted in matched
conditions (same training/test conditions).
The ELISA system on XM2VTS is based on the
LIA system presented to the NIST 2002 speaker
recognition evaluation. The speaker verification
system uses 32 parameters (16 LFCC + 16 ¨LFCC).
Silence frame removal is applied before centering
(CMS) and reducing vectors.
For the world model, 128 Gaussian-component
GMM was trained using Switchboard II phase II data
(8 kHz landline telephone) and then adapted (MAP
[18], mean only) on XM2VTS data (25 evaluation
impostor set). The client models are 128 Gaussian-
component GMM developed by adapting (MAP, mean
only) the previous world model. Decision logic is
based on using the conventional log-likelihood ratio
(LLR).
4) Packet loss and compression effects over IP
networks. This section proposes a methodology for
evaluating the voice packets over IP networks. The
idea is to duplicate an existing database (XM2VTS or
CSTAR120) used for vocal recognition by passing its
speech signals through different coders and different
network conditions representative of what can occur
over the Internet.
Packet loss simulation and speech compression:
There are two main transport protocols used on IP
networks. These are UDP and TCP. While UDP
protocol does not allow any recovery of transmission
errors, TCP include some error recovery processes.
However, the transmission of speech via TCP
connections is not very realistic. This is due to the
requirement for real-time (or near real-time)
operations in most speech related applications [19]. As
a result, the choice is limited to the use of UDP which
involves packet loss problem. The process of audio
packet loss can be simply characterized using a Gilbert
model [20] consisting of two states (Fig. 1).
p
1-p 0 1 1-q
q
Figure 1. Gilbert model.
One of the states (state 1) represents a packet loss
and the other state (state 0) represents the case where
packets are correctly transmitted. The transition
probabilities in this statistical model, as shown in Fig.
 1, are represented by p and q. In other words, p is the
probability of going from state 0 to state 1, and q is the
probability of going from state 1 to state 0.
Different values of p and q define different packet
loss conditions that can occur on the Internet. The
probability that n consecutive packets are lost is given
by p(1-q)n-1. If (1-q)>p, then the probability of losing a
packet is greater after having already lost a packet than
after having successfully received a packet. This is
generally the case in data transmission on the Internet
where packet losses occur as bursts. Note that p + q is
not necessarily equal to 1. Different values of p and q
representing different network conditions considered
in our experiments were used.
Two degraded versions of database (CSTAR120
in speech recognition case and XM2VTS in speaker
verification case) were obtained by applying G711 and
G723.1 codecs alone, without any packet loss.
Six degraded versions of XM2VTS were obtained
using simulated packet loss conditions: two conditions
(average/bad) and three speech qualities
(clean/G711/G723.1). The simulated average and bad
networks conditions considered in this study
correspond to 9% and 30% speech packet losses,
respectively. Each packet contained 30ms of speech,
which was consistent with the duration proposed in
RTP (real-time protocol used under H323.).
The CSTAR120 database (in speech recognition
case) was duplicated giving several versions,
according to the Gilbert model with 5 different
conditions (Table I).
Table I.
Different packet loss conditions
condition 1 2 3 4
p 0.10 0.05 0.07 0.20
q 0.14 0.06 0.10 0.4
Real-conditions packet loss. In order to
investigate the effects of real network conditions, it
was decided to play and record the whole database
(CSTAR120 in speech recognition case and XM2VTS
in speaker verification case) through the network. This
was carried out by playing the speech dataset into a
computer which was set up for videoconferencing. For
this purpose, a transatlantic connection was
established between France and Mexico using
videoconferencing software. The microphone on the
French site was then replaced with the audio output of
a computer playing the speech signals database. Due
to numerous networks breakdowns, the transmission
of speech signals had to be conducted using several
different connections established on different days and
different times. This, of course, provided variations in
Proceedings of the Electronics, Robotics and Automotive Mechanics Conference (CERMA'06)
0-7695-2569-5/06 $20.00 © 2006
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on February 20, 2009 at 05:43 from IEEE Xplore. Restrictions apply.
network conditions that occur in the case of real
applications.
5) Reconstruction strategies. Against packet loss, the
typical mechanisms belong to two classes. Firstly, the
mechanisms of Automatic Repeat request (ARQ)
based on the retransmission of packet that did not
arrive at its destination. Unfortunately, these
mechanisms can not be used for the audio
transmission in real-time [3][8]. Secondly, the
Forward Error Correction mechanisms (FEC) based on
the transmission of redundant information [3][8], like
XOR technique. Another approach in audio
transmission is the technique of interleaving. This
mechanism is based on the redistribution of samples,
of such a way that if a packet is lost, the data can be
reconstructed by a repair mechanism [3], for example
the interpolation calculus, the Lagrange interpolation,
the repetition technique, or the XOR technique.
Interleaving. The technique of interleaving
distributes the effect of the lost packets in order to
reduce the bursting effects over IP connections; that
means that the information of a speech part is
distributed in the other packets. The original data units
are not combined in the original sequential order as
produced by the coder, but interleaved by the
transmitter as shown in Fig. 2. The data units are
regrouped in crossed form before their transmission, in
such a way that the units are redistributed and
separated to supply a distance between the lost ones.
In the receiver, they are rearranged into their original
sequence. As we can see, the lost of a packet results in
the lost of several speech units distributed in the other
packets.
Repetition. The repetition technique consists in
5 placing the lost packets by copies of the last received
0.25 packet. Here we have a tradeoff between the low
0.62 complexity and a reasonably good performance [8].
We can observe the repetition process in Fig. 3.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
1 5 9 13 2 6 10 14 3 7 11 15 4 8 12 16
1 5 9 13 Lost 3 7 11 15 4 8 12 16
1 3 4 5 7 8 9 11 12 13 15 16
Figure 2. Interleaving technique
 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 conditions: 31.8 % for G723, 29.8 % for G711 and
30.5 % for PCM, which means that the same quantity
of signal is lost in average. But, as we can see in table
1 2 3 4 Lost 9 10 11 12 13 14 15 16
II, the highest WER (Word Error Rate) is for G723,
with an average of 81.8 %, then 62.9 % for G711, and
1 2 3 4 1 2 3 4 9 10 11 12 13 14 15 16 53.5 % for PCM. Thus, for a same packet loss rate, the
highest compression level is, the higher the value of
Figure 3. Repetition technique WER will be. This difference may be due to the effect
of the compression itself, but also to the fact that in the
case of real transmissions with G723 (the highest
Packet I-1 Packet I Packet I+1 compression degree), one packet lost represents a
bigger quantity of consecutive voice information lost
than when G711 codec or no codec (PCM) are used.
Then, the lost information occurs dramatically as
broad bursts for G723 whereas it is more spread for
Lost G711 and PCM data transmitted. The correlation
between WER and PLR was also measured and the
results show that the real conditions do not really lead
to the same ideal and predictable results obtained in
Paket I=((Packet I-1)+(Packet I+1))/2 simulated conditions (in which a value of 0.98 was
obtained) since the correlation between WER and PLR
Figure 4. Interpolation technique is less important (0.64 instead of 0.98) and tends to
decrease with additional factors like speech
Simple Interpolation. Simple interpolation or compression.
calculation of the average consists in interpolating by
using the packets after and before the lost packet, as is B. Reconstruction performance
shown in Fig. 4. The advantage of this technique is its
simplicity; nevertheless its efficiency decreases as the When we recovering techniques, it is easier
number of lost packets increases. recover isolated losses than to recover losses of long
duration. The interleaving technique has the advantage
3. Results to cut the long duration losses (bursts losses) in
smaller pieces (small time-disjoint intervals). Another
A. Degradation with real transmissions advantage is that interleaving technique does not
increase the network load. Unfortunately, if a great
This experiment was performed to shown the amount of consecutive packets are used in
influence of the degradation due to real transmissions interleaving, the introduced delay might be
over IP, different audio bitstreams were used: G723, prohibitive.
G711 and PCM (no codec). The speech recognition Combined with interleaving technique, we can
performance was assessed and we show a summary of use other techniques to calculate the characteristics of
results in table II. the lost voice units. Interpolation and repetition have
as their main advantage their simplicity and low delay.
Table II. When near neighbor units are used to calculate lost
Results of WER and packet loss rate (PLR) in real units they give a very good performance. With these
VoIP conditions techniques we gain about 77 % of recognition
performance.
Audio bitstream G723 G711 PCM
Mean PLR 31.8 29.8 30.5 C. Speaker verification results
Mean WER 81.8 62.9 53.5 Table III.
Correlation coef. Results in EER % of speaker verification experiments
0.28 0.49 0.64
(WER, PLR)

In real VoIP conditions, there is the addition of Network condition Clean G711 G723
three problems: noise due to our experimental No packet loss 0.25 % 0.25 % 2.68 %
transmission protocol (not quantified here), Average condition
0.25% 0.25 % 6.28 %
degradation due to speech compression, and packet p=0.1, q=0.7
Bad network condition
losses. Comparable PLR’s were found for the three p=0.25, q=0.4
0.50 % 0.75 % 9%

Proceedings of the Electronics, Robotics and Automotive Mechanics Conference (CERMA'06)

0-7695-2569-5/06 $20.00 © 2006

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on February 20, 2009 at 05:43 from IEEE Xplore. Restrictions apply.
 Comparing these results with those for speech [13] T. Zeppenfeld, M. Finke, K. Ries, M. Westphal, A.
recognition, it can be said that the speaker verification Waibel, Recognition of conversational telephone
performance is far less sensitive to packet loss. On the speech using the Janus speech engine, IEEE
other hand, the lat column of table III shows that the International Conference on Acoustics, Speech and
Signal Processing, Munich, Germany, 1997.
speaker verification performance is adversely affected
[14] L.F. Lamel, J. L. Gauvain, M. Eskenazi, BREF, a
by the compression effects. large Vocabulary Spoken Corpus for French,
Eurospeech, Genes, Italy, Vol 2, pp. 505-508, 24-26
4. Conclusion September 1991.
[15] D. Vaufreydaz, M. Akbar, M. Rouillard, J. Caelen,
Internet documents: a rich source for spoken
In our experiments, speaker verification
language modeling, ASRU’99, Keystone Colorado
performance is more sensitive to compression effects (USA), pp. 277-280.
than packet loss; on the other hand speech recognition [16] I. Magrin-Changnolleau, G. Gravier, and Blouet for
is more sensitive to packet loss effects. Recovering the Elisa consortium, Overview of the 2000-2001
strategies show to be performance using interleaving ELISA consortium research activities, in 2001: A
to distribute the lost speech units, and reconstruction Speaker Odyssey-The Speaker Recognition
techniques like interpolation o repetition. Workshop, pp. 67-72, Crete, Greece, June 2001.
[17] Luttin J. and Maitre G., Evaluation Protocol for the
XM2VTSDB Database (Lausanne Protocol),
5. References Technical Report 05, IDIAP Communication,
Martigny-Valais-Suisse, October 1998.
[1] F. Metze, J. McDonough and H. Soltau, Speech [18] Gauvain, J.L. and Lee C.-H., Maximum a posteriori
Recognition over NetMeeting Connection, Eurospeech, estimation for multivariate Gaussian mixture
September 2001, Scandinavia. observations of Markov chains, IEEE Trans. Speech
[2] L. Besacier, C. Bergamini, D. Vaufreydaz, E. Castelli, Audio Process., April 1994, 2, pp. 291-298.
The effect of speech and audio compression on speech [19] Black U.: Voice over IP, Prentice Hall, 2001.
recognition performance, in IEEE Multimedia signal [20] Bolot J-C. and Fosse-Parisis S., Adaptive FEC-based
Processing Workshop, Cannes, France, October 2001. error control for Internet telephony, Proc. IEEE
[3] P. Mayorga-Ortiz, R. Lamy, L. Besacier, Recovering of Infocom’99, New York, NY, USA, March 1999.
pscket loss for distributed speech recognition, Proc.
EUSIPCO 2002, Toulouse, France, Sep. 2002.
[4] B. Milner and S. Semnani, Robust Speech Recognition
Over IP Networks, ICASSP2000, Istambul (Turkey).
[5] N. W. D. Evans, J. S. Mason, R. Aukenthaler and R.
Staper, Assessment of speaker verification degradation
due to packet loss in the context of wireless mobile
devices, COST275 Workshop on Biometrics over the
Internet, Rome, Italy, November 7-8, 2002.
[6] L. Besacier, P. Mayorga, JF. Bonastre, C. Fredouille,
Methodology for Evaluating Speaker Verification
Robustness over IP Networks, COST275 Workshop on
Biometrics over the Internet, Rome, Italy, November 7-
8, 2002.
[7] Besacier L., Mayorga P., Bonastre J-F., Fredouille C.,
Meignier S., Overview of compression and packet loss
effects in speech biometrics, 2003 IEE Proceedings
Vision, Image & Signal Processing-Special issue on
Biometrics on the Internet. Vol. 150, no6, December
2003.
[8] C. Perkins, O. Hodson, V. Hardaman, A survey of
Packet loss Recovery Techniques for streaming audio,
IEEE Network Magazine, pp. 40-48, Sept./Oct. 1998.
[9] M. Yajnik, S. Moon, J Kurose and D. Towsley,
Measurement and Modeling of temporal Dependence in
Packet Loss, IEEE Infocom’99, New York, USA,
March 1999.
[10] http://www.c-star.org.
[11] http://www.ee.surrey.ac.uk/Research/VSSP/xm2vtsdb.
[12] http://cost.cordis.lu.

Proceedings of the Electronics, Robotics and Automotive Mechanics Conference (CERMA'06)

0-7695-2569-5/06 $20.00 © 2006

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on February 20, 2009 at 05:43 from IEEE Xplore. Restrictions apply.