Using the Lyapunov Exponent from Cepstral
Coefficients for Automatic Emotion Recognition
Marius Dan Zbancioc1,2
Institute of Computer Science, Romanian Academy
Technical University “Gheorghe Asachi” of Iasi
Iaşi, România
zmarius@etti.tuiasi.ro
Abstract— The main goal of this paper is to establish the
relevance of nonlinear parameters (Lyapunov exponents) in the
automatic classification of emotions, for the Romanian language.
The Largest Lyapunov Exponent - LLE was computed for the
MFCC mel frequency cepstral coefficients and the LPCC linear
prediction cepstral coefficients. The Support Vector Machine -
SVM classifier provides better results than Weighted K-Nearest
Neighbors - WKNN classifier in emotion recognition for feature
vectors that contains LLE (around 75%). The best recognized by
using SVM classifier was the neutral tone, followed by the
sadness, fury and the weakest recognized was the joy. For
features vectors which include LLE the best results was obtained
in combination with LAR - Log Area Ratio coefficients,
respectively PARCOR - partial correlation coefficients.
Keywords - Largest Lyapunov exponent, cepstral coefficients,
automatic emotion recognition
I. INTRODUCTION
Recently, the automatic emotion recognition plays an
important role in the research area. The emotion recognition
from speech has many applications in various areas such as: in
the speaker recognition, in improving the synthesizers, in
learning of a new language, in health care, etc. In the analysis
of speech emotion recognition one of the important stage is
how to extract and select better the speech features. The
common features extracted are: the pitch, the formants, the
energy, and some spectrum features for example Mel-
Frequency Cepstrum Coefficients (MFCC), Linear Prediction
Cepstrum Coefficients (LPCC), Log Frequency Power
Coefficients (LFPC), Perceptual Linear Predictive (PLP), Mel-
energy spectrum dynamic coefficients (MEDC).
The researchers [1] used in the emotion recognition as the
classifier the SVM and as the vector features: the pitch, the
energy and the coefficients MFCC, LPCC, MEDC. They tried
to classify three emotional states: happy, sadness and neutral
tone and the emotional databases used were Berlin Database of
Emotional Speech and their self-built Chinese emotional
databases. The best accuracy rate (91.3% for the Chinese
database and 95.1% for the Berlin database) was obtained by
combining MFCC+MEDC+ Energy.
In [2], it was used as the important speech features MFCC,
LPC - Linear Prediction Coefficients, LPCC. The results
obtained with the SVM classifier was 84.2% and with NN-
Monica Feraru1
Institute of Computer Science, Romanian Academy
Iaşi, România
monica.feraru@iit.academiaromana-is.ro
neural network 80.8%. The best emotion recognized applying
SVM was anger and was confuses the happiness with the
sadness state and boredom with neutral tone. Appling NN was
obtained better performance for the sadness and neutral tone,
and “perfectly” distinguishes between happiness and boredom.
In order to classify the emotional state, in [3] was used the
SVM classifier on the features extracted from the IITKGP-
SESC. The extracted parameters were the prosodic features –
the energy, the pitch, and the formants and the spectral features
- MFCC and LPCC. They concluded that using both (prosodic
and spectral) features, the recognition rate is better than only
one. The recognition rate of the combinations made on German
model was: energy+pitch (33.33%), MFCC+MEDC (86.66%),
MFCC+MEDC+LPCC (86.66%), MFCC+MEDC+Energy+
pitch (90%), MFCC+MEDC+Energy (91.30).
In [4] the researchers said that MFCC and LPCC “are used
as the correlates of vocal tract information” in the emotion
recognition process. They used the simulated emotional speech
corpus (IITKGP-SESC) and the Berlin emotional speech
database (Emo-DB). As the classifier they applied the GMM -
Gaussian Mixture Models and AANNs - Auto-Associative
Neural Networks. The results obtained show that the using of
LPCC is better (69%) than the results of the MFCC. The happy
state is recognized ”well by most of the features”. They
concluded that this can be explained because the LPCC
“represent the speech production characteristics, by analyzing
all frequency components in a uniform manner”. The emotion
recognition was better by using GMM than AANNs.
Regarding to the classification of seven emotions from the
emotional Berlin database, the researchers [5] used SVM and
the accuracy obtained was of 68%. In [6] they concluded that
the combination of two or more features offer a good accuracy
to the results. Using as features MFCC [7] and KNN as the
classifier, the accuracy rate obtained was 67%. Using MFFC
and SVM the accuracy rate was of 65% in [8].
In order to classify five emotions (anger, happiness,
sadness, surprise and neutral state) using SVM and as the
parameters: MFCC, the periodicity histogram and the
fluctuation pattern, in [9] for the Danish Emotion Speech
(DES) database it was obtained the recognition rate between
55.40% and 68%. In paper [10] the accuracy rate archived
using HMM, in order to recognize five emotions were 88.7%
for the emotion of Mandarin database. They selected six III. FEATURE VECTORS EXTRACTION
parameters (LPCC, MFCC, LFPC, jitter, PLP and LPC) and in Because the emotion recognition is made on the feature
order to obtain the best recognition of the emotion they used vectors which contain the prosodic information as fundamental
single and different combination of these parameters. Using frequency F0 and the formants F1-F4, the extraction of the
only MFCC and LPCC the accuracy was of 68.21% and other parameters was performed only on the vowel phonemes
respectively 68.68%; by combining these two parameters they (vowels and voiced consonants depending of the context). In
obtained 68.97% recognition; by combining all six parameters the following when we refer to consonants only means to
the accuracy was 83.91%. unvoiced/voiceless areas.
The paper is structured as follows. In the next section, some We mention this because in the emotion recognition are
brief information regarding the emotional corpus SROL are important the amplitude of sounds in voice, the accent, and the
presented and in section 3, the features vector extraction inflections of the voice. The pitch is present only in the vowels
algorithm are discussed. The analysis of the results obtained areas and not in the consonants areas. It were considered from
and the concluding remarks are presented in the last section. the beginning as interest areas in speech recognition only those
which contain F0. If it would have done the speaker
II. SROL DATABASE – EMOTIONAL CORPUS recognition then it will be necessary also the information from
The SRoL project (Sound of Romanian Language) is a the consonant (unvoiced) areas.
collection of sound files and of instruments for speech Recently, we proposed a robust method for the C/V
processing. It can be freely accessed at the webpage: segmentation based on the predictive neural networks
www.etc.tuiasi.ro/sibm/romanian_spoken_language/index.html described extensively in [12], [15]. After the method
. The collection contains utterance from normal people which optimization it was significantly reduced the running times so
are not professional voices (like actors for example). For the that the segmentation can be made in real time.
emotional corpus the speakers pronounced seven sentences
with four emotional states: sadness, joy, fury and neutral tone. The functional blocks of the application for the feature
vectors extracting that will sets the testing and the training test
Because this emotional database do not contain of the classifiers are described in the following figure:
professional voices it was necessary to validate the expressed
emotions for each file. After validation from a total of 396 files Preprocessing
(each file contains many pronunciations of some sentences) it s(t)
Vowel / sw(t) Pre- sp(t) Hamming sˆw (t )
consonant Framing
was kept 145 sound files with 545 sentences. From every emphasis windowing
segmentation
sentence it was extracted a feature vector with a number of 208
parameters. Only 203 parameters are useful in classification,
the other 5 helps to identify the sound file and the speaker. ΔMFCC
mfcc Logarithm mel Mel filter FFT
ΔΔMFCC
cepstrum + DCT spectrum bank
The recorded sentences from the SRoL corpus are: S1. extraction
”You came back to me / Ai venit iar la mine”, S2. “Yesterday Feature MFCC
evening / Aseară”, S3. “Who done that? / Cine a făcut asta?”, vectors Mel Energies
S4. “You will get the desired place / Îţi vei câştiga locul dorit”, AC 14th AC
S5. “My man done it / Omul meu îl lucră”, S6. “Anyway, you PARCOR
LAR Log area PARCOR Levinson
can get the desired place / Oricum îţi poţi câştiga locul dorit”, LPCC ratio -Dublin
Auto-
S7. “Mother is coming / Vine mama”. LPCC LPC coefficient method correlation
conversion LPC
LPC
TABLE I. THE FEATURE VECTORS
Figure 1. Cepstral coefficients extraction block diagram
Index Parameter
1, 2 sentence name, number of appearances The preprocessing is made in four stages:
3-5 mean, standard deviation, median of F0
P1. The C/V segmentation – It was extracted the voiced
6-8, 9-11, 12-14, mean, standard deviation, median of first formants
areas where the signal is quasi-periodic and it was eliminated
15-17 F1 (6-8), F2, ... , F4(15-17)
18,19,20 jitter extracted with 2 methods, shimmer the consonants areas, where we have a noise type signal and
21-33, 33-44 mean of 12 MFCC mel frequency cepstal coef.
where the spectral energy is concentrated in the high
frequencies area. The segmentation method based on the neural
45-56, 57-68 derivative parameter ΔMFCC, ΔΔMFCC
networks predicts / approximate better the signal in the periodic
69-89 mean of 21 LPCC linear predictive cepstral coef.
areas of the vowels than in consonants areas, where the
90-104, 105 LPC linear predictive coef., prediction error (105)
prediction error is significantly higher. The number of samples
106-119, PARCOR partial correlation coef.,
120-133,134-148 LAR – log area ratio and AC autocorellation coef. in each frame is N=564, for a sample frequency f s =22.05kHz
149-160, 161-181 Lyapunov from MFCC, respectively from LPCC
182-193 Energy from 12 mel frequency bands P2. Framing / windowing – from the vowel areas it was
194-205 Lyapunov from energy extracted window analysis s w (t ) / frames of length 25.6ms.
206,207,208 ID of recording, sentence and speaker The overlapping of two consecutive frames is 60%, which
means the windows are located at 10 ms distance each to other.
 P3. Pre-emphasis – it was applied a filter which increase
the amplitude of high frequencies. This filter is useful in our
application because in the vowel regions the spectral energy is
predominantly in low frequency bands [0-2000]Hz. The filter
coefficient α has the value 0,7.
s p [k ] = s w [k ] − α ⋅ s w [k ] , k = 1, N (1)
P4. Hamming windowing – by multiplying the s p window
with this tapering function the samples located close to the
frame borders are reduced in amplitude, compared with the
samples situated in the center of the frame, which receives
higher weights. The rectangular window is not recommended
to be used because introduces false information in the
spectrum. The hamming window moderates the undesirable
effects on DFT (Discrete Fourier Transform) produced by the
discontinuities from the borders of analysis window. The
values of parameters a and b are: a=0.54, b=1-a;
⎛ k −1 ⎞
hw [k ] = a + b ⋅ cos⎜ 2π ⎟ , k = 1, N (2)
⎝ N −1 ⎠
sˆw [k ] = s p [k ] ⋅ hw [k ] (3)
After the preprocessing stages from the windows sˆw (t ) it
can be extracted the cepstral coefficients: LPCC and MFCC.
For the extraction of the mel-frequency cepstral coefficients is
necessary to make the transition to the frequency domain. The
processing steps are noted by M1-M4 and are described below:
M1. FFT – computes the spectrum S(f) for each frame.
This spectrum is the convolution of the input signal and of the
hamming window. The frequencies are linear distributed in the
interval [0, fs/2] in N/2+1 points.
M2. Mel-frequency bank filtering – The frequency scale is
mapped into a logarithmic scale. The mel scale is used in
perceptual analysis of vocal signal and reflect the manner of
how the human ear process the sounds. The conversion of
normal frequencies into mel-frequencies is made by the next
formula:
f mel = 1127 ⋅ ln(1 + f / 700 )  (4)
Until the corner frequency 700 Hz the bandwidth of the
triangular filters are approximately equals, but because after
this value the bandwidths grows logarithmically a
normalization of the filters is required. This operation is
detailed in [12, 13].
M3. Logarithm + DCT – The cepstrum computation require
the inverse Fourier transform of the logarithm of the squared
spectrum of signal. This formula explains the necessity to
apply the logarithm. The MFCC values are obtained with the
classical formula of the Discrete Cosine Transform:
nf
⎛ ⎞
vMFCC (c) = ∑ log10 (E j ) ⋅ cos⎜ c ⋅ ( j − 0.5) π ⎟ (5)
j =1 ⎝ nf ⎠
where E j represent the mel-energies of n f = 40 triangular
filters. There are computed a number of 13 coefficients, but
because the first coefficient vMFCC (0) is the mean of the
input set, there are kept only 12 values ( c = 1,12 ).
M4. Delta MFCC and delta-delta MFCC extraction –
represent the derivate of the cepstral vector. These derivative
coefficients are computing from windows situated at distance
of 50ms (the step between two consecutive windows is 10ms).
vΔMFCC (c, k ) = vMFCC (c, k − 2) − vMFCC (c, k + 2) (6)
The ΔΔMFCC are computed with the same derivative
formula but applied on the ΔMFCC.
The extraction of LPCC requires the following steps:
L1. Autocorrelation computing - with the classical formula:
N −1− i
Css [i ] = ∑ sˆ [k ] ⋅ sˆ [k + i]
w w , i = 0, nAC (7)
k =0
L2. Levinson-Durbin method – apply a recursive algorithm
in order to solve a prediction problem. Each autocorrelation
coefficient is predicted using the other nAC=14 values. The
values obtained at the last iteration give the linear prediction
coefficients LPC.
nac
Css [i ] = ∑ α k ⋅ Css [ i − k ] , i = 1, nac
(8)
k =1
vLPC [0] = 1 , vLPC [k ] = −α k (9)
L3. Log are ratio coefficients – are computed from vLPC.
The Levinson-Durbin method and the computation of
PARCOR and LAR coefficients is better described in [14]
L4. LPCC conversion – the computing of cepstral
coefficients LPCC are different for the first values until the nAC
order and the high order until nLPCC=21:
vLPCC [k ] = vLPC [k ] , k < nAC (10)
k −1
j
vLPCC [k ] = ∑ ⋅ vLPCC [ j ] ⋅ vLPC [mod(k − j , nAC )]
j =1 k
From this sets of parameters we have computed the LLE –
Largest Lyapunov Exponent varying the embedding lag and the
embedding dimension. The algorithm is detailed in [12].
 Because the input series applied to the procedure for
computing the nonlinear parameter of Lyapunov are too short,
we have used a method proposed by Rosenstein [11]. This
algorithm was specially projected for small data sets and is fast
and robust to changes and to noise.
IV. RESULTS AND CONCLUSIONS
The emotional corpus from SROL contains at this moment
around 400 files made from 25 speakers. In each files there are
pronounced 3-5 sentence. The feature vectors were extracted
only from 145 files / 545 sentences left in database after
validation. The training data sets include 75% of the feature
vectors, and the testing set contains 25%. The average error /
global error is computed after 50 iterations, at each iteration the
training and testing sets are mixed, keeping the ratio 75-25%.
In our previous research, we have observed that KNN
classifier has good performance for cepstral coefficients, but do
not offer a good classification of emotions when in the feature
vectors are included Lyapunov exponents. We have verified
this conclusion for the new training data sets. The smallest
global error for KNN classifier was around 50% for {90-104
LPC coef., 161-181 Lyap. LPCC}, respectively {21-32 MFCC,
149-160 Lyap. MFCC} Because the results for KNN classifier
was modest, we present in Table II only the results obtained
with the SVM classifier:
TABLE II. SVM CLASIFIER – EMOTION RECOGNITON ERROR WITH
FEATURE VECTORS THAT CONTAIN LLE EXPONENTS
Parameters Global Joy Fury Sad- Neu-
error ness tral
149-160 Lyapunov from MFCC coef. 26.77 32.16 30.07 22.39 15.67
161-181 Lyapunov from LPCC coef. 27.33 32.48 30.30 23.31 17.14
194-205,149-181 Lyap. Energy + Lyap 29.46 36.39 31.88 24.96 17.44
from MFCC and LPCC coef.
194-205, 149-160 Lyap Energy+MFCC 27.40 33.88 29.70 22.99 16.04
194-205, 161-181, Lyap Energy+LPCC 28.54 35.19 30.30 24.59 17.52
21-32 MFCC, 149-160 Lyap MFCC 26.03 29.78 30.22 20.52 18.21
33-44 MFCC stdev, 149-160 Lyap MFCC 26.35 29.18 31.19 22.69 16.57
45-56 delta MFCC, 149-160 Lyap MFCC 26.91 33.66 28.58 22.61 16.04
182-193 Energy, 149-160 Lyap MFCC 26.17 33.43 28.21 20.45 15.60
90-104 coef LPC, 161-181 Lyap LPCC 26.20 29.17 28.80 24.44 17.82
106-119 coef Parcor, 161-181 Lyap LPCC 24.51 26.77 26.54 22.71 18.72
120-133 coef LAR, 161-181 Lyap LPCC 23.35 25.49 24.59 22.26 18.27
182-193 Energy, 161-181 Lyap LPCC 25.79 30.75 26.09 24.14 17.67
The SVM classifier provides a global error around 27%
when the feature vectors contain only the LLE (Largest
Lyapunov Exponent) extracted from cepstral coefficients
MFCC or LPCC. The emotion classification accuracy is
improved when the feature vectors contain supplementary
parameters. In this case the best recognition 76.65% (error
23.35) is obtained for Lyapunov from LPCC + LAR
coefficients. A good recognition of 75.49% was also obtained
for PARCOR + Lyapunov from LPCC.
The best results presented in this paper are comparable with
the results obtained in our previous research when the
Lyapunov was extracted from the energy of mel-frequency
bands and when the smallest global error was 23.59% obtained
for {106-119 PARCOR + 192-205 Lyap. energy}. The neutral
tone has the best recognition percent and the weakest
recognized was for the joy state. The emotion recognition
accuracy obtained by us is comparable to those reported by
others, with the mention that our database does not contain
professional voices.
ACKNOWLEDGMENT
We acknowledge to Romanian Academy priority research
grant “The cognitive systems and applications” and to entire
research team of the project SRoL “Sounds of the Romanian
language”
REFERENCES
[1] Y. Pan, P. Shen, L. Shen, “Feature Extraction and Selection in Speech
Emotion Recognition”, vol. 2, pp. 64-69, 2012.
[2] T.-L. Pao, Y.-T. Chen, J.-H. Yeh, P.-J. Li, “Mandarin Emotional Speech
Recognition Based on SVM and NN”, 18th Int. Conf. on Pattern
Recognition, vol. 1, pp. 1096-1100, 2006.
[3] B.S. Yalamanchili, K.K Anusha, K. Santhi, P. Sruthi, B.
SwapnaMadhavi, “Non Linear Classification for Emotion Detection on
Telugu Corpus”, Int. Journal of Computer Science and Information
Technologies, vol. 5 (2) , pp. 2443-2448, 2014.
[4] K.S. Rao and S.G. Koolagudi, “Robust Emotion Recognition using
Spectral and Prosodic Features”, SpringerBriefs in Speech Technology,
pp. 17-46, 2013.
[5] A Milton, S. Sharmy Roy, S. Tamil Selvi, “SVM Scheme for Speech
Emotion Recognition using MFCC Feature” Int. Journal of Computer
Applications, vol. 69, no.9, 2013.
[6] S. Koolagudi and K. Sreenivasa Rao, “Emotion recognition from speech
using source, system and prosodic features”, Int J Speech Tech, pp. 265
– 289, 2012.
[7] Y. Han, G. Wang, Y. Yang, “Speech Emotion Recognition Based on
MFCC”, Journal of Chong Qing University of Posts and
Telecommunication, Natural Science Edition, vol. 20(5), 2008.
[8] E. Albornoz, D. Milone and H. Rufiner, “Spoken emotion recognition
using hierarchical classifiers”, Computer Speech and Language, vol. 25,
pp. 556 –570, 2011.
[9] V. Chavan, V.V. Gohokar, “Speech Emotion Recognition by using
SVM-Classifier”, Int. Journal of Engineering and Advanced Technology
- IJEAT, vol. 1, issue 5, 2012.
[10] T.-L. Pao, Y.-T. Chen, J.-H. Yeh, and W.-Y. Liao, “Detecting Emotions
in Mandarin Speech”, Computational Linguistics and Chinese Language
Processing, vol. 10, no. 3, pp. 347-362, 2005.
[11] M.T. Rosenstein, J. J. Collins and C. J. De Luca, “A practical method
for calculating largest Lyapunov exponents from small data sets”,
Physica D., 1993.
[12] M. Feraru and M. Zbancioc, “Emotion Recognition using Lyapunov
Exponent of the Mel-Frequency Energy Bands”, Int. Conf. of
Electronics, Computers and Artificial Intelligence - ECAI 2014
Bucureşti, România, in press.
[13] M. Zbancioc and M. Feraru, “A Study about MFCC Relevance in
Emotion Classification for SROL Database”, Proc. of 4th Int. Symp. on
Electrical and Electronics Engineering, ISEEE 2013, Galaţi, România,
ISBN 978-1-4799-2441-7, IEEE Catalog Number CFP1393K-USB.
[14] M. Feraru and M. Zbancioc, “Emotion Recognition in Romanian
Language using LPC Features”, The 4th IEEE Int. Conf. on E-Health
and Bioengineering Grigore T. Popa University of Medicine and
Pharmacy, Iaşi, Romania, ISBN 978-1-4799-2372-4, pp. 1-4, 2013.
[15] M. Zbancioc and M. Feraru, “The automatic segmentation of the vocal
signal using predictive neural network”, Int. Symposium on Signals,
Circuits, and Systems - ISSCS2013, ISBN: 978-1-4799-3193-4, pp.1-4