Multiple Input Audio Denoising Using Deep Neural Networks
*Nelson Yalta (Waseda University), Kuniaki Noda (Waseda University),
Kazuhiro Nakadai (Honda Research Institute Japan Co., Ltd), and Tetsuya Ogata (Waseda University)
1. INTRODUCTION
Robots should have auditory functions[1] in order to
have a natural interaction. Maintaining robustness and
adaptively to the daily dynamically-changing
environments, is the principal topic of the methods
proposed in [2]. In fact, Robots must be able to adapt,
recognize, and separate the sources information by
reducing the noise from the source in order to work
properly in the real world. This process is called
denoising. In this paper, we evaluate the use of Deep
Neural Network (DNN) for doing a denoising task, and
also using the DNN after a previous denoising stage
performed by the existing source separation methods.
2. SOUND SOURCE SEPARATION
2.1. Independent Component Analysis
Blind Source Separation (BSS), is the separation of a
set of source signals from a set of unknown mixed
signals. For this problem the typical solution is to apply
an Independent Component Analysis (ICA). ICA is
defined as a method for separating a multivariate signal
into additive subcomponents. This is done by assuming
that the subcomponents are non-Gaussian signals and
that they are statistically independent from each other.
An important note to consider is that if N sources are
present, at least N observations are needed to recover
the original signal.
2.2. Speech Enhancement and Sound Source
Separation for dynamic environments
Geometrically constrained High-order Decorrelation
based Source Separation with Adaptive Step-size
control (GHDSS-AS) is a Sound Source separation
technique proposed by Nakadai et al.[2]. GHDSS-AS
can be applied to track the dynamic changes for robot
audition system.
GHDSS is a hybrid algorithm that approaches
beamforming and BSS by performing directivity
formation of the decorrelation and sound direction
between the sound source signals from a microphone
array.
A Noise Reduction (NR) method called Histogram-
based Recursive Level Estimation (HRLE) is also
proposed. It enhances speech with a small number of
parameters in consideration of dynamic environments.
2.3. Task
Since GHDSS is a linear separation technique, it
does not handle non-linear mixed signals. Meanwhile,
a proposed model representation for a noisy nonlinear
ICA based on the formulation of maximum likelihood
estimation [3], which the generation model of the
observed signal were assumed, presents a limit on
capacity and scalability.
3. IMPLEMENTATION
3.1. Deep Neural Network (DNN)
DNN is a neural network with multiple hidden layers
of units between the input and the output layers that
can model complex nonlinear representations. One of
its principal characteristics is the ability of self-
organizing sensory features from large amount of
training data, which the recognition performance
exceeds the performance from conventional models on
the speech recognition task [4]. Moreover, it can show
high performance as a model for Noise Reduction tasks
[5].
In this paper, we aim to use DNN to achieve both,
SSS and NR performance over the prior methods.
3.2. Extraction Method of Acoustics features
We prepare the data using the Mel filter bank
features extracted from the audio data. For the analysis
of continuous speech, we perform the following:
As a specific extraction method, first the
recording is performed by an arrangement of N
array microphones.
The recorded audio signal is converted into Mel
Filter Bank (MFB) of D dimensions features.
The obtained MFB features are sampled cutting
on with a T frames sliding window.
Finally, the feature are concatenated on N parts
channels.
This generated D x T x N noised acoustic feature
become the input dimension for DNN described in the
next section.
3.3. Proposed Model
We use a DNN as shown in Fig. 1, with supervised
learning. For modeling the separation filter we use the
MFB features from the 1 channel clean data and as
input depending on the task we use the 8 channels
signal data, and a 2 channels HRLE + GHDSS output
data. The DNN has multiple layers of which dimension
is compressed from the input to output layers.
 Fig. 1. DNN - Proposed model
4. EVALUATION
4.1. Audio Dataset
For the training and test stage we use the JNAS
dataset, which includes almost 70,000 speech sentences
from different speakers, at different sound noise rate in
Japanese language and different speech length.
For preparation of this dataset was used a hearbo
robot equipped with an 8 Channel array microphone
(Fig. 2). A speech sentence was sampled every 10 deg.
around the robot, starting at 0 deg. And a noise input
was used the Fan Noise from the robot.
Fig. 2. Hearbo
4.2. Training the DNN
For our experiments, from the audio data using MFB
we get 27 dimensions and use a sliding window of 11
frames, so the input 297 dimensions x Channel, and the
output become a size of 297 dimensions. For the first
experiment we use the 8 Channels noisy data, so the
input becomes to 2376 dimensions; and for the second
experiment using the data from then HRLE GHDSS
cleaned output as input, the DNN input becomes to 594
dimensions.
We evaluated multiple type of DNN, and found the
most suitable structure with three Hidden Layers of
which neuron size are 1600, 800, and 400 respectively.
As for the training data, we use random information
from the JNAS dataset without information of the
speaker location angle.
Fig. 3. Results
4.3. Results
We show the results of our experiments in Fig. 3
showing the accuracy rate of the speech recognition by
Julius engine for multiple audio inputs from different
direction. We can see that both, the GHDSS-HRLE and
the DNN methods has almost the same average result.
However, applying DNN for denoising after a GHDSS-
HRLE previous process, enhance the previous accuracy rate.
5. CONCLUSION
In this paper, we proposed a SSS method using DNN. As
show in the results, a combined method of DNN + GHDSS-
HRLE enhanced the recognition rate that using only one
method. However, is important to remark the DNN can
achieve a recognition rate of two different process in one
DNN process without no more information that the noisy
information and the clear information.
ACKNOWLEDGEMENTS
The work has been supported by MEXT Grant-in-Aid for
Scientific Research (A) 15H01710.
REFERENCES
[1] K. Nakadai, T. Lourens, G. O. Hiroshi, and H. Kitano,
Active audition for humanoid, Proc. Natl. Conf. Artif.
Intell., pp. 832839, 2000.
[2] K. Nakadai, G. Ince, K. Nakamura, and H. Nakajima,
Robot audition for dynamic environments, 2012 IEEE
Int. Conf. Signal Process. Commun. Comput. ICSPCC
2012, pp. 125130, 2012.
[3] S. Maeda and S. Ishii, A noisy nonlinear independent
component analysis, Proc. 2004 14th IEEE Signal
Process. Soc. Work. Mach. Learn. Signal Process. 2004.,
pp. 173182, 2004.
[4] G. Hinton, L. Deng, D. Yu, G. E. Dahl, A. Mohamed, N.
Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N.
Sainath, and B. Kingsbury, Deep Neural Networks for
Acoustic Modeling in Speech Recognition, IEEE Signal
Process. Mag., no. November, pp. 8297, 2012.
[5] X. Feng, Y. Zhang, and J. Glass, Speech feature
denoising and dereverberation via deep autoencoders for
noisy reverberant speech recognition, ICASSP, IEEE Int.
Conf. Acoust. Speech Signal Process. - Proc., pp. 1759
1763, 2014.