SPECTRO-TEMPORAL NEURAL FACTORIZATION FOR SPEECH DEREVERBERATION

Abstract

This study presents a spectro-temporal neural factorization (STNF) for speech dereverberation. Traditionally, a contextual window of spectro-temporal reverberant speech was unfolded into a one-way vector which was fed into a neural network to estimate the spectra of source speech at each time frame. Model parameters were trained by using the vectorized error backpropagation algorithm. System performance is constrained because contextual correlations and common factors in frequency and time horizons are disregarded. To compensate this weakness, a spectro-temporal factorization is incorporated to preserve the structural information in neural network training based on bi-factorized error backpropagation where the spectral and temporal factor matrices are estimated. Affine transformation in one-way neural network is generalized to the bilinear decomposition in bi-factorized neural network. The spectro-temporal features are extracted and forwarded to fully-connected layers for regression outputs. Such a STNF is further improved by merging with long short-term memory layer to capture the temporal features. Experiments results on 2014 REVERB Challenge demonstrate the meaningfulness of the factorized features and the merit of integrating these features for speech dereverberation.