Browsing by Author "Chi, Tai-Shih"

Now showing 1 - 20 of 46

A 2.17-mW Acoustic DSP Processor With CNN-FFT Accelerators for Intelligent Hearing Assistive Devices
(2020-08-01)
Lee, Yu-Chi
;
Chi, Tai-Shih
;
Yang, Chia-Hsiang
;
電機工程學系
;
Department of Electrical and Computer Engineering
This article presents an acoustic DSP processor containing a neural network core for intelligent hearing assistive devices. The processor includes the accelerators for convolutional neural networks (CNNs) and fast Fourier transform (FFT). The CNN-based speech enhancement algorithm predicts the desired mask for the Fourier spectrogram of the speech signal to enhance speech intelligibility. Several design techniques are applied to enable efficient hardware mapping. The computational complexity for the CNN can be reduced by 23.6% by frame sharing, and a fast mask generation + partial sums pre-computation technique further reduces output latency by up to 64%. The size of the memory for the model is reduced by 75% using weight quantization. FFT is implemented by leveraging the packing algorithm to reduce the computational complexity by 43%. Reconfigurable processing elements are shared to support both FFT and CNN, realizing a saving in the area of 42%. In addition, input sharing and output sharing are used to, respectively, reduce data movements by 94% and 75%. A reordered FFT structure also eliminates up to 256 multiplexers. Fabricated in a 40-nm CMOS technology, the chip's core area is 4.2 mm(2) and the power dissipation is 2.17 mW at a clock frequency of 5 MHz from a 0.6-V supply. The embedded CNN accelerator supports both convolutional and fully connected (FC) layers and achieves a comparable energy efficiency with state-of-the-art CNN accelerators, despite the flexibility for FFT. The speech intelligibility is enhanced by up to 41% in the low SNR regime.
A 2.17mW Acoustic DSP Processor with CNN-FFT Accelerators for Intelligent Hearing Aided Devices
(2019-01-01)
Lee, Yu-Chi
;
Chi, Tai-Shih
;
Yang, Chia-Hsiang
;
電機工程學系
;
Department of Electrical and Computer Engineering
This paper proposes an acoustic DSP processor with a neural network core for speech enhancement. Accelerators for convolutional neural network (CNN) and fast Fourier transform (FFT) are embedded. The CNN-based speech enhancement algorithm takes the speech signals spectrogram as the model's input, and predicts the desired mask of speech to enhance speech intelligibility after passing through the CNN model. An array of multiply-accumulator (MAC) and coordinate rotation digital computer (CORDIC) engines are deployed to efficiently compute linear and nonlinear functions. Hardware sharing is applied to reduce hardware area by leveraging the high similarity between CNN and FFT computations. The proposed DSP processor chip is fabricated in a 40-nm CMOS technology with a core area of 4.3 mm(2). The chip's power dissipation is 2.17 mW at an operating frequency of 5 MHz. The CNN accelerator supports both convolutional and fully-connected layers and achieves an energy efficiency of 1200-to-2180 GOPS/W, despite the flexibility for FFT. The speech intelligibility can be enhanced by up to 41% under low SNR conditions.
A BINAURAL ALGORITHM FOR SPACE AND PITCH DETECTION
(2011)
Chou, Wen-Sheng
;
Cheong, Kah-Meng
;
Chi, Tai-Shih
;
電機工程學系
;
Department of Electrical and Computer Engineering
A binaural algorithm to simultaneously detect the azimuth angle and the pitch of the sound source is proposed in this paper. This algorithm is extended from the stereausis model with two-dimensional coincidence detectors in the joint Space-Pitch domain. In our simulations, sounds from different locations are produced by passing through the Head-Related-Transfer-Function (HRTF). Simulation results show that estimated azimuth angles from our proposed algorithm are more accurate than those from the stereausis model in the single sound source testing condition. Satisfactory results in streaming sound sources from the two-sound mixture by using estimated Space-Pitch information are also demonstrated in our pilot experiments.
A Bone-Guided Cochlear Implant CMOS Microsystem Preserving Acoustic Hearing
(2017-01-01)
Qian, Xin-Hong
;
Wu, Yi-Chung
;
Yang, Tzu-Yi
;
Cheng, Cheng-Hsiang
;
Chu, Hsing-Chien
In this paper, a bone-guided cochlear implant (BGCI) SOC microsystem is proposed and designed. The BGCI microsystem uses four or more electrodes placed on the bone surface of the cochlea and one on the round window to preserve partially the acoustic hearing. The external SOC of the BGCI processes the acoustic signals and generates stimulation patterns and command that are transmitted to the implanted chip through the 13.56MHz wireless power and bidirectional data telemetry. The implanted chip with 4-channel high-voltage-tolerant stimulator generates biphasic stimulation currents up to 800 mu A and sends back the power indication and electrode impedance error signals to the external SOC. Electrical tests on the fabricated BGCI has been performed. The in-vivo animal tests on guinea pigs has shown the evoked Wave III of EABR waveforms. The proposed BGCI can be applied to patients with high-frequency hearing loss, tinnitus, and dizziness.
A frequency bin-wise nonlinear masking algorithm in convolutive mixtures for speech segregation
(2012-05-01)
Chi, Tai-Shih
;
Huang, Ching-Wen
;
Chou, Wen-Sheng
;
電機工程學系
;
Department of Electrical and Computer Engineering
A frequency bin-wise nonlinear masking algorithm is proposed in the spectrogram domain for speech segregation in convolutive mixtures. The contributive weight from each speech source to a time-frequency unit of the mixture spectrogram is estimated by a nonlinear function based on location cues. For each sound source, a non-binary mask is formed from the estimated weights and is multiplied to the mixture spectrogram to extract the sound. Head-related transfer functions (HRTFs) are used to simulate convolutive sound mixtures perceived by listeners. Simulation results show our proposed method outperforms convolutive independent component analysis and degenerate unmixing and estimation technique methods in almost all test conditions. (C) 2012 Acoustical Society of America
A GENERATIVE AUDITORY MODEL EMBEDDED NEURAL NETWORK FOR SPEECH PROCESSING
(2018-01-01)
Lo, Yu-Wen
;
Shen, Yih-Liang
;
Liao, Yuan-Fu
;
Chi, Tai-Shih
;
電機工程學系
Before the era of the neural network (NN), features extracted from auditory models have been applied to various speech applications and been demonstrated more robust against noise than conventional speech-processing features. What's the role of auditory models in the current NN era? Are they obsolete? To answer this question, we construct a NN with a generative auditory model embedded to process speech signals. The generative auditory model consists of two stages, the stage of spectrum estimation in the logarithmic-frequency axis by the cochlea and the stage of spectral-temporal analysis in the modulation domain by the auditory cortex. The NN is evaluated in a simple speaker identification task. Experiment results show that the auditory model embedded NN is still more robust against noise, especially in low SNR conditions, than the randomly-initialized NN in speaker identification.
A HEARING MODEL TO ESTIMATE MANDARIN SPEECH INTELLIGIBILITY FOR THE HEARING IMPAIRED PATIENTS
(2015)
Tsai, Pei-Chun
;
Lin, Shih-Ting
;
Lee, Wen-Chung
;
Hsu, Chung-Chien
;
Chi, Tai-Shih
A hearing model, which is parameterized by hearing thresholds, degrees of loudness recruitment and reductions of frequency resolution of a hearing-impaired (HI) patient, is proposed in this paper. The model is developed in the filter-bank framework and is flexible for fitting hearing-loss conditions of HI patients. Psychoacoustic experiments were conducted under clean and noisy conditions to validate the model\'s capability in predicting Mandarin speech intelligibility for HI patients. Statistical analysis on the hearing-test results suggests that the proposed model can predict Mandarin speech intelligibility for HI patients to a certain degree.
A HEARING MODEL TO ESTIMATE MANDARIN SPEECH INTELLIGIBILITY FOR THE HEARING IMPAIRED PATIENTS
(2015-01-01)
Tsai, Pei-Chun
;
Lin, Shih-Ting
;
Lee, Wen-Chung
;
Hsu, Chung-Chien
;
Chi, Tai-Shih
A hearing model, which is parameterized by hearing thresholds, degrees of loudness recruitment and reductions of frequency resolution of a hearing-impaired (HI) patient, is proposed in this paper. The model is developed in the filter-bank framework and is flexible for fitting hearing-loss conditions of HI patients. Psychoacoustic experiments were conducted under clean and noisy conditions to validate the model's capability in predicting Mandarin speech intelligibility for HI patients. Statistical analysis on the hearing-test results suggests that the proposed model can predict Mandarin speech intelligibility for HI patients to a certain degree.
A HYBRID NEURAL NETWORK BASED ON THE DUPLEX MODEL OF PITCH PERCEPTION FOR SINGING MELODY EXTRACTION
(2018-01-01)
Chou, Hsin
;
Chen, Ming-Tso
;
Chi, Tai-Shih
;
電機工程學系
;
Department of Electrical and Computer Engineering
In this paper, we build up a hybrid neural network (NN) for singing melody extraction from polyphonic music by imitating human pitch perception. For human hearing, there are two pitch perception models, the spectral model and the temporal model, in accordance with whether harmonics are resolved or not. Here, we first use NNs to implement individual models and evaluate their performance in the task of singing melody extraction. Then, we combine the NNs to constitute the composite NN to simulate the duplex model, which complements the pitch perception from unresolved harmonics of the spectral model using the temporal model. Simulation results show the proposed composite NN outperforms other conventional methods in singing melody extraction.
A Multi-Scale Fully Convolutional Network for Singing Melody Extraction
(2019-01-01)
Gao, Ping
;
You, Cheng-You
;
Chi, Tai-Shih
;
電機工程學系
;
Department of Electrical and Computer Engineering
The melody extraction can be considered as a sequence-to-sequence task or a classification task. Many recent models based on semantic segmentation have been proven very effective in melody extraction. In this paper, we built up a fully convolutional network (FCN) for melody extraction from polyphonic music. Inspired by the state-of-the-art architecture of the semantic segmentation, we constructed the encoder in a dense way and designed the decoder accordingly for audio processing. The combined frequency and periodicity (CFP) representation, which contains spectral and cepstral information, was adopted as the input feature of the proposed model. We conducted performance comparison between the proposed model and several methods on various datasets. Experimental results show the proposed model achieves state-of-the-art performance with less computation and fewer parameters.
A Non-uniformly Distributed Three-microphone Array for Speech Enhancement in Directional and Diffuse Noise Field
(2014-01-01)
Hsu, Chung-Chien
;
Cheong, Kah-Meng
;
Chi, Tai-Shih
;
電機資訊學士班
;
Undergraduate Honors Program of Electrical Engineering and Computer Science
This paper proposes a non-uniformly distributed three-microphone array speech enhancement system to suppress directional interferences and diffuse noise simultaneously. Each pair of microphones is designed to tackle one kind of noise. Unlike other hybrid systems, which combine different noise suppression techniques derived in different domains, the proposed system integrates two noise suppression techniques derived in the unified power spectral density domain. In terms of objective LLR and PESQ measures, the proposed hybrid system is demonstrated more effective in eliminating directional and diffuse noise under various SNR conditions than each individual suppression technique.
A precedence effect based far-field DoA estimation algorithm
(2013)
Chou, Wen-Sheng
;
Chi, Tai-Shih
;
電機資訊學士班
;
Undergraduate Honors Program of Electrical Engineering and Computer Science
A robust far-field DOA estimation algorithm is proposed in this paper. The algorithm is inspired by the precedence effect which results in humans robust capability in localizing sound sources. The algorithm implements the concept of the precedence effect by applying a proper threshold and an onset detection mechanism in the cross-correlation domain. Experiment results show that by cascading the proposed algorithm to a conventional cross-correlation based DoA algorithm, the accuracy of the DoA estimation is significantly improved in far-field test conditions.
A Spectral Modulation Sensitivity Weighted Pre-emphasis Filter for Active Noise Control System
(2016-01-01)
Cheong, Kah-Meng
;
Wang, Yuh-Yuan
;
Chi, Tai-Shih
;
電機工程學系
;
Department of Electrical and Computer Engineering
Psychoacoustic active noise control (ANC) systems by considering human hearing thresholds in different frequency bands were developed in the past. Besides the frequency sensitivity, human hearing also shows different sensitivity to spectral and temporal modulations of the sound. In this paper, we propose a new psychoacoustic active noise control system by further considering the spectral modulation sensitivity of human hearing. In addition to the sound pressure level (SPL), the loudness level is also objectively assessed to evaluate the noise reduction performance of the proposed ANC system. Simulation results demonstrate the proposed system outperforms two compared systems under test conditions of narrowband and broadband noise in terms of the loudness level. The proposed algorithm has been validated on TI C6713 DSP platform for real-time process.
A Two-stage Singing Voice Separation Algorithm Using Spectro-temporal Modulation Features
(2015)
Yen, Frederick Z.
;
Huang, Mao-Chang
;
Chi, Tai-Shih
;
電機學院
;
College of Electrical and Computer Engineering
A two-stage singing voice separation algorithm using spectrotemporal modulation features is proposed in this paper. First, music clips are transformed into auditory spectrograms and the spectral-temporal modulation contents of all time-frequency (T-F) units of the auditory spectrograms are extracted using an auditory model. Then, T-F units are sequentially clustered using the expectation-maximization (EM) algorithm into percussive, harmonic and vocal units through the proposed two-stage algorithm. Lastly, the singing voice is synthesized from clustered vocal T-F units via time-frequency masking. The algorithm was evaluated using the MIR-1K dataset and demonstrated better separation results than our previously proposed one-stage algorithm.
Acoustic Scene Change Detection by Spectro-Temporal Filtering on Spectrogram using Chirps
(2016)
Lin, Chang-Hong
;
Cheong, Kah-Meng
;
Huang, Mao-Chang
;
Chen, Ming-Yen
;
Chang, Chen-Kuei
Indoor acoustic scene change detection systems using periodic impulse signals were developed in the past. Comparing with impulse signals, the chirp signal is more robust against environmental noise due to its specific spectro-temporal structure, which can be easily detected using a spectro-temporal modulation filtering mechanism. In this paper, we demonstrate a system which employs the two-dimensional spectro-temporal filtering mechanism on Fourier spectrogram to measure the total energy of reverberations of the chirp signal and compares the energy difference between consecutive chirps with a predefined threshold to automatically detect the change of the acoustic scene. Simulation results show the proposed system is very effective in detecting the acoustic scene change in a real living room with great robustness against noise recorded from real-world.
Auditory cortical receptive fields: Stable entities with plastic abilities
(2007-09-26)
Elhilali, Mounya
;
Fritz, Jonathan B.
;
Chi, Tai-Shih
;
Shamma, Shihab A.
;
電信工程研究所
To form a reliable, consistent, and accurate representation of the acoustic scene, a reasonable conjecture is that cortical neurons maintain stable receptive fields after an early period of developmental plasticity. However, recent studies suggest that cortical neurons can be modified throughout adulthood and may change their response properties quite rapidly to reflect changing behavioral salience of certain sensory features. Because claims of adaptive receptive field plasticity could be confounded by intrinsic, labile properties of receptive fields themselves, we sought to gauge spontaneous changes in the responses of auditory cortical neurons. In the present study, we examined changes in a series of spectrotemporal receptive fields (STRFs) gathered from single neurons in successive recordings obtained over time scales of 30-120 min in primary auditory cortex (A1) in the quiescent, awake ferret. We used a global analysis of STRF shape based on a large database of A1 receptive fields. By clustering this STRF space in a data-driven manner, STRF sequences could be classified as stable or labile. We found that > 73% of A1 neurons exhibited stable receptive field attributes over these time scales. In addition, we found that the extent of intrinsic variation in STRFs during the quiescent state was insignificant compared with behaviorally induced STRF changes observed during performance of spectral auditory tasks. Our results confirm that task-related changes induced by attentional focus on specific acoustic features were indeed confined to behaviorally salient acoustic cues and could be convincingly attributed to learning-induced plasticity when compared with "spontaneous" receptive field variability.
AUTOENCODING HRTFS FOR DNN BASED HRTF PERSONALIZATION USING ANTHROPOMETRIC FEATURES
(2019-01-01)
Chen, Tzu-Yu
;
Kuo, Tzu-Hsuan
;
Chi, Tai-Shih
;
電機工程學系
;
Department of Electrical and Computer Engineering
We proposed a deep neural network (DNN) based approach to synthesize the magnitude of personalized head-related transfer functions (HRTFs) using anthropometric features of the user. To mitigate the over-fitting problem when training dataset is not very large, we built an autoencoder for dimensional reduction and establishing a crucial feature set to represent the raw HRTFs. Then we combined the decoder part of the autoencoder with a smaller DNN to synthesize the magnitude HRTFs. In this way, the complexity of the neural networks was greatly reduced to prevent unstable results with large variance due to overfitting. The proposed approach was compared with a baseline DNN model with no autoencoder. The log-spectral distortion (LSD) metric was used to evaluate the performance. Experiment results show that the proposed approach can reduce LSD of estimated HRTFs with greater stability.
Binary Mask Estimation Based on Frequency Modulations
(2014-01-01)
Hsu, Chung-Chien
;
Chien, Jen-Tzung
;
Chi, Tai-Shih
;
電機工程學系
;
Department of Electrical and Computer Engineering
In this paper, a binary mask estimation algorithm is proposed based on modulations of speech. A multi-resolution spectro-temporal analytical auditory model is utilized to extract modulation features to estimate the binary mask, which is often used in speech segregation applications. The proposed method estimates noise from the beginning of each test sentence, a common approach seen in many conventional speech enhancement algorithms, to further enhance the modulation features. Experimental results demonstrate that the proposed method outperforms the AMS-GMM system in terms of the HIT-FA rate when estimating the binary mask.
Categorical Rating of Narrowband Mandarin Speech Quality
(2013)
Huang, Kuan-Lang
;
Chi, Tai-Shih
;
電機資訊學士班
;
Undergraduate Honors Program of Electrical Engineering and Computer Science
Speech quality is postulated to consist of several perceptual attributes. Psychoacoustic experiments for Mandarin monosyllables were designed and conducted to investigate the relations between five abstract attributes, including intelligibility, clarity, naturalness, continuity and noise intrusiveness, and perceived integral speech quality. Experimental results demonstrate a good speech quality estimate can be obtained using a simple multivariate linear regression method. The linear regression analysis shows clarity has the most impact on speech quality, while intelligibility contributes little in the subjective assessment. These findings could be used to develop categorical-rating based objective speech quality measures in the future.
CNN BASED TWO-STAGE MULTI-RESOLUTION END-TO-END MODEL FOR SINGING MELODY EXTRACTION
(2019-01-01)
Chen, Ming-Tso
;
Li, Bo-Jun
;
Chi, Tai-Shih
;
電機工程學系
;
Department of Electrical and Computer Engineering
Inspired by human hearing perception, we propose a two-stage multi-resolution end-to-end model for singing melody extraction in this paper. The convolutional neural network (CNN) is the core of the proposed model to generate multi-resolution representations. The 1-D and 2-D multi-resolution analysis on waveform and spectrogram-like graph are successively carried out by using 1-D and 2-D CNN kernels of different lengths and sizes. The 1-D CNNs with kernels of different lengths produce multi-resolution spectrogram-like graphs without suffering from the trade-off between spectral and temporal resolutions. The 2-D CNNs with kernels of different sizes extract features from spectro-temporal envelopes of different scales. Experiment results show the proposed model outperforms three compared systems in three out of five public databases.