Browsing by Author "Chiang, Chen-Yu"

Now showing 1 - 20 of 29

A New Approach of Speaking Rate Modeling for Mandarin Speech Prosody
(2012)
Hsieh, Chiao-Hua
;
Chiang, Chen-Yu
;
Wang, Yih-Ru
;
Yu, Hsiu-Min
;
Chen, Sin-Horng
A new approach of Mandarin-speech prosody modeling to consider the effects of speaking rate is proposed. The approach is a modification of our previous prosody labeling and modeling method to take speaking rate as a continuous independent variable and let prosodic-acoustic features and some parameters of prosodic models depend on it in order to count its influences. A speaking rate-dependent hierarchical prosodic model is hence constructed from four speech corpora of a single female speaker with fast, normal, medium and slow speaking rates. An analysis of the effects of speaking rate on the model parameters showed that they agreed well with our prior knowledge. So, the proposed approach provides a systematic and effective way to quantify the effects of speaking rate on Mandarin-speech prosody.
A New Model-based Mandarin-speech Coding System
(2011-01-01)
Chiang, Chen-Yu
;
Yang, Jyh-Her
;
Liu, Ming-Chieh
;
Wang, Yih-Ru
;
Liao, Yuan-Fu
In this paper, a new model-based Mandarin-speech coding system is proposed. It employs a prosody-enriched ASR with a hierarchical prosodic model (HPM) to generate from the input speech enriched transcriptions, including linguistic features, prosodic tags and spectral parameters in the encoder. By sending these features to the decoder, we can first reconstruct the prosodic-acoustic features of syllable pitch contour, syllable duration, syllable energy level, and inter-syllable pause duration by HPM using the linguistic features and prosodic tags; and then combined with spectral parameters to reconstruct the input speech signal by an HMM-based speech synthesizer. Experimental results show that the reconstructed speech has good quality at a low data rate of 543 bits/s.
A New Model-based Prosody Coder for Mandarin Speech
(2013)
Chiang, Chen-Yu
;
Hung, Yu-Ping
;
Chen, Sin-Horng
;
Wang, Yih-Ru
;
電機工程學系
In this paper, a novel parametric prosody coding approach for Mandarin speech is proposed. It employs a hierarchical prosodic model (HPM) as a prosody generating model in the encoder to analyze the speech prosody of the input utterance to obtain a parametric representation of four prosodic-acoustic features of syllable pitch contour, syllable duration, syllable energy level, and syllable-juncture pause duration for encoding. In the decoder, the four prosodic-acoustic features are reconstructed by a synthesis operation using the decoded HPM parameters. The reconstructed prosodic features are lastly used in an HMM-based speech synthesizer to help to generate the reconstructed speech. Experimental results show that the reconstructed speech has good quality at low data rates of 114.9 bits/s for a speaker-dependent task. An informal listening test confirmed decoded speeches sounded very fluently.
A New Prosody-Assisted Mandarin ASR System
(2012-08-01)
Chen, Sin-Horng
;
Yang, Jyh-Her
;
Chiang, Chen-Yu
;
Liu, Ming-Chieh
;
Wang, Yih-Ru
This paper presents a new prosody-assisted automatic speech recognition (ASR) system for Mandarin speech. It differs from the conventional approach of using simple prosodic cues on employing a sophisticated prosody modeling approach based on a four-layer prosody-hierarchy structure to automatically generate 12 prosodic models from a large unlabeled speech database by the joint prosody labeling and modeling (PLM) algorithm proposed previously. By incorporating these 12 prosodic models into a two-stage ASR system to rescore the word lattice generated in the first stage by the conventional hidden Markov model (HMM) recognizer, we can obtain a better recognized word string. Besides, some other information can also be decoded, including part of speech (POS), punctuation mark (PM), and two types of prosodic tags which can be used to construct the prosody-hierarchy structure of the testing speech. Experimental results on the TCC300 database, which consists of long paragraphic utterances, showed that the proposed system significantly outperformed the baseline scheme using an HMM recognizer with a factored language model which models word, POS, and PM. Performances of 20.7%, 14.4%, and 9.6% in word, character, and base-syllable error rates were obtained. They corresponded to 3.7%, 3.7%, and 2.4% absolute (or 15.2%, 20.4%, and 20% relative) error reductions. By an error analysis, we found that many word segmentation errors and tone recognition errors were corrected.
A Novel Model-based Pitch Conversion Method for Mandarin Speech
(2009)
Hwang, Hsin-Te
;
Chiang, Chen-Yu
;
Sung, Po-Yi
;
Chen, Sin-Horng
;
電信工程研究所
In this paper, a novel model-based pitch conversion method for Mandarin is presented and compared with other two conventional conversion methods, i.e. the mean/variance transformation approach and the GMM-based mapping approach. Syllable pitch contour is first quantized by 3(rd) order orthogonal expansion coefficients; then, the source and target speakers' prosodic models are constructed, respectively. Two mapping methods based on the prosodic model are presented. Objective tests confirmed that one of the proposed methods are superior the conventional methods. Some findings in informal listening tests and objective tests are worthwhile to further investigate.
A SPEAKING RATE-CONTROLLED MANDARIN TTS SYSTEM
(2013)
Hsieh, Chiao-Hua
;
Wang, Yih-Ru
;
Chiang, Chen-Yu
;
Chen, Sin-Horng
;
電子工程學系及電子研究所
In this paper, a new speaking rate-controlled Mandarin TTS system based on a speaking rate-dependent hierarchical prosodic model (SR-HPM) [6] is proposed. In the training phase, a data-driven approach is employed to automatically build the SR-HPM directly from a large prosody-unlabeled speech database containing utterances of various speaking rates. The SR-HPM comprises 15 sub-models designed to describe various relationships among 3 types of prosodic-acoustic features of speech utterances, two types of prosodic tags specifying a 4-layer prosody hierarchy, linguistic features of various levels of the associated texts, and the speaking rates. In the test phase, the SR-HPM is employed to generate 4 prosodic-acoustic features, including syllable pitch contours, syllable durations, syllable energy levels, and syllable juncture pause durations. Combining these prosodic features with the spectral features generated by the HTS synthesizer, the system can generate natural speech for any speaking rate in a wide range of 0.15-0.3 seconds/syllable. A distinct feature of the system to control the occurrence frequencies of breaks of various types as well as their pause durations according to the given speaking rate was demonstrated. A subjective test showed that MOS scores of 3.35, 3.44 and 3.28 were achieved respectively for fast (SR= 0.17 sec/syllable), medium (SR=0.2 sec/syllable) and slow (SR= 0.25 sec/syllable) synthetic speeches.
A STUDY ON CROSS-LANGUAGE KNOWLEDGE INTEGRATION IN MANDARIN LVCSR
(2012)
Chiang, Chen-Yu
;
Siniscalchi, Sabato Marco
;
Wang, Yih-Ru
;
Chen, Sin-Horng
;
Lee, Chin-Hui
We present a cross-language knowledge integration framework to improve the performance in large vocabulary continuous speech recognition. Two types of knowledge sources, manner attribute and prosodic structure, are incorporated. For manner of articulation, cross-lingual attribute detectors trained with an American English corpus (WSJ0) are utilized to verify and rescore hypothesized Mandarin syllables in word lattices obtained with state-of-the-art systems. For the prosodic structure, models trained with an unsupervised joint prosody labeling and modeling technique using a Mandarin corpus (TCC300) are used in lattice rescoring. Experimental results on Mandarin syllable, character and word recognition with the TCC300 corpus show that the proposed approach significantly outperforms the baseline system that does not use articulatory and prosodic information. It also demonstrates a potential of utilizing results from cross-lingual attribute detectors as a language-universal frontend for automatic speech recognition.
Advanced Unsupervised Joint Prosody Labeling and Modeling for Mandarin Speech and Its Application to Prosody Generation for TTS
(2009)
Chiang, Chen-Yu
;
Chen, Sin-Horng
;
Wang, Yih-Ru
;
電信工程研究所
;
Institute of Communications Engineering
Motivated by the success of the unsupervised joint prosody labeling and modeling (UJPLM) method for Mandarin speech on modeling of syllable pitch contour in our previous study, in this paper, the advanced UJPLM (A-UJPLM) method is proposed based on UJPLM to jointly label prosodic tags and model syllable pitch contour, duration and energy level. Experimental results on the Sinica Treebank corpus showed that most prosodic tags labeled were linguistically meaningful and the model parameters estimated were interpretable and generally agreed with other previous study. In virtue of the functions given by the model parameters, an application of A-UJPLM to the prosody generation for Mandarin TTS is proposed. Experimental results showed that the proposed method performed well. Most predicted prosodic features matched well to their original counterparts. This also reconfirmed the effectiveness of the A-UJPLM method.
An Automatic Prosody Labeling Method for Mandarin Speech
(2007)
Chiang, Chen-Yu
;
Yu, Hsiu-Min
;
Wang, Yih-Ru
;
Chen, Sin-Horng
;
電信工程研究所
A new model-based automatic prosody labeling method for Mandarin speech is proposed. It first introduces four models to describe the relationships of the prosody tags to be labeled, the prosodic features of the speech signals, and the linguistic features of the associated texts. It then employs a sequential optimization procedure to estimate parameters of these four models and find all prosody tags. Experimental results on the Sinica Tree-Bank corpus showed that most prosody tags labeled were meaningful and the estimated parameters of these four models matched well with our a priori knowledge about Mandarin prosody.
An Exploration of Local Speaking Rate Variations in Mandarin Read Speech
(2018-01-01)
Liou, Guan-Tin
;
Chiang, Chen-Yu
;
Wang, Yih-Ru
;
Chen, Sin-Horng
;
電機工程學系
This paper explores speaking rate variation in Mandarin read speech. In contrast to assuming that each utterance is generated in a constant or global speaking rate, this study seeks to estimate local speaking rate for each prosodic unit in an utterance. The exploration is based on the existing speaking rate-dependent hierarchical prosodic model (SR-HPM). The main idea is to first use the SR-HPM to explore the prosodic structures of utterances and extract the prosodic units. Then, local speaking rate is estimated for each prosodic unit (prosodic phrase in this study). Some major influence factors including tone, base syllable type, prosodic structure, and speaking rate of the higher prosodic units (utterance and BG/PG) are compensated in the local SR estimation. A syntactic-local SR model is constructed and use in the prosody generation of Mandarin TTS. Experimental results on a large read speech corpus generated by a professional female announcer showed that the generated prosody with local speaking rate variations is proved to be more vivid than the one with a constant speaking rate.
An Investigation on the Mandarin Prosody of a Parallel Multi-Speaking Rate Speech Corpus
(2009)
Chiang, Chen-Yu
;
Tang, Cheng-Chang
;
Yu, Hsiu-Min
;
Wang, Yih-Ru
;
Chen, Sin-Horng
In this paper, the prosody of a parallel multi-speaking rate Mandarin read speech corpus is investigated. The corpus contains four parallel speech datasets uttered by a female professional announcer with various speech rates (SRs) of 4.40 (fast), 3.82 (normal), 2.97 (median) and 2.45 (slow) syllables/second. By using the unsupervised joint prosody labeling and modeling (PLM) method proposed previously, the relationship between SR and various prosodic features, including pause duration, patterns of three high-level prosodic constituents, and the break labels, are investigated. The analyses reported in this study could be very informative in developing prosody generation mechanism for text-to-speech and prosody modeling for automatic speech recognition in various SRs.
ENRICHING MANDARIN SPEECH RECOGNITION BY INCORPORATING A HIERARCHICAL PROSODY MODEL
(2011)
Yang, Jyh-Her
;
Liu, Ming-Chieh
;
Chang, Hao-Hsiang
;
Chiang, Chen-Yu
;
Wang, Yih-Ru
This paper presents a new probabilistic framework of Mandarin speech recognition by incorporating a sophisticated hierarchical prosody model into the conventional HMM-based system. The prosody model describes the relations of linguistic cues of various levels, break types and prosodic states which represent the prosody hierarchical structure, and prosody-related acoustic features. Aside from producing the recognized word sequences, the system also decodes other information including word's part-of-speech, punctuation marks, inter-syllable break types, and prosodic states of syllables. Experimental results on the TCC300 corpus, which consists of paragraphic utterances, showed that the proposed system significantly outperformed the baseline system. The word and character error rates decreased from 24.4% and 18.1% to 20.7% and 14.4% (or 15.2% and 20.4% relative improvements), respectively.
Exploration of high-level prosodic patterns for continuous Mandarin speech
(2008)
Chiang, Chen-Yu
;
Yu, Hsiu-Min
;
Wang, Yih-Ru
;
Chen, Sin-Horng
;
電信工程研究所
In this paper, the high-level prosodic patterns of prosodic word (PW), prosodic phrase (PPh) and breath group/prosodic phrase group (BG/PG) for syllable pitch-level and duration are explored using an automatic joint prosody labeling and modeling method. Experimental results on a treebank speech corpus showed that the explored high-level prosodic patterns not only matched well with our a priori knowledge about Mandarin prosody, but also conformed well to other previous studies. They can therefore be integrated to form a meaningful Mandarin prosody hierarchy.
Hierarchical prosody modeling for Mandarin spontaneous speech
(2019-04-01)
Lin, Cheng-Hsien
;
You, Chung-Long
;
Chiang, Chen-Yu
;
Wang, Yih-Ru
;
Chen, Sin-Horng
In this paper, a hierarchical prosody model (HPM)-based method for Mandarin spontaneous speech is proposed. First, an HPM is designed for describing relations among acoustic features of utterances, linguistic features of texts, and prosodic tags representing the underlying hierarchical prosodic structures of utterances. Subsequently, a sequential optimization algorithm is employed to train the HPM based on a large conversational speech corpus, the Mandarin Conversational Dialogue Corpus (MCDC), which features orthographic transcriptions and prosodic event annotations. In this unsupervised training method, all utterances of the MCDC are labeled with two types of prosodic tags, namely, break and prosodic states, automatically and simultaneously. After training, the HPM parameters are examined to identify critical prosodic properties of Mandarin spontaneous speech, which are then compared with their counterparts in the read-speech HPM. The prosodic tags on the studied utterances enable mapping of various prosodic events onto the hierarchical prosodic structures of the utterances. Prosodic analyses of some disfluent events are conducted using the prosodic tags affixed to the MCDC. Finally, an application of the HPM to assist in Mandarin spontaneous-speech recognition is discussed. Significant relative error rate reductions of 9.0%, 9.2%, 15.6%, and 7.3% are obtained for base-syllable, character, tone, and word recognition, respectively. (C) 2019 Acoustical Society of America.
Hierarchical Prosody Modeling of English Speech and its Application to TTS
(2014-01-01)
Tsai, Chung-Yao
;
Kuo, Chin-Kuan
;
Wang, Yih-Ru
;
Chen, Sin-Horng
;
Liao, I-Bin
In this paper, a hierarchical prosody modeling approach for English speech is proposed. It is an extended version of the HPM approach proposed previously for Mandarin speech. It first designs a syllable-based, statistical prosodic model to describe various relationships of prosodic-acoustic features of the speech signal, linguistic features of the associated text, and prosodic tags representing the underlining prosody structure of the speech. It then employs a prosody labeling and modeling algorithm to estimate the model parameters and label the prosodic tags of all training utterances simultaneously from a prosody-unlabeled speech corpus. Experimental results on a corpus containing many paragraphic utterances of a female English-majored Chinese speaker show that the inferred parameters of the model are all meaningful. We then use the trained model to generate prosodic information for a TTS system. An informal listening test shows that the synthetic speech sounds quite natural.
Improvements on Punctuation Generation Inspired Linguistic Features for Mandarin Prosody Generation
(2016-01-01)
Chiang, Chen-Yu
;
Hung, Yu-Ping
;
Liou, Guan-Ting
;
Wang, Yih-Ru
;
電機工程學系
This paper proposes two types of machine-extracted linguistic features from unlimited text input for Mandarin prosody generation. One is the improved punctuation confidence (iPC) which is a modified version of the previously proposed punctuation confidence that represents likelihood of inserting major punctuation marks (PMs) at word boundaries. Another is the quotation confidence (QC) which measures likelihood of a word string to be quoted as a meaningful or emphasized unit. Since major PMs are highly correlated with prosodic breaks, and a quoted Chinese word string plays an important role in human language understanding, the two features potentially could provide useful information for prosody generation. The idea is realized by employing conditional random field-based models to predict major PMs, quoted word string structures, and their associated confidences, i.e. iPC and QC. Then the predicted confidences are combined with traditional linguistic features to predict prosodic-acoustic features. Both objective and subjective tests showed that the prosody generation with the proposed linguistic features performed better than the ones without the proposed features.
Knowledge Integration for Improving Performance in LVCSR
(2013-01-01)
Chiang, Chen-Yu
;
Siniscalchi, Sabato Marco
;
Chen, Sin-Horng
;
Lee, Chin-Hui
;
電機工程學系
This paper presents a knowledge integration framework to improve performance in large vocabulary continuous speech recognition. Two types of knowledge sources, manner attribute and prosodic structure, are incorporated. For manner of articulation, six attribute detectors trained with an American English corpus (WSJO) are utilized to rescore hypothesized phones in word lattices obtained by a baseline ASR system. For the prosodic structure, models trained with an unsupervised joint prosody labeling and modeling (PLM) technique using WSJO are used in lattice rescoring. Experimental results on the American English WSJ word recognition task of the Nov92 test set show that the proposed approach significantly outperforms the baseline system that does not use articulatory and prosodic information. The results also demonstrate the effectiveness and usefulness of the PLM technique in constructing prosodic models for American English ASR.
Latent prosody model of continuous mandarin speech
(2007)
Chiang, Chen-Yu
;
Wang, Xiao-Dong
;
Liao, Yuan-Fu
;
Wang, Yih-Ru
;
Chen, Sin-Horng
The major difficulty of prosody modeling and automatic tone recognition of continuous Mandarin speech is the complex interaction of tones and prosody/intonation on F0 contours. In this study, we propose a latent prosody model (LPM) aiming to jointly model the affections of tone and prosody state on F0. The main purposes are twofold including (1) automatic prosody state labeling and (2) improving tone recognition accuracy. The basic idea is to introduce latent prosody state variables into an additive statistic model of F0 which already considers the affecting factors of tone and speaker. Experiments on the Tree-Bank corpus showed that LPM not only gave meaningful prosody state labeling results but also improved the average tone recognition rate from 80.86% of a multi-layer perceptron (MLP) baseline to 82.55%.
Modeling of Speaking Rate Influences on Mandarin Speech Prosody and Its Application to Speaking Rate-controlled TTS
(2014-07-01)
Chen, Sin-Horng
;
Hsieh, Chiao-Hua
;
Chiang, Chen-Yu
;
Hsiao, Hsi-Chun
;
Wang, Yih-Ru
A new data-driven approach to building a speaking rate-dependent hierarchical prosodic model (SR-HPM), directly from a large prosody-unlabeled speech database containing utterances of various speaking rates, to describe the influences of speaking rate on Mandarin speech prosody is proposed. It is an extended version of the existing HPM model which contains 12 sub-models to describe various relationships of prosodic-acoustic features of speech signal, linguistic features of the associated text, and prosodic tags representing the prosodic structure of speech. Two main modifications are suggested. One is designing proper normalization functions from the statistics of the whole database to compensate the influences of speaking rate on all prosodic-acoustic features. Another is modifying the HPM training to let its parameters be speaking-rate dependent. Experimental results on a large Mandarin read speech corpus showed that the parameters of the SR-HPM together with these feature normalization functions interpreted the effects of speaking rate on Mandarin speech prosody very well. An application of the SR-HPM to design and implement a speaking rate-controlled Mandarin TTS system is demonstrated. The system can generate natural synthetic speech for any given speaking rate in a wide range of 3.4-6.8 syllables/sec. Two subjective tests, MOS and preference test, were conducted to compare the proposed system with the popular HTS system. The MOS scores of the proposed system were in the range of 3.58-3.83 for eight different speaking rates, while they were in 3.09-3.43 for HTS. Besides, the proposed system had higher preference scores (49.8%-79.6%) than those (9.8%-30.7%) of HTS. This confirmed the effectiveness of the speaking rate control method of the proposed TTS system.
On Finding Word-Level Break-Type Formation Rules For Mandarin Read Speech
(2015)
Kung, Fu-Ja
;
Lee, Pa-Hwa
;
Wang, Yih-Ru
;
Chen, Sin-Horng
;
Chiang, Chen-Yu
This paper presents a study on exploring word-level break-type formation rules for Mandarin read speech. A 4-layer hierarchical structure with seven break types is adopted to represent the prosody of utterance. The work is based on the break-type tags labeled on a large read-speech database by the prosody labeling and modeling algorithm (PLM) proposed previously. Occurrence frequencies of seven break types for pre- and post-boundaries of several types of function words are calculated and taken as the inferred statistical break-type formation rules. Linguistic interpretations of the most likely break types occurred at pre- and post-boundaries of each function word are discussed. Some exceptions that deviate from the most likely break types are also examined.