【作者】 黄浩；

【导师】 朱杰；

【作者基本信息】 上海交通大学 ， 电路与系统， 2008， 博士

【摘要】 汉语是一种带调语言,声调在汉语语音中具有非常重要的意义。相同的声母和韵母构成的音节随声调的不同而具有完全不同的意义,对应着不同的方块字。特别是当语言模型上下文缺失的情况下,声调在汉语普通话中承担着重要的构字辨义的作用。因此,将声调信息应用于汉语普通话的语音识别系统当中,将会有效地提高识别系统的性能。近年来,基于区分性原理的机器学习方法已成为模式识别特别是自动语音识别研究领域的热门研究方向之一。利用区分性原理在模型训练以及特征优化方面提出的一些方法,在小规模的分类任务以及大词汇连续语音识别系统中都显示了优越的性能。本文以汉语普通话大词汇连续语音识别系统为应用背景,旨在根据汉语声调发音的特点,从区分性原理的角度来讨论汉语语音的声调建模以及声学建模中的声调信息利用问题。回顾了语音识别技术的发展历史,介绍了声调在汉语语音识别中的作用,系统性描述了区分性训练准则以及应用比较成功的区分性模型与方法,并由此提出了不同模型下改进声调识别性能以及利用声调信息改进声学建模性能的区分性方法,为汉语语音识别中声调问题的解决提供了新的研究思路。这些方法可概括如下:首先从区分性训练的角度研究了基于隐马尔可夫模型的声调建模方法。为了提高汉语声调识别率,从模型空间中利用区分性训练的参数更新方法对模型参数进行重估。在汉语普通话中,由于协同发音的存在,连续语音的声调识别较孤立语音声调识别复杂。声调协同发音体现为对当前音节的声调感知高度依赖于上下文声调。基于上述原理,在特征空间的区分性训练方面,提出区分性声调特征提取方法。该方法根据区分性线性特征补偿的思想,根据区分性目标函数训练得到的线性变换,将上下文基音频率进行映射并补偿至当前音节基音频率特征。实验表明区分性声调特征提取显著提高了声调识别率,声调特征提取基础上的模型参数联合训练进一步提高了声调识别的性能。并从识别率以及特征变换参数的角度进行分析,说明特征提取方法与传统声调特征归一化的本质不同。条件随机场（conditional random fields,CRFs）是近年来在自然语言处理领域使用的成功的数学模型。论文采用条件随机场的一种扩展-隐条件随机场对汉语语音声调进行显式建模,提出一种对传统动态特征的扩展-广义动态特征来更好地捕捉基音频率曲线的动态变化。声调识别实验表明采用相同的特征和结构,隐条件随机场较最大似然训练的隐马尔可夫模型声调识别率有显著提高,加入广义动态特征之后声调识别率有一致性改进。隐条件随机场区别于HMM的重要特性在于无须对特征采用统一的利用方式,这使得该模型非常适合于处理汉语语音中基音频率在浊音段连续、清音段不连续的声学现象。提出了隐条件随机场对断续F₀进行直接建模的隐式声调建模方法,带调音节分类实验表明在隐条件随机场下对断续基音频率序列的直接建模较使用清音段平滑F₀特征的识别率有明显的提高,该实验结果对利用隐条件随机场在大词汇连续语音识别系统下,声学建模中对断续基音频率序列的直接建模提供初步的实验依据。讨论了大间隔（large margin）高斯混合模型的声调建模方法,根据大间隔区分性训练准则对模型参数进行区分性训练。对于参数的更新,针对基于Quasi-Newton梯度下降方法收敛速度慢的缺点,提出一种扩展Baum Welch（extended Baum Welch,EBW）形式的大间隔高斯混合模型的参数更新方法,该方法借助弱辅助函数的原理对高斯参数进行优化,实验表明该方法与基于Quasi-Newton的梯度方法相比只需要几次迭代就可以达到相同甚至更高的识别结果。另一方面,对于基于段特征的高斯混合模型,选取什么样的特征能够达到更好的识别率往往需要反复试凑得到最优的识别结果。本文利用线性判别分析方法来对声调特征进行降维,通过线性判别分析得到更加适合于声调区分的段特征,声调识别实验上表明在维数缩减特征基础上的高斯混合声调模型,较传统的重叠双音调高斯混合模型在声调识别性能方面有明显的提高,这表明线性判别分析获得的特征要优于人工选取的超音段声调特征。最后讨论了一种区分性模型权重的训练方法,将显式训练的声调模型加入大词汇量连续语音识别系统中来提高汉语连续语音识别率。该方法根据最小音子错误（minimumphone error,MPE）准则,区分性地训练模型相关的概率权重。利用这些权重对传统基于传统谱特征的HMM模型概率以及声调模型概率进行加权,通过调整模型之间的作用程度提高系统识别率。推导了利用扩展Baum-Welch算法的权重更新公式。根据汉语上下文相关声学建模的特点,由此提出了带调音节相关、韵母模型相关、模型组合相关和整词相关的模型权重策略。对不同模型权重组合策略进行了评估。在实验中,由于训练语料的有限性,各种权重策略随着可训练参数增多,容易受到过训练的影响。具体表现在:对训练数据目标函数增大,但是测试数据识别率反而下降。提出利用权重之间的平滑的方法来克服权重训练过拟合的问题。分别通过大词汇连续语音的带调音节输出和汉字输出两种识别任务来验证区分性模型权重训练的性能。实验结果表明在两种识别任务上,使用区分性的模型权重较使用全局模型权重显著地降低了误识率,这表明了区分性模型权重对提高声调模型集成性能的有效性。更多还原

【Abstract】 Chinese is a tonal language and tones are of fundamental importance to Mandarin speech recognition.Tones can be as important as phonemes when contextual information is limited or missing.Utilization of tone information to improve performance in Mandarin speech recognition has been widely studied in recent research.Significant improvements have been achieved on various scale speech recognition tasks in both clean and noisy en-vironment.In recent years,discriminative machine learning method has been one of the hottest direction in pattern recognition and especially in automatic speech recognition research. Several model parameter estimation and feature extraction methods based on discriminative principles have shown to be successful in both classification and continuous speech recognition tasks.This dissertation aims at solving tone problems which are unique in Mandarin speech recognition,and hence improving the performance of large vocabulary speech recognition system,by taking advantage of the recently proposed discriminative training criteria,models and methods.An systematic overview of the discriminative training criteria,models and correspondingly derived discriminative techniques is provided.Several discriminative ap-proaches to tone problem solving in Mandarin speech recognition are proposed,which can be summarized as follows:Traditional tone modeling based on hidden Markov models is firstly investigated from a new,discriminative training perspective.To improve tone recognition accuracy,discriminative training in both the model space and the feature space is proposed.In the model space, the model parameters are trained by using an objective function termed as minimum tone error,which is a smooth approximation of tone recognition accuracy.In the feature space, based on the fact that Mandarin tones are greatly influenced by the context tones,a tonal feature extraction method for HMM based tone modeling is inroduced.The method uses linear transforms to project F₀（fundamental frequency） features of neighboring syllables as compensations,and adds them to original F₀ features of current syllable.The trans-forms are discriminatively trained according to the same objective function.Experiments show the new tonal features achieve significant tone recognition improvement,compared with baseline using maximum likelihood trained HMM on normal F₀ features.The overall discriminative training on the new features introduces further improvement.It is also found the DTFE method brings additional improvements to traditional F₀ normalization technique.Conditional random fields（CRFs） should be one of the most successfully applied mathematical models in the research field of natural language processing.Tone modeling using the extension of CRFs,hidden conditional random fields（HCRFs） is explored.To better capture the F₀ contour,a generalized dynamic feature is introduced.Experimental results on tone recognition have shown the HCRFs based tone model outperform both the maximum likelihood and discriminatively trained HMM tone models when using the same model structure and observations.The generalized dynamic features introduces consistent gain over the normal dynamic features.It has been pointed out that a key advantage of CRFs or HCRFs is their great flexibility to include a wide variety of arbitrary,non-independent features of the input.In Mandarin speech recognition,unlike the spectral features,no F₀ is observed in unvoiced region.The discontinuity between voiced and unvoiced segments has traditionally made tone modeling difficult.Thus the model of HCRFs is more suitable for dealing with this special phenomenon.A preliminary evaluation of HCRFs for embedded tone modeling in Mandarin speech recognition is presented.Experimental results on tonal syllable classification tasks have shown HCRFs on discontinuous F₀ features is better than using smooth F₀ feature.The large margin methods have attracted a lot of research attentions in the field of machine learning.The fact that it is the margin in classification rather than the raw training error that matters has become a key tool in recent years when dealing with discriminative classifiers.We build segmental feature based tone classifier on Gaussian mixture model.A discriminative objective function termed as large margin criterion is adopted to train Gaus-sian mixture parameters.A novel model parameter updating equation using the weak-sense auxiliary function is formulated to obtain an efficient iterative training approach of the Gaussian parameters.Linear discriminant analysis feature reduction algorithm is applied to extraction critical segmental feature of the tones.Experimental results on tone recog-nition tasks have shown the margin based discriminative criterion is better than empirical risk based objective function.The proposed Extended Baum Welch（EBW） like updating algorithm have achieve a comparable performance when using only several iterations.The GMMs trained on LDA derived features are better than the previously proposed overlapped di-tone Gaussian mixture models.When integrating explicitly trained tone models into lattice based rescoring,a discriminative framework of tone model integration is proposed.The method is to use model dependent weights to scale probabilities from various models:the HMM based on spectral features and tone models based on F₀ related tonal features.The weights are discriminatively trained by the minimum phone error（MPE） criterion and update equation of model weights based on the EBW algorithm is derived.Various schemes of model weight combination such as tonal syllable dependent,final model dependent,model combination dependent and word dependent are evaluated and a smoothing technique is introduced to make training robust to over fitting.The proposed method is evaluated on tonal syllable output and character output speech recognition tasks.Experiments results show the proposed method has obtained significant relative error reduction than global weight on the two tasks due to a better interpolation of the given models.更多还原