【作者】 李济洪；

【导师】 梁吉业； 刘开瑛；

【作者基本信息】 山西大学 ， 计算机应用技术， 2010， 博士

【摘要】 为了给大规模的汉语框架语义资源的构建提供一个自动标注工具,本文基于山西大学自主开发的汉语框架语义知识库(CFN),在给定句子中的一个目标词及目标词所属框架情形下,将其语义角色(框架元素)的自动标注问题通过IOB策略转化为整个句子上的词序列标注问题,使用条件随机场模型(CRF),采用统计学中的正交表实验方案,研究了汉语框架语义角色的自动标注模型。本文的全部实验语料使用的是现有的CFN中选出25个框架的6692个例句。将语料均匀分为4份,分3组作2-fold交叉验证,以3组交叉验证的平均F1-值作为系统性能评价指标。本文给出了系统性能评价指标的方差估计,以及两个标注系统性能差异的显著性检验方法。本文以词为基本标注单元,将标注步骤分为1)边界识别、2)角色分类、3)后处理三个步骤。分别采用了边界识别与角色分类一起进行,以及先边界识别,再角色分类两种标注策略。在后处理步骤上,对输出的标注序列要求在整个句子上满足IOB序列合法性约束,并以所有合法序列中概率最大的序列作为最后的标注输出。本文总共提取了26个特征,对每个特征设定若干可选的窗口,组合构成CRF模型的各种特征模板。为了选出较好的特征模板,本文基于统计学中的正交表给出了一种模板选优方法,并采用三种方案进行了实验。方案一：基于11个词层面特征,其特征包括词、词性、词相对于目标词的位置、目标词等,实验选用正交表L32(49×24)；方案二：基于全部的26个特征,包括11个词层面的特征和基本块的句法标记、结构标记等15个特征,选用正交表L54(21×325)。其中基本块特征提取使用的是清华大学周强的自动分析器；方案三：分批正交表实验,即先用正交表L32(49×24),在11个词层面特征选出的最好模板基础上,再加入15个基本块特征,使用正交表L54(21×325),通过适当选择正交表的水平以确保性能不低于前一批实验结果。对每种方案的实验进行了详细分析。本文对正交表模板选优方法与传统的基于贪心算法的方法进行了比较。也比较了本文的基于词序列标注方法和采用完全句法分析树的方法,也对选用不同标注模型,如支持向量机(SVM)模型和最大熵模型的实验结果进行了比较。实验结果表明：(1)在基于11个词层面特征上(方案一),最好结果(平均F1-值)达到61.61%,比基于完全句法分析树,将角色标注看做句法成分的分类问题的结果显著高。与传统的贪心算法特征选择方法比较,本文的正交表模板选择方法与其在标注性能上没有显著差异,但正交表方法的计算更简单,且在通用模板的选择上更适宜。(2)加入15个基本块特征(方案二)可以显著提高标注模型的性能。这类特征主要对角色分类有显著作用,对角色的边界识别作用不显著。(3)分批正交表实验(方案三)比实验方案二在性能上有显著提高。(4)每个框架训练一个模型,边界识别与角色分类一起进行,与先边界识别,再角色分类两个步骤在标注性能上没有显著差别,但由前者得到的标注性能有较小的方差。(5)基于条件随机场标注模型(CRF)与基于支持向量机(SVM)模型的标注结果没有显著差异,但显著好于基于最大熵(ME)模型的标注结果。(6)在全部25个框架的所有实验中,语义角色边界识别最好的结果(平均F1-值)为71.68%；在给定语义角色边界下,角色分类的最好结果(平均精确率)为84.08%；在给定句子中的目标词以及目标词所属的框架情况下,最好结果(平均F1-值)达到63.26%.本文的创新之处主要是首次系统地研究汉语框架语义角色的自动标注模型,给出了一种采用正交表的模板选优方法,在计算上,该方法比基于贪心算法的模板选择方法更简单。对于一般的序列标注中的特征选择问题,本文的正交表特征模板选优法也适用。在标注性能上,本文的结果优于基于句法分析树的语义角色标注的结果。更多还原

【Abstract】 In order to provide an automatic labeling tools for developing a large-scale resource of Chinese FrameNet, based on the semantic knowledge base of Chinese FrameNet(CFN) self-developed by Shanxi University, this paper study the semantic role(frame element) automatic labeling for given a target word in a sentence and its known frame name.The task of semantic role automatic labeling is conversed into sequential tagging problem at word-level within the entire sentence by IOB strategy. The conditional random fields model (CRF), and the orthogonal array experiment in statistics are employed.The experimental corpus in the paper, selected from current CFN corpus, include 6692 annotated sentences of 25 frames. The corpus is uniformly divided into four parts. Therefore,2-fold cross validation experimrnt can be engaged in the three different groups. We take the cross-validation average F1-value on three groups as the system performance measure. This paper presents the estimator of variance of the system performance measure as well as the significant test method for two different labeling system.Using word as the basic tagging unit, the tagging procedure is divided into three steps:1)identification,2) classification,3)post-processing. The two IOB strategies are adopted, one is conjunction of identifying and classifying, and the other is firstly identifying then classifying. In post-processing step, the final output of the sequential labels is choosen by the largest probability of all labels with a logical IOB sequence in entire sentence.This paper totally extract 26 features, and for each feature set some optional windows. The combination of various features with different windows form the feature templates of CRF model.The best template selection method is given based on orthogonal array in statistics.The three schemes of experiment are adopted. Scheme I: based on 11 word-level features, including word, POS of word, position of word relative to the target word, and the target word etc, the experiment is arranged in orthogonal array L32 (49×24); SchemeⅡ:based on all 26 features, including 11 word-level features and 15 base chunk features about shallow syntax, arranged in orthogonal array L54 (21×325).The base chunk features are automaticly extracted by automatic analyzer of Tsinghua University Zhou; Scheme III:batch orthogonal array experiment, i.e. first using the orthogonal array L32 (49 x 24) on 11 words-level features acquire the best templates, and then join the 15 base chunk features into orthogonal array L54 (21×325). Through the appropriate selection of the levels in orthogonal array L54 (21×325), it ensure that performance measure is not lower than the previous results.Each experiment conduct a detailed analysis.The paper compares the template selection method upon orthogonal array with traditional greedy algorithm, and compares the sequential tagging method at word-level with method upon syntactic parses tree, and also compares with different tagging models, such as support vector machine (SVM) model and maximum entropy(ME) model.Experimental results show:1)Based on 11 word-level features (schemeⅠ), the best average F1-value reach 61.61%.The result is significantly higher than method upon syntactic parses tree which regard the role labeling as a classification of syntactic constituents.On template selection, comparing with traditional greedy algorithm, the two methods dose not have significant differences, but orthogonal array method is relatively sample in calculation, and has some advantage in choice of the general template for any frame.2) Adding the 15 base chunk features (schemeⅡ) can significantly improve the performance. These features mainly have significant effects on role classification, not significant on role identification.3) Batch orthogonal array experiment (schemeⅢ) has a significant higher performance than the schemeⅡ.4) Two IOB stratagies, i.e. each frame training a model with role identification and classification together, or the firstly identifying then classifying, are no significant difference in performance, but the former has less variance of performance measure.5) The experimental results have no significant difference in annotation upon the conditional random fields model (CRF) and upon support vector machine (SVM) model, but CRF model is significantly better than a maximum entropy (ME) model. 6) In total 25 frames of all experiments, for semantic role identification the best average F1-value reach 71.68%. Given semantic role boundary, for the role classification the best average accuracy achieve 84.08%.Given target word in the sentence and its known frame name, the best average F1-value obtain 63.26%. Main innovations in the paper is the first systematic studies of Chinese FrameNet semantic role of automatic labeling, and proposes to adopt orthogonal array to select template. The method is more simple than the greedy algorithm template selection method. For feature selection problems in general sequential annotation, the orthogonal array feature template selection method is also applicable.For labeling performance, this paper’s results is better than those based on syntactic parses tree.更多还原