【作者】 刘万里；

【导师】 刘三阳；

【作者基本信息】 西安电子科技大学 ， 应用数学， 2008， 博士

【摘要】 统计学习理论为研究小样本情况下机器学习问题提供了有力的理论基础。它使用结构风险最小化原则,综合了统计学习、机器学习和神经网络等方面技术,在最小化经验风险的同时,有效地提高了算法泛化能力。支持向量机是在该理论体系下产生的一种新的、非常有力的机器学习方法。它较好地解决了以往困扰很多学习方法的小样本、非线性、过学习、高维数、局部极小点等实际问题,具有良好的潜在应用价值和发展前景。目前,统计学习理论和支持向量机作为小样本学习的最佳理论,受到越来越广泛的重视,成为人工智能和机器学习领域新的研究热点。本文综述了支持向量机的研究现状,针对目前存在的几个问题:不平衡调整问题、大样本减样和除噪问题、两种支持向量算法即Support Vector Machines(或SVM)与Support Vector Domain Description(或SVDD)的结合问题、核心向量的性能和应用问题以及SVDD算法在不确定型决策中的应用等问题进行研究。本文的主要工作如下:1.研究了不平衡支持向量机的调整方法。不平衡数据集的学习问题被公认为机器学习领域的难题之一,其困难主要来自于不平衡数据集本身的特点:例如,样本数量少的类其样本不足,样本的分布并不能很好地反映整个类的实际分布。因此标准支持向量机在应用于不平衡数据集时,往往把少数类的样本错分,尽管整体的分类精度比较高,但数量少的类的分类精度非常低。本文针对支持向量机中两类不平衡数据的分离超平面提出一种调整算法。该算法根据样本投影分布和样本容量所提供的信息给出两类惩罚因子比例,从而得到一个新的分离超平面。实验结果显示了该方法的良好性能。2.研究了样本的减样和除噪问题。在使用支持向量机分类时,存在以下两个问题:一是当两类训练样本中存在野点(噪点)时,分类的精度较低;二是对大规模样本集,所占用的内存空间较大,训练时所需时间较长。针对以上问题,我们分别基于欧氏距离和核距离,根据概率论的知识定位分析了野点(噪点)及多余样本点的一般比例情况,给出一种减样方法。实验结果表明该方法与标准SVM相比,能保持或提高分类精度;对于大样本来说不仅能保持精度不减,而且还能较大地提高分类速度,具有较强的实用性。3.将支持向量机与支持向量域描述结合起来,提出一种分类器。支持向量机在学习阶段,所有样本参加训练,因此需要较大的内存空间和较长的训练时间;而支持向量域分类器(Support Vector Domain Classifier,或SVDC),只训练一类样本点,因此,分类时训练时间较短,但精度较低。为了减少SVM的训练时间,提高SVDC的精度,我们建立一种新的分离超平面,即基于支持向量域的分离超平面。该算法是从整体上考虑分类信息,实现了SVDD和SVM的结合。实验结果显示了该方法的有效性。4.提出了核心向量的重要概念,并把核心向量集应用于支持向量机的改进。为了有效提取样本类信息,基于SVDD算法依据参数选择,剔除支持向量,找核心向量。为了研究核心向量的性能,分别使用线性以及径向基核函数对样本数据进行描述,从理论上证明了核心向量在样本集中,在对应参数下具有最大密度值,因而得出核心向量包含最大信息量的重要结论。因此,核心向量不仅可以作为样本的期望点估计,而且可以提炼控制向量,改善SVM的分类效果。5.将支持向量域描述算法应用在不确定型群决策中。分别研究了模糊判断和区间判断两种逆判问题。对于模糊判断的逆判问题,是以模糊互反判断为准,使用SVDD算法,寻找公共信息,根据信息的贡献量决定专家的评判权重。对于区间判断的逆判问题,通过对区间判断矩阵的点向量分解,采用径向基核函数,使用SVDD算法提取群体的公共信息,同样根据信息贡献量决定专家的权重。该研究充分利用了SVDD的描绘功能,抓住主要信息,比较适合于不确定型的群决策问题。它不仅开拓了SVDD的研究领域,而且为不确定型群决策的研究提供了有效的技术。更多还原

【Abstract】 Statistical Learning Theory (SLT) provides a powerful theoretical basis for machine learning in studying small sample. By using the Structural Risk Minimization to integrate techniques of the statistical learning, machine learning and neural network etc and efficiently improve generalization ability of algorithm under Empirical Risk Minimization. Support Vector Machine (or SVM) is a new and very powerful machine learning method generated under such a theoretical system. SVM is of good potential applicability and development prospect, for it can solve well many practical problems that puzzle many existing learning methods, such as small sample, nonlinearity, over learning, high dimensional number and local minimal point etc. Currently, SLT and SVM, as the best theory for small sample learning, has receiving wide attention, and becoming a new research hotspot in machine learning and artificial intelligence. In this paper, we firstly illustrate existing algorithm and application researches on SVM. Secondly, we study in detail some existing problems concerned now, such as the unbalanced problems, de-sampling and de-noising problems, combination problems of two kinds of support vector algorithms SVM and Support Vector Domain Description(or SVDD), performances and applications of core vectors and application problem of SVDD algorithm to uncertain group decision etc. The main works in this paper include contents as follows:1. Study the adjustment method for unbalanced support vector machines. The learning of unbalanced data set is regarded as one of the open difficult problems in the area of machine learning, where the difficulty comes mainly from the feature of the unbalanced data itself. For instance, the class with few samples lacks samples, which can not reflect well the practical distribution of whole class. Therefore, the standard SVM often makes mistakes when separating the samples from the class with few samples in application to unbalanced data set. This results in the fact that the class with few samples has low precision though the whole classification precision is high. This paper proposes an adjustment algorithm for the separating hyperplane of two classes of unbalanced data in SVM. We use the information provided by sample projection distribution and sample size to determine the ratio of two classes of penalty factors and then obtain a new separating hyperplane. Experiment results show the good performances of the method.2. Study the de-sampling and de-noising problems. There exist two problems in using SVM to perform classifications as follows: One is the low classification precision due to the existence of outliers (or noises) in sample set; another is long training time for a large scale sample set due to requiring great memory space. For above problems, according to probability theory we analyze in location general proportions of outliers (or noises) and surplus samples, and propose a de-sampling method based on Euclid distance and kernel distance, respectively. The experiments show that the proposed method can keep or improve classification precision compared with SVM generally; for large sample, the method can not only keep precision, but also improve classification speed greatly, which is of strong practicality.3. A kind of classifier is presented by combining SVM and SVDD. Since all the samples participate in training for using SVM, it needs great memory space and long training time; while SVDC (Support Vector Domain Classifier) is low in classification precision though it needs a relatively little time in classification. To reduce training time of SVM and to speed SVDC, we build a new separating hyperplane, namely separating hyperplane based on SVDD. The algorithm considers classification information as a whole, and implements the combination of SVM and SVDD. Experiments show the efficiency of the method.4. Propose the concept of core vector, and apply core vector set to improve SVM. To extract sample information efficiently, we delete all the support vectors and find core vectors by choosing parameters based on SVDD. Linear kernel and radial basis kernel function are applied to describe sample data respectively to study the performance of core vector. It is proved theoretically that a core vector is of maximal density with respect to corresponding parameters in given sample set, hence we obtain the important conclusion that a core vector contains maximal information in the sample set. Therefore, any core vector can be evaluated the expectation point of a sample set, further more, core vector set can be trained to find control vector to improve SVM.5. Apply SVDD to uncertain group decision. Respectively Study the two kinds of inverse judgment problems of fuzzy judgment and interval judgment. For the fuzzy judgment, we choose fuzzy reciprocal judgment as the standard to determine expert weight according to the informational contribution by using SVDD to find common information. For the interval judgment, expert weights are determined in terms of the information contribution by using SVDD to extract group information, in which interval judgment matrices are decomposed as point vectors, and radial basis kernel function is applied. This research makes full use of the description performance of SVDD, holds main information, which is well suitable for uncertain group decision problems. This method not only enlarges the research area of SVDD, but also provides an efficiently technique for studying uncertain decision.更多还原