【作者】 唐发明；

【导师】 陈绵云； 王仲东；

【作者基本信息】 华中科技大学 ， 控制理论与控制工程， 2005， 博士

【摘要】 传统的统计学研究的是假定样本数目趋于无穷大时的渐近理论,现有的机器学习方法大多是基于这个假设。然而在实际的问题中,样本数往往是有限的。现有的基于传统统计学的学习方法在有限样本的情况下难以取得理想的效果。统计学习理论是在有限样本情况下新建立起来的统计学理论体系,为人们系统地研究小样本情况下机器学习问题提供了有力的理论基础。支持向量机是在统计学习理论基础上开发出来的一种新的、非常有效的机器学习新方法。它较好地解决了以往困扰很多学习方法的小样本、非线性、过学习、高维数、局部极小点等实际问题,具有很强的推广能力。目前,统计学习理论和支持向量机作为小样本学习的最佳理论,开始受到越来越广泛的重视,正在成为人工智能和机器学习领域新的研究热点。本论文研究的主要内容包括以下几个方面:支持向量机算法、多输出支持向量回归、多类支持向量机分类、支持向量机算法以及支持向量分类和支持向量回归的应用。论文主要研究工作有: 1.标准的支持向量机算法,其最优分类超平面与正负两类是等距的,在处理一些特殊分类问题时,会存在不足。在对支持向量机算法进行研究和分析之后,提出了基于不等距分类超平面的支持向量机算法,并对算法进行了简要的理论推导和仿真。2.支持向量回归算法是针对单输出回归问题提出的,对于多输出系统的回归估计,传统的方法是对各个输出独立地建立单输出支持向量回归模型,其缺点是忽略了各个输出之间实际存在的联系,并且不能保证各输出误差和最小化。针对这些问题,通过增加误差和约束条件,且在同一个优化公式中考虑所有输出的回归估计,提出了一种多输出支持向量回归算法,从而可以考虑到各个输出之间的联系,并能提高整个回归模型的回归估计精度。3.支持向量机的训练算法需要解决一个大的二次规划最优化问题,传统的二次规划数学算法在求解大数据的二次规划问题时,需要巨大的内存空间,所以并不能更多还原

【Abstract】 Traditional statistics is based on assumption that samples are infinite, so are most of current machines learning methods. However, in many practical cases, samples are limited. Most of existing methods based on traditional statistical theory may not work well for the situation of limited samples. Statistical Learning Theory (SLT) is a new statistical theory framework established from finite samples. SLT provides a powerful theory fundament to solve machine learning problems with small samples. Support Vector Machine (SVM) is a novel powerful machine learning method developed in the framework of SLT. SVM solves practical problems such as small samples, nonlinearity, over learning, high dimension and local minima, which exit in most of learning methods, and has high generalization. Currently, being the optimal learning theory for small samples, SLT and SVM is attracting more and more researcher and becoming a new active area in the field of artificial intelligent and machine learning. This dissertation studies multi-output Support Vector Regression (SVR), multiclass SVM, support vector machines algorithm, and applications of SVM and SVR. The main results of the dissertation are as follows: 1.After the original formulation of the standard SVM is studied and analyzed, a new learning algorithm, Non-equidistant Margin Hyperplane SVM (NM-SVM), is proposed to handle some frequent special cases in pattern classification and recognition. The separating hyperplane of NM-SVM is not equidistant from the closest positive examples and the closest negative examples. 2. Support vector regression builds a model of a process that depends on a set of factors. It is traditionally used with only one output, and the multi-output case is then dealt with by modeling each output independently of the others, which means that advantage cannot be taken of the correlations that may exist between outputs. The dissertation extends SVR to multi-output systems by considering all output in one optimization formulation. This will make it possible to take advantage of the possible correlations between the outputs to improve the quality of the predictions provided by the model. 3.For the study of SVM training algorithm, training a Support Vector Machine requires the solution of a very large Quadratic Programming (QP) optimization problem. Traditional optimization methods cannot be directly applied due to memory restrictions. Up to now, several approaches exist for circumventing the above shortcomings and work well. The dissertation explores the possibility of using Particle Swarm Optimization (PSO) algorithm for SVM training. 4.For lager-scale samples, based on Rough Sets (RS) theory and SVM, an integrated method of classification named RS-SVM is presented. Using the knowledge reduction algorithm of RS theory, the method can eliminate redundant condition attributes and conflicting samples from the working sample sets, and evaluates significance of the reduced condition attributes. Eliminating the redundant condition attributes can cut down the sample space dimension of SVM, and SVM will generalize well. Deleting the conflicting samples can reduce the count of working samples, and shorten the training time of SVM. 5.The methods constructing and combining several binary SVMs with a binary tree can solve multiclass problems, and resolve the unclassifiable regions that exist in the conventional multiclass SVM. Since some existing methods based on binary tree didn’t use any effective constructing algorithm of binary tree, several improved multiclass SVM methods based on binary tree are proposed by using class distance and class covering of clustering. 6.The study of SVM and SVR application. An approach based on voice recognition using support vector machine (SVM) is proposed for stored-product insect recognition. Adaline adaptive noise canceller is used as voice preprocessing unit, feature vectors are extracted from audio signals preprocessed of known insect samples, and used to train multiply SVMs for insect recognition. The operation is very convenient, only requiring the insect’s audio signals collected by sensors without insect images or samples. Focusing on the difficulty of scattered data approximation, two methods of surface approximation based on SVR are presented, which have been applied to reconstruct temperature fields of large granaries.更多还原