【作者】 刘叶青；

【导师】 刘三阳；

【作者基本信息】 西安电子科技大学 ， 应用数学， 2009， 博士

【摘要】 支持向量机成为一种主要的机器学习技术已经有十多年了,然而它的大部分学习算法都是在对偶空间针对其对偶问题提出的。近年来的研究表明,直接在原始空间对支持向量机的原始问题进行求解也是训练支持向量机的一种有效途径。随着人们在原始空间对支持向量机研究的深入,实际应用中碰到的各种问题也开始在原始空间进行求解,如半监督学习问题等。但总体来说,支持向量机在原始空间中的研究还不是很多,也不够完善。因此,本文重点研究了原始空间中支持向量机分类算法的以下四个问题。1.针对光滑支持向量机中现有的光滑函数逼近精度不高的问题,将正号函数变形并展开为无穷多项式级数,由此得到了一族多项式光滑函数,并证明了这类光滑函数的优良性能,它既能满足任意阶光滑的要求,也能达到任意给定的逼近精度。最后将得到的多项式光滑函数用于求解广义支持向量机。2.半监督支持向量机利用大量的未标记样本和少量的标记样本共同学习以改进其泛化性能,最后得到一个非凸优化问题,对其优化采取两种策略:组合优化和连续优化。组合优化的具体方法是给出了一个自训练半监督支持向量机分类算法,它的子程序是用前面得到的多项式光滑函数在原始空间求解标准支持向量机。接下来用连续优化的方式给出了一个多项式光滑的半监督支持向量机分类算法,给出的多项式函数有严格的理论基础,并且在样本的高密度区逼近精度高,而当逼近精度低时,则出现在样本的低密度区。3.直接方法是一类常用的无约束优化技术,简便实用,它和之前用于支持向量机的循环算法不同,不是一次更新w的所有分量,而是每次通过解一个单变量的子问题来更新w的一个分量。本文分别用Hooke and Jeeves模式搜索法、Rosenbrock转轴法和Powell方向加速法求解线性支持向量机,并分析了算法的复杂性。4.支持向量机采用的线性Hinge损失函数对噪声样本产生的损失没有限制,这是支持向量机对噪声敏感的根本原因。由于特殊的损失函数能有效抑制噪声产生的损失,本文据此给出了一个全新的双曲正切损失函数,并在此基础上给出了相应的健壮支持向量机。实验表明上述方法和结果在支持向量机算法中均具有较好的学习性能。更多还原

【Abstract】 Support vector machine (SVM) has been a dominant machine learning technique for more than a decade, however, most of its algorithms were proposed in allusion to its dual problem in the dual space. Research indicates that solving the primal problem is also an effective approach for training SVM in recent years. As people make an intensive study of SVM in the primal space, various problems meted in application were solved in the primal space, such as the problem of semi-supervised learning. But as a whole, the research on SVM is not familiar and perfect in the primal space. Therefore, this dissertation mainly focuses on the study of the following four problems of SVM classification algorithm in the primal space.To deal with the problem of poor approximation performance of smoothing functions available of smoothing support vector machines, plus function was transformed into an equivalent infinite series. Thus a family of polynomial smoothing functions were derived. The properties of them were discussed. It is shown that the approximation accuracy and smoothing rank of polynomial functions can be as high as required. Finally, the polynomial smoothing functions were used to solve the generalized support vector machine.Semi-supervised SVM makes use of the large collection of unlabeled data jointly with a few labeled examples for improving generalization performance. Finally, a non-convex optimization problem was obtained. We adopted two strategies for minimizing above optimization problem: combinatorial optimization and continuous optimization. The way of combinatorial optimization is presenting a self-training semi-supervised SVM classification algorithm, whose subprogram use the above obtained polynomial smoothing functions to solve the standard SVM in the primal. After that, a polynomial smooth semi-supervised support vector machines classification algorithm was presented in the way of continuous optimization. The introduced polynomial functions have a good command of theory and have high approximation accuracy in high density regions of samples and poor approximation performance appear in low density regions of samples.Direct search method is a common unconstrained optimization technique, it is different from the cyclic algorithms used in SVM former, which update all components of w at a time. However, direct search method updates one component of w at a time by solving a one-variable sub-problem. On account of the simpleness and practicability of direct search method, three algorithms of the method were used to solve linear SVM. The three algorithms are Hooke and Jeeves pattern search algorithm、Rosenbrock coordinate-turning method and Powell’s direction acceleration method. The detailed solving algorithm was given and the complexity of the algorithm was analyzed.The essential reason of the sensitivity of SVM to noise is that the adopted linear loss function has no limits on the penalty loss of noise samples. According to the fact that special loss function is able to control the loss value caused by noise samples, a novel hyperbolic tangent loss function was constructed, and based on the new loss function, the corresponding robust SVM- hyperbolic tangent SVM was proposed.Experiments were performed to verify the above four problems in SVM classification algorithm, experimental results show they can obtain satisfactory learning performance.更多还原