【作者】 吴青；

【导师】 刘三阳；

【作者基本信息】 西安电子科技大学 ， 应用数学， 2009， 博士

【摘要】 支持向量机是借助优化方法解决机器学习问题的新工具.近年来,支持向量机越来越受到人们的广泛关注,在其理论研究和算法实现方面都取得了重大进展,成为机器学习领域的前沿热点课题.支持向量机将机器学习问题转化为优化问题,并应用优化理论构造算法.优化理论是支持向量机的重要理论基础之一,本文主要从优化理论和方法的角度对支持向量机进行研究.主要内容如下:1.对最小二乘支持向量机进行研究.提出一类训练最小二乘支持向量机的条件预优共轭梯度法.当训练样本的个数较大时,最小二乘支持向量机需要求解高阶线性方程组,利用分块矩阵的思想将该高阶线性方程组系数矩阵降阶,为了提高收敛速度,克服数值的不稳定性,采用条件预优共轭梯度法求解低阶的线性方程组,大大提高了最小二乘支持向量机的训练速度.2.对光滑支持向量机进行研究.无约束支持向量机模型是非光滑不可微的,许多优化算法无法直接用来求解该模型.采用CHKS函数作为光滑函数,提出了光滑的CHKS支持向量机模型,并用Newton-Armijo算法来训练该模型.该算法通过批处理训练来提高训练速度,节省存储空间,可以有效求解高维、大规模的分类问题.3.基于优化理论中的KKT互补条件,分别建立了支持向量分类机和支持向量回归机的无约束不可微优化模型,并给出了有效的光滑化近似解法.建立了支持向量分类机的无约束不可微优化模型,给出了求解支持向量分类机的调节熵函数法.该方法不需要参数取值很大就可以逼近问题的最优解,避免了一般熵函数法为了逼近精确解,参数取得过大而导致数值的溢出现象;调节熵函数法同样可以用来训练无约束不可微的支持向量回归机,提出了求解支持向量回归机的调节熵函数法,有效避免了数值的溢出现象.这两个算法分别为求解支持向量分类机和支持向量回归机提供了新的思路.4.对模糊支持向量机进行研究.针对支持向量分类机对训练样本中的噪声和孤立点特别敏感的问题,提出了一类基于边界向量提取的模糊支持向量机方法.选择可能成为支持向量的边界向量作为新样本,减少了参与训练的样本数目,提高了训练速度.样本的隶属度根据边界样本和噪声点与所在超球球心的距离分别确定,减弱了噪声点的影响,增强了支持向量对支持向量机分类的作用;为了克服最小二乘支持向量机对于孤立点过分敏感的问题,将模糊隶属度概念引入最小二乘支持向量机中,提出了基于支持向量域描述的模糊最小二乘支持向量回归机.新的隶属度的定义减弱了噪声点的影响.把所要求解的约束凸二次优化问题转化为正定线性方程组,并采用快速Cholesky分解的方法求解该方程组.在不牺牲训练速度的前提下,比支持向量机和最小二乘支持向量机具有更高的预测精度.5.对半监督支持向量机进行研究.为了改进?TSVM的分类性能,引进了一个光滑分段函数,给出了光滑分段半监督支持向量机模型.光滑分段函数的逼近性能优于高斯近似函数.根据光滑分段半监督支持向量机的非凸特性,首次采用保证收敛的线性粒子群算法来训练半监督支持向量机,光滑分段半监督支持向量机在分类性能上优于?TSVM.更多还原

【Abstract】 Support vector machine (SVM) is a new approach that can solve machine learning problem with optimization methods. In recent years, there has been a surge of interest in SVM. It has achieved a prodigious progress in the theory research and algorithm implement, thus has been an active research area in machine learning.SVM translates the machine learning problems into the optimization problems and applies the optimization theory to construct algorithms. The optimization theory is the important theory foundation of SVM. This dissertation mainly does researches on SVM with the optimization theory and the optimization methods. All of the research results can be described as follows.1. The study on least squares support vector machine (LSSVM). A preconditioning conjugate gradient method for LSSVM is proposed. LSSVM has to solve a large scale linear system of equation when the number of the training simples is large. Block matrix is applied to reduce the system of equations. In order to improve the rate of convergence and overcome instability of numerical value, a preconditioning conjugate gradient method is presented for solving the reduced system of linear equations. The training efficiency of LSSVM is improved greatly by the method.2. The study on smooth SVM. The objective function of the unconstrained SVM model is non-smooth and non-differentiable. So a lot of optimization algorithms can not be used to find the solution of the model. To overcome the difficulty, a novel smoothing method using Chen-Harker-Kanzow-Smale functions for SVM (CHKS-SVM) is proposed. The Newton-Armijo method is adopted to train the smooth CHKS-SVM. Using the proposed method, the optimal separating hyperplane is trained in batches, both the training time and memories needed in the training process are saved. So the novel method can efficiently handle large scale and high dimensional problems.3. Based on KKT complementary condition in optimization theory, two unconstrained non-differential optimization model for SVM and support vector regression (SVR) are proposed respectively. A smooth approximate method is given to deal with the proposed optimization problems. An adjustable entropy function method is given to train SVM. The proposed method can find an optimal solution with relatively small parameters, which avoids the numerical overflow in the traditional entropy function methods. The adjustable entropy function method can be used to train SVR analogously, which avoids the numerical overflow effectively. It is a new approach to solve SVM and SVR.4. The study on fuzzy SVM. A fuzzy SVM based on border vector extraction is presented, which overcomes the disadvantage that traditional SVM are so sensitive to noises or outliers in the training samples. Select possible support vectors for border vectors to train SVM, so as to reduce training samples and improve training speed. The fuzzy membership, which is defined according to the distance between the center of their spheres and border vectors and outliers respectively, both diminishes the effect of noises and outliers and improves the role of support vectors to design a classifier. The conception of fuzzy membership is introduced into LSSVM in order to overcome the disadvantage that LSSVM is much sensitive noises or outliers in the training samples. And then fuzzy LSSVM (FLSSVM) is proposed based on support vector domain description. The new defined fuzzy membership can reduce the effect of outliers. The constrained convex quadric programming problem can be translated into positive definite linear equation system. The fast Cholesky decomposition is applied to solve the linear equation system. The regression performance of FLSSVM is superior to that of SVM and LSSVM.5. The study on Semi-supervised SVM (S3VM). A piecewise function is used as a smooth function and smooth piecewise semi-supervised support vector machine (SPS3VM) is given. The approximation performance of the smooth piecewise function is better than that of the Gaussian approximation function. According to the non-convex character of SPS3VM, a converging linear particle swarm optimization is first used to train S3VM. Experimental results illustrate that our proposed algorithm improves ?TSVM in terms of classification accuracy.更多还原