【作者】 罗林开；

【导师】 林成德；

【作者基本信息】 厦门大学 ， 控制理论与控制工程， 2007， 博士

【摘要】 由Vapnik等人提出的支持向量机(Support Vector Machine,SVM)技术,由于具有极强的模型泛化能力,不会陷入局部极小点,以及很强的非线性处理能力等特点,近十年来取得了全面飞速的发展,获得了大量成功的应用,已成为模式识别中最为活跃的研究领域之一。当前,选择合适的核函数及其参数(核选择)已成为SVM进一步发展的关键点和难点。核函数决定了SVM的非线性处理能力,也决定着分类函数的构造,而对具体问题而言,选择合适的核函数及其参数,还存在着许多的实际困难。针对SVM中的核选择问题,本文对SVM的模型问题、特征空间线性可分的结构问题、核学习中基核的选择问题、以及核函数及其参数的评判准则问题开展了深入的探讨,主要的工作有:1.在SVM的模型方面,给出了L2-范数下平分最近点原理问题;然后得到了它的解与最大间隔原理问题的解之间的关系,建立了它与最大间隔原理的等价性;指出它还具有模型性质更好、几何意义更直观、能利用求解凸包之间距离的内点算法等优点;最后给出了它的SMO(Sequential minimal optimization)求解算法。2.在特征空间线性可分的结构方面,利用平分最近点原理模型,通过对核矩阵零空间的深入分析,得出特征空间中样本线性可分与核矩阵零空间关系的一个充要条件。3.在基核矩阵的选取方面,首先提出矩阵的秩空间差异性(Rank Space Diversity,RSD)概念,其次将其作为基核矩阵的差异性度量,由此导出选择基核矩阵的一个定量规则“基核矩阵的秩空间差异性越大越好”。我们还给出了基于L2-范数下平分最近点原理的核学习模型和模型求解算法;最后通过实验验证了基核矩阵选择规则的有效性。4.在核函数及其参数的评判准则方面,首先从分类函数抗样本扰动的“泛化性能”出发,分析了传统最大间隔原理的不足,提出了分类函数的鲁棒度概念;探讨了鲁棒度的性质;并提出用最大鲁棒度作为核选择的评判准则;通过与经典的交叉验证方法和最小支持向量数方法的实验对比,表明最大鲁棒度准则克服了交叉验证方法时间代价高,最小支持向量数方法测试准确率不稳定的缺点,获得了很好的结果。5.在核学习方面,提出了按单属性设计基核,以最大鲁棒度为优化目标的核学习方法,给出了鲁棒度的梯度计算公式和模型的求解算法,并用实验表明了该方法的有效性和优越性。更多还原

【Abstract】 In the last ten years there have been very significant developments in the theoretical understanding of Support Vector Machines (SVMs) , proposed by Vapnik and others, as well as algorithmic strategies for implementing them, and applications of the approach to practical problems.Nowadays, the selection of the SVM-kernel with suitable form and parameters (Kernel Selection) has become a key-point both in theoretical research and application consideration. In fact, the nonlinear processing ability of SVM and the structure of the separating function are both largely decided by the choice of individual kernel function, and actually there are still a lot of difficulties on practice.As a research work focused on kernel selection of SVM, this paper has mainly discussed the following problems:1. On the modeling of SVM, the principle of bisecting closest points under L2-norm is firstly introduced. The relation between the solutions based respectively on the bisecting closest points principle and the maximum margin principle is then deduced, and the equivalence is established on these two solutions. The advantage of bisecting closest points method is showed, including of the better model character, the more intuitive geometric significance, and the optional nearest point algorithm. A SMO typed algorithm for the model based on bisecting closest points principle under L2-norm is also presented.2. On the aspect of linear separable structure of sample set in feature space, a necessary and sufficient condition is obtained based on null space of kernel matrix.3. On the aspect of selecting the base-kernels in kernel learning, a new concept of rank space diversity of matrices is firstly proposed; it is considered as a diversity measure for the base-kernel matrices.“Rank space diversity of base-kernel matrices should be as big as possible”is then deduced as a rule for the selection of base-kernel matrices. The kernel learning model based on bisecting closest points principle under L2-norm, as well as it’s solving algorithm, are given, and the validity of this rule is showed by some experiments. 4. On the aspect of the criterion of kernel evaluation, a robustness concept on separating function is firstly proposed based on the anti-disturbance ability of samples. By its properties, the maximum robustness of separating function is proposed to be a criterion for kernel evaluation. Experiments on the comparison among classic k-fold cross validation, minimum support vectors and maximum robustness methods show that our proposition is efficiency, which overcomes the shortages of high time cost for k-fold cross validation and the unstable testing accuracy for minimum support vectors.5. On the aspect of kernel learning, a new method is proposed, in which the base-kernels are designed on each attribute and the robustness of separating function is maximized. The corresponding solving algorithm of this kernel learning model is presented, and the validation and advantages of our method is shown by some numerical experiments.更多还原