【作者】 厉小润；

【导师】 赵光宙；

【作者基本信息】 浙江大学 ， 控制理论与控制工程， 2007， 博士

【摘要】 模式识别是一门以应用为基础的学科,模式识别研究的理论和方法在很多方面得到了成功的应用,所有这些应用都是和问题的性质密不可分的,至今还没有发展成统一有效的可应用于所有问题的模式识别方法。由于大量实际的模式识别问题是具有多类别的高维复杂模式的识别,因此研究复杂模式的分析和分类方法是必要而且有意义的。基于核函数的学习方法(简称为核方法)是从统计学习理论中发展出来的较新的学习方法,它有效克服了传统模式识别方法的局部极小化和不完全统计分析的缺点。核方法本质上是非线性的信息处理工具,它在处理具有非线性关系的高维复杂模式识别问题时,有着其它学习方法无法比拟的优越性。核方法的研究和应用目前正方兴未艾,新的算法不断地被提出,但是作为一种尚未成熟的技术,仍然存在着许多不完善和有待解决的问题,如核函数的构造和选择、多类分类等问题,因此研究基于核方法的复杂模式识别理论具有重要的意义。本论文研究的内容主要侧重于如何用核方法实现高维多模式对象的特征提取和模式分类,论文所作的工作包括以下几部分内容:1.针对奇异情况下如何更好地解决核Fisher描述分析中非线性最优鉴别矢量集的求解问题,提出了改进的核直接描述分析(IKDDA)方法。根据再生核理论,定义核类内散度矩阵和核类间散度矩阵,将高维特征空间中的Fisher描述准则函数转化为核Fisher描述准则函数。基于同构映射原理和奇异值分解定理,在一个更小的空间内将核Fisher描述准则函数的极大值问题转化为其倒数的极小值问题,使最终的解不需要分开考虑核类内散度矩阵的零空间和非零空间。在ORL和UMIST人脸库上的实验结果表明了IKDDA方法与其他方法相比具有较低的误识率和较快的运行速度。2.针对高维、小样本模式识别中的特征提取问题,提出了一种约束线性描述分析方法(CLDA)。以线性变换后样本的类内距离与类间距离之比最小作为准则函数,同时加上约束条件使变换后的样本中心沿着特定的正交方向,通过白化变换、Gram-Schimdt正交化和正交子空间投影求解约束准则函数得到最优变换矩阵。针对人脸识别的小样本问题,根据奇异值分解定理实现白化变换。运用核技巧,将CLDA推广到非线性的约束核描述分析(CKDA),给出了原理与算法过程。对ORL和UMIST人脸库进行了仿真研究,结果表明CLDA方法和CKDA方法的有效性。3.针对如何有效地设计决策树支持向量机(SVM)多类分类器的层次结构这个关键问题,提出了一种基于向量投影的类间可分性测度的设计方法,并给出了基于该类间可分性测度设计决策树SVM多类分类器层次结构的偏二叉树方法和完全二叉树方法。为了加快每个SVM子分类器的训练速度且保持其高推广性,将改进的基于向量投影的支持向量预选取方法用于每个子分类器的训练中。对不同类型的数据的仿真实验,结果表明新方法有效地提高了决策树SVM多分类器的分类精度和速度。4.提出了改进的基于投影和三角不等式的k近邻搜索法以及改进的基于向量投影的边界向量预选取方法。针对样本总体分布已知的分类问题,提出了一种新的分类方法。通过非线性映射将训练样本映射到高维特征空间,基于向量投影法从训练样本中选择边界向量,运用k近邻搜索法确定每个边界向量同类中的k近邻,运用统计理论中的大数定理估计样本的类条件概率密度函数,由边界向量与相应的密度函数构成新的训练样本对。对每一类建立一个径向基函数(RBF)网络,以相应类的边界向量作为中心,通过训练,最终以RBF网络来估计样本的类条件概率密度。在此基础上,基于最小错误率的贝叶斯决策实现分类。对机器学习数据的仿真研究结果表明该方法能快速有效地实现多类分类。更多还原

【Abstract】 Pattern recognition is an application oriented subject, its theories and methods have been successfully applied in many areas, and all these applications are consanguineous with the property of idiographic problem. Up to present, no method can be applied to all problems. It is necessary and significative to study pattern recognition method for problems with complex patterns owing to the fact that most practical problems comprise high dimension and complex multi-class patterns.Kernel-based learning method is a new tool developed from statistic learning theory, it has effectively overcome the problems of local minima and overfitting. Kernel-based learning method is essentially a nonlinear information processing tool, and has been proven more effective than other learning methods on the pattern recognition of high dimension complex problem with nonlinear pattern. The research and application of Kernel-based learning method is just in the ascendant and many new algorithms are continually proposed. But as an immature technology, it still has to cope with many problems in the structure, function, classification and so on. Generally speaking, the research in complex pattern recognition based on Kernel learning method is of great significance.The main content of this paper contains kernel-based extraction and pattern classification methods. The main contributions are as following:1. In order to find a better method to compute the optimal discrimination vectors for the kernel-based Fisher discrimination analysis in the singular cases, In this paper, an improved kernel direct Fisher discrimination analysis (IKDDA) is proposed. Based on the theory of reproducing kernel, the kernel within-class and kernel between-class scatter matrices are defined, the Fisher discrimination criterion in the high feature space is transformed to kernel Fisher discrimination criterion. Based on the theory of isomorphic mapping and singular value decomposition theorem, the maximum of the kernel Fisher discrimination criterion can be skillfully acquired by solving the minimum of its reciprocal in a small space, and the final solution is acquired without taking account of the null space and non-null space of the kernel within-class scatter matrices separately. Experiment results on the ORL and the UMIST face image database indicate that the proposed methodology is able to achieve lower error rate and quicker speed compared with other methods.2. A constrained linear discrimination analysis (CLDA) method is proposed for the feature extraction in the pattern recognition of problems with high dimension and small samples. Applying whitening process and Gram-Schimdt orthogonalization and orthogonal subspace projection, an optimal transformation matrix is designed to minimize the ratio of intra-class distance to inter-class distance while imposing the constraint that different class centers after transformation are along specifically directions that are orthogonal each other. For the small sample problem of face recognition, the whitening process is realized by singular value decomposition. Using kernel tricks, the CLDA is generalized to constrained kernel discrimination analysis (CKDA). Experimental results on face images show that both CLDA and CKDA are effective and feasible.3. Designing the hierarchical structure is a key issue for the decision-tree-based (DTB) support vector machines multi-class classification. Inter-class separability is an important basis for designing the hierarchical structure. A new method based on vector projection is proposed to measure inter-class separability. Furthermore, two different DTB support vector multi-class classifiers are designed based on the inter-class separability: one is in the structure of DTB-balanced branches and another is in the structure of DTB-one against all. Experiment results on three large-scale data sets indicate that the proposed method speeds up the decision-tree-based support vector machines multi-class classifiers and yields higher precision.4. An improved k-nearest-neighbor search method based on projection and triangular inequality is presented, some inequalities are used to delete impossible data points and reduce distance computations. An improved method based on vector projection is proposed to address the problem of pre-extracting boundary vectors. A novel data classification method was prompted for the classification problem about samples with known distribution. A nonlinear function was chosen to map the input to a higher-dimensional space, vectors near the boundaries were pre-extracted from the training samples. By the law of large numbers in statistics, the value of the class-conditional probability density function of each boundary vector was estimated by k-nearest-neighbor estimation method. The learning algorithm constructed a radial basis function network with the boundary vectors as the network centers to approximate the class-conditional probability density function of each class of the objects in the training data set. The classification was conformed by the minimum error rate Bayesian decision rule. The experiment on machine learning data sets proved that the proposed algorithm can quickly and effectively carry out data classification with more than two classes of objects.更多还原