基于非线性偏最小二乘的特征提取方法研究

【作者】 周琳；

【导师】 孙权森；

【作者基本信息】 南京理工大学 ， 模式识别与智能系统， 2011， 硕士

【摘要】 特征提取是模式识别,尤其是图像识别中最为核心的问题之一,提取的结果直接影响最终的分类识别效果。作为“第二代多元统计分析技术”的偏最小二乘(PLS)因为综合了多元线性回归分析、主成分分析和典型相关分析的基本功能,在图像识别的特征提取方面有着绝对的优势。它的基本思想就是在自变量空间和因变量空间中分别找出各自具有特征代表性的主成分,且这对成分之间的相关性最大。本文以传统的偏最小二乘为基础,从外部样本变换和内部成分间非线性映射这两种思路出发,对其进行了非线性扩展,并将其应用于特征提取和图像识别方面,取得了较好的效果。本文的主要工作和研究成果如下：(1)对多种偏最小二乘实用算法进行了对比、分析和改进,并通过实验进行验证。分别从自变量的预处理、因变量的生成和自变量与因变量的约束关系角度出发对典型算法进行了改进,提出了基于正交约束条件的偏最小二乘算法。该方法以非迭代的奇异值分解方法为基础,首先使用正交约束去除自变量中与因变量正交的成分,然后使用共轭正交条件进行偏最小二乘建模,最后通过多组实验验证了算法的可行性和有效性。(2)基于核函数思想,深入研究了基于外部样本变换的非线性偏最小二乘方法。通过核函数方法解决了样本映射时的维数灾难和非线性形式难于表达的难题,并从核函数的选择和构造两方面探讨了核函数的确定问题。通过核校准和将全局核与局部核组合的方法得到最佳的核函数,然后在偏最小二乘中使用核技巧,得到了可以更好地提取非线性特征的核偏最小二乘。基于非迭代的奇异值分解法提出了一种新的核偏最小二乘,并加入正交约束对其进行改进,提高了算法性能。(3)以内部成分间非线性映射的思想为基础,深入研究了模糊偏最小二乘、神经网络偏最小二乘和模糊神经网络偏最小二乘等方法。分别通过级联和相互融合的方式对模糊模型和神经网络模型进行结合,得到了不同的模糊神经网络模型。将单独的模糊模型、神经网络模型和综合的模糊神经网络模型分别嵌入到迭代形式的偏最小二乘中对成对的成分进行非线性映射,得到了不同形式的非线性偏最小二乘方法,并通过对自变量和因变量成分进行相同的迭代更新对其进行改进,进一步提高了各种方法提取非线性特征的能力。多次实验表明这类算法是有效的,尤其是基于模糊神经网络的偏最小二乘算法的性能最好。更多还原

【Abstract】 Feature extraction is one of the core problems in pattern recognition, especially in image recognition. Extracted result will directly affect the final classification and recognition result. Partial least squares(PLS) which is called "the second generation of multivariate statistical analysis" synthesizes basic functions of multiple linear regression analysis, principal component analysis and canonical correlation analysis, so it has an absolute advantage in the feature extraction of image recognition. Its basic idea is to find principal components which represent the characteristics of independent variables space and dependent variables space respectively, and meanwhile correlation between the components is the largest. In this paper, based on the traditional partial least squares, it will be expanded to nonlinear with methods of transforming samples externally and modeling nonlinear mapping between components internally. And apply it to feature extraction and image recognition, a better effect will be achieved.The main work and research results of this paper can be summarized as follows:(1)A variety of practical algorithms about Partial least squares are compared, analyzed, improved and verified by experiment. From the points of preprocessing independent variables, producing dependent variables and making restraint on them, we improved typical algorithms and proposed a new algorithm:partial least squares based on orthogonal constraint. The algorithm which is based on the non-iterative singular value decomposition partial least squares method, removes the components which are orthogonal to dependent variables from independent variables firstly, and then use the PLS method to model with condition of conjugate orthogonal. At last, through several experiments, we demonstrated the feasibility and effectiveness of the algorithm.(2)Based on the idea of kernel function, we deep researched the Kernel partial least squares, the typical one of nonlinear PLS algorithms, which are based on the idea of transforming samples externally. With the method of kernel function, we solved the dimension disaster of sample mapping and the difficult problem of expressing the nonlinear form. Besides, we researched the problem of identifying kernel function from both the choice and the construction of kernel function in depth. With the method of kernel alignment, we get the best kernel function by combining the global kernel and the local kernel together. Then in process of partial least squares, by using the best kernel, we get a kernel partial least squares which can extract nonlinear features better. At last, based on the non-iterative singular value decomposition PLS method, we proposed a novel kernel partial least squares algorithm. And added the orthogonal constraint to it, performance of the algorithm was improved.(3)Based on the idea of modeling nonlinear mapping between components internally, we deep researched several kinds of nonlinear PLS algorithms, such as fuzzy partial least squares(FPLS), neural network partial least squares (NNPLS) and fuzzy neural network partial least squares(FNNPLS). By combining fuzzy model and neural network model with method of cascade and mutual integration respectively, we obtained different fuzzy neural network models. When embed separate fuzzy model, neural network model and integrated fuzzy neural network model into the iterative partial least squares, we will find different types of nonlinear mapping and get different forms of nonlinear PLS method accordingly. Besides, we make the same iterative update on both independent variables and dependent variables to improve ability of extracting nonlinear features of the algorithms. Many experiments show that these methods are all effective. And especially the FNNPLS method can give the best result.更多还原