【作者】 吴松松；

【导师】 杨静宇；

【作者基本信息】 南京理工大学 ， 模式识别与智能系统， 2012， 博士

【摘要】 面对日益增长的海量数据,人们越来越多地依赖计算机智能化地从数据中得到问题解决所需要的有用信息。作为智能化数据分析的重要手段,维数约简技术不仅有效减低了处理过程的计算复杂度,也显著提高了数据分析的准确性和有效性。维数约简技术广泛应用于模式识别和计算机视觉领域,其中基于维数约简的特征提取已成为解决诸多相关问题的关键因素。尽管关于维数约简的研究已取得丰富的成果,但当前数据呈现出的高维数和多模态特点带来了新的挑战。在人脸图像识别、视频序列分析、文本与图像检索等实际应用的驱动下,维数约简技术通过对现有方法进行完善或探索新的理论方法获得了进一步的发展。本文立足于当前的数据形势,深入研究了关于向量数据和高阶数据的维数约简技术及其在特征提取中的应用问题。作为维数约简研究的一些新成果,本文提出的算法在数据可视化和人脸识别中得到了较好的应用。流形嵌入是目前非监督向量数据降维的研究热点,在探索数据的潜在结构上该方法具有优势。然而流形嵌入方法无法获得数据空间到低维特征空间的显式映射关系,故难于对新数据进行维数约简。针对这个问题,本文提出了面向流形的随机近邻投影(MSNP)用于非监督特征提取。MSNP算法在随机近邻嵌入(SNE)算法的启发下提出,基本想法是改善SNE算法的非线性流形展开能力和用显式的线性投影近似流形嵌入的非线性映射以适于特征提取任务。本文分析了SNE算法的不足,在以下三个方面进行了改进和完善：(1)提出在数据空间中用测地线距离代替欧氏距离构建随机近邻选择概率,从而提高了描述数据相似关系的准确性。(2)提出在低维特征空间使用柯西分布代替学生t分布构建随机近邻选择概率,以增强算法对数据的适应性。(3)在近邻概率分布保持的原则下引入线性投影得解决了新样本的维数约简问题,同时基于共轭梯度的迭代解法简明直观并具有比SNE更快的收敛速度。本文通过数据可视化、人脸识别和掌纹识别实验考察了MSNP的算法性能,包括投影基的性质、算法收敛性和特征提取能力。实验结果证明本文所提出的MSNP算法是一种有效的非监督向量数据降维方法,具有挖掘数据复杂模式的能力。在监督化向量数据降维方面,局部化线性鉴别分析方法考虑了数据的局部结构信息,提取鉴别特征的能力强于传统的全局线性鉴别方法。经过深入研究,本文发现已有的局部化线性鉴别方法普遍存在模型参数多且不易设置的问题。针对此问题,本文对局部线性鉴别分析的自适应问题进行了探索研究。本文提出了由同类局部近邻样本确定的局部邻域概念,其中的异类近邻样本能够自动被确定。在新的局部邻域内,同类和异类样本的分布反映了数据集不同局部的数据特性。据此,本文发展出了一种自适应的局部线性鉴别方法。该方法采用差分鉴别模型,模型中反映同类样本和异类样本权重的参数由局部近邻的特性自动设置。由于只需要同类近邻样本数这一个参数,本文提出的算法大大提高了局部线性鉴别分析在特征提取上的可用性。通过对人脸识别实验结果的分析,本文发现与已有的局部线性鉴别算法相比,所提出的自适应算法多数情况下能提升所得特征的鉴别能力,即便在训练数据极端少的情况下也取得了与已有方法相当的识别效果。对于如图像和视频这一类的高阶数据,近几年兴起的张量化维数约简方法基于数据的张量模型利用多重线性投影以获得数据的线性结构。尽管也出现了如张量LPP和张量NPE这样的方法试图获取张量数据的非线性结构信息,但基于多重线性投影的降维方式导致非线性结构信息在降维过程中不可避免地遭到损失。本文从张量数据采样自低维非线性流形的假设出发,提出了一种直接获得张量数据的低维嵌入(即参数化坐标)的非监督降维算法。该算法利用局部秩一张量投影所得到的低维向量来刻画张量数据的局部线性结构,然后在局部坐标全局化排列的原则下由局部仿射变换得到张量的全局性低维表示。由于维数约简过程依赖于一个非线性映射,本文提出的算法能有效挖掘张量数据的非线性结构。此外,为了方便特征提取,本文基于数值插值方法给出了张量嵌入的一种泛化方案。数据可视化的实验结果表明,本文提出的张量嵌入方法能有效发现张量数据流形的潜在结构,而人脸图像识别上的实验结果证明所提算法经过泛化扩展后能从张量数据中提取出有效的模式特征。本文针对张量数据的监督化特征提取问题,提出了一种新的张量化降维算法---“局部鉴别化正交秩一张量投影”(LDOROTP)。该算法的目标是从张量数据中提取出紧凑的特征并同时赋予特征相当的鉴别能力。LDOROTP算法通过正交秩一张量投影获得张量数据的向量形式的特征,并通过局部鉴别分析求取最优的投影张量基。与已有的算法相比,本文所提出的算法创新点在于：(1)局部鉴别分析采用所有的同类样本和适当数量的异类近邻样本；(2)在局部邻接图中引入新的加权函数对局部鉴别信息进行编码。LDOROTP算法的目标函数建立在差分鉴别模型上,避免了难以处理的奇异矩阵求逆问题。除此之外,LDOROTP算法对于秩一张量的正交性约束提出了轮换正交的策略,增强了算法结果的稳定性。本文提出的算法在人脸图像特征问题上进行了验证,实验结果证明LDOROTP算法在提取张量数据的鉴别特征方面是有效的。更多还原

【Abstract】 Faced with the growing massive data, people increasingly rely on computers to intelli-gently get useful information from the data to solve problems. As an important technique of in-telligent data analysis, dimensionality reduction not only effectively reduces the computational complexity of the processing procedure, but also significantly improve the accuracy and validity of data analysis. Dimensionality reduction techniques are widely used in applications of pattern recognition and computer vision, which makes feature extraction become the key to solve re-lated problems. Despite the research for dimensionality reduction has made fruitful results, but the new characteristics of current data with high dimensionality and multi-modal features being new challenges to dimensionality reduction target. Driven by the requirements from applica-tions such as face image recognition, video analysis and multispectral remote sensing image processing, dimensionality reduction technique has made further development through improv-ing and perfecting the existing methods or exploring new theories and techniques. Standing on the reality of data analysis demand in current situation, this thesis studies some theoretical and algorithmic problems in dimensionality reduction for vector data and high-order data, as well as some practical problems of feature extraction in applications. We develop some new algo-rithms as regards it, and these algorithms are verified to be effective in the application of data visualization and face identification.Manifold embedding methods have advantages in discovering the underlying structure of data, thus it is currently a research focus in unsupervised dimensionality reduction for vec-tor data. However, manifold embedding methods cannot obtain explicit mapping between data space and feature space, which leads to the difficulties when one try to apply the methods to new data. To solve this problem, we develop a novel algorithm named manifold oriented stochastic neighbor projection(MSNP) is developed for unsupervised feature extraction. This algorithm is inspired by the stochastic neighbor embedding(SNE) algorithm and owns the characteristic that it uses an explicit linear projection from data space to feature space to approximate the nonlinear manifold mapping. Based on the analysis results about the deficiency of SNE al-gorithm, we achieves improvement and perfection on the following three aspects:(1) MSNP models the structure of manifold by stochastic neighbor distribution in the high-dimensional observation space with geodesic distance rather than Euclidean distance, improving the accu-racy of estimating the pairwise similarity of data. (2)In order to enhance the adaptability to data with variable Characteristics, MSNP used Cauchy distribution in low-dimensional feature space instead of the Student t distribution to calculate stochastic neighbor selection probability. (3) MSNP benefits from stochastic neighbor distribution preserving strategy and linear projec-tion manner to endow with nice properties, with a fast convergence speed given by the solution based on conjugate gradient operation. We evaluate the effectiveness of our MSNP method for dimensionality reduction, including the property of basis vector, convergence speed and the ability of feature extraction. The experimental results on ORL, Yale, AR, and PolyU databases demonstrate that MSNP can substantially enhance the quality of data visualization compared with many competitive manifold learning algorithms and improve the recognition accuracy in biometrics recognition task. This verifies that the proposed MSNP is an effective feature ex-traction method.Local linear discriminant analysis takes data structure information into account for unsu-pervised feature extraction of vector data, so that it obtains feature with more discriminative power than the global discriminative analysis method. We make a thorough study on the exist-ing local linear discriminative methods, then find that there exist multiple parameters in most of these methods and it is difficult to set the proper parameters. Aiming at this problem, we make exploration and research on the automation of local discriminative analysis for dimensional-ity reduction. We develop a novel algorithm named adaptive local discriminative analysis, in which there is only one parameter needed to be set. This algorithm performs discriminative analysis by using the difference model based on a novel local neighborhood, in which the local inter-class neighbors are determined automatically. We design the adaptive algorithm under the principle that the model parameter is automatically determined according to the local data distribution of inter-class neighbors and intra-class neighbors. The experimental results show that the proposed adaptive achieves higher recognition accuracy than the existing methods in most case, and in the special case of 2 samples available per class it gets comparable results. Considering that the adaptive algorithm need to set only one parameter, the results provide proofs that our algorithm is effective for supervised feature extraction.Several tensor based dimensionality reduction methods are proposed recently to reduce the dimension of high-order data, such as images and videos. Although some algorithms such as Tensor LPP and Tensor NPE consider the nonlinear structure of data, the manner of linear projection restricts the existing tensor methods on the ability of discovering nonlinear structure. In this paper, we develop a novel unsupervised dimension reduction algorithm for the purpose of exploring the nonlinear structure information from high-order data. We design the algorithm under the assumption of tensor manifold, and combine local rank-one tensor projection and global alignment strategy to obtain the embedding of tensor manifold. Besides, we provide a scheme of numerical interpolation to solve the out-of-sample problem. The experimental results of data visualization show that the proposed tensor manifold embedding algorithm can discover the underlying nonlinear structure of high-order data. And its effectiveness of feature extraction is proved by the experimental results of face recognition.For the supervised feature extraction task to high-order data, we propose a new tensor based algorithm, called local discriminant orthogonal rank-one projection(LDOROTP). The goal of LDOROTP is to learn a compact feature for images meanwhile endow the feature with prominent discriminative ability. LDOROTP achieves the goal through a serial of rank-one tensor projections with orthogonal constraints. To seek the optimal projections, LDOROTP carries out local discriminant analysis, but differs from the previous works on two aspects: (1)the local neighborhood consists of all the samples of the same class and partial local sam-ples from different classes; (2)a novel weighting function is designed to encode the local dis-criminant information. The criterion of LDOROTP is built on the trace differences of matrices rather than the trace ratio, so the awkward problem of singular matrix do not emerges. Besides, LDOROTP benefits from an efficient and stable iterative scheme of solution and a data pre-processing called GLOCAL tensor representation. LDOROTP is evaluated on face recognition application on two benchmark databases:Yale and PIE, and compared with several popular projection techniques. Experimental results suggest that the proposed LDOROTP provides a supervised image feature extraction approach of powerful pattern revealing capability.更多还原