【作者】 陶大鹏；

【导师】 金连文；

【作者基本信息】 华南理工大学 ， 信息与通信工程， 2014， 博士

【摘要】 小样本学习（Small Sample Learning，SSL）是模式识别领域中非常重要的研究主题。在可穿戴设备、移动互联网以及视频监控等智能应用方面受到了广泛关注。这些应用有一个共同的特点：嵌入在一个高维空间中可用于训练模型的样本非常少。在过去的几十年，研究人员提出了很多算法来减少这个问题带来的影响，并学习得到一个鲁棒的模型。本文目的在于进一步改善在实际应用问题中嵌入在高维空间的小样本学习的有效性和稳定性。我们考虑这个问题的如下几个方面并提出了对应的解决策略：首先，提出了排序信息保留鉴别分析（Rank Preserving DiscriminantAnalysis，RPDA）来探索排序信息对鉴别学习的性能提升。具体说来，RPDA采用块配准框架对类内样本的局部排序信息以及类间样本的鉴别信息进行建模。然而，同其他监督流形学习算法一样，RPDA算法仍有一些超参数，难以选择到最优的设置。我们进一步提出了一个新的降维算法—集成流形排序信息保留（Ensemble Manifold Rank Preserving，EMRP）来回避这一问题。EMRP寻求多个配准矩阵最优的线性组合来近似存在数据中的本质流形。我们将这两种算法应用于基于加速度的人体行为识别以得到鲁棒和高效的低维表达。然后，提出了稀疏排序信息保留（Rank Preserving Sparse Learning，RPSL）。该方法考虑保留排序信息和获得稀疏投影矩阵两个方面，因此RPSL可以减少集中测量现象的影响以及获得计算上的简约性。另外，为了有助于随后的分类，建模过程也考虑了分类错误最小化。通过一系列等价变换，我们将RPSL的目标函数转换为基于Lasso惩罚的最小平方问题。另外，在我们基于Kinect的场景分类研究中，我们对RGB-D图像样本提取SIFT特征，并采用局部约束线性编码对其进行特征表达，随后采用RPSL和一个简单分类器对场景进行分类。与其他经典的降维算法相比较，RPSL得到模型有着较好的解释性，另外在测试阶段可以节约计算方面的资源。其次，提出了一个全新的半监督分类器—Hessian正则化支撑向量机（HessianRegularized Support Vector Machines，HesSVM）。我们详细论证了利用Hessian正则化对边缘分布紧支集局部几何特性进行建模的合理性，并且证明了再生核希尔伯特空间中的HesSVM等效于核主成分学习的主分量张成的空间进行HesSVM学习。另外，我们提出了在云计算环境下进行图像标注的框架：通过Hamming压缩感知将压缩后的图像传输到云上，随后采用HesSVM进行语义标注。我们在公开的PASCALVOC’07数据集上验证了HesSVM分类器对大规模图像标注的有效性。最后，研究了弱监督度量学习。我们注意到KISS度量学习小样本训练中存在对协方差矩阵的逆估计不稳定的情况，从而会导致性能变差等问题。本文提出了正则光滑KISS度量学习（Regularized Smoothing KISS，RS-KISS），该方法将光滑和正则化技术无缝的结合用于估计协方差矩阵。RS-KISS算法优于KISS算法，是因为RS-KISS能够采用有效的办法放大协方差矩阵中小特征值估计不足，以及减少协方差矩阵中大特征值被高估的情况。另外，KISS的协方差矩阵采用的是极大似然估计。一般认为随着训练样本数量的增加，基于最小分类误差准则的鉴别学习比经典的极大似然估计更加可靠。因此我们进一步提出一个新的算法—最小分类误差KISS度量学习（MinimumClassification Error KISS，MCE-KISS）。这两个方法在VIPeR和ETHZ数据集上进行了充分试验。结果表明MCE-KISS算法准确性更高，而RS-KISS计算更加有效。因此，我们需要依据实际情况选择适用的算法。更多还原

【Abstract】 Small sample learning (SSL) is a hot topic in pattern recognition. It has receivedintensive attention because of its widespread use in intelligent systems, such as wearablecomputing, mobile and internet entertainment, and video surveillance. These applicationsshare a common characteristic, namely that the sample embedded in a high-dimensional spaceand available for model training is of small size; this is known as the ‘small sample size’(SSS)problem. Over the past few decades, many algorithms have been proposed to reduce the SSSeffect and learn robust models. This thesis aims to further improve the efficiency and stabilityof SSL in practice, specifically by exploiting SSL for data embedded in high-dimensionalspaces. We consider the following aspects of this problem and propose the followingsolutions:First, we propose rank-preserving discriminant analysis (RPDA) to exploit rank orderinformation and improve discriminant learning. In particular, RPDA encodes local rankinformation of within-class samples, and discriminative information of between-class samples,under the ‘Patch Alignment Framework’. However, like other supervised manifold dimensionreduction algorithms, RPDA has several hyper-parameters, the optimal settings for which arenot trivial to choose. We therefore propose a new dimension reduction algorithm to avoid thisproblem, termed ensemble manifold rank preserving (EMRP). EMRP finds the optimal linearcombination of the alignment matrices to approximate the intrinsic manifold in the data. Weapply these two schemes to acceleration-based human activity recognition, and achieve arobust and effective low-dimensional representation.Second, we propose rank-preserving sparse learning (RPSL), which preserves the rankorder information and obtains a sparse projection matrix, and in doing so reduces theconcentration of the measured phenomena and obtains parsimony in computation. In addition,we consider minimization of classification error to facilitate classification. By utilizing aseries of equivalent transformations, we can transform the objective function of RPSL into alasso-penalized least squares problem. In addition, in our Kinect-based scene classificationstudies, we apply locality-constrained linear coding (LLC) to local SIFT features to representRGB-D samples, and classify scenes through the cooperation between RPSL and a simple classification method. Compared to other classical dimension reduction algorithms, RPSLresults in an interpretable model and saves computational costs in the testing stage.Third, we propose a novel semi-supervised classifier, termed the Hessian-regularizedsupport vector machine (HesSVM). We carefully explain the rationale for using Hessianregularization to encode the local geometry of the compact support of the marginaldistribution, and prove that using HesSVM in the reproducing kernel Hilbert space isequivalent to conducting HesSVM in the space spanned by the principal components of thekernel principal component analysis. In addition, we present a scheme for image annotation inthe cloud, in which mobile images compressed by Hamming-compressed sensing aretransmitted to the cloud, and semantic annotation is conducted in the cloud using a novelHesSVM. We conduct experiments on the PASCAL VOC’07dataset and demonstrate theeffectiveness of HesSVM for large-scale image annotation.Finally, we investigate weakly-supervised metric learning. We noticed that KISS metriclearning estimates the inverse of a covariance matrix to be unstable, and the resultingperformance can therefore be poor. Thus, we present regularized smoothing KISS metriclearning (RS-KISS), which seamlessly integrates smoothing and regularization techniques torobustly estimate covariance matrices. RS-KISS is superior to KISS because it can effectivelyenlarge underestimated small eigenvalues, and reduce overestimated large eigenvalues, in theestimated covariance matrix. In addition, the covariance matrices of KISS are estimated bymaximum likelihood (ML) estimation. It is known that with an increasing number of trainingsamples, discriminative learning based on the minimum classification error (MCE) is morereliable than classical ML estimation. Thus, a new scheme is presented, termed the minimumclassification error KISS (MCE-KISS). These two algorithms are used in thorough validatoryexperiments on the VIPeR and ETHZ datasets, and the results show that MCE-KISS is muchmore accurate and RS-KISS is computationally much more efficient. Therefore, onealgorithm needs to be chosen according to the practical situation.更多还原