【作者】 彭宏京；

【导师】 陈松灿；

【作者基本信息】 南京航空航天大学 ， 计算机应用技术， 2002， 博士

【摘要】 基于存储器的神经网络模型(如N-tuple神经网络(NTNN)、稀疏分布存储器模型(SDM)等)，因其结构简单，硬件易实现；查表式的学习算法，运行速度快，因此受到众多研究者的关注。在许多领域获得了成功应用，也是神经网络走向商品化的基础模型。然而，也正是它们“无权值(weightless)计算”的结构等诸原因使它们表达非线性映射的能力不强，且其行为缺乏理论分析，因此影响了它们的进一步应用。本论文的目的是从修改它们的结构和学习算法着手进行推广性研究。通过学习能力分析、与相关模型的比较到人脸识别应用的研究都证实了推广后的新模型可行和有效。并且获得了更多原始模型不具有的性质，应用范围从原来只能进行二值模式的识别(非二值模式必须二值化)拓展到能直接处理实数向量输入模式来实现函数逼近、灰度人脸的识别。本论文的研究主要包括以下几方面的内容： 1、针对一类回归RAM式神经网络进行推广。提出了一个新型自适应模式识别系统一基于稀疏RAM的N-tuple神经网络模型(SR-NTNN)，它既可以用于模式识别，也能用于函数逼近，具有比较一般性的特点。增加的可调参数使新模型表现更灵活，同时减少了存储开销，抑制了RAM式神经网络容易饱和的问题。NTNN、SDM都可以作为其特例。最后，通过实验证实了它所具有的函数逼近能力。 2、在保留SDM的稀疏分布存储特征的基础上修改了SDM的结构和学习算法，提出逼近型的SDM模型。拓展了原SDM只能应用于联想记忆的应用范围，并且对新模型进行了学习能力的理论分析，分析表明它具有与小脑模型关节控制器(CMAC)相当的学习能力，尽管它们的量化方式完全不一样。逼近型SDM有优于CMAC的许多特点：不需要哈稀(Hashing)技术、不存在分块效应以及更易于理解和实现。理论分析和示例表明了该改进模型的合理性和有效性，函数逼近的效果优于CMAC。 3、经典的N-tuple分类器及单层查找感知器模型(SLLUP)由于结构简单、运算速度快、便于硬件实现而得到广泛应用。但由于它们均是基于RAM的结构而必须二值化输入样本，因此在进行大维数样本的识别问题时受到许多限制。为此将稀疏分布存储的概念引入SLLUP,提出了基于稀疏RAM的逼近型N-tuple模型。该模型从结构上看，囊括了逼近型SDM和SLLUP，但这决非只是结构上的简单推广，当稀疏地址编码直接采用实数向量时，新模型获得了SLLUP根本不可能具有的特点，即直接对输入模式进行N-tuple采样，使得处理大维数的样本数据真正成为可能，从而表现出较SLLUP明显的优势和更大的灵活性。函数逼近的实验表明该新模型通过选择适当的参数，逼近效果远优于SLLUP和逼近型SDM。 4、将CMAC、逼近型SDM、SLLUP以及基于稀疏RAM的逼近型N-tuple模 基于稀疏RAM的神经网络及其人脸识别应用研究型等纳入到一个称为一般存储器神经网络（GMNN）的模型框架中，其共同的特点是由三部分构成：输入空间量化、存储器地址产生器、查表式某种组合输出。其本质是通过地址选择隐含地实现从低维到高维空间的映射从而能更好地解决非线性的分类和回归问题，显示出核（kernel）方法特点。特别当产生的地址个数是有限常数且网络输出是加权线性和时，证明了此类网络可以收敛到最小平方误差解，从而为更好地应用和拓展此类网络提供了理论基础。 5、提出了一系列基于Ntuple特征子模式划分的人脸识别方法。在人脸识别应用上的研究不仅证实了本文所提模型的直接处理大维数样本的能力，而且展示了一类区别于基于显式地进行特征提取的人脸识别方法的新颖性。单个NtUple的分类或逼近能力是弱的，但通过不同的组合各个Ndple的方式可改善系统的性能。由此提出了结构式组合和输出组合的两种组合方式。稀疏RAM逼近型Ntuple网络即属结构式；通过结合Naive七ayes规则和直接投票的方法等进行的组合属输出组合方式；另外采用Boosting方法集成若干N－tuple，获得了结构组合输出再组合的高级集成方法。实验是基于Benchmark人脸数据库ORL的，其特点：l人 既不需要图象的完整特征，也无需进行特征提取预处理，只需少量N－tUple特征，再按所提方法进行组合；n、输出组合方法对 N－tuple特征划分方式以及大小不敏感；3人 Boosting NtUples与常规Boosting方法的区别在于不仅仅每个基本分类器使用不同的训练集，而且同一个训练样本提交给不同基本分类器不同的特征。实验结果证实所有提出的方法均获得了较好的识别效果（误识率在6．10％左右），所使用的特征数占整幅图象的10－100％不等，但实验表明更多的特征并没有更好的效果。事实上，所提的各种组合方法，尤其是 Boosting NtuPles方法，开辟了一条进行特征选择的新途径。更多还原

【Abstract】 The memory-based neural networks, such as N-tuple neural network (NTNN) and Sparse distributed memory model (SDM) etc., have attracted great attention by virtue of their simple architectures for easy hardware implementation, of lookup tables algorithm for fast operation. Their successful applications in many areas make them act as the basis of a commercial product. It is just their "weightless" or "RAMnet" architectures that results in poor nonlinear map, and at the same time few theoretical analyses about their behaviors have been gotten, so that further applications are limited. This dissertation starts by modifying their structures and algorithms for improving their performances. The extensive studies of analyses for learning ability, comparisons with related models and face recognition applications confirm that our generalized models are feasible and effective. Many good properties are obtained that original models can’t have, the scope of applications has been expanded from binary pattern recognition (if not, need to be converted to binary string) to function approximation and gray face recognition, all these benefit from the novel models’ abilities to dealing with real vector inputs directly. Several contributions have been made as follows:1. We generalize a class of regress NTNN with RAM and present a novel adaptive pattern recognition system桝 N-tuple neural network model with sparse RAM that can be applied to pattern recognition as well as function approximation task. The increased adjustable parameters make new model flexible, cut down memory requirement and restrain NTNN’s defect of easy saturation. It is a general model to some extent in which both NTNN and SDM can be regarded as a special case. Finally, experiments have shown its ability of approximating functions.2. The approximate SDM has been presented by modifying original SDM’s structure and algorithm, retaining original SDM’s characteristic of sparse distribution. It exceeds original SDM in application for original SDM is only applied to the associate memory. The theoretical analysis about the novel SDM shows that its learning ability is in common with CMAC while both quantification manners aren’t the same. Furthermore, no block effect appears and Hashing technology is not used in new model but the reverse in CMAC. Theoretical analysis and example have shown this improved model effective and reasonable, better performance in function approximation than CMAC does.3. SLLUP(single-layer lookup perceptrons) as well as classical N-tuple classifier is extensively used in many regions because of their simple architecture, fastoperation and easy realization to hardware. At the same time, this kind of architecture based on RAM results in that input sample need to be converted to binary vector. Consequently, their applications in large dimension sample recognition are limited. Therefore, the approximate N-tuple model based on sparse RAM has been presented by integrating SLLUP with sparse distributed memory, SLLUP as well as approximate SDM becomes a special case there. This is not simple generalizing architecture, since it is really possible that the novel model can deal with large dimensional samples directly when sparse address code is immediately real vector one, that is, N-tuple sampling can be operated on input examples directly that can’t in SLLUP. Function approximation experiment demonstrates that this new model can get better performance than CMAC and approximate SDM by selecting parameters appropriately.4. There is a kind of networks, such as CMAC, approximate type SDM, SLLUP and approximate type N-tuple network based on sparse RAM, titled a general memory neural network (GMNN). It consists of: input space quantification, memory address generator, combined output by memory lookup operations. The essence of its operation, analogous to kernel method, is that a nonlinear map can be achieved by address selection operation leading to a higher dimension space, so that better classification or regression performance can be obtaine更多还原