【作者】 刘悦；

【导师】 吴耿锋； 张博锋；

【作者基本信息】 上海大学 ， 控制理论与控制工程， 2005， 博士

【摘要】 学习方法的泛化能力、学习效率和易用性是机器学习及其应用过程中所面临的三个关键性挑战问题。神经网络集成学习通过训练多个神经网络并将其结果进行合成,显著地提高了学习系统的泛化能力,成为近年来机器学习领域一个重要的研究方向。本文在分析神经网络集成方法研究现状的基础上,以实验设计、粗集理论、特征加权以及并行技术等为支撑,围绕神经网络集成学习方法的易用性、泛化能力和学习效率等问题展开研究,提出了更有效的神经网络集成方法,并将其应用到地震预报领域。神经网络集成的结构(个体网络的数目和个体网络的结构)和个体网络的训练参数(如训练次数和学习率等)关系到集成性能的好坏,同时影响着集成是否易于被使用。本文首先研究了实验设计在神经网络集成中的应用,提出了一种简单、科学地确定神经网络集成结构和个体网络的训练参数的方法。使用者可以用较少的实验次数,分析影响神经网络集成泛化能力的因素以及确定各因素用什么水平搭配起来对集成的泛化能力最佳。同时,通过最近邻聚类算法自动确定个体网络的隐层节点,得到具有较大差异度的异构的个体网络,从而提高集成的泛化能力。其次,研究了构造算法和选择性方法的结合,提出了一种构造选择性神经网络集成方法。自动地确定了神经网络集成中个体网络的数目、个体网络隐层节点的个数及其训练次数等;并且采用多目标的个体网络选择方法,既保证了个体网络的精度又保证了个体网络之间的差异度。用户只需要简单地指定一些参数的初始值即可构造出集成,提高了神经网络集成的易用性。泛化能力是机器学习关注的基本问题之一。集成特征选择通过特征选择技术产生差异度大的个体,提高了集成的泛化能力。有效地生成属性子集是其需要解决的核心问题。本文就此展开了相关的研究,提出了基于粗集约简的神经网络集成方法。该方法充分考虑到了各属性之间的依赖性和相关性,利用基于可辨识矩阵的粗集约简方法有效地生成属性子集,能够构造出具有更高精度和差异度的个体,从而提高神经网络集成的泛化能力。特征加权能够细致地区分特征对结果影响的程度,已经成为当今流行的提高学习器的预测精度的方法之一。本文着重研究了如何将特征加权技术应用于提高神经网络集成的泛化能力,提出了一种基于特征加权的神经网络集成方法。该方法通过自适应遗传算法的优胜劣汰机制为输入属性确定了特征权值,提高了集成中各个体网络的精度和差异度,从而提高了神经网络集成的泛化能力。提高学习效率是机器学习永远的追求。本文结合最新的并行计算编程技术,提出了一种神经网络集成方法的并行实现方案,显著地提高了集成的学习效率;更多还原

【Abstract】 Generalization ability, efficiency, and convenience are the three major topics in the field of machine learning and its applications. Neural network ensemble is a learning paradigm, in which a collection of a finite number of neural networks is trained for the same task. Recently, it has become a hot topic in the machine learning community because of its high generalization ability. In this paper, several novel neural network ensemble methods were proposed and applied to the field of earthquake prediction on the basis of the following advanced techniques i.e. Design Of Experiment, Rough Set Reducts, Feature Weighting, and Parallel Computing.The architecture of the ensemble and the training parameters of individual neural networks are closely relative to the performance of the ensemble and the convenience of the ensemble creation. This paper firstly employed Design Of Experiment to guide users with little experience of using neural networks to design the ensemble architecture and adjust the training parameters of individual neural networks. At the same time, the nearest-neighbor clustering algorithm was used to create the heterogeneous individuals with different hidden nodes. Secondly, the constructive algorithm and selective algorithm were utilized to make the users without any experience of neural networks expediently and automatically construct the ensemble, and then the convenience of the ensemble creation was improved.Generalization ability is the principle issue in the field of machine learning. Feature selection for ensembles has shown to be an effective strategy for improving the generalization ability of the ensemble. In this paper, we focused on how to select the appropriate feature subsets, and employed an optimized approach based on discernibility matrix for determining rough set reducts. Finally, the ensemble with high generalization ability was build up on the projections of the training set. Feature weighting is the general case of feature selection, which has the potential of performing better than (or at least as well as) feature selection. In this paper, a new self-adaptive genetic algorithm was used to conduct a search for the weight vector, which could optimize the classification accuracy of the individual neural networks to improve the generalization ability of the ensemble.Efficiency is another principle issue of machine learning. This paper proposed a parallelization strategy for the neural network ensemble by using MPI (Message Passing Interface) techniques to reduce the complexity and improve the efficiency of更多还原