【作者】 沈艳军；

【导师】 汪秉文；

【作者基本信息】 华中科技大学 ， 控制理论与控制工程， 2004， 博士

【摘要】 神经网络是高度非线性动力学系统,又是自适应自组织系统,可用来描述认知、决策及控制等的智能行为,使得智能的认识和模拟成为神经网络理论研究的一个重要方面;同时,神经网络理论又成为信息并行处理的基础,PDP(并行分布处理)成为80年代中后期的一个研究新热点,它进一步拓展了计算概念的内涵,使神经计算、进化计算成为新的研究领域。引起了包括计算机科学、人工智能、认识科学、信息科学、微电子科学、自动控制与机器人、脑神经科学等学科与领域的科学家的巨大热情和广泛兴趣。然而,传统神经元的M-P模型用连接权值和非线性激活函数分别模仿神经元的突触和细胞体的作用。在训练过程中,权值是可以调节的,但激活函数不能调节,是事先确定的。这种模型过于简单,其性能受到极大的限制。基于此,人们提出激活函数可调的神经元模型(TAF)以及由此模型构成的多层前向神经网络(TAF-MFNN),与传统网络相比,采用这种新型的神经元模型来解决某些问题时,所需要的网络会更为简单,网络性能会更好,推广泛化能力也会更强。但是,用来训练这种网络的BP算法收敛速度慢,容易陷入局部极小。众所周知,RLS算法具有收敛速度快,收敛精度高等优点,本文将激活函数可调的神经网络进行变形,使得变形后的网络和原网络等价,给出了网络学习的快速算法。通过仿真实验,改进的算法大大提高网络的收敛速度和精度。在此基础上,本文将激活函数可调的神经网络进行改进,提出了多输出神经元模型(MO)以及由此模型构成的多层前向网络(MO-MFNN),并给出了网络的RLS算法、LM算法、LMAM算法和OLMAM算法。通过仿真实验,得到结论:在训练样本不太大时,选择利用LM算法、LMAM算法和OLMAM算法来训练网络;在训练样本很大时,选择利用RLS算法来训练网络。在神经网络的训练中大多用如下均方误差函数(MS)作为网络的训练目标函数,直到现在,大多数快速算法都是基于MS基础上得到的。但是MS有两个主要缺点,一方面MS误差函数面存在很多的次最优解,在网络的训练过程中,易于陷<WP=5>入这些次最优解;另一方面,MS误差函数是一个能满足各种不同方面应用的通用判决标准,利用MS误差函数训练网络容易出现过拟和现象。为了提高网络的性能,如降低测试误差和提高网络的泛化能力,在一些特殊的应用中,需考虑额外的假设和启发性的信息。其中一个利用先验知识的技巧是正则化方法,即构造一个正则化的目标函数。本文研究了MO-MFNN带正则化因子的学习算法,仿真实验表明,利用MO-MFNN可以减小计算复杂度和存储量。神经网络具有并行性和自然容错性的能力,非常适合于非线性系统的实时自适应控制。到目前为止,还没有一种较为成熟的快速在线学习算法,而快速在线学习算法是实时控制的关键。多输出神经元模型的多层前向神经网络具有极大的网络容量,泛化能力强,网络结构简单,待定参数少,在线学习速度快,仿真实验表明,利用MO-MFNN进行非线性系统控制可以大大减小CPU时间,使其应用实时控制成为可能。更多还原

【Abstract】 Neural networks is a high nonlinear dynamical and adaptive and self –organizational system. It can be used to describe the intelligent activation of cognition, decision and control e.t., which makes the intelligent cognition and simulation become the major facet of neural networks theoretic research. At the same time, neural networks becomes the bedrock of information parallel distributed processing (PDP). PDP becomes a new hot domain in the medium and upper term of 80’s last century. It ulteriorly extends the connotation of computation, which makes neural computation and evolution computation become a new research domain. It arouses enormous passion and broad interesting of scientist including computer science, artificial intelligent, cognition science, information science, micro-electronics, automatic control and robot, brain neurology e.t..However, the traditional M-P neuron model which used connective weights and a nonlinear activation function simulate the operation of neural’s synapse and soma respectively. During the training process, weights are tunable while the function is settled beforehand. Obviously this model is a simplified one comparing with that of the biology neural. Thus its capability is limited. Based on this, a tunable activation function neuron model (TAF) and a multilay feedforward neural networks with this neural model (TAF-MFNN) are presented. Compared to the traditional multilayer feedforward neural network (MFNN), the TAF-MFNN can deal with more difficult problems easily. Also it can simplify network’s architecture and achieve excellent network performance and generalization capability. However, the speed of convergence of BP algorithm which is used to train the networks is slow. The other shortcoming of this algorithm is that it is prone to get into a local optimum. It is well known, The RLS algorithm uses a modified BP algorithm to minimize the mean squared error between the desired output and the actual output with respect to the summation. Therefore, in our research, we propose to transform the architecture of the TAF-MFNN and enable it with a faster learning algorithm. The modified neural networks is equilvalent to the original networks. By simulations, the modified algorithm can improve the convergent speed and accuracy. Based on this, we have ameliorated the TAF model and presented a new neural networks with multi-output neural model (MO-<WP=7>MFNN). The algorithms such as RLS algorithm, LM algorithm, LMAM algorithm and OLMAM algorithm are used to train MO-MFNN. It is obtained the conclusion: when the training samples is small, LM algorithm or LMAM algorithm or OLMAM algorithm is selected to train the networks; When the training sample is very large, RLS algorithm is used to train the networks. The mean squared error function is used extensively in the training of backpropagation neural networks. Until now, most of the fast learning algorithms were derived based on the MS error function. Despite the popularity of the MS error function, there are two main shortcomings in applying those MS error based algorithms for general applications. On the one hand, there are many sub-optimal solutions on the MS error surface. The networks training may easily stall because of being stuck in one of the sub-optimal solutions. On the other hand, the MS error function, in general, is a universal objective function to cater for harsh criteria of different applications. However, there is a common view that different applications may emphasize on different aspects. To have an optimal performance such as a low training error and high generalization capability, additional assumptions and heuristic information on a particular application have to been included. One of the technique to absorb the a priori knowledge is regularization. Namely, a regularized index function is constructed. The learning algorithms for MO-MFNN with regularization are researched in this paper. By simulations, it is shows that it can decrease the computation complexity and storage by using of MO-MFNN. Neural networ更多还原