【作者】 史志伟；

【导师】 韩敏；

【作者基本信息】 大连理工大学 ， 控制理论与控制工程， 2008， 博士

【摘要】 混沌理论是现代科学技术中的一朵奇葩,它揭示了复杂系统的内在运行规律。复杂系统常呈现出高度的非线性和初始状态敏感性,初始条件的细微差异也会导致截然不同的系统运行状态。然而这些纷纭杂沓的自然现象都遵循着某种潜在的运行规律,它就是混沌理论所揭示的有序性和确定性。二十余年间,国内外学者着力于混沌时间序列的预测问题研究,其成果颇丰。然而,现存方法大多停留于理论,而忽略实际观测过程中的不确定性,与此同时,预测模型的提取缺乏有效的非线性处理机制,泛化性能及鲁棒性能也存在诸多问题。作为一种针对动态系统的机器学习方法,储备池在混沌时间序列的预测问题中性能卓越;近年来,国内外学者对储备池的预测机制展开研究,但某些性质仍无法较好地解释。基于此,本文以混沌时间序列的储备池预测方法为题进行研究,以期探索储备池的非线性处理机制,另觅新的机器学习方法,主要研究内容包括:1、分析和建立混沌时间序列的储备池预测模型。对于一些确定性混沌序列,基于储备池的迭代预测方法性能卓越,但其结构设置缺乏合理解释,而且到目前为止,这种方法较好地应用于含噪声的混沌时间序列。针对这些问题,本文首先证明此类模型对非线性系统状态轨迹的逼近特性,并探讨初始状态设置的任意性。其次,本文将储备池模型分为三类:常规状态反馈结构、输出反馈结构和前馈(静态)结构,而著名的迭代预测方法则可由输出反馈结构加以分析。通过进一步对比这三类结构,本文提出了基于储备池的混沌序列直接预测方法,该方法利用预测原点和预测时域之间的关系直接构建预测器。相对于已有的迭代方法,本文所建直接预测器的稳定性可预先加以保障,遂避免由于网络附加回路闭合而产生的稳定性和误差积累问题。2、提出储备池正则化学习方法。在现有的储备池学习方法中,存在较为严重的不适定性,表现为奇异值分布较连续、条件数较大,得到的输出权值幅值较大,从而为储备池的应用埋下了隐患。针对这个问题,本文提出储备池的正则化学习方法。该方法可通过奇异值截断或惩罚方法实现,其中,截断方法直接处理病态的系数矩阵,通过矩阵的奇异值截断,舍弃较小的奇异值以解决不适定问题;惩罚方法则采用岭回归形式,改善待因子化矩阵的性质,使其对称正定,因而可通过高效的Cholesky或高斯消元法进行求解。此外,本文还探讨了正则化方法应用于含噪声混沌序列预测的理论问题。假设时间序列所含的噪声有界,从变量含误差(Errors-in-variables)模型的角度,可得到由噪声所引起的最坏预测误差。通过最小化该误差,便得到含噪混沌序列的鲁棒最优预测模型,该模型具有惩罚正则化的形式。3、基于储备池方法,提出无核非线性支持向量机模型。传统的核方法实现了一种静态映射,但较难实现递归结构,因此无法直接处理动态模式。储备池具有“递归核”的功能,并可较好地应用于动态系统辨识。基于此,本文结合储备池的特点和传统支持向量机的处理方法,提出一种不依赖核的非线性支持向量机—支持向量回声状态机(Support Vector Echo-State Machines,SVESMs)。SVESMs的主要特点是不依赖核方法构建隐式的特征空间,它采用随机生成的储备池来处理非线性系统建模问题,在高维的储备池状态空间中进行计算。这种方法便于实现结构风险最小化(StructuralRisk Minimization),并可根据问题的不同引入不同的代价函数,当采用鲁棒损失函数时,SVESMs可处理包含异常点的预测问题。SVESMs可工作在递归模式和前馈模式。相对于传统递归神经网络,工作在递归模式的SVESMs易于训练,不存在局部最小点问题,且预测器精度高、泛化能力强;此外,工作在前馈模式的SVESMs可应用于静态模式识别问题,它具有与传统支持向量机类似的超参数和容量控制方法,但形式上与传统的前馈网络相同,从而建立起神经网络和支持向量机之间的内在联系。更多还原

【Abstract】 Chaos theory serves an important part in modern science and technology, which reveals the inherent laws governing complex system. A complex system is a nonlinear chaotic system, which exhibits the sensitive dependence on the initial conditions. Two such systems with however small a difference in their initial state eventually will end up with a large difference between their states. The objective of the Chaos theory is to explore the order and determinacy behind these phenomenons. It has been twenty years since scientists started the research work of time series prediction based on Chaos theory, and many results have been obtained. However, many prediction methods stay in laboratories, and ignore the uncertainty in practical situations. Meanwhile the access to accurate prediction model relies on machine learning tools, which are expected to be efficient in dealing nonlinearity, excellent in generalization and reliable in robustness. As an emerging machine learning tool for dynamical system, reservoir method has been shown the excellent performance in chaotic time series prediction. It draws many interests but the mysteriousness still exists. Based on the above observations, this paper will focuses on reservoir method and try to find new techniques for chaotic time series prediction and machine learning applications. It covers:1. Analyze and construct a reservoir prediction model for chaotic time series. Reservoir can be used as a good predictor for some chaotic systems, however, the predictor is in the absence of reasonable explanation and difficult to apply to noisy situations at present. Based on the problem, the approximation ability of this kind of recurrent neural networks is verified in this study, and the emphasis is placed on the problem of trajectory learning and initial state setting. The reservoir model structure is classified into three categories: general state-feedback structure, output-feedback structure and the feed-forward structure. The existing iterative predictor can be well explained in the perspective of output-feedback structure, and it is shown that this structure will lead to two difficulties in modeling chaotic time-series: error accumulation and underlying stability problems. To eliminate these problems, a direct prediction method based on general state-feedback structure is proposed, and it relates the prediction origin and horizon directly. The stability can be assured before the network training, and the prediction error does not accumulate since there is no output-feedback loop.2. Propose a regularized learning method for reservoir. The existing reservoir training method is ill-posed, which has many symptoms, such as continuous singular value spread, large condition number and output weights. Regularized learning method is then proposed to curve the difficulties in this study. Regularization can be realized by truncated singular value decomposition(TSVD) or penalty methods. In TSVD, the small singular values are discarded to improve the solution; In penalty method, the ridge regression is used to improve the matrix structure to be factorized. The computation problem is also addressed, and it is shown that the penalty method can be more efficient than the truncation method. In addition, the theory analysis is carried out to the noisy chaotic system prediction problem, the worst prediction error caused by noise is computed in the style of Error-In-Variables model. Given a perturbation bound of the noise, it is shown that the regularization learning based on penalty method can result in an optimal robust solution.3. Propose a novel nonlinear support vector machines(SVMs) without a kernel. Based on the similarity between kernel and reservoir, a kernel-free SVMs—Support Vector Echo-State Machines(SVESMs) is proposed in this study. Classic SVMs rely on kernel method to compute the inner product, by contrast, reservoir method allows the direct creation of nonlinear mapping by the mechanisms of echo state property. The computation in the reservoir state space is straight-forward and it is easy to realize the structural risk minimization principle and introduce different loss functions for various applications. Robust loss functions can be used in SVESMs so that the model is insensitive to outliers, which is very suitable for practical time-series modeling. SVESMs can work both in recurrent and feed-forward modes. In recurrent mode, SVESMs are easy to train and do not get trapped in local minimums, and the generalization ability is well supported by the statistical learning theory. SVESMs working in the feed-forward mode behave much like the classic SVMs in many ways, such as hyper-parameter searching and capacity controlling, and feed-forward SVESMs serves a bridge from feed-forward neural networks to SVMs.更多还原