【作者】 易泽湘；

【导师】 潘昊；

【作者基本信息】 武汉理工大学 ， 计算机应用技术， 2007， 硕士

【摘要】 近年来，随着以TCP／IP为核心协议的互连网的快速发展，IP技术作为一种承载业务和沟通传输的中间体得到了极大的发展。基于IP的业务应用发展迅速，网络的规模日趋庞大。网络的可用性与网络规模膨胀的矛盾显得更加突出。结构越来越复杂和规模越来越庞大的网络系统更需要网络管理软件来保证系统的正常运作，网络管理的质量会直接影响到网络的运行质量，管理好一个网络与网络的建设同等重要。网络管理软件通过对网络数据的分析达到对网络资源、性能、故障的管理，而网络数据的获取是网络管理软件中最为重要的部分，本文详细阐述了IP网络流量数据采集与存储系统的设计与实现。文章首先介绍了网络管理的一些基础知识，包括网络管理的基本概念、网络管理的五大功能以及网络管理的体系结构。其次，文章分析了网管数据采集系统所使用的主要协议，包括SNMP、NetFlow、sFlow以及RMON等。接着，文章介绍了IP网络海量数据的存储方法，并比较了几种存储方案的优劣。文章最后结合项目实践详细描述了IP网络数据采集与存储系统的设计与实现。鉴于被管网元的多样性和管理类别的不同，我们在数据采集层上采取了不同的协议采集不同的网管数据，把数据采集分为主动数据采集和被动数据采集。NetFlow、sFlow、SNMP TRAP属于被动采集的范畴，在flowd里面自动实现数据的采集、告警和入库操作：而SNMP、CLI属于主动采集，必须让采集服务器使用轮询操作来及时取得网管数据。为了避免海量数据存取导致的平凡I／O操作，在数据存储上我们采取了嵌入式数据库，并将原始数据和处理了的业务数据进行分库存储；另外为了提高系统的整体效率，我们对系统进行分层处理，每一层对应着不同的进程，数据采集层为flowd进程和arpd进程、业务处理层为flow-mining进程，而前台展示层我们则使用J2EE规范技术实现。为了提高数据接收和处理效率，我们用C语言编写了数据采集层和业务处理层，而前台展示层我们则用跨平台性良好的JAVA技术实现。更多还原

【Abstract】 With the great development of Internet , the IP network technology ,as a carrier of business and communication, having got rapid growth in recent years. The business based on IP technology also make good time ,which make the network become more and more enormous, and increase the inconsistency between availability and size of the network. The quality of the network management can directly affect the network’s transport, the construction is as important as management to network..Network management software manage the network resource、performance、fault by analyzing the network data, how to get network data is the most important part during the course of developing the network management software, This article will elaborate on the design and implement method of data acquisition and storage system.This paper first introduces some base theory of network management technology, including base conception, main functions and framework of network management.Secondly, this paper analyzes some protocols used by data acquisition, including SNMP, NetFlow and RMON.After that this paper gives a brief introduction about how to storing big data of IP network, at the same time, To show differences we compare many kinds of storing means.Finally , the design and implementation of the data acquisition and storage system allied project practice is discussed in detail at chapter 5. Taking different protocols collecting different network data in data acquisition layer for network cell and management class difference, In upper level, data acquisition is divided into active acquisition and passive acquisition.NetFlow、SNMP TRAP belong to passive acquisition, and in the flowd module acquisitive the data , save the data and transfer the data to alarm module. However, SNMP and CLI, belonging to active acquisition , must be polled regularly by polling server. In order to avoiding frequently operate about I/O, using embedded database store collected data, for origin data and business data disposed store in difference databases. In order to improve the whole efficiency of the whole system, divide the whole system into three layers , each layer corresponding with a process. Data collect layer is flowd and arpd process, business dispose layer is flow-mining, then for the front disposal we take J2EE technology.In order to promote the efficiency of receiving and disposing, we program with the C language for data collecting layer and business disposing layer . for front disposal layer we program with the JAVA language for it working different platform.更多还原