Internet环境下的资源管理模型及算法研究

【作者】 刘乃文；

【导师】 刘方爱；

【作者基本信息】 山东师范大学 ， 管理科学与工程， 2009， 博士

【摘要】 网络管理理论与技术是管理科学与工程的一个重要研究领域,随着互联网技术日益成熟,网络资源越来越丰富,从管理科学角度研究如何有效地进行网络资源管理、提高网络资源利用率具有重要意义,因此新一代网络应用平台和网络管理理论及关键技术是目前网络管理技术的一个重要研究方向。迄今为止,现存的技术主要为资源集中管理提供了有效的管理方法,但很多情况下资源都分布存放在不同地点,需要实施分布管理的方法和技术。例如,考虑我国中小学教育资源共享问题,目前各个学校拥有丰富的教学资源,包括各学科教学课件、优秀教学案例、实验案例、精品课程的教学录像以及各类教学管理软件等,如何利用网络资源管理技术高效的共享这些资源,避免盲目重复开发,是迫切需要研究的问题。因此,研究基于分布式资源管理的网络体系结构,开发标准的网络资源管理平台具有重要意义。作为一个典型应用,可以利用该平台实现中小学教育资源的分布管理,为我国基础教育资源管理提供一个通用开放的软件支撑平台,对于加强各地中小学之间的交流与合作,提高整体教学水平具有重要作用。目前,不同机构的资源一般都集中存储在各自的服务器上,以Web方式供用户访问。这种集中式资源管理具有很高的效率,但是Web服务器瓶颈问题、资源副本管理问题、资源发现问题和网络带宽问题等若干因素给这种应用模式带来很大的挑战,诸多技术问题亟待解决。网格作为一种日益流行的网络计算平台,为网络资源管理提供了一种有效的解决方案。在不改变现有资源存储模式的前提下,建立一个分布式资源管理平台来整合这些资源,需要研究网络环境下分布式资源管理的网络体系结构、资源管理机制和服务理论。论文以我国中小学教育资源管理为应用背景,重点研究了网络环境下进行资源共享的系统拓扑结构、资源发现机制、资源感知模型及资源副本管理技术等,为建立互联网络环境下网格资源管理平台探索有效的方法和途径。论文主要研究内容及创新工作包括:1.研究了资源共享系统的拓扑结构,以网络节点处理能力和物理网络参数为依据构建了RP(k)C-Grid系统,设计了高效的消息路由算法MR-RP(k),解决了现有拓扑结构的系统瓶颈、可扩展性差及拓扑失配问题。目前网络资源管理系统主要有集中式、分布式非结构化和分布式结构化等结构形式。这些结构各有其不足:集中式结构的服务器是系统瓶颈;分布式非结构化网络基于泛洪的查询机制容易造成网络拥塞,系统可扩展性差;分布式结构化网络目前主要采用DHT机制组织网络节点和资源,这种方式使得物理上相邻的节点经过哈希算法处理后逻辑上可能相距很远,降低了网络的运行效率。分布式结构化网络的DHT路由策略、节点的动态离开与加入给系统带来的震荡也是需要深入研究的问题。由于RP(k)结构充分利用了Petersen的稳定性和环的简单性,具有短的网络直径、简单的拓扑结构及高效的路由策略,基于RP(k)构建的RP(k)C-Grid系统和其它拓扑结构相比具有更好的稳定性和更高的通信效率。2.针对资源搜索问题,设计了RP(k)C-Grid上的PFS资源发现算法,提出了查询登记机制避免重复泛洪。和其它搜索算法相比,PFS算法有效降低了资源搜索延迟时间,减少了查询消息跳数,在系统规模增大时有更高的搜索效率。目前资源搜索方法主要有集中式方法和分布式方法,集中式资源搜索效率较高,但可扩展性差,存在单一故障点;基于DHT的分布式系统资源定位方法可伸缩性好,但分布式哈希表的拓扑失配问题降低了系统的资源搜索效率。PFS搜索算法充分利用了RP(k)C-Grid系统的结构特点,有效发挥簇头节点的性能优势,采取簇头探察、簇头泛洪和RP(k)C轮询相结合的方法减少泛洪次数。查询登记机制使得对同一资源的重复查询请求可以在簇头节点进行高效处理而避免重复泛洪。3.为提高簇头节点的资源感知能力,设计了资源目录全局广告算法,使得簇头节点能够快速获取全局资源目录视图,以良好的资源感知能力实现对资源搜索的快速响应。资源需求者对共享资源信息的感知能力是影响资源发现效率的重要因素。广告使得供应商以最快捷的方式把商品介绍给用户,用户也以最小的搜索代价从广告中获取满意的商品信息。基于这种市场模式的启发,建立了资源目录广告模型,设计了RP(k)C上的资源目录全局广告算法RP(k)C-DA,该算法分两部分实现:Petersen内目录广告和RP(k)C目录广告。针对不同系统规模进行了实验分析,和其它文献的方法进行了比较,验证了资源目录全局广告算法的优势。4.从提高副本利用率、降低副本维护代价角度入手,提出了基于资源访问热度队列策略的自适应关口副本算法,把访问热度高的资源在其它节点建立副本,均衡访问负载,降低当前资源节点的副本替换频率。资源副本是网络环境下文件资源共享时保证资源可用性和降低访问迟延的有效方式,但是副本如何放置才能达到更高的使用效率而不浪费网络带宽和存储空间,这需要在特定拓扑结构下进行分析。针对RP(k)C-Grid拓扑结构,提出了自适应关口副本机制,对成功跨簇查询的资源文件在簇头节点建立副本以提高查询响应速度。由于频繁的关口副本建立与替换会增加簇头节点的存储代价及性能损耗,提出采用资源访问热度队列策略对副本进行管理维护,实验证明该策略有效改善了系统副本管理性能。5.以教育资源共享管理为应用背景,利用研究的网络资源管理技术建立了一个教育资源网格系统RP(k)C-Grid,实现了网络环境下的资源管理、副本管理以及资源搜索等。根据都柏林元数据标准和CELTS-42规范提出了一种实用的基础教育资源元数据模型,用来表示教育资源网格中文件资源的元数据信息。目前我国各省市及县区的中小学和各级教育管理部门一般都拥有较高性能的服务器,并积累了大量的教学资源。这些服务器在物理位置的分布上相对规则,适合建立结构化拓扑。利用研究的网络资源管理技术设计了一个教育资源网格应用实例,分析了网格构建的关键技术,给出了系统主要模块设计。RP(k)C-Grid系统的建立和运行方式更适合目前我国中小学教育资源管理现状。本文的研究进一步丰富和完善了网络环境下资源管理技术的模型及算法,为网络环境下资源共享探索有效的方法。更多还原

【Abstract】 Theories and technologies of network management is an important research field in management science and engineering. With the development of the Internet, network resources are getting richer day by day. And the research on how to carry out network resources management effectively from the perspective of management science has an important significance in improving resource utilization ratio. Therefore, a new generation network application platform and theories as well as key technologies of network management is a significant research direction on present network management. Up to now, existing technologies focuse on providing effective ways for centralized management resources. In most circumstances, resources are stored in different places and call for methods and techniques of distributed management. For instance, every school, nowadays, occupies a variety of educational resources, including courseware, excellent teachers’teaching cases, experiment cases, first-rate classes video tapes and education management software, etc. So, it’s an urgent problem to be discussed that how to share those resources efficiently by means of network management and avoid developing repeatedly and blindly. It is necessary to do some research on network architecture based on distributed resource management and develop standard platform of distributed resource storage and management. As a typical application, it can make use of the platform to realize the distributed management of educational resources in primary and secondary schools and provide a universal software support platform to our country’s basic education resource management. This platform will play an important role in the exchange and cooperation between primary and secondary schools across the country, and help to reach a higher teaching level.At present, most of the institutions’resources are used to store in their own Web servers for user access. It’s an effective resource management method for the centralized files, but the web server bottleneck problem, the resource replica management problem, the problems of resource discovery and the network bandwidth bring challenges to this application model, and there are still many technical problems to be solved. Grid, as a network computing platform, is becoming more and more popular, and it will come up with an effective way for network resource management. On the premise of unchanging resource storage model, it is considerable to build a distributed resource management platform to integrate these resources. Therefore, we need to study the network architecture, resource management mechanism and service theory in network environment.My thesis choose the primary and secondary schools’educational resource management as background, focuses mainly on resource sharing system topology, resource discovery mechanism, resource awareness model, resource replica management model and algorithm in order to explore an effective method of resource grid platform under the Internet environment. Followings are the main details and new ideas.1. Topologies of resource sharing systems are investigated, the RP(k)C-Grid system according to network nodes processing ability and physical network parameters is built up, efficient message routing algorithm- MR-RP(k) is designed, and system bottleneck, limited extendibility as well as topology mismatch of existing topology structures are solved. Present resource management systems mainly have forms of topology including centralized, decentralized unstructured and structured mode, which have their own weaknesses: the central server is system bottleneck of centralized structure; the decentralized structure network based on the flooding mechanism is easy to lead network congestion and has limited extensibility; while structured network mainly adopts DHT mechanism to organize system nodes and resource, which may enlarge the distance with the physical adjacent connection nodes and decline the network efficiency. At the same time for the routing policy of DHT, dynamical node departure and arrival bring instabilities to the system, which call for a further research. RP(k) structure takes full advantage of Petersen’s stability and ring’s simplicity, which has short network diameter, simple topology structure and efficient routing policy. Compared with the other topologies, the RP(k)C-Grid system has better stability and higher communication efficiency.2. PFS resource discovery algorithm of RP(k)C-Grid is designed in the point of problem on resource searching, and the query registration mechanism is proposed to avoid repeated flooding. Compared with other searching algorithms, PFS algorithm decreases searching delay effectively, reduces hop count of searching messages, and has an excellent searching efficiency especially when the system scale is increasing.Centralized and distributed methods are the two main ways of present resource searching. The centralized resource searching has a lower extensibility and single-point failure though it is more efficient; while the distributed resource searching method based on DHT has good scalability but bad efficiency in searching on the reason of topology mismatch. PFS searching algorithm makes full use of structure characteristic of the RP(k)C-Grid system and performance advantage of the cluster head nodes, adopt a combined mechanism of cluster head probe exploration, cluster head flooding and RP(k)C scan to decrease flooding times. The query registration mechanism makes it possible for the repeated query of same resource to be responded efficiently on the cluster head nodes and make the repeated flooding avoidable.3. A global advertising algorithm of resource directory is designed to improve the resource awareness ability of the cluster head nodes, which makes nodes get a global view of the resource directory rapidly and realize quick respond of resource searching.It’s an important factor of resource discovery efficiency that how the resource requester apperceives the shared resource information. By using advertisement, the supplier can promote their products in the fastest way to the consumer, and the consumer can get product information with a lowest cost through advertisement. Inspired by this marketing mode, a resource directory advertising model is proposed and a global resource directory advertising algorithm of RP(k)C-DA is designed, which is divided into two parts including Petersen inner directory advertisement and RP(k)C directory advertisement. From experiments of different system scales and comparison with methods of other papers, the advantages of the global advertising algorithm of resource directory is verified.4. A self-adaptive gateway replica algorithm based on resource access popularity queue policy is proposed from the perspective of improving replica utilization ratio and decreasing replica maintenance cost. By creating replica in other nodes for resources having hot popularity, it can balance access load and reduce replica replacement frequency of current resource node. Resource replica is an efficient way to guarantee the availability and decrease the access delay in network environment. However, it’s really a tough work to deal with the replicas placement in order to achieve its highest-performance and avoid storage waste, which asks for analysis under particular topology. In the point of topology structure of RP(k)C-Grid, a self-adaptive gateway replica mechanism is proposed which creates replica in the cluster head nodes for resources from successful cross-cluster query to improve system response speed. For frequent gateway replica may increase the storage cost and the performance loss of the cluster head nodes whereas the system uses resource access popularity queue policy to manage the replica. Experiment proves that this strategy improves the system replica management performance efficiently.5. On the application background of educational resource sharing, the educational resource grid is built supported by network resource management technologies of my thesis, which realizes resource management, replica management, and resource query under network environment.On the basis of the DC metadata standard and the CELTS-42 norm, an educational resource data model is put forward to describe file resource metadata. At present, primary and secondary schools as well as educational management departments of various provinces and cities generally have high-performance servers, and accumulate amount of teaching resources. These servers have relative rule distribution in physical location, which is fit to build the structured topology. An educational resource grid instance, RP(k)C-Grid, is constructed based on research technologies of network resource management, the key technologies are analyzed and the main system models are designed. The establishment and operation way of RP(k)C-Grid system are more suitable to the present management situation of educational resources sharing for primary and secondary schools of our country.My thesis steps further to enrich and perfect the resource management model and algorithm in network environment, and provide an effective method to the exploration of network resources sharing.更多还原