【作者】 宋振华；

【导师】 孙凝晖；

【作者基本信息】 中国科学技术大学 ， 计算机系统结构， 2010， 硕士

【摘要】 随着CPU、RAM、磁盘阵列、高速互连网络等硬件技术的发展,以及以虚拟化技术、并行计算和分布式系统等软件技术的发展,网格计算、效用计算、服务计算、云计算等新兴计算模式不断涌现,并得到了迅猛发展。虚拟化技术提供的服务整合、资源动态分配技术、增强的安全隔离等特性可以很好地解决这些新兴计算模式所遇到的安全隔离、资源分配等问题。与此同时,随着全球数据信息量的持续爆炸性增长,企业和个人对数据存储空间的需求也越来越大,用于购买存储设备的支出、以及存储设备在总的IT开支中的比例也不断增大。但是一方面是快速增长的存储设备开支、较高的设备能耗,另一方面却是较低的存储空间利用率,这些都对存储管理技术提出了新的需求和挑战。为此,出现了自动精减配置、空闲空间回收、重复数据缩减、基于云计算模式的云存储等许多新的存储管理技术和存储服务模式。云存储模式同其它云计算一样,基于虚拟化技术,通过整合众多异构、廉价的服务器和存储设备,构建众多的虚拟化的实例,为企业和个人用户提供持续不断的应用和存储服务,不仅可以满足广大中小企业和个人的计算和存储需求,而且也有效地减少了中小企业的存储开支和能耗。但设备开支和能耗压力又转移到了云存储服务端。自动精减配置是目前许多存储管理产品都具备的功能,基于存储虚拟化技术,减少每个虚拟卷的初始配置大小,仅在必要的时候才进行扩展,以此来提高存储空间利用率。因此在云存储服务端结合自动精简配置技术管理存储资源,以有效提高存储利用率,应是云存储服务端存储资源管理的重要趋势。虽然自动精减配置技术可以大幅度提高存储空间利用率,但是由于大量的文件删除等操作,虚拟卷中仍然存储大量的空闲空间无法利用。目前仅有较少的几种自动精减配置技术配有空间回收技术,而且其中有许多都是要在迁移时才能进行空闲空间的回收,因而广泛存在回收效率不高、影响用户使用、应用范围狭窄等缺点,在实际使用时无法真正有效地回收利用空闲空间。本文针对目前云存储系统中存在的提高存储利用率的需求,扩展了自动精减配置技术,提出一个虚拟存储在虚拟机间按需动态流动模型(Virtual Storage Flow on-Demand, VSFoD)。该模型将一台服务器拥有的所有存储资源都整合到一个统一的存储池中,从存储池中创建逻辑卷,作为每个虚拟化实例所拥有的虚拟存储空间。基于该模型,虚拟存储空间可以根据需要动态地扩展;另一方面,当整体存储空间紧张时,为了响应其它虚拟化实例的虚拟存储空间的扩展与创建需求,也可以收缩某一虚拟化实例所拥有的虚拟存储空间,回收其一部分空闲空间;若回收了空闲空间后资源仍然紧张,则可以向网络中的邻居节点请求共享邻居节点的空闲空间。即存储空间可以由一个空闲空间较多的虚拟化实例“流”向空闲空间紧张的虚拟化实例。由此,该模型比起现有的自动精简配置技术,更能有效地提高存储利用率,进而降低存储开支和能耗,也更加适合于云存储模式。Xen是一个拥有较高性能的开源的服务器虚拟化技术,许多企业的产品和科研项目都基于Xen,包括在业界非常有代表性的亚马逊(Amazon)公司和EC2(弹性计算云)和S3(简单存储服务)。本文中实现的VSFoD模型的原型系统也是基于Xen。在原型系统中,本文首先改进了现有的逻辑卷管理器,使其能够考虑到各个物理存储设备的性能差异,提高逻辑卷的读写性能,实验结果表明,应用该改进方法,可以显著地提高所创建的逻辑卷的读写性能;另一方面,能够在创建逻辑卷时,可以指定所映射的物理存储区域,增强逻辑卷管理器的管理功能。然后,本文改进了Xen的虚拟块设备管理,使得扩展或收缩某一虚拟块设备后,它所在的虚拟机能够感知其大小的变化。否则,只能重启Xen虚拟机才能更新它的虚拟块设备大小,而这样就会影响到用户的数据访问操作。此外,本文也给出一种更加适合于Xen等虚拟化环境下的虚拟机间IO性能虚拟化方法,以保障各个用户之间对服务器IO带宽的公平共享,避免相互影响。实验结果表明,比起Xen中原有的IO调度方法,这种性能隔离方法,可以有效地保障各虚拟机间公平地共享磁盘IO带宽,也能够根据不同虚拟机的权重,为各虚拟机按比例地分配磁盘带宽。为了使用户能够感知并正常使用动态变化了的虚拟块设备,必须使文件系统的大小与之匹配,并尽量降低对用户读写操作的影响。本文最后给出了一种对Linux系统中最常用的也是默认的ext2/3文件系统进行在线收缩的方法,并给出了在Xen虚拟机中实现ext2/3文件系统在线收缩的方法。实验结果表明,该方法可以正确地收缩文件系统,并且不阻塞用户读写,对于读写性能的影响在最坏情况下也只有5%~10%。更多还原

【Abstract】 With the development of computer hardware (CPU, RAM, disk array, high-speed network and so on) and software technology (virtualization, parallel computing and distributed system), emerging computing patterns such as grid computing, service computing and cloud computing developed rapidly. Virtualization can provide service consolidation, dynamic allocation of resource, enhanced security isolation to those new computing patterns to resolve problem such as security, isolation and resource allocation.At the same time, with the explosive growth of global information, the requirement for data storage space of enterprises and individuals is increasing more and more, and the payout for storage device and its proportion in IT cost are both growing larger. Now the rapidly growing payout for storage device and high energy consumption of device, and on the other hand the low utilization of storage space poses new demands and challenges for the storage management technology. So a lot of new storage management technologies and service patterns have arisen, such as thin provisioning, free space reclaimation, data de-duplication and cloud storage.As other cloud computing, cloud storage integrate many heterogeneous and cheap servers and storage devices to construct a lot of virtual OS instances for enterprises and individuals to provide continuously application and storage service. It can not only satisfy the storage requirement of many middle & small scale enterprises and individuals but also cut down the payout and energy consumption of storage device. But then the stress of expenditure and energy consumption will be transfered to servers of cloud storage. Thin provisioning can purchase less storage capacity up front, defer storage capacity upgrades in line with actual business usage, so it’s a method for optimizing utilization of available storage. It’s an important trend to apply thin provisioning for cloud storage platform to optimize utilization of storage device.Although thin provisioning can optimize utilization of storage device, there will also be amont of free space because of frequent file delete operations. Currently only a frew thin provisioning support space reclaimation, and most of them reclaim free space only when volume is migrated to another server, so those reclaimation technology can’t reclaim free space effectively.To increase storage utilization farther, in this thesis proposed a Virtual Storage Resource Flow on-Demand Model (VSFoD) to strengthen thin provisioning. VSFoD integrate all storage resource of a server into a unified storage pool, and from which construct logical volume as the virtual block device (VBD) for every virtual instanc (virtual machine, VM). Based on this model the VBD can extend on demand, and when the storage is in short supply, some VBDs of VMs can be shrinked and even the storage of other nodes in the same LAN can be shared to satisfy the extend and creation requirement of other VMs. That is to say the storage resource can "flow" from a VM to another, so VSFoD can optimize utilization of storage device more effectively than thin provisioning, and then cut down the expenditure and energy consumption of storage device more effectively, and are more suitable for cloud storage.Xen is a high performance and open-source server virtualization project, many commercial products and scientific research projects are all based on Xen, including typical Amazon EC2 (Elastic Compute Cloud) and S3 (Simple Storage Service) in cloud computing. The prototype of VSFoD described in this thesis is also realized based on Xen.First of all the prototype improved the current logical volume manager(LVM), and when creating or extending a logical volume it can consider the different performance of every physical storage device to improve read/write performance, the evaluation result warrant the conclusion. On the other hand when creating or expending logical volume user/administrator can customize the expected mapped physical storage area, and then optimize the LVM.Then the prototype improved the virtual block device management of Xen and make VM can modify its VBD capacity after the VBD is extended or shrinked, otherwise the VM need to reboot to know the updated capacity of its VBD, and this will interfer users’ access. In addition, the prototype also give a new performance isolation method among VMs which is more sutiable for Xen, evaluation results demonstrated that this method can guarantee the disk IO bandwidth be proportional shared by all VMs according their weights more effectively than other IO schedulers in Xen.After the VBD is resized, the upper file system must be also resized to match the updated capacity of the VBD, at the same time the file system resize should try to reduce the interference to user’s read/write, and then VM can utilize the dynamic updated VBD. This thesis will describe a method to shrink the most common file system for Linux - ext2/3 online, and realize the shrink online into Xen. Evaluation results demonstrated that the method can correctly shrink ext2/3 online, and won’t suspend or block user’s read/write process, and in the worst it only bring down the IO performance only 5%~10%.更多还原