【作者】 肖亮；

【导师】 谢长生；

【作者基本信息】 华中科技大学 ， 计算机系统结构， 2009， 博士

【摘要】 随着信息化数据的数量和重要性不断增大,对计算机存储系统的容量、I/O性能以及可用性,可靠性,安全性等方面提出越来越高的挑战。基本的解决思路是通过构建具有标准接口的层次性存储系统,使之能够合理的集成更多软硬件部件以满足对存储系统诸多方面的要求。但是在传统存储系统中,不同层次之间的存取接口隐藏了应用、主机系统和设备各自的细节,上层丰富的语义信息无法为存储系统所获取和利用,降低了存储系统高效管理和组织数据的能力。另一方面,存储设备本身所具有的计算能力并没有被充分利用以改善系统性能。而存储对象及其属性管理是能够解决上述问题。并且基于对象存储架构具有智能的存储设备能够感知各种不同用户各自的存取特征,以保障其存储服务质量(QoSS,Quality of Storage Service)。存储服务质量是存储系统在提供数据传输过程中需要满足用户应用需求的一系列服务请求,旨在为用户应用提供服务分区和性能保证。具体可量化为存储容量、数据可用性、I/O速度、可扩展性和服务成本等。服务分区是根据不同应用需求为其提供不同质量保证的存储访问;性能保证则要解决诸如带宽和延迟等性能指标的保证问题。目前网络存储的许多技术和思想本质上是I/O性能的优化、可管理性问题。实施存储系统的服务质量机制能有效解决存储系统规模与系统管理之间的矛盾。首先以对象存储技术和属性管理为基础,借鉴网络通信系统的QoS控制和管理机制并考虑存储系统自身的特点,通过QoS分类学建立了基于服务质量的存储资源管理体系,改进了基于属性的存储服务质量描述和相应的对象服务质量体系的实施框架,涵盖了QoSS提供机制以及QoSS控制和管理机制,这些是属性存储实施和优化的基础。对一个对象存储系统的QoSS构架及相关的QoSS优化机制和策略进行了分析,并研究了一种基于请求拆分的QoSS优化方法。在此基础上,总结了一些典型对象存储服务质量优化方法,对TCP延迟性能模型在理论上进行了探索。接着针对大量小文件复制和迁移性能较差的现象,尤其在分布式环境下这种现象极为突出,提出了一种批量小文件服务质量优化方法。在ext3文件系统基础上对于批量小文件复制和迁移过程进行了研究,并引入多种优化策略。实验表明,串行读并行写过程在本地复制中具有最佳的表现;而聚合复制方法在网络复制中具有最好性能,同时也获得了元数据操作相关的实验数据,为进一步优化文件系统性能打下良好基础。最后,在对象存储系统原型(AMSS)的基础上,以QoSS控制和管理机制为目标,提出了基于对象延迟和带宽属性的存储优化策略。在iSCSI协议和面向对象的扩展SCSI命令集的基础上,定义了符合OSD T10标准的对象延迟和带宽扩展属性页,实现了基于属性的对象访问接口,作为属性传递机制的基础。测试结果表明,从聚合输出带宽来看,与没有采用QoSS优化策略的基本系统相比,采用QoSS优化策略的性能提高了28-38%。与基于iSCSI的系统相比,AMSS能支持更多的客户端。通过有效的QoSS管理和控制机制,能获得比不采用QoSS管理和控制机制更好的带宽和端对端延迟的QoS保证。更多还原

【Abstract】 With the explosive growth of the data and the importance of the information, more and more challenges on capacity, I/O performance, availability, reliability, security of the storage system have been put forward. The solution to this issue is to establish the hierarchical storage system with standard interface so as to integrate abundant software and hardware resource and achieve the user application requrements. However, the access interfaces that exist in the different hierarchies of the traditional storage system have covered the detailed information from the top user between application, host and device and the abundant semantic information can not be obtained by the storage system, which decreases the manageability and the ability on data organization in network storage system. The computing ability of the storage device can not be fully utilized for the improvement of system performance. The object based storage and attribute management hold great potential expectation for the solutions of the above problems.Moreover, the intelligent storage devices in object based storage framework are able to aware the access patterns from diverse user applacations, which is avalaible for the QoSS (Quality of Storage Service) gurantee. Quality of storage service is a general metric for the system prformance evaluation, which provides service partition and guaranteed performance for the users. QoSS can be quantitatively described as a series of parameters, such as capability, availability I/O throughput, scalability, service cost and etc. Service partition aims to provide different guaranteed quality storage access in accordance with the different application requirements; while guaranteed performance means to solve the problems of performance parameters, such as bandwidth, delay and etc. In some sense, many techniques and ideas which involved with the network storage are mainly focused on the issues of optimization and manageability. Enforcing QoSS efficiently can decrease the complexities and difficulties in the large scale storage.Firstly, based on the object storage technology and attribute management, learning the QoS control and management mechanism from the network communication system and considering the character in storage system, the storage resource management system based on QoS is eatablished in accordance with the QoS methodlogy, meanwhile, the attribute based QoSS description and relative enforcing framework are improved, including the QoSS providing mechanism as well as the QoSS control and management mechanism, which constructs the base for implementation and optimization of the attributes based storage. A QoSS framework based on object storage and relative QoSS mechanism are introduced and a QoSS optimization approach based on request breakdown is analyzed, which summarizes the typical optimization strategies about the QoSS. The beneficial research on TCP delay performance model has been carried out theoretically, which is meaningful for the further research.Secondly, considering the phenomenon of replicating batch small files always represents poor performance in systems, especially in the distributed system. A novel method on QoS optimization of batch small files is proposed. Parallel, consecutive, aggregating and other polices have been implemented in the study and optimization of the replication and emigration process for batch small files on ext3 file system. The experiment shows that the algorithm of consecutive reading source files and parallel writing target files have the best performance in local replication, and aggregating algorithm also do in network replication. Some relevant data about metadata operation have also achieved in the experiment, which will be helpful for the further optimizing file system performance.Lastly, in order to demonstrate and evaluate the proposed strategy, an attribute-managed storage prototype system with guaranteed QoSS called AMSS is designed and implemented in accordance with the QoSS control and namagement mechanism. The storage strategy based on delay and bandwidth attributes is proposed, which is the base for the attribute transmission mechanism. The object based access interface based on extension of OSD and iSCSI protocols is implemented. The experiment result shows that the performance of aggregate output bandwidth in the system QoSS optimization strategies increased by 28～38% than the general system without QoSS optimization strategies. Compared with the iSCSI based storage system, AMSS can maintain more client number. By effective QoSS control and management mechanism, AMSS can achieve the better performance and end to end delay guaranteed QoSS than general system.更多还原