节点文献
云计算环境下高分辨率遥感影像存储与高效管理技术研究
Technologies of Storage and Efficient Management on Cloud Computing for High Resolution Remote Sensing Image
【作者】 康俊锋;
【导师】 刘仁义;
【作者基本信息】 浙江大学 , 地图学与地理信息系统, 2011, 博士
【摘要】 随着对地观测技术的发展,我国已初步形成天地一体化的遥感应用体系,各行业和科研机构已积累了大量的高分辨率遥感影像数据。然而,这种按行业分散存储及管理的方式导致形成的“信息孤岛”,大大制约了高分辨率遥感数据共享及应用的发展。因此,如何整合并管理现有高分辨率遥感影像数据资源,及在分布式环境下为高分辨率遥感影像共享服务及高性能应用服务提供技术支撑是一个亟待解决的问题。近年来,云计算技术的不断成熟和发展,为高分辨率遥感影像快速获取及高效处理提供了一种新的有效方法。论文在分析高分辨率遥感影像特点及应用的基础上,结合云计算的虚拟化、分布式存储、分布式计算技术,设计云计算环境下的高分辨率遥感影像存储模型、管理平台,以及高性能计算等服务,并将本平台与土地利用规划业务结合实现了一个原型实验系统。本文的主要研究内容如下:(1)设计云计算环境下的高分辨率遥感影像存储模型C-RSM在分析和对比当前主流云平台基础上,提出整合已有云平台Hadoop及Eucalyptus,并围绕遥感影像数据共享及地图服务等应用的特点,设计了基于Hadoop云平台下的高分辨率遥感影像数据组织方法;版本变更管理机制;并提出了Hadoop云平台下高分辨率遥感影像数据划分及存储策略;及设计了Hadoop云平台下高分辨率遥感影像存取算法,存取算法主要讨论了两种算法,分别是Hadoop下默认透明方式的高分辨遥感影像存取算法,和在提出Hadoop下的基于Block的金字塔模型索引基础上,设计了Hadoop下遥感影像地图服务的访问算法。(2)在C-RSM模型基础上,设计高分辨率遥感影像管理平台C-RSMP在分析和对比当前分布式环境下的空间数据管理技术基础上,提出采用云计算技术管理高分辨率遥感影像的优势,并在分析了当前云GIS发展不足后,着手设计C-RSMP的体系结构、服务结构,以及C-RSMP中的高分辨率遥感影像基础服务,包括云计算环境下的高分辨率遥感影像数据共享服务、地图服务、及高性能计算服务。在服务设计中首次提出将GPU的并行计算能力与云计算技术结合,并在设计GPU加速的影像重采样算法及金字塔模型创建算法基础上,设计了Hadoop下GPU加速的影像重采样算法、及高分辨率遥感影像缓存管理算法,另外高性能计算服务的研究内容分为基于Eucalyptus虚拟资源管理的分布式任务、及基于MapReduce的高性能计算。(3)将C-RSMP与土地利用规划业务结合实现原型实验系统:讨论了系统构架及系统中云计算环境的部署,实现并展示了系统中土地利用规划业务、云平台管理、高分辨率遥感影像数据共享及地图服务、高性能计算服务等功能模块。选取浙江省部分土地利用规划数据进行效率测试,主要针对系统提供的高分辨率遥感影像存取服务、地图服务、及高性能计算中关键算法在本实验平台上进行了多组对比效率测试,实验表明平台中的算法正确,性能可靠。研究结果表明,本文采用基于遥感影像应用来设计云计算环境下高分辨率遥感影像存储模型、及管理平台的研究方法,可以解决当前遥感影像分散存储与管理的问题。本研究提出的两种高性能计算服务,其中基于虚拟资源管理的高性能计算可为其他串行算法移植到本平台提供基础,而基于MapReduce的GPU加速遥感影像算法可为其他遥感影像处理串行算法移植到本平台提供借鉴。本文将C-RSMP与土地利用规划行业应用结合,同样可为其他行业应用迁移到云计算环境提供一种新途径。
【Abstract】 With the development of earth observation technologies, integrated earth observation system of remote sensing satellites and ground systems has been established preliminary in China, and numerous high resolution remote sensing image data has been accumulated by various industries and research institution. However, the "Information silos" caused by dispersed storage and management, has seriously restricted the development of high resolution remote sensing image data sharing and application. The issues of how to integrate and manage the high resolution remote sensing image resources, provide technical support for high resolution remote sensing image sharing services and high performance application services in distributed computing environment, is urgently to be solved. In recent years, the development of cloud computing technology provides a new effective method for rapid accessing and efficient processing of high resolution remote sensing image.Based on the analysis of characteristics and applications of high resolution remote sensing images, combining cloud computing technology, which including virtualization, distributed storage, and distributed computing; the paper designed a model of high resolution remote sensing image storage, and then designed a management platform and some high-performance computing services. Finally, a prototype experimental system was developed by combined land plan services with this platform. The main research works are as follows:1. Designed a high resolution remote sensing image storage model C-RSM on cloud computing:by analyzing and comparing current mainstream cloud computing platforms, and focusing on characteristics of remote sensing image data sharing and map service etc., we designed high resolution remote sensing image organization method on Hadoop, and spatiotemporal management mechanism; then proposed storage strategy of high resolution remote sensing image data on Hadoop, and designed an algorithm of storing-accessing high resolution remote sensing image on Hadoop.2. Designed a management platform of high resolution remote sensing image based on C-RSM, which is called C-RSMP:after analyzing and comparing the spatial data management technology on current distributed computing environment, we summarized the advantages of using cloud computing technology to managing high resolution remote sensing images, and we integrated two cloud computing platform (Hadoop and Eucalyptus) for managing spatial data, then by analyzing the deficiency of current cloud GIS development, we designed the C-RSMP’s architecture structure, services structure, and designed C-RSMP’s high resolution remote sensing image fundamental services, including sharing service, map service, and high performance computing services of high resolution remote sensing image data in cloud computing environment. We proposed an approach that combined the parallel computing capability of GPU to cloud computing for the first time. Based on designing the algorithms of resampling and building pyramid model, which are accelerating by GPU, we redesigned these algorithms on Hadoop. The high performance computing services included distributed tasks based on Eucalyptus virtual resources management and high performance computing based on MapReduce.3. Achieve the prototype experimental system of land plan services on C_RSMP: firstly we discussed the system architecture and system deployment in cloud computing environment, then demonstrated the system’s main modules, including land plan affairs, cloud computing platform management, sharing service, map service, and high performance computing service of high resolution remote sensing image data. Finally, we mainly tested the key algorithms of storing-accessing service, map service, high performance computing service of high resolution remote sensing image, by using land plan data of several parts in Zhejiang province. The results of experiments proved the high performance of our platform.The results showed that our research works can solve the problem on distributed storage and management of remote sensing image. The proposed high-performance computing services can be referenced for other serial algorithms migrating to our platform. Our research work on combining C-RSMP and land plan industry can also be applied to other industries to migrate to the cloud computing environment.
【Key words】 Cloud Computing; High Resolution Remote Sensing image; Land Plan; Storage and Management; High Performance Computing; Virtualization; MapReduce; Hadoop; Eucalyptus;