【作者】 石中博；

【导师】 吴枫； 李厚强；

【作者基本信息】 中国科学技术大学 ， 信号与信息处理， 2014， 博士

【摘要】 数字图像视频的压缩技术已经研究超过二十年,并取得了巨大的成功。然而,多年的开发使得以“预测-变换”为主的传统编码框架越来越接近其性能极限。我们有必要从新的角度分析并理解数字图像视觉内容,开发新的编码压缩方法。最近十年中,计算机视觉领域的快速发展启发我们可以从视觉内容分析出发,发掘图像的视觉相关性,改进图像视频编码性能。在本论文中,我们重点研究视觉内容分析技术与图像视频编码技术的结合,借助视觉内容分析技术对图像间视觉相关性进行分析,消除图像和视频中不同层面的视觉冗余,从而提高图像视频的编码效率。本论文的主要贡献可以总结为三个部分的工作。在第一部分工作中,我们提出了一种基于视觉模式分析的图像编码方法。该方法通过先验视觉模式描述图像低频和高频之间的视觉相关性,并由此自适应地在编码端丢弃图像中的某些高频视觉冗余,改善编码性能。同时在解码端,根据视觉模式包含的先验信息估计并恢复丢失的高频细节,改善图像重建质量。此外,我们进一步地将基于视觉模式的分析技术扩展至可伸缩视频编码应用,提出了一种新的基于视觉模式分析的层间预测方法。该方法借助视觉模式的搜索和映射,同时发掘可伸缩视频序列在时域和空域的视觉相关性,分别生成两个高质量的层间预测信号,改善可伸缩视频编码性能。此外,我们还采用了基于参数分析的预测方法,通过分析基本层已编码的信息(比如HEVC编码中的四叉树信息)来实现复杂度较低的层间预测。我们的方法通过结合多种内容分析机制,同时提供了多环路和单环路的系统实现,在编码性能和复杂度之间取得更好的平衡。在第二部分的工作中,我们提出了一种基于图像特征的高效图像编码方法。该方法通过图像局部特征匹配建立起更紧密的图像局部视觉联系,再配合像素层面的相关性分析,更有效地消除视觉冗余。具体地说,我们利用多尺度小波变换和SIFT特征提取,首先将输入图像分解为全局信息和子带内局部信息,并进行编码压缩。全局信息是对输入图像的基本描述,包含有限的视觉冗余；而子带内局部信息则是从不同的小波子带中提取的SIFT局部特征。在解码端,我们利用解码的SIFT特征从云端图像数据库中,检索出一组视觉相似的图像片。然后,结合基于视觉模式的分析和映射,将这些相似图像片中的信息与解码的全局信息融合起来,重建目标图像。根据子带内SIFT特征建立的视觉联系,我们利用基于视觉模式的映射从最低频的子带开始,由低频至高频,依次将视觉相似图像片内的信息融合进入对应的子带,恢复图像不同频带内的局部细节,直到图像完整重建。我们的方法通过结合局部特征分析技术和视觉模式分析的优点,取得更高效的图像编码性能。在第三部分的工作中,我们提出一种基于图像特征全局相似度分析的图像集整体优化编码方法。根据图像局部特征的整体统计特性,我们定义特征距离来分析图像与图像间的全局相似度。在此基础上,我们将图像集聚类为若干个相关性更强的子集,并将每个子集中图像间的关系描述为一个加权有向图。图中每个节点代表一幅图像,每一条边由特征距离加权。通过寻找该有向图的最小权值生成树,可以得到具有最小预测代价的优化编码结构。为了进一步增强图与图之间的相关性,我们提出了一种全新的基于特征的图像间三步预测方法。首先,我们利用SIFT特征匹配和多模型几何运动估计,消除不同区域的几何形变。其次,我们引入光度变换消除图像间由于光照变化带来的差异。最后,我们利用基于块的运动补偿机制生成局部优化的预测信号。我们提出的基于图像特征的方法充分利用多种内容分析技术的优点。基于特征的全局分析技术有效地确定了优化编码结构；基于局部特征匹配的图像变换增强了图像与图像间的区域相关性；基于像素的运动补偿生成了更精确的预测信号。因此,我们的方法有效地发掘了相关图像间的视觉冗余,提高了图像集整体编码的效率,同时为进一步研究大数据和云存储环境中的大尺度图像视频编码提供了新的思路。更多还原

【Abstract】 Image and video coding have been studied for more than twenty years and achieved huge successes. However, it is more and more difficult to keep improving coding efficiency in traditional ways. It is necessary to analysis and understand visual contents from a different angle. Luckily, the latest developments in computer vision inspire us to utilize visual content analysis technologies for exploiting visual correlations.In this thesis, we mainly study the collaboration on visual content analysis technologies and image/video coding approaches. These visual content analysis technologies focus on exploiting the visual correlations and help us to further remove visual redundancy. The main contributions of this thesis can be described as three parts.In the first part of our work, we propose a novel visual pattern based image coding mode. This mode utilizes pixel-level visual pattern analysis to exploit the pixel-level visual correlation. According to prior knowledge established by visual patterns, we can adaptively discard some high-frequency redundancy on the encode side, and then restore them on the decoder side. In this way, both of higher compression ratios and better visual quality can be achieved. Besides, we extend the visual pattern based approach into scalable video coding scenario to better balance the coding performance and scalabilities. We propose a novel visual pattern based inter-layer prediction approach. This approach utilizes searching and mapping among visual patterns to exploit spatial-temporal correlations and produce two enhanced signals to improve the precision of inter-layer prediction. We also adopt the parameter analysis based approach (e.g. using the coded base-layer HEVC quadtree information) to achieve high efficient inter-layer predictions as well as limited decoding complexity increases. By involving multiple content analysis technologies, our scheme supports both multi-loop and single-loop implements.In the second part of our work, we propose a novel feature based image compression scheme. This scheme utilizes robust image feature matching to achieve closer local correlations. We combine pixel based analysis and region-based analysis together to further reduce visual redundancy. Specifically, we first decompose an image into the global information and local information via multi-scale wavelet transform and SIFT feature extraction, and then compress them accordingly. The global information is the basic description of an image with limited redundancy. The local information is the SIFT feature extracted in different subbands. On the decoder side, we use decoded SIFT features to establish visual correlations with images in the cloud and further generate a group of visually similar image patches. Then, we employ aforementioned visual pattern based learning and mapping to merge these patches into global information and reconstruct the target image. The reconstruction begins from the lowest subband to the highest one iteratively. Taking advantage of two kinds of analysis approaches (the pixel-level visual pattern and region-level image feature), our method demonstrates good visual quality at very high compression ratios.In the final part of our work, we propose a novel feature based image set compression scheme. According to the global statistical characteristic of robust local features, we utilize feature distance to analysis visual similarities among images. According to feature distances, we divide a large image set into more compact subsets first. Then, we model the relationship among images in a subset as a directed graph. Each node of this graph denotes an images and the weighted value of each edge is the feature distance. By searching the minimum spanning tree with the smallest edge costs, we can achieve an optimal coding structure for a given image set. In order to further enhance visual correlations between images, we propose a feature based three-step inter-image prediction approach. At the first step, we involve multi-model estimation algorithm to estimate multiple geometric models for different image regions, and then reduce geometric distortions in these regions accordingly. Following this, we involve photometric transformation to eliminate variances caused by illumination changes. At the final step, we use block-based motion compensation to improve the local prediction precision. Our feature based scheme takes advantage of multiple content analysis approaches. The feature based global analysis determines more efficient coding structure; the feature based local matching enhances inter-image region-level correlations; the pixel based compensation produces more precise predictions. Thus, our scheme demonstrates much better performance, meanwhile shows potentials towards future big data analysis and compression in cloud environments.更多还原