【作者】 赵寅；

【导师】 虞露；

【作者基本信息】 浙江大学 ， 通信与信息系统， 2013， 博士

【摘要】 三维视频是近十年来视频处理领域的一大热点。它将传统的平面图像生动地拓展到三维空间,带给观众更为真切的视觉体验。继传统的双目立体视频系统之后,一种新式的基于深度的三维视频系统正逐渐兴起：它通过虚拟视点合成由参考视点的纹理和深度图像产生目标虚拟视点的纹理图像,从而可由参考视点和虚拟视点图像共同构成具有合适视差的立体视频,实现用户自定义的、更为舒适的三维立体视觉体验,因此被ITU和MPEG等组织确立为第二代三维电视系统。三维视频的视觉质量较二维视频更为复杂。双目立体观看中,不同的双目图像将激发深度感知、双目融合和双目竞争等多种双目视觉特性,从而引入双目感知相关的视觉质量影响因素。因此,三维视频的视觉质量被认为是包含图像质量和深度感受等若干因素的多维问题；求解这个复杂问题,可能需要对众多因素分而治之,依据视觉特性建立定量计算模型,从而逐步完善理论基础。三维视频系统包括内容获取、压缩传输、三维重现等多个环节,其中每一环节均可能扭曲理想的三维视频信号、引入视觉失真。三维视觉质量增强既可以从消除视觉敏感的某些类型失真出发,直接提升图像的视觉自然度,也可以从更好地保持受损信号与理想信号之间的信号保真度出发,降低失真的整体强度,间接地增强主观质量。本文第一章对比传统双目立体视频系统和新一代基于深度的三维视频系统,并简要阐述三维视觉质量的多种影响因素以及三维视频系统中视觉失真的引入途径。随后,总结回顾近年来三维视频视觉质量和三维视频处理领域的发展状况,进而阐明三维视觉质量的研究思路和三维视觉质量增强处理的两个可行方向。第二章详细阐述三项关于三维视觉质量影响因素建模的研究工作,这些研究包括：1)创新地提出结合人眼视觉敏感度、视觉暂留、最近效应等多个重要视觉特性,通过时空域联合分析受损视频视觉失真强度的客观质量评价算子,其系统地模拟主观质量评价过程,并且在多个质量评价数据库上稳定地取得了世界领先的评价准确度；2)率先提出基于双目融合特性和掩蔽效应的双目最小可辨别差异模型,它将传统的双目最小可辨别差异模型拓展为更完善的双目模型,为双目信号失真感知提供理论基础；3)建立了世界首个双目竞争失真的主观样本数据库,其指明三维视频处理可能引起人眼敏感的双目竞争现象,并为双目竞争失真的检测和抑制方法研究提供主观实验样本。第三章围绕虚拟视点合成模块,通过深入的理论分析,回答了两个重要问题：为什么合成图像中物体边缘区域容易产生显著的视觉失真?以及怎样通过视点合成使三维视频能够自动地呈现舒适的深度感?基于这些问题的分析结果,提出了两项视觉质量增强方法：1)抑制物体边缘失真的纹理图像改善方法,它从根本上分析并防止边缘失真的形成,相比于传统方法具有更好的理论性能和实际效果；2)在不同显示器上保持相同舒适深度感受的三维体验增强方法,该方法已经被JCT-3V组织采纳为三维视频后处理的一种辅助增强方法。以上两项工作分别从纹理真实度和深度舒适度出发,提高三维视频的视觉自然度。第四章以三维视频编码为中心,提出面向重建三维视频保真度的理论模型与增强方法。一方面,通过理论推导,建立不引起视点合成图像变化的深度失真容限模型,该模型是面向目标虚拟视点图像质量的深度压缩等处理的理论基础,并基于该深度失真容限模型提出改善深度编码效率的两项编码预处理方法。另一方面,通过深度指示的视间对应关系,率先提出在非对称纹理编码应用中,以重建的高质量视点信号增强低质量视点图像保真度的编码增强后处理,该方法明显提升低质量视点的信号保真度,并帮助抑制其中的振铃效应和块效应。这两项研究工作目标在相同的传输代价下产生具有更高保真度的解码端图像,因而也有助于提高三维视频系统的运行效率和服务质量。第五章首先总结本文的研究工作；之后,对三维视频质量分析和增强处理的未来动向作简要分析,并对三维视频产业的发展进行初浅的前瞻。综上所述,本文的研究工作以认识和提高三维视频的视觉质量出发,通过主观实验和理论分析,增进了对三维视频视觉质量的理解,提出了三维视频质量评价和增强处理方面的一些创新理论和技术方法,帮助解决三维视频视觉质量改善方面的多个热点科学问题,为今后三维视频的广泛应用提供理论积淀和技术储备。更多还原

【Abstract】 3D video (3DV) is a hot topic in the field of video processing in the past decade.3DV vividly extends2D images into a third dimension, thus creating immersive viewing experience. Preceded by stereoscopic video system, a new depth-based3D video system has received increasing interest. The depth-based3DV system employs view synthesis to generate virtual views, and can realize user-defined depth sensation by composing stereo pairs of arbitrary baseline with synthesized virtual views. Owing to this benefit, depth-based3DV is considered a strong candidate for the second-generation3DTV by several organizations, e.g., International Telecommunication Union (ITU) and Motion Picture Expert Group (MPEG).Visual quality of3D video is surely more complex than that of2D video. In stereoscopic viewing, two different images for left and right eyes may evoke binocular properties, such as the stereopsis, binocular fusion and binocular rivalry, in which the activated binocular properties can further impact the overall3D visual quality. Therefore,3D visual quality is generally considered as a high-dimension problem involving at least both image quality and depth quality. To solve this complicated issue, it may be necessary to separate attributes of3D visual quality and to establish computational models for them. These research efforts are believed to contribute to future development of3D visual quality theory.3DV system includes several components to make a real3D scene be shown to remote viewers, including content acquisition, data compression and transmission, and3D visualization. Each of them may distort ideal3D video signal and introduce visual impairments, thus resulting in visual quality degradation. There are two ways to promote3D visual quality:1) to enhance the naturalness of3D video by eliminating specific sensitive visual distortions, and2) to improve fidelity of processed3D video signal to reduce the overall distortion strength and thus promote visual quality.Chapter1introduces framework of conventional stereoscopic system and the new depth-based3DV system, and elaborates contributing factors to3D visual quality as well as the cause of visual distortions in3DV. Then, recent research progress in3D visual quality and quality-enhancing processing is reviewed, and the research directions for3D quality analysis and enhancement presented in this thesis are introduced.Chapter2discusses three works related to the modeling of3D visual quality attributes, including:1) a video quality model based on many monocular visual properties (e.g., visual sensitivity, visual persistence and recency effect), which systematically simulates subjective quality evaluation process and consistently outperforms state-of-the-art metrics on several VQA databases;2) a binocular just-noticeable-difference (JND) model which considers binocular fusion and masking effects and extends conventional JND model into binocular model, and3) establishment of the world’s first subjective database for binocular rivalry in real processed images, which points out the existence of binocular rivalry in processed3D video and collect some subjective samples of binocular rivalry artifacts for future development of binocular rivalry detection and suppression methods.Chapter3highlights the view synthesis module. Two questions are answered based on deep analysis:why salient visual distortions in synthesized images are prone to appear at object boundaries, and how to create comfortable depth sensation with synthesized views. Based on the answers, we develop a method to improve texture quality by suppressing boundary artifacts in synthesized images, which is theoretically and experimentally superior to existing methods, and also propose a scheme to maintain the same comfortable depth sensation on different3D displays, which is adopted by JCT-3V as a recommended post-processing technique for3D video. The two works enhance the naturalness of3D video from image quality and depth sensation, respectively.Chapter4proposes a theoretical model and a quality enhancing technique oriented at3DV signal fidelity. First, we develop a model describing critical depth distortions that still results in the same synthesized images, which is the theoretical fundamentals of depth processing optimized for target synthesized views, and propose its applications in depth map coding. On the other hand, with the inter-view correspondences indicated by depth data, we attempt to use the high-quality view to enhance the low-quality view in the context of asymmetric coding of texture video in3DV, which significantly improves signal quality of the low-quality view and efficiently suppresses blocking and ringing artifacts in the more-severely-distorted view. These two algorithms contribute to better signal fidelity of the coded and synthesized views, at the same data transmission cost.Chapter5first concludes the thesis, and then points out some future movements in the field of3D visual quality and related enhancement processing. The prospect of3DV applications is also briefly discussed.In summary, this paper is oriented at the understanding and enhancement of visual quality of3D video. With psychophysical experiments and theoretical derivation, we enrich the comprehension of3D visual quality, and propose novel theories and techniques for visual quality evaluation and enhancement, which help solve the key issues in improving3D visual quality and also contribute to the development of3DV systems.更多还原