【作者】 刘晓军；

【导师】 张家树；

【作者基本信息】 西南交通大学 ， 计算机技术， 2013， 硕士

【摘要】 随着显示设备的发展,3D电视在市场上普及开来,裸眼3D电视也正在进入人们的生活。3D显示给观众带来了更真实更震撼的体验,供给裸眼显示设备的多视点内容却远远不能满足广大消费者的需求。3D内容的欠缺是目前制约3D推广发展的因素之一,因此2D转3D或者双目转多目技术就成为3D视频生成的关键技术。基于深度图的绘制(Depth Image Based Rendering,DIBR)因为兼容性好、质量较高、灵活性强等特点成为3D内容制作的重要方法。利用通常的图像数据和对应的深度图可以合成虚拟视点图像,虚拟视点图像和原图像有可调节的视差,多幅虚拟视点图像就和原图一起组成了多视点内容,在立体显示设备播放。利用已有的视频内容生成供裸眼显示设备播放的多视点内容成为市场的迫切需求。基于深度图绘制的虚拟视点生成主要分为三步：深度图获取及处理、3D变换、空洞处理。由于场景中物体具有不同的景深及遮挡关系,在当前视点中被遮挡的物体或背景,在虚拟视点中可能暴露出来。暴露的区域因缺乏有效信息,常称为空洞。空洞区域处理不当,会严重影响观众的视觉体验。视频中通常有各种各样的场景,情况复杂,合成空洞区域真实的内容非常具有挑战性。因此空洞区域的处理就成为DIBR过程中重要的环节。空间一致性和时域稳定性是虚拟视点的两个重要因素,满足这两点就可以给观众良好的视觉体验。采用简单的算法填补空洞区域,常会造成纹理错位、模糊,边缘瑕疵,帧间闪烁等问题。本文针对基于深度图绘制方法生成的虚拟视点存在空洞这一问题,分析总结已有算法,将图像邻域及视频帧间信息用于填补空洞,以改善虚拟视点图像质量。本文详细讨论实现了深度图质量对虚拟视点合成的影响、基于图像修复的快速填补和基于样例的空洞填补方法,改善了虚拟视点的质量。本文主要从以下几个方面展开了工作：1)深度图预处理方面,重点实现比较了双边滤波,联合双边滤波、时空域结合的滤波对深度图质量的改进,实验通过深度图预处理改进虚拟视点质量的途径。2)针对边缘插值法在填补空洞时瑕疵明显的缺点,利用图像快速修复技术,对空洞区域进行填补,修复过程中利用方向化的快速行进法,从背景向前景进行修复,保证了距离初始边界较近的点优先修复。该方法为邻域方法,计算相对简单,纹理简单区域能较好修复；修复过程不受深度图质量影响。3)对视频中空洞,采用基于图像样例块的修复方法,对空洞区域进行帧间或邻域采样,寻找与空洞点邻域最相似图像块来填补当前空洞。利用视频的帧间信息,对当前视点遮挡的区域,通过在其他帧中搜索对应信息的出现,填补此类空洞。本文主要分析改进了基于样例方法中优先度的计算、匹配代价的计算等关键问题,提高了匹配的准确度。利用视频的帧间相关性,对空洞区域所需信息在帧间进行搜索匹配,将最佳候选信息用做空洞区域的填补,保证虚拟视点的时空一致性,实现较高质量的虚拟视点的合成。4)结合上述主要算法,开发了一款基于MFC和OpenCV的虚拟视点生成演示软件。更多还原

【Abstract】 With the development of display devices,3DTV is popular on the market and auto-stereoscopic TV is coming into our life. The3D display brings more realistic and shocking experience to us, but the contents for auto-stereoscopic are far from enough to customer satisfac-tion. The short of3D content is one limitation of3D technology generalization, so2D to3D or stereoscopic to multi-view conversion becomes the key technology for3D video generation. Due to the advantages of backward compatible, high quality and flexibility, the depth image based view generation becomes the important method for3D creation. Using the texture image and corresponding depth map we could rendering virtual views, which has adjustable parallax with the original. Original views plus their virtual views compose multi-view contents for the auto-stereo-scopic display device. The method of using existing2D video generate multi-view video for auto-stereoscopic display devices plays the urgent needs of the market. Generally, the depth image based rendering is divided into three steps: depth map acquisition and processing,3D warping, disocclusion handing. Due to objects of scene usually have different depth and occlusion, the object or the background which is occluded in the current view, may be exposed in the virtual view. Disocclusion areas are often defined as holes due to lack of effective information. The quality of these filled hole areas will seriously affect the audience’s visual experience. Because of the wide variety of complex scene of video, synthesis disocclusion area’s real content is very challenging. Handling of the disocclusion area is one of the most important parts of the DIBR. The spatial coherence and the temporal stability are two important factors of the virtual view, accomplish these factors could bring good visual experience to the audience. Simple algorithms to fill the hole areas, fast but often cause the texture dislocation, blur, edge defects and interflicker.This thesis, aims at the problem of filling disocclusion in the virtual view which is generated by DIBR, analyzes existing algorithms, tries to use image neighborhood and video interframe information to fill the hole, to improve the quality of the virtual viewpoint. This paper discusses the impact of the depth map quality with virtual view synthesis and employs fast image inpainting based method and the exemplar based disocclusion filling method, finally improves the quality of the virtual view. This paper’s work is mainly composed with the following aspects:1) preprocessing of depth map, focuses on the affect of bilateral filtering, joint bilateral filtering, and spatial-temporal filtering to generate a virtual viewpoint, explores ways to improve the quality of the virtual view by preprocessing depth map. 2) the edge interpolation method to fill the hole area always attains defects, utilize fast image inpainting technology to fill the area. As a neighborhood approximate method, the calculation is relatively simple, simple texture region can get satisfactory result and the repair procedure is not affected by the quality of depth map.3) For the hole areas of video, use the method of exemplar block of image, tries to use informa-tion from inter frame. The occlusion area of the current view may appear in other frames, so searches corresponding information to fill these holes and samples the inter-frames or neighborhoods to find the most similar image blocks with hole area. This thesis mainly analyzes the key issues of priority calculation and the calculation of the matching cost of exemplar based method. Utilizes frame coherence of video, searches the required information and matches the best candidate to fill the area, make sure spatial and temporal consistency of the virtual view to achieve synthesis of the virtual viewpoint with high quality.4) Based on these main algorithms, develops a presentation software of virtual view genera-tion based on MFC and OpenCV.更多还原