【作者】 许小艳；

【导师】 廖晓峰；

【作者基本信息】 重庆大学 ， 计算机应用技术， 2011， 博士

【摘要】 3D电视是近年来兴起的一种能够增强视频观赏体验的新型媒体介质。多视点视频技术是3D电视的基础,它能够通过基于深度图像绘制(DIBR)技术来合成一定范围内的任意视点的视图,从而使3D电视系统具有交互性和立体感。本文研究的目的,就是通过对多视点视频中基于深度图像绘制的视图合成的原理、技术和方法进行研究,为它在3D电视中的应用提供理论和技术支撑。论文首先介绍了多视点视频中视图合成的理论基础——针孔摄像机模型;然后在该模型的基础上,研究了深度图像获取和基于DIBR视图合成所涉及的关键技术。在深度图像获取方面,重点研究立体图像对的极线校正;在视图合成方面,论文首先提出了一种空洞填充算法;然后研究了基于图像重投影的视图合成,以及任意视点的基于DIBR的视图合成方法;最后研究了基于深度图像的3D视频去隔行。论文的主要研究成果包括:①针对深度图像获取过程中的极线校正问题,研究了图像重投影,并在此基础上提出了一种立体图像对的极线校正方法。该方法结合了基于基本矩阵的极线校正方法和无需基本矩阵的极线校正方法,有效地避免了最优化时陷入局部最优的可能,同时又不过分依赖基本矩阵的精度。实验表明该方法具有较高的校正精度,图像的扭曲也比较小。②提出了一种新的空洞填充算法。该算法总是先填充那些4邻域像素集中不空的像素大于等于本轮域值的空洞点;并且每填完一遍空洞点之后都重新搜索4邻域像素集中所有像素都不空的空洞点以开始下一轮填充。实验表明该算法能较好地填充较小的空洞,特别是对纹理单一区域中的空洞能产生良好的填充效果。该算法适用于经过平滑深度图像处理后剩余空洞的填充,也便于硬件实现。③针对摄像机绕光心旋转的特殊情形,给出了单应矩阵具有的性质,并结合三维图像变换技术提出了一种摄像机绕光心旋转时新视图的合成方法。由于充分利用了单应矩阵的性质,该方法不但能够生成质量较好的新视图,而且绘制速度也得到了加快。④提出了一种基于DIBR的视图合成方法。该方法执行两次三维图像变换,第一次变换采用平滑后的深度图像,得到含有较小空洞的目标图像;第二次变换采用原始的深度图像,得到含有较大空洞的目标图像。然后以第二次变换得到的目标图像为基准将两幅目标图像融合,最后对得到的目标图像进行空洞填充。该方法能在消除空洞的同时保证不会降低非空洞区域图像的质量,从而尽可能地保留视图的真实性,适用于3DTV中的立体图像对的合成。⑤研究了基于深度图像的3D视频如何在现行的电视系统中传输的问题,提出了3D视频隔行化的概念,并提出了一种适用于DIBR的去隔行算法。由于该算法采用了基于方向的场内插值算法,因而能得到画质较高的逐行新视图,适用于基于深度图像的3D视频的去隔行。作为一项具有创新性意义的工作,论文在研究方法与思路上力求有所突破,其研究成果将为多视点视频的视图合成在3D电视中的应用提供具有实用价值的新方法。更多还原

【Abstract】 3D TV is a novel medium developed in recent years that is able to enhance video viewing experience. Based on multi-view video technology, which can synthesis a range of arbitrary view by depth-image-based rendering (DIBR), a 3D TV system can provide the viewer the interactivity and stereoscopic perception.The purpose of this dissertation is to probe into view synthesis from multi-view video based on DIBR, which will provide the theories and technical-support for its applications to 3D TV.In this dissertation, pin-hole camera model, the theoretical basis of view synthesis from multi-view video, is firstly introduced. Based on this model, two main aspects are studied: depth image acquisition and the key technologies of view synthesis based on DIBR. As for depth image acquisition, the epipolar line rectification for stereo pairs is thoroughly studied. As for view synthesis, a new hole filling algorithm is firstly proposed; then view synthesis based on image re-projection is discussed, and the method for arbitrary view synthesis based on DIBR is also presented; finianlly, de-interlacing of depth-image-based 3D video is elaborated.The main research results of this dissertation are summarized as follows.①With respect to epipolar line rectification for depth image acquisition, the theory of image re-projection is discussed; and based on image re-projection, a new method for epipolar line rectification for stereo pairs is given. The method combines the rectification method which needs the fundamental matrix with the rectification without the fundamental matrix, avoiding in this way local minima as well as the dependence on the precision of the fundamental matrix. Experimental results show that the presented method works accurately with low image distortion.②A new hole-filling algorithm is proposed. The algorithm guarantees that the holes which satisfy the following condition will be filled first: the number of non-holes in 4-neighborhood-pixels set is larger than or equal to the threshold of the current iteration. After each iteration, the algorithm will re-search the holes whose 4-neighborhood-pixels are all non-holes for the next iteration. Experiment results show that the algorithm can be employed efficiently for the small holes, especially for the holes at lowly-textured area. The results suggest that the algorithm adapts to the hole filling after depth image smoothing during the view synthesis procedure. In addition, the algorithm is facilitative for hardware implementation.③As for the special case when the camera rotates around its optical center, the properties of the homography in this case is thoroughly discusses. Combining the properties of the homography with the 3D image warping technique, the paper introduces a new way to synthesize novel views when camera rotates around its optical center is proposed. Due to the use of the properties of the homography, the proposed method can generate novel views with better image quality as well as faster rendering speed.④A new method of view synthesis based on DIBR is proposed. The method performs 3D image warping two times: the first warping uses the smoothed depth image, and the destination image contains holes with small hole-size; the second warping uses the original depth image, and usually big holes appear in the destination image. After that, two destination images are merged based on the imaged formed from the second warping. Finally, hole-filling algorithm is applied to the merged image. This method can avoid degrading the image quality in non-hole area when filling the holes, thus the“authenticity”of the image is guaranteed. Experiment results suggest that the method adapts to the stereo pair synthesis for 3DTV.⑤The transmition of the depth-image-based 3D video in current TV system is discussed. The conception of interlaced 3D video is proposed. Based on this concept, a de-interlacing algorithm for depth-image-based 3D video is given. Experiment results show that, as an intra-field interpolation algorithm based on direction is applied, the proposed de-interlacing algorithm can provide progressive novel views with quite good image quality, which also suggests that the method adapts to the de-interlacing of depth-image-based 3D video.As an innovative work, this thesis tried it’s best to break through on research methods and research thoughts, and the research results will provide new methods for the practical applications of view synthesis from multi-view video in 3D TV.更多还原