【作者】 朱云芳；

【导师】 顾伟康；

【作者基本信息】 浙江大学 ， 信号与信息处理， 2006， 博士

【摘要】 随着电子信息产业的发展和技术进步，能够获取、记录视频信息的设备如摄像机，摄像头等日益普及，并随之产生大量原始视频数据。如何有效的利用计算机对这些视频进行检索和再编辑引起了研究者的广泛注意。由于视频是由一系列相互联系的图像帧构成，要达到让计算机自动处理视频的目的，必须建立起不同图像帧之间的联系，随之产生的图像匹配分割问题是视觉和图像处理领域的热点和难点之一。考虑到视频场景的内在联系，本文提出了通过构造视频的全景图，利用手工与计算机相结合的方法对全景图进行编辑，实现对视频内容的再编辑。建立全景图需要完成运动参数估计，运动物体分割，图像拼接等计算过程。而在拼接完成的全景图上对视频内容进行编辑将变得简单有效并且易于实现。本文依次研究建立全景图过程中的主要技术问题，并给出在全景图基础上对视频进行编辑的具体方法。实现视频全景图拼接的第一步是对摄像机运动参数的求解并建立视频中各帧图像之间的关系以及二维图像与三维空间对应关系。本文第二章对此问题进行了详细讨论，提出利用各帧图像的Harris角点特征，进行匹配，建立起对应关系，并采用RANSAC对匹配的结果进行投票选择来保证匹配结果的鲁棒性和准确性的方法。利用匹配结果，可以求出各帧图像间的透视变换矩阵和摄像机拍摄时的内外参数。由于视频序列中可能会有运动物体存在，这会对全景图拼接过程产生干扰。本文第三章研究了当视频中的运动物体有较大范围的运动时，对其进行分割的方法。本文提出一种两步算法，首先利用帧差法(Frame difference)来快速定位运动物体的位置，之后利用均值偏移法(Mean-Shift)准确估计运动物体的边缘并利用图切割(Graph-Cut)方法建立两者之间联系。考虑到视频的运动连续性，本文同时引入前一帧分割结果来约束当前帧的分割。该算法同时利用了帧差法，均值偏移法的优点，能够快速准确的分割在视频场景中出现的运动物体。得到运动参数和运动物体分割结果之后，可从视频帧中合成全景图，本文第四章讨论了二种图像拼接采用的模型：平面投影模型和柱面投影模型。柱面投影模型假设摄像机的光心固定，摄像机在同一平面内转动。实际视频拼接过程中，经常会有因为手持摄像机转动时出现的光轴倾斜而导致拼接后的图像发生卷曲的现象。对此，本文给出了一个求柱面投影时圆柱最佳中心轴，来抵消卷曲的解决方案。同时，考虑到在摄像机运动过程中由于光照的影响，成像时白平衡和曝光补偿量不同而导致图像颜色不一致的情况，本文给出了利用有效对应点的直方图匹配求出图像的校正参数，对视频各帧图像进行颜色校正的解决方法。与传统方法相比，该方法可以消除错误对应点对校正参数带来的影响。得到视频全景图后就将视频序列转变成了全景图表示。因此，对视频进行编辑包含对全景图像进行编辑的过程。本文第五章讨论比较并改进了三种图像编辑方法：手工交互图像移植(healing brush)、平滑图像半自动修复(In-painting)和纹理图像半自动恢复(texture synthesis)。其中第一种算法适用于为修改区域指定填充信息。第二种算法适用于编辑和修复平滑区域或者比较窄的带状区域，第三种算法适用于普通或者含有纹理的图像区域。本文改进了平滑图像修复算法，使之可以实时实现。同时对于纹理图像的编辑算法，定义了新的距离度量，减小其对颜色的依赖性。文中进行的实验证明了算法的有效性。本文第六章给出了在得到全景图后，实际进行视频编辑方法的三种应用：运动全景图生成、视频中运动物体的去除、视频图像的修复与编辑。在实际处理视频的过程中，可能会遇到视频抖动及运动补偿导致的黑边等问题。针对这些具体问题，本文分别讨论了相应的算法和解决方案，并给出了实验结果。最后，在第七章中对全文的工作做一小结并对今后可能的后续工作进行了展望。更多还原

【Abstract】 The rapid advance of the information technology and electronics has introduced many devices can capture and record video, such as, Video camera or Web camera. These devices generate huge amount of video data, and raise requirements of video processing technologies. To manually process such amount of video data is impracticable, many researchers study on automatically video indexing and video editing technologies. Considering a video sequence is a set of connective image frames, to correctly manipulate the video sequence we should keep the correspondence between frames and thus to extract the correspondence between video frames at first. The image correspondence and the along with segmentation are two of the most hot and difficult problems in computer vision and image processing domain. We deal with the problems with a new appliance, to create video scene panorama. By exploiting the intrinsic structure of the video scene, we propose to construct panoramas of video sequences. With the panoramas, a user can easily and efficiently edit video sequence with manual and automatic approaches. The process of establishing video panorama encompasses three steps, which are motion parameter estimation, motion object segmentation and image mosaic. In this paper, we study the technical problems of each step in detail and propose algorithms of the generated panorama editing. Moreover, we describe several practical appliances of panorama based video editing.To construct a video panorama, the first step is to estimate the camera motion parameters and establish the project correspondence between following video frames and projections from 3d space to the frame planes. Chapter 2 describes the approach to solve the problem. We propose to extract Harris corner features from each video frames and match those features between frames with RANSAC. Our approach uses voting to ensure the robustness and correctness of corresponding results. And with the results, we can compute the projective matrix of video frames and the intrinsic and extra parameters of cameras.Moving objects in the video sequence interfere in the result of video panorama. The chapter 3 describes how to segment and remove the moving objects from video sequence. A two steps approach is proposed. Firstly, we compute Frame difference to estimate the initial positions of the moving objects. Secondly, we use the Mean-shift to segment the video frames and combine the segment results by refining the initial estimated moving object with the computation of a graph cut. During the calculation, the segmentation result of the previous frame is introduced as a restriction of the current frame segmentation. And thus it assures the continuous and smooth segment results. Our approach incorporates the advantages of the Frame difference and Mean-shift. It can quickly locate the moving object and precisely segment the moving objects from video frames.After motion estimation and motion segmentation, we can mosaic the video frames to derive the panorama. Chapter 4 introduces two mosaic models, planar projection model and cylindrical projection model. The cylindrical projection model assumes the camera did in-plane motion during the capturing process. But in real circumstance, the hand-held camera may often tilt and it raises the image curl during the mosaic process. For this, we propose an optimization algorithm to compute an optimal cylinder axis. The approach successfully eliminates the image curl. Furthermore, luminance of video frames may change and it introduces the difference of white balance and exposure parameters among video frames. Then the color spectrum of the video frames is not consistent. We proposed an efficient algorithm to rectify the color spectrum. Itextracts the corresponding points and uses the histogram of corresponding points to compute the color correspondence. Our approach exceeds traditional approaches in robustness because it can reduce the errors introduced by the wrong correspondences.With the constructed panorama we can edit the video sequence by editing the panorama image. The chapter 5 represents three image editing methods: Manually interactive image clone; Image in-painting; Image texture synthesis. We consider the first method could be used to fill the assigned region; the second approach could be used to edit and restore the smooth or narrow band region; the third approach could be applied on image areas with normal textures. We adapt Image in-painting algorithm with a real time implementation and define a novel distance measurement to enhance the texture synthesis algorithm with the reducing color dependency. The experiments demonstrate the efficiency of our algorithms.Chapter 6 gives three real appliance of video editing with video panorama. They are motion panorama, moving objects removing and video restoring and re-editing. Several problems existed in the real video editing procedure. The problems are, for example, video twitter and black edges introduced by motion compensation. We provide the approaches to tackle the problems and demonstrate the performance of the approaches with extensive experiments.The final Chapter, chapter 7 summarizes the whole paper, draws conclusions and proposes several potential future directions based on the current works..更多还原