【作者】 朱娟娟；

【导师】 郭宝龙；

【作者基本信息】 西安电子科技大学 ， 电路与系统， 2009， 博士

【摘要】 电子稳像技术EIS(Electronic Image Stabilization)是视频处理、计算机视觉等的研究热点,它可以提高运动载体摄像的稳定性和清晰度,在军事领域以及民用摄像领域均有广泛的应用。对于摄像系统尤其是运动载体摄像系统来说,载体姿态的无规则抖动将会导致视频序列的不稳定和模糊现象,而EIS的实质是应用数字图像处理的方法来直接确定图像的偏移并进行运动补偿,从而实现图像重组达到序列的平滑稳定。本文主要以研究EIS为主线,系统研究了稳像关键技术及其理论分析,并针对EIS的基本模块:运动估计模块、运动决定模块和运动补偿模块,进行了深入地研究;尤其是结合应用场景的不同摄像机抖动方式,分类设计了三种电子稳像算法,有针对性的实现各种抖动模式下的全局运动检测和抖动参数补偿;同时,针对传统运动补偿难以跟随摄像扫描并有延迟现象,和补偿图像边界的不完整等问题,文中设计了一种基于自适应滤波的全景图像补偿方法,以提高输出视频的观测质量。最后,将电子稳像的研究成果应用到摄像机运动-目标运动的目标检测领域中,以实现在背景配准下利用帧间差分确定运动目标的目的。本文的主要研究工作及贡献如下:(1)对电子稳像中的运动估计和补偿理论进行了较为详细的分析。首先概述了成像变换原理和图像运动的坐标变换模型;然后对摄像机抖动产生的原因和类型做了详细的分析,指出摄像机各运动方式的统计特性,表明图像背景的全局运动具有良好的一致性,并给出了快速判定帧间运动的方法;接着分别讨论了电子稳像中运动估计和运动补偿的关键技术,并对其理论进行了详细分析,最后总结了电子稳像中的关键问题并提出了改进思路。(2)针对视频序列的小平移抖动,传统块匹配算法速度太慢,且固定分块容易导致过多误匹配的问题,本文利用快速位平面匹配的思路,提出了一种基于分层位平面匹配HBPM (Hierarchical Bit-plane Matching)的电子稳像算法。算法将多分辨率运动估计和位平面匹配相结合,在每层搜索时选择不同的位平面,在减少匹配运算量的同时保证匹配的准确性;且结合运动矢量在层间和层内的相关性,预测搜索起点,并判断其精度来提前结束搜索,从而实现由粗到细地快速平移运动检测;再统计所有子块运动矢量获得最终的全局运动矢量;最后累积检测出的运动参数作为抖动参数以逐帧补偿,以实现稳定视频的输出。实验结果表明,与经典块匹配全搜索算法相比,运动检测速度提高了20倍,且平均误差小于半个像素,稳定视频能够完全去除抖动。(3)针对视频序列的大平移抖动,传统整幅图像的投影算法难以克服局部运动物体的影响,且其全相关搜索策略,会造成计算时间浪费的问题,本文提出了一种基于分区快速投影FPA(Fast Projection Algorithm)的电子稳像算法。首先,对图像进行直方图均衡化,以增强图像的对比度;然后对图像进行分区投影运动估计,排除局部运动区域的影响;同时,在投影曲线相关搜索时,提出了三点自适应搜索策略,使得搜索快速沿着正确的方向找到极小值点;然后对各子区域运动矢量进行中值滤波,确定最终的全局运动矢量;最后对检测的帧间平移运动进行自适应均值滤波,并结合运动矢量修正措施来进行图像的补偿,以实现大平移抖动的去除。实验结果表明,算法检测每帧大平移运动的平均时间12ms,比经典块匹配全搜索提高了10倍,其精度与传统投影算法几乎一致,而计算量仅为1/3,且分区处理提高全局运动估计精度,同时在补偿时考虑摄像系统可能的扫描分量,去除抖动同时跟随扫描运动。(4)针对视频序列的复杂抖动如平移、旋转和变焦运动,传统特征点提取易过于集中于前景运动物体上,匹配步骤复杂导致速度过慢,且将所有特征点直接代入参数计算,从而导致局部点和误匹配点影响计算精度的问题,本文提出了基于特征点迭代的电子稳像算法GPI(Global Points Iteration)。利用Harris算子分区均匀提取参考帧中的特征点,然后采用特征窗匹配以提高匹配速度;并结合摄像机运动方式,对特征点进行预处理,来确定候选全局特征点对,减少无效特征点对的不利影响;再通过最小二乘解的迭代运算,逐步去除位置误差较大的点对,从而大大提高了全局运动估计的精度;最后文中设计了一种基于自适应Kalman滤波的全景图像补偿方法,确定抖动参数并结合图像拼接技术来实现全景补偿,提高输出视频的稳定性和完整性。实验结果表明,该算法对特征点的预处理和迭代步骤,保证了运动检测的全局性,具有较高的精度,与真实值的误差在很小范围内(水平垂直方向均小于0.5像素),且补偿结果能够实时跟随摄像系统的扫描运动,输出稳定完整的视频序列。(5)将电子稳像的关键技术应用于运动目标检测,针对摄像机运动-目标运动的视频序列,其运动目标实时检测存在的主要问题是:帧间直接差分不利于前景运动的保留;简单阈值判断不能自适应分割运动区域,本文结合人眼的视觉特性提出了一种基于帧间补偿的可变块差分算法VCBD (Variant Compensated Block’s Difference)。算法首先通过对相邻帧间进行全局运动估计,并补偿当前帧以实现背景的校正;然后对补偿后的相邻帧进行可变块的差分运算,通过判定块的平均绝对误差来决定是否进行更细划分,直到划分到块的最小尺寸为止;最后将所有误差较大的块在原始图像中标定,结果以外接矩形表示,实现运动目标的检测。实验结果充分证明,对于摄像机扫描和存在抖动的视频序列,VCBD算法均可以实时准确地检测运动目标。更多还原

【Abstract】 Video stabilization in dynamic sequence is an important research area of video processing and computer vision. It can improve the stability and fidelity of video sequence recorded from cameras on moving carriers, and has been widely used in military affairs and civil cameras. Electronic image stabilization (EIS) is the new trend of the video stabilization, which is the technique to determine the motion between frames and compensate it with image processing methods. In this dissertation, the author mainly focuses on the research of EIS on three aspects: motion estimation、motion decision and motion compensation. In especial, three image stabilization algorithms are proposed to deal with different motion types of camera systems. At the same time, the research results of global motion estimation is applied to moving objects detection in dynamic sequences, which is realized with background matching and frame difference. The main research work in the dissertation is as follows:1. The theories and key techniques in image stabilization are detailed. The basic geometrical principles of the camera and the transform models of image motion are introduced. Thereafter, the reasons and types of inter-frame motions are analyzed and a fast method to determine the camera motion type is proposed. Then, the motion estimation and compensation methods are detailed and analyzed. Lastly, the difficulties of EIS are discussed and the improving thoughts are summarized.2. Aiming to deal with the small dithering at the fixed scene, a hierarchical bit-plane matching (HBPM) is presented to realize fast image stabilization. The speed is highly improved due to the simple XOR operations instead of the SAD of block matching. It makes full use of multi-resolution pyramid matching with different bit-planes to find the translation vector. According to the high correlation of adjacent blocks’motion vectors and father-children of hierarchical blocks, the initial search point is adaptively chosen. The threshold is also selected to detect the motion accuracy to finish the search in advance. With all the local motions as a set, the motion with the largest number is detected as the global motion. Lastly, the dithering motion is computed as the accumulation of the inter-frame translations and compensated frame by frame. Experimental results show that this algorithm excels the block matching in speed while the error is below 0.5pixel and the result sequence is smooth.3. Aiming to deal with large translation dithering, a fast projection algorithm (FPA) is proposed based on gray projection of divided blocks. Firstly, the histogram equalization is applied in the original images to enhance the contrast. Then, the image is divided into separate blocks and projected in rows and columns, respectively. In the process of projection correlation, the three points’adaptive search method is presented to conduct the search towards the right direction to find the single minimum peak. Then, the median filter is applied on all the block motions to find the correct global motion with the largest number. Lastly, the inter-frame motions are filtered with the adaptive average filter and the error control is applied to select the original frame to be compensated. Experimental results show that this algorithm excels the traditional projection algorithm in speed while maintains the same accuracy. Even when the translation is as large as the 1/3 of the image size, it also accomplishes the real-time applications and high veracity.4. Aiming to deal with complex dithering, a global points’iteration (GPI) algorithm is proposed based on point matching. It is able to deal with translation, rotation and zooming and it’s robust to local moving objects in the scene. Firstly, the points are selected in the reference frame and then each feature window is matched to find the corresponding points in the current frame. Then, considering the local moving objects and the covered or the disappearing points, the distance criteria is applied to eliminate the mismatched and the local points. Thirdly, the candidate points are brought into the motion model to make iteration with the least-square method. As a result, the points with large error are deleted gradually and the global points are sustained. Lastly, the Kalman filter is applied to smooth the motion vectors to obtain the dithering vector and the image mosaic is used to compensate each current frame. Experimental results show that this algorithm accomplishes high precision and speed. The error with the true motion is below half the pixel and the result sequence is smooth with high integrity.5. The frame difference can not preserve the foreground moving objects and the simple threshold can not realize adaptive segmentation of moving objects. Aiming to solve the problem, the variant blocks difference based on compensation (VCBD) is proposed to detect moving objects. Firstly, the global motion is estimated between adjacent frames to compensate the current frame. Thereafter, the difference based on variant blocks instead of pixels is made from-coarse-to-fine. Large size is used to divide the images and the median absolute difference is compared with the threshold to make fine divisions till the smallest size. At last, the blocks with large difference are labelled as the foreground moving objects. Experimental results show this algorithm is able to detect the moving objects from dithering video sequence.更多还原