【作者】 孙国强；

【导师】 田彦涛；

【作者基本信息】 吉林大学 ， 控制理论与控制工程， 2010， 博士

【摘要】 在计算机视觉领域,从二维图像序列恢复物体的三维结构和运动是一项重要而困难的工作。本文主要对一般透视投影模型下和存在数据丢失情况下由图像序列重建非刚性物体的三维结构及摄像机运动信息问题进行了比较深入的研究。主要内容包括:1.提出了一种迭代算法,从图像序列恢复非刚性物体的结构和运动,将非刚体因式分解法从以往的弱透视投影假设扩展到一般透视投影的情况,从而提高了重建结果的精度。2.提出了一种准透视投影模型来恢复刚体和非刚体的结构和运动。首先,在摄像机距物体较远且旋转较小的情况下,提出并证明了成像过程可通过准透视投影模型来近似。由于投影深度隐含在形状矩阵中,此模型比仿射摄像机模型更加精确并和仿射情况一样容易计算。其次,对模型进行因式分解并通过准透视投影假设建立了刚体和非刚体因式分解的框架。此外,提出并证明了一种鲁棒方法来恢复变换矩阵,将因式分解方法拓展到欧式空间。3.提出了一种带正交约束的幂因式分解算法,将正交约束和运动矩阵的重复模块直接合并到迭代算法中,克服了以往基于奇异值分解方法的局限,不但便于实现,而且可以处理丢失数据的跟踪矩阵的问题。此外,提出一种连续的因式分解方法来实时恢复新图像的形状和运动。4.提出了两种算法来改进透视因式分解的性能。首先,提出通过做较大运动的摄像机的两个射影重建的结构来初始化射影深度,然后通过最小化反投影残差来迭代地优化该深度。其次,提出一种基于Kruppa约束的自标定方法来处理更一般的摄像机模型,然后通过标准化的跟踪矩阵因式分解法来恢复欧式结构。更多还原

【Abstract】 One of the study object of computer vision is recovering both structure and motion from 2D image sequences. During the last two decades, many approaches have been proposed for different applications. Among them, factorization based methods are widely studied and attracted much attentions due to their good robustness and accuracy.The traditional methods of 3D reconstruction work mainly for static rigid object. While in real world, most objects and scenes are non-rigid and dynamic. Many previous methods on this problem are hardly applied to reconstruction of this object. How to recover both structure and motion of non-rigid object from image sequences is a study hotspot of computer vision and pattern recognition.There exists two bottlenecks in reconstruction of recovering structure and motion of 3D non-rigid object:one is the choice of model, the other is system robustness. Previous reconstruction methods are all based on affine camera model. This is a zero-order(weak perspective)or first-order(paraperspective)approximation to the general perspective projection model and is only valid when the depth variation of the object is small compared to the distance between the object and the camera. Usually, it does not satisfy the assumption for many image sequences in real life. Therefore, there exists large reconstruction errors for the result, especially the depth variation of the object is large or the distance between the object and the camera is small. Otherwise, the methods of 3D reconstruction for non-rigid object all assume that the match of tracked feature points is known and these features are visible in all images. However, match mistakes or missing of features in some frames are often appear due to occlusions or tracking methods failure.In this case, the existent methods can’t be used directly.Aimed at above problems, the paper addresses the problem of recovering structure and motion of 3D non-rigid object from uncalibrated image sequences under perspective projection model. In addition, the paper also studys robustness reconstruction methods to solve the problem of tracking error of features and missing data.1. The paper proposes a recursive algorithm to estimate 3D structure and motion of non-rigid object from a monocular video sequences and update previous non-rigid factorization methods from weak perspective assumption to the case of perspective projection. Accordingly, the precision of the reconstruction result is improved.2. The paper proposes a quasi-perspective projection model to recovery structure and motion of rigid and non-rigid objects. First, under the assumption that the camera is far away from the object with small rotations, we propose and prove that the imaging process can be modeled by quasi-perspective projection. The model is more accurate than affine camera model since the projective depths are implicitly embedded in the shape matrix. However, it is computationally as cheap as affine. Second, we apply the model to the factorization algorithm and establish the framework of rigid and non-rigid factorization under quasi-perspective assumption. Third, we propose a new and robust method to recover the transformation matrix that upgrades the factorization to the Euclidean space.3. The paper proposes a constrained power factorization algorithm that combines the orthonormal constraint and the replicated block structure of the motion matrix directly into the iterations. The proposed algorithm overcomes the limitations of previous SVD based methods. It is easy to implement and can even cope with the tracking matrix with missing data. Based on the solutions of the CPF, a novel sequential factorization technique is proposed to recover the shape and motion of new frames in realtime.4. The paper proposes two new algorithms to improve the performance of perspective factorization. First, we propose to initialize the projective depths via a projective structure reconstructed from two views with large camera movement, then optimize the depths iteratively by minimizing reprojection residues. The algorithm is more accurate and converges quickly. Second, we propose a self-calibration method based on Kruppa constraints to deal with more general camera model. The Euclidean structure is then recovered from factorization of the normalized tracking matrix.Extensive experiments on synthetic data and real sequences validate the effectiveness of the proposed algorithm and show noticeable improvements over the previous methods and provide reference to the further study and application.更多还原