【作者】 郑颖；

【导师】 汪增福；

【作者基本信息】 中国科学技术大学 ， 模式识别与智能系统， 2009， 博士

【摘要】 随着计算机软硬件水平的不断提高,三维模型已经深入到我们生活、工作、娱乐的各个方面。如何获取逼真的三维模型,是计算机视觉与计算机图形学领域的一个重要研究课题。基于图像的建模是三维建模中的一个重要手段。本文以人脸为重建对象,研究从二维图像中快速、精确地重建人脸三维模型的方法。三维人脸建模与一般物体的三维建模无论在方法学层面上,还是在应用层面上都有着显著的不同。一方面,三维人脸模型在人脸识别和人脸动画等研究课题中有着广泛而重要的应用,与这些相关课题研究之间有着相辅相成、密不可分的关联关系。另一方面,由于人脸是一类具有很大共性的对象。在相关研究中得到的有关人脸共性的知识可以对人脸三维重建工作起到很好的辅助和补充作用。除了计算机视觉领域中的相关方法以外,模式识别和统计学习等领域中的相关研究成果也会对三维人脸建模研究产生积极和正面的影响。本文在对前人工作进行详细总结分析的基础之上,对三维人脸建模问题进行了深入的研究。主要的研究工作包括以下几个方面。在人脸图像预处理技术方面,提出了局部选择投影方法,并将该方法应用于眼睛定位。同现有方法相比,基于局部选择投影的眼睛定位方法精确性更高,对光照、姿态变化的鲁棒性更强。利用该技术,可以精确地定位眼睛,为面部其他特征点的定位提供了很好的初始化依据。而面部特征点的定位正是三维人脸建模中的重要一环。在人脸三维建模方面,本文从不同的侧面,依据不同的图像来源,对相关的三维建模方法进行了探讨。本文提出了一种基于从明暗恢复形状与径向基函数内插相结合的三维人脸建模方法,可以从单幅正面人脸图像中快速重建逼真的三维人脸模型。该方法从传统的从明暗恢复形状的方法入手,利用其提供的在特征点上较为准确的三维信息,通过径向基函数内插的方式,修改一个通用的三维人脸模型,最后经过纹理映射,达到合成特定人脸模型的目的。该方法可以以很小的计算代价获取具有一定真实感的三维人脸模型。本文对传统的从明暗恢复形状方法进行了创新。利用统计学习的方法,从大量的纹理图像和深度图像中学习从纹理图像到深度图像的映射关系,并进一步利用该映射关系从人脸图像恢复深度,进而重建三维人脸。通过将人脸配准技术引入到图像预处理中去,使得深度图像像素与纹理图像像素精确对应,极大地提高了深度估计精度。同时,通过引入核方法,解决了学习纹理图像到深度图像的映射关系这样高维问题。针对基于图像的三维建模可能遇到的受光照变化影响的问题,本文提出了从单幅近红外图像中恢复三维模型的方法。近红外图像由于其成像装置的特殊性,使得无论环境光照如何变化,其图像的光照条件基本一致。这就避免了可见光图像易受光照影响的问题。我们采集了近红外图像和对应的三维人脸数据,并提出了近红外图像与三维人脸数据之间的配准算法。利用统计学习的方法得到从近红外图像到深度图像的映射关系,进而重建三维人脸模型。对于传统的立体视觉方法,本文针对三维人脸重建这一特定问题,做了相应的改进。在方法上,突破了传统立体视觉方法基于纹理的左右图像直接匹配的方式。替代地,利用一个参考人脸的三维模型估计输入立体图对中人脸的位姿参数,从而生成相同姿态下的参考人脸的虚拟图像对。由于虚拟图像之间的对应是已知的,通过计算同一姿态下输入人脸与虚拟人脸之间的对应关系,将虚拟图像对应扩展到输入立体图像的对应。这种方法大大改进了人脸立体图对的匹配效果,使由立体视差计算得到的人脸三维点云数据更为准确。立体视觉的方法可以提供精确的深度信息,同时也会带来一定的噪声点或奇异点。为此,本文进一步将立体视觉的方法与Morphable Model的方法相结合。后者是一种统计方法,它通过将一系列的真实三维人脸模型配准,将主元分析应用于配准后的三维人脸数据中,抽取形状主成分和纹理主成分。这样,一个特定的三维人脸的形状可以表示成平均形状与形状主成分的线性组合。利用Morphable Model提供的形状约束,我们可以获得平滑的三维人脸模型。通过将立体视觉与Morphable Model相结合,利用Morphable Model匹配点云数据,最终可以获得立体视觉点云数据的Morphable Model表示,使得最终重建的三维人脸模型逼真、平滑。本文对三维人脸建模这一课题做了各种尝试,改进了现有方法的一些不足,提出了一些新颖的方法。希望本文的工作和结论能够对相关领域的研究提供帮助。更多还原

【Abstract】 The development of computer hardware and software has made the 3D model become more and more popular in our life,work,and entertainment.How to acquire realistic 3D model is a major research topic in either computer vision or computer graphics.The image-based modeling is one of the major 3D acquisition measures. This thesis concentrates on the image-based 3D modeling of faces.Although face is just one kind of objects being modeled,the 3D modeling of faces has its unique characteristics compared to other objects.On one hand,3D face models play important roles in subjects such as face recognition and face animation,thus are highly correlated to these subjects.As a result,we should incorporate the successful research results of these subjects into the 3D face modeling.On the other hand,human face is an object which shares a lot of commonness across different individuals. This commonness is very helpful in extracting prior information or knowledge about faces to facilitate the 3D modeling procedure.This means that 3D face modeling involves methodologies borrowed from pattern recognition,statistical learning,not just the computer vision.Based on a thorough review of the state of art,we perform our research work on the following aspects.Regarding the face image preprocessing techniques,we propose a novel projection algorithm called Locally Selective Projection,and apply this algorithm to eye location. Compared with existing eye location methods,our method based locally selective projection shows higher performance against the changes in illumination and pose,and exhibits more precise eye detection results.We propose a fast 3D face modeling method based on the combination of shape-from -shading and radial-basis-function interpolation.This method utilizes the accurate 3D information provided by shape-from-shading on several facial feature points,and uses the 3D information of these feature points to drive a modification process from a general 3D face model to a specific one,based on the radial-basis-function interpola-tion. The proposed method can construct realistic 3D face model with low computa-tional expense. We also propose a novel shape-from-shading method which uses the statistical learning to learn a mapping from texture to depth images,and further uses the mapping to estimate depth from texture for 3D face modeling.We incorporate the face alignment techniques into our learning process to dramatically promote the depth estimation accuracy.We also introduce the kernel based learning to deal with the high dimension learning problem.Regarding the problem that the image-based 3D face modeling can be greatly influenced by illumination changes,we propose a solution of 3D face modeling from near infrared images.The near infrared images,due to its unique capture hardware,are immune from illumination changes,thus can be a qualified source for depth estimation. We build a database containing 3D faces and corresponding near infrared images,and propose an algorithm for aligning these two kinds of data.We then use statistical learning to learn the relation between near infrared images and depth images,and further use this relation for depth estimation and 3D face reconstruction.Stereo is an important 3D reconstruction method.Because the human faces are lowly textured,the conventional stereo methods based on texture correlation can not give satisfying 3D face reconstruction results.We propose a stereo matching method based on virtual stereo correspondences.The virtual face images with known correspondences are first synthesized from a reference 3D face.Then the known correspondences are extended to the incoming stereo face images,using face alignment and image warping.The 3D face point cloud can thus be reconstructed from stereo images reliably.Although the 3D point cloud provided by stereo is accurate,it is not smooth enough for the realistic rendering of 3D face.We present a novel and efficient method to obtain dense 3D face from stereo images by combining stereo vision and morphable model.Using morphable model as the reference face for the calculation of stereo matching,we can obtain the correspondences between morphable model and stereo point cloud.Then the stereo point cloud is further registered with the morphable model to obtain a refined face surface.The reconstructed face surface shows high fidelity and smoothness.In conclusion,in this thesis,we perform a thorough research on the topic of image based 3D face modeling.We hope that our work can be helpful for the relating researches.更多还原