【作者】 方小勇；

【导师】 魏小鹏； 滕弘飞；

【作者基本信息】 大连理工大学 ， 机械设计及理论， 2010， 博士

【摘要】 人脸运动捕捉是运动捕捉(Motion capture, MoCap)领域的一个新兴分支,是基于运动捕捉的数据采集方式捕捉人脸表情运动、处理和分析运动捕捉数据以及对人脸表情进行动画仿真的一门学科。该学科是人体工程学、计算机图形学、图像处理、数据处理等多个学科相互交叉与渗透而产生的新兴研究领域,是当前计算机科学的研究热点。人脸表情运动捕捉的研究不仅具有一定的理论研究意义,还具有广泛的实际应用价值。到目前为止,人脸表情运动捕捉已经广泛应用于现代影视动画、游戏制作、医学分析、虚拟现实等多个领域。本文针对人脸运动捕捉领域的若干关键问题进行了深入的研究,取得了一定的理论成果,开发了一个原型系统。论文工作主要包括以下几部分内容：1.基于空间几何柔性的人脸模板自动匹配方法：针对由三维坐标点集构成的具有相同排布方式、局部非刚性变形以及分布误差的不同人脸表情模板,运用启发式方法将模板空间进行归一化,构建了人脸拓扑结构,在模板匹配过程中,计算局部标记点(Marker)的运动对匹配进行矫正,使用临时反馈方法(Temporary feedback method, TFM)提高标记点的匹配可靠性,实现了由局部到整体的人脸模板自动匹配。实验证明该方法具有一定的鲁棒性,能对具备较大表情差异的模板进行快速准确的自动匹配,解决了在序列首帧模板生成过程中需要手动干预的实际问题。2.基于人脸运动捕捉数据的在线(Online)数据处理与轨迹光顺方法：针对运动捕捉数据中存在的数据缺失和噪声,提出了动态时空耦合的在线处理方法。分析了噪声繁殖问题,使用基于自适应Kalman滤波的噪声繁殖处理模块(Noise propagation solution module, NPS)对其进行抑制。提出了基于3D人脸拓扑结构的精练非刚性运动解析机制(Sophisticated non-rigid motion interpreter, SNRMI),结合动态追踪方法,能有效地追踪非缺失数据以及修复长时间缺失的标记点数据。最后,使用语义检测算法对开放结构(嘴、眼等)的错误追踪进行检测和修正。另外,针对轨迹光顺问题,提出了一种在线曲线的构造算法。实验证明,该方法能够自动处理具有噪声和长时间相邻多点缺失的运动捕捉数据,以及对运动轨迹进行在线光顺。3.基于运动捕捉数据的人脸动画仿真方法：基于对人脸不同功能分区进行模型驱动的思想,构建了基于径向基函数方法(Radial Basis Functions, RBF)的交叉映射,实现了不同人脸模型的运动数据生成。在模型驱动研究中,添加虚拟标记点并求解其运动,作为模型驱动数据,采用RBF方法实现了基于不同个性人脸模型的真实感动画生成。设计了预计算算法提高了实时仿真的效率,减少了计算消耗。实验证明,该方法能够将同一演员的运动捕捉数据应用于不同的个性人脸模型,生成逼真的人脸表情动画。4.基于人脸运动捕捉的数据处理及动画重构原型系统开发：系统定位为拥有自主知识产权的交互性软件系统,整合了论文的理论研究成果,构建了统一的底层数据结构,提供了强大的交互接口,设计了友好的操作界面。为使运动捕捉数据处理及表情动画仿真能够直观高效,系统将模板构建、数据处理及光顺和模型驱动三个主要过程模块无缝整合在一起。实验证明,该系统能够有效验证本文方法,并且全程交互处理运动捕捉数据及重构人脸表情动画,处理效率高,动画仿真效果真实感强。综上所述,本文针对人脸运动捕捉的数据处理及动画重构过程中所面临的几个主要问题做出了深入的理论研究,结合提出的算法理论,设计和开发了人脸表情运动捕捉数据处理及表情动画重构原型系统,在大量实验测试的基础上,验证了各个算法的有效性、鲁棒性、仿真程度及执行效率。更多还原

【Abstract】 Facial motion capture is an emerging branch of the motion capture (MoCap) field, which is a science of capturing facial motion by MoCap data acquisition method, processing and analyzing MoCap data, then generating facial animation from the processed data. Facial MoCap is a multi-disciplinary intersection and penetration of human body engineering, computer graphics, image processing, data processing and so on, and is the current research focus in computer science. Facial MoCap is an important issue with not only theoretical significance but also application value. The facial MoCap has been widely used in modern film and television animation, game production, medical analysis, virtual reality and other fields. This thesis systematically studied several key issues in facial MoCap, and made some academic achievements, at last, a prototype system for facial MoCap data processing and facial expression reconstruction was developed. The main work of this thesis includes the following aspects:1. Automatic facial template matching based on spatial geometric flexibility:According to two facial templates of identical cardinality with global similarity but local non-rigid deformations and distribution errors, the proposed method used heuristic methods to normalize the templates, the motions of local markers to correct local match, and the temporary feedback method (TFM) to improve reliability of the match, then achieved an automatic process to match the templates from local to global. The experiments proved that the method is robust, and it can fast and effectively match the different templates under different facial expressions in a deterministic and automatic process, which resolved the requirements of troublesome manual interventions during template generation.2. Online data processing and trajectory smoothing for facial MoCap:According to the data missing and noise of raw MoCap data, an online processing method based on dynamic spatial-temporal information was proposed. First, an analysis of noise propagation problem is proposed, and a noise propagation solution module (NPS) based on adaptive Kalman filter is used to suppress the noise propagation. Second, a 3D facial topology-based sophisticated non-rigid motion interpreter (SNRMI) is put forward, together with a dynamic tracking method, which could not only track the valid non-missing data effectively but recover several adjacent markers under long time missing. Third, to rule out wrong tracks generated from the markers in open structures (such as mouth, eyes), a semantic correction method is proposed. Lastly, an online curve modeling method is proposed to construct curves online to smooth marker trajectories. Experiments show that the method can automatically process raw data with noise and long time missing problems, and can simultaneously smooth trajectories.3. Simulation of facial animation based on facial MoCap data:Based on the idea of driving facial model from different functional regions, a cross-mapping algorithm based on radial basis functions (RBF) method is constructed for the generation of motion data for different facial models. During model driving, virtual markers are added and their motion data are computed, then, RBF is used to drive the different personalized facial models to generate realistic facial animation. A pre-computing algorithm is proposed to reduce computational cost during real-time simulation. The experiments proved that the method can not only map the MoCap data of one subject to different personalized face but generate realistic facial animations.4. Development for the prototype system of facial MoCap data processing and facial animation reconstruction:The system is designed and positioned to a interactive graphics system with software intellectual property rights, the proposed algorithms of the thesis are integrated, a unified underlying data structures is built, a friendly and powerful interactive interface is provided. To make the MoCap data processing and facial expression simulation intuitive and efficient, the system seamlessly integrates the three main modules:template construction, data processing and smoothing, and model-driven process. Practice has shown that the system can effectively prove the proposed methods, and process MoCap data and reconstruct facial expression animation by full interaction, moreover, the processing is efficient and the simulated animation is realistic.To sum up, several major issues in facial MoCap data processing and facial animation. reconstruction have been studied, and under the support of the proposed algorithms, prototype system for facial MoCap data processing and facial expression animation reconstruction has been designed and developed. With a great deal of experiments, the effectiveness, robustness, credibility level and efficiency of each algorithm were verified and proved.更多还原