【作者】 潘绍松；

【导师】 左洪福；

【作者基本信息】 南京航空航天大学 ， 交通信息工程及控制， 2012， 博士

【摘要】 民用航空器维修保障是一项技术难度高、劳动强度大的工作，如何降低维修难度、避免维修差错、提高工作效率、压缩维修成本始终为航空公司所关心，因而以此为课题展开研究无论是在理论层面还是在应用层面都具有重大意义。本文以构建面向过程的维修作业系统为目标，以应用增强现实技术为基础，深入研究了在实际构建该系统时所必需的若干关键技术问题。该系统的功能是：以维修任务为中心、以维修过程为线索向现场维修人员提供所需的文字和图像，帮助他们理解维修任务、提高认知能力。现将本文的主要研究内容和创新点总结如下：(1)根据旋转矩阵是SO3流形的事实，文章提出了在2D-3D点对应情形下用流形优化的方法获取对象位姿的新颖算法，算法利用泰勒展开和收缩映射消除了用正交投影模型代替针孔投影模型所引入的理论误差，减小了连续视频对象位姿跟踪中累积误差的传播，实验表明流形优化法得到对象位姿精度和所需的迭代次数均优于正交迭代，在综合数据试验中噪声方差为１时，前者的旋转轴、旋转角的相对误差为0.5%和0.25%，仅为后者相对误差的1/5和1/2；随后文章针对工业对象表面通常缺乏点特征，但具有共面圆特征一类常见工业对象的事实，研究了利用共面圆特征作为对象坐标系，依据绝对二次曲线的射影理论求解对象位姿的新方法，真实对象的试验表明该方法得到的位姿最大相对误差分别只有:旋转轴2.7%,旋转角1.7%,平移1.7%；最后针对特征点3D坐标不易获取的实际问题，文章给出了重构稳定匹配的SIFT特征点3D坐标的方法。(2)动态地跟踪对象位姿是实现连续视频图像注册的前提。本文改进了具有直线边缘轮廓对象位姿跟踪方法，提出了基于共面圆轮廓特征的对象位姿跟踪方法。文章利用前一帧图像的轮廓位置，采用计算轮廓的梯度归一化互相关系数的方式隐式缘搜索直线边缘或椭圆轮廓，建立2D-3D的轮廓对应关系，为提高隐式轮廓搜索的效率文章还采用扩展卡尔曼滤波，一方面预测当前帧图像的对象位姿，另一方面限制直线或共面圆轮廓特征搜索起点和搜索位置。对于直线边缘对象，文章综合采用Tukey估计子和权重直线拟合构造位姿计算目标函数并用SVD法求解，实验表明本文基于直线的位姿跟踪方法的旋转角最大的平均偏差0.29o，绝对偏差为1.90o，跟踪速度大约为10帧/秒，综合性能明显优于基于LM的位姿跟踪算法；对共面圆轮廓图像，文章则采用Sampson距离剔除野值、采用各项异性数据回归通过提高椭圆拟合精度获得高精度对象位姿，实验表明该方法跟踪对象位姿的最大角度误差为1.4o、平移为3.5mm，跟踪速率达10~12帧/秒。实验还表明利用扩展卡尔曼滤波的可以有效地将位姿跟踪速度提高近一倍。(3)标定光学透视式头盔(HMD)是实现信息增强的基础工作，文章改进了Azuma所提出的HMD标定方法，克服了其标定方法中的缺陷。利用前两章共面圆位姿的研究成果，首先在标定过程用相机替代电磁位置跟踪器的HMD的位姿；其次用3D重构的方式确定对象坐标系与世界坐标系间的变换，免除了使用标定工作台确定该变换关系所需的高昂代价；最后将HMD的内、外参数分开标定，且认为外参数固定不变，减轻普通用户标定HMD的负担。标定性能评估实验中HMD标定单眼最大平均误差为2.15像素，说明本文改进的HMD方法是有效的、可行的，假设外参数固定不变是正确的；随即文章还简单地研究了3D模型获取过程中涉及到的位姿变换以及比例因子的问题。(4)维修环境中的指令增强和人机自然交互可以有效地免除维修人员注意力频繁切换的弊端。基于此本文设计与实现了利用菜单式人机自然交互的方式，用于切换虚拟的维修指令，具体内容包括：将维修任务中的指令信息抽象成维修任务菜单和维修任务子菜单、菜单的内容以虚拟成像的方式出现在HMD上、菜单的输入焦点通过对象位姿跟踪和虚拟成像的方式固定在对象表面、输入焦点区域的点击动作则来自于手势识别、最后则采用菜单状态转换有限状态机建立手势输入与菜单内容切换的联系。实验表明该方式的人机交互成功率可达76%，输入速度达8~10帧/秒。(5)文章最后以实现面向过程的维修原型系统为目标，首先研究了如何利用现有的民航维修技术出版物获得指令增强、2D图像增强信息，提出利用RM模型构建3D信息增强以及对象位姿跟踪所需要的信息；然后研究了实现这样的一个原型系统在软件结构和软件实现上所应考虑的问题；最后以APU点火嘴安装为例，实现了一个小型的原型演示系统，以直观的形式演示本文的研究目标以及面向过程的维修在民用航空器维修领域的作用。更多还原

【Abstract】 Maintenance support for civil aircraft is an intensive and difficult labor. The eternal problemconcerned by Airline Company is how to reduce difficulty, improve efficiency, avoid error duringmaintenance and compress direct cost. Thus, research the theory and technology on the problem issignificant from theoretical and technological perspective. This thesis aims at construction ofprocess-oriented maintenance guiding system to make an intensive study on informative enforcementand information organization that are two inevitable contents for constructing the mentioned system.Functions of the system are to provide suggestive information including texts&graphics that arebased on maintenance task and process so that maintenance personnel can improve quickly theircognitive competence and understand easily maintenance task. Main contents and innovativehighlights of this research are now generized as follows:(1)According to the fact that rotation matrix is SO3manifold, this thesis proposes the novelalgorithm of estimation object pose based on optimization over manifold under the condition ofestablished2D-3D correspondences. The approach exploits Taylor Expansion and retraction mappingto eliminate the principal deviation imported by the substitution orthogonal projection model with thatof pin-hole projection and degrade the propagation of accumulated error when pose tracking inconsecutive video images. Experiments show that object pose obtained by optimization on manifold isbetter at precision and the number of iteration than orthogonal iteration，i.e., when noise varianceequals1, the relative deviation of new algorithm with synthetical datum is0.5%for rotation axis and0.25%for rotation angle respectively, which are only fifth and half of those for orthogonal iteration.In industrial environment, the object, generally short of point features on its surface, but with coplanarcircles, represents a category of industrial object. Thus, this thesis deliberates on how to constructobject coordinate frame and estimate its pose from coplanar circles using projective theory of absoluteconic. Experiment on real object illustrates that the relatively maximum errors obtained by coplanarcircles are respectively: is only2.7%for rotation axis,1.7%for rotation angle and1.7%fortranslation. Considering the difficult problem of retrieving3D coordinates for feature points,reconstruction of3D coordinates for stable matched SIFT points is showed as an applied example ofpose estimated from coplanar circles that is full of engineering significance.(2)Tracking dynamically object pose is the prerequisite for image registration. This thesisimproves object pose tracking with straight lines and proposes a tracking algorithm based on coplanar circles. Contour’s location in previous frame is utilized to calculate normalize cross correlationefficiency of image gradient for search contour points implicitly. In order to promote searchingefficiency, Extended Kalman Filter (EKF) is used to predict object pose in current frame on one handand limit searching range and departure location. For object with straight line, Tukey Estimator andweight line fitting are exploited to construct objective function and obtain object pose via SVD.Experiments show that line-base tracking approach in this thesis can achieve the performance thatmaximum mean error for rotation angle is0.29o, absolute angle error is1.90o, and tracking speed isabout10frames per second. This explains that the comprehensive performance of ours is better thanthe LM-based one. For object with coplanar circles, Samposon distance becomes the standard ofremoving outliers and heteroscadastic regression is imported to improve accuracy of ellipse fitting forresolving object’s pose. Experiments for coplanar circles’ tracking illustrate that the maximumdeviation is1.4ofor angle,3.5mm for translation and tracking speed is10~12frames per second.Evidences’ from experiments verify the effectiveness of using EKF that promote tracing rate morethan100percent.(3)Calibration of optical See-through Helmet Display (HMD) is a fundamental work inimplementation Augmented Reality. This thesis improves the calibration method and eliminates thedisadvantage proposed by Azuma. Basing on proceeding research perform, we firstly substituteelectromagnetic tracker with camera when calibrating to obtain HMD’s extrinsic pose. Secondly,3Dreconstruction is used to determine the transform relationship between world coordinate system andobject ones which exempt the procedure from expensive calibration workbench; at last, intrinsic&extrinsic parameters are individually calibrated and the hypothesis that extrinsic parameters isconstant to reduce the burdens who takes parting in calibration. Evaluation of calibration gives us thatthe single maximum error is only2.15pixels that shows the promoted calibration method is effective,feasible and the hypothesis is right. Subsequently, this thesis studies how to calculate the posetransformation and scale factor during obtaining3D models for pose tracking.(4)In maintenance environment, enhanced instructions and human machine interaction (HMI)can exempt maintenance personnel from the defect of switching attention focus repeatedly. Because ofthe background, this thesis designs and implements a menu-typed HMI to modify instructions’registration. At first, the HMI translates maintenance task instructions into two layer menus anddisplays menu items on HMD as virtual image. Secondly, virtual images that act as focus and acceptclicking are fixed on object’s surface with the help of object pose tracking. At last, finite state machineis utilized to drive relationship between gesture input and alteration of menu content. Experiments illustrate the success rates of the proposed HMI achieve76%, and input capacity achieves8~10frames per second.(5)Aiming at implement prototyped system of process-orientated, this thesis interprets how toobtain instructions and2D images for enhanced information from existing technical publications andpropose to construct necessary information for3D enhancement and object pose tracking; then thinkssome practical problems when actually implementing the system. Demo system is implemented viainstallation of oil warm-up torch on APU to exhibit intuitively the research objectives of this thesisand the effect of process-orientated maintenance would be taken in civil aircraft maintenance fields.更多还原