【作者】 田元；

【导师】 王乘；

【作者基本信息】 华中科技大学 ， 系统分析与集成， 2010， 博士

【摘要】 随着信息技术的发展,增强现实(Augmented Reality,简称AR)技术越来越广泛地引起了研究人员的注意,其研究内涵与应用范围也不断拓展延伸。它是一种将计算机系统产生的虚拟物体、场景或系统提示信息叠加到真实世界中实现对真实世界有效扩充和增强的技术。增强现实技术与虚拟现实(Virtual Reality,简称VR)技术密切相关,但存在区别。不同于VR的纯虚拟环境,AR强调虚拟对象与真实环境的完美融合。在早期的研究中,研究者们多将注意力集中在解决跟踪注册、三维重建等问题上,并取得了突破性的进展。但随着研究的不断深入,虚实遮挡处理成为增强现实系统所必须解决的又一关键性问题。当将虚拟物体叠加到真实场景中时,虚拟对象与真实场景间存在一定的空间位置关系,即遮挡与被遮挡的关系。目前,大多数增强现实系统只是简单的将虚拟对象叠加到真实场景图像上,以致真实场景始终被虚拟物体遮挡。当场景中的虚拟物体被真实物体遮挡时,用户很难正确判断虚实物体间的位置关系。针对上述问题,本文详细分析了现有方法存在的缺点与不足,吸取了数字图像处理、模式识别、计算机视觉和非线性优化等学科理论的最新成果,围绕虚实遮挡处理中的半自动实时处理方法、自动实时处理方法和非刚性物体虚实遮挡处理方法等技术内容展开了深入、系统的研究与实践,目标旨在针对不同的应用场合,研究相应的虚实遮挡处理方法,在提高增强现实系统可用性的同时最大限度保证虚实遮挡处理的效果和系统的实时性。本文的主要研究工作如下：(1)基于轮廓跟踪的虚实遮挡处理方法研究针对现有基于模型的虚实遮挡处理方法中,要求用户重建三维场景或遮挡物体,以及基于深度的虚实遮挡处理方法仅适用于真实场景比较固定的情况,设计一种基于轮廓跟踪的虚实遮挡处理框架,使系统在未知场景或遮挡物体三维模型的情况下,能够实时的完成虚实遮挡处理,并且适用于视角发生变化的情况。(2)半自动实时虚实遮挡处理方法的研究与实现设计一种基于用户手工指定与系统自动划分相结合的半自动实时虚实遮挡处理方法,在完成第一帧虚实合成图像遮挡处理的同时为后续帧的遮挡判断提供可靠的依据,最大限度的降低操作的复杂程度,在后续帧中,采用一种基于水平集的快速跟踪方法,实时、精确的跟踪目标轮廓,不仅保证了虚实遮挡处理的有效性,也保证了系统的实时性和可用性。(3)自动实时虚实遮挡处理方法的研究与实现针对增强现实系统无人工干预的要求,设计一种自动实时虚实遮挡处理方法。在无人工参与的情况下,利用立体匹配技术计算得到的初始帧图像中像素点的深度信息,判断真实物体与虚拟物体间的空间位置关系,保证判断的准确性。另外,采用一种基于光流法和最大流／最小割的轮廓跟踪方法,使系统在场景复杂、前景背景颜色相似的情况下也能实时、精确的跟踪目标轮廓,且当初始轮廓与目标边界相差较大时,也能快速收敛到目标边界,从而保证了后续帧图像中遮挡处理的有效性和实时性。(4)非刚性物体虚实遮挡处理方法的研究与实现针对真实场景中存在非刚性物体的情况,借助因式分解技术和基于活动外观模型的轮廓跟踪技术,设计一种非刚性物体虚实遮挡处理方法,在准确求取摄像机外部参数的同时完成虚实遮挡处理,使系统在无标识物的条件下,能够完成三维注册和遮挡判断,降低系统对标识物的依赖,扩大系统的应用范围。更多还原

【Abstract】 People pay more and more attention to augmented reality (AR) along with the development of information technology. Its range of application expanded significantly recently. AR is a technology that superimposes the computer generated virtual objects, scene and information to the real world. It expands and augments the real world with additional information. The AR has close relationship to virtual reality (VR), but they have distinctions. All the objects in the VR system are computer generated. Different with the VR system, AR system emphasizes the fusion between the virtual objects and the real worlds.The early researches on augmented reality mainly focus on the registration between virtual and real worlds and the 3D reconstruction technology. People have made significant improvements in these areas. With the development of research, occlusion handling has become another crucial problem we need to solve. When we overlay the virtual objects on the real scene, there are spatial relationships between the virtual and real objects. Sometimes the virtual objects occlude the real object and sometimes on the contrary. At present, most AR systems simply draw virtual objects on the composite images regardless of the spatial relationships between the virtual and real objects. This leads to that the real objects are occluded by the virtual objects all the time. It will make the user difficult to get the correct spatial relationships between the real and virtual objects. Focusing on this problem, we analyze the advantages and disadvantages of the existing approaches. Then we take full advantages of digital image processing, pattern recognition, computer vision and non-linear optimization technology to investigate the real-time semi-automatic occlusion handling method, real-time automatic occlusion handling method and non-rigid occlusion handling method. The goal is to get correct spatial relationships between real and virtual objects in real time and improve the availability of AR systems.The main studies and achievements of the thesis are listed as following:(1) A new framework handling occlusion problem based on contour trackingIn the existing model-based occlusion handling approaches, the 3D models of the real scene or the occluding real objects need to be reconstructed. This is difficult in the complex scene. And the existing depth-based methods are only suitable to the static scene. To enlarge the application range, we propose a novel framework based on contour tracking to handle occlusion problem. Our approach can successfully handle the occlusion problem without 3D reconstruction and is suitable to the case when the camera changes over a wide range of viewing angles and volumes.(2) A real-time semi-automatic occlusion handling methodWe propose a novel real-time semi-automatic occlusion handling method. Our method is divided into three steps:occluding real object specifying, object tracking and occlusion handling. In occluding real object specifying step, user specifies the occluding object using interactive segmentation method. Then the contour of the specified object will be tracked in the subsequent frames using the fast level set contour tracking method which is accurate and real-time. In occlusion handling step, all the pixels on the tracked object are redrawn on the unhandled augmented image to produce a new synthesized image in which the relative position between real and virtual objects is correct. The proposed method has several advantages. Firstly, it is robust and stable as it remains effective when the camera is moved through large changes of viewing angles and volumes or when the object and the background have many similar colors. Secondly, it is fast as the real object can be tracked in real-time. Lastly, the seamless mergence is improved by utilizing smoothing technique. Several experiments are provided to validate the performance of the proposed method.(3) A real-time automatic occlusion handling methodTo satisfy the requirement of minimizing the human intervention in AR systems, we propose a new real-time automatic approach that resolves occlusion problem. The main interest is that it can automatically obtain the proper spatial relationships between virtual and real objects in real time. Our approach is divided into two steps:off-line disparity map constructing and on-line occlusion handling. In the off-line stage, the disparity map of the real scene is constructed using the global stereo matching method prior and then the disparities are refined by means of the fast mean shift method. Because the depth values of objects in different positions are different, the real object that occludes the virtual object can be specified according to the depth value. In the on-line stage, the contour of the specified object is tracked using the real time object tracking method with the combination of feature tracking method and minimum s-t cut method. The augmented image with correct occlusions is produced by redrawing all the tracked object pixels on the augmented image. Compared with the existing methods, the proposed approach can automatically resolve occlusion problem in real time. The effectiveness of our method is demonstrated with several experimental results.(4) An occlusion handling method for non-rigid objects in augmented reality systemsThere are always non-rigid objects existing in the real world. Different with rigid objects, the shape of the non-rigid objects will change randomly. So the general occlusion handling methods used for rigid objects are not suitable to the non-rigid objects. We propose an occlusion handling method for non-rigid objects in augmented reality systems based on active appearance models (AAM) and factorization method. This method can not only calculate the extrinsic parameter but also solve the non-rigid occlusion problem.更多还原