【作者】 张锐；

【导师】 秦学英；

【作者基本信息】 山东大学 ， 数字媒体技术和艺术， 2014， 博士

【摘要】 虚拟现实是近年来信息领域迅速兴起的一种技术,具有重要的理论价值和广泛的应用价值。增强现实是虚拟现实的一个分支,借助于计算机图形学技术、交互技术、传感技术、三维显示技术、计算机视觉技术,将计算机生成的虚拟景物实时叠加到真实场景中,增强用户对真实场景的理解,达到超越现实的感官体验。随着技术的不断进步,增强现实已经应用到尖端武器及飞行器的研制与开发、精密仪器制造和维修、医疗研究与解剖训练、工程设计和远程机器人控制、文化遗产保护以及教育娱乐等诸多领域。通过将虚拟物体和真实场景无缝融合,呈现给用户一个感官效果真实的新环境是增强现实技术的一个重要研究内容。但迄今为止,这项技术仍有大量的问题尚未解决。近年来,增强现实系统主要的研究工作集中在跟踪、注册和交互技术上,光照一致性方面的研究相对较少,而光照一致性是确保虚拟物体表面光照效果的真实感、并使虚实物体无缝融合的关键技术,这就造成了增强现实系统中虚拟物体真实感程度不高的现状。由于室外场景的几何复杂性和光照复杂性,其光照估计是当前尚未解决的难点和热点问题。此外,室外光照估计也是计算机视觉的重要研究内容之一,物体的分割与识别、视频跟踪、阴影检测等算法都不同程度地受到变化光照的影响。因此,在场景三维模型未知的条件下,如何根据室外场景的光照特点,从输入的视频图像估计光照条件对计算机视觉和计算机图形学具有十分重要的意义。在场景三维几何信息未知的情况下,本文围绕固定视点下室外场景视频图像的光照展开研究,分析基于室外场景光照线性分解模型的光照估计算法的求解性质,避开重建大规模室外场景所面临的重重困难,省去繁琐的几何建模工作,提出新的解决策略,使光照估计算法更为实用,以实现增强现实场景中虚实物体的光照一致性的目标。本文在室外场景光照估计研究的主要工作和创新点如下：1)证明了在室外场景基图像分解模型下,同一太阳方位在不同天气条件下的三幅图像具有线性相关性,使得基图像方程欠约束,导致基图像无法自动求解。为此,利用不同太阳方位不同天气条件下的采样图像,提出自动求解基图像的新算法。其基本策略是根据阴影中的像素估算天空光,由基图像分解的特点计算太阳光,并利用太阳光与天空光基图像的像素色调一致性优化基图像及太阳光和天空光光照参数。2)提出了基于对反射率没有任何限制的整体光照明模型的室外静态场景图像线性分解模型,并证明了在此模型下任意一幅静态室外场景图像都可以分解为太阳光基图像和天空光基图像的线性组合；分析研究了基图像的性质,证明了基图像包含场景的几何信息和材质反射率,且是场景的不变量,将太阳光和天空光基图像定义为太阳光和天空光单位强度下的场景整体光照明效果；输入静态场景图像通过最小化二次能量方程得到基图像。3)提出了基图像的约束和先验条件的室外场景光照估计新方法,将静态室外场景的延时视频序列的每一帧自动分解成太阳光基图像和天空光基图像的线性组合；使用k-means聚类算法分析每个像素的延时曲线检测出阴影像素,利用基图像线性分解模型得到太阳光和天空光基图像,并基于基图像的约束和先验条件提出过程式改进方法,优化光照参数。本文算法均不需要场景的任何三维几何信息,避免了室外场景的大规模重建；对场景中物体的材质和纹理没有要求,也不需要场景中存在特殊物体或者特殊表面,适用于一般的室外场景；除了场景视频图像不需要额外输入,也不需要用户交互,便于应用到增强现实系统。更多还原

【Abstract】 In recent years, virtual reality grows rapidly, which is of great theoretical and applied significance. Augmented reality is developed on the basis of virtual reality. With the aid of computer graphics technology, interactive technology, sensor technology, three-dimensional display technology and computer vision technology, the augmented reality system can real-timely overlay computer-generated objects onto video images of real scenes, to enhance the user’s sensory experience. Along with the advance of mature concept and technology, augmented reality has been applied to research and development of sophisticated weapons and aircrafts, manufacture and repair of precision instruments, medical research and anatomical training, engineering and remote robot control, protection of cultural heritage and educational entertainment, and many other areas.Through the seamless integration of virtual objects and the real scene, augmented reality can enhance the display of the real world. However, many open problems still remain in augmented reality. In recent years, research on augmented reality mainly focused on tracking, registration and interactive technology, and research on illumination consistency was relatively less. But illumination consistency plays an important role in achieving high realism. So the realism of virtual objects was not high in many augmented reality systems. Due to the complexity of geometry and lighting, the illumination estimation of outdoor scenes is a difficult and hot problem at present. Also, it is one of research topics in computer vision. Varying illumination usually severely degrades the performance of algorithms proposed in object recognition and segmentation, video tracking, shadow detection, etc. Therefore, without the information of3D scene geometry, the real-time illumination estimation of outdoor scenes is of great importance for both computer graphics and computer vision.In the case that3D geometric information of the scene is unknown, the dissertation focuses on the illumination estimation of outdoor video images captured under a fixed view, and analyzes the property on solving illumination parameters based on the linear decomposition model of outdoor scene lighting. To get rid of the difficulties in reconstructing the3D geometry of large-scale outdoor scenes, we propose the new resolution strategy, which makes the outdoor illumination estimation more practical. The contributions of the dissertation are as follows:1) We prove that three images captured with the same sun position under different weather conditions show linear correlation, based on the theory of the basis image decomposition of the sunlight and the skylight for fixed outdoor scenes. Therefore, the basis image equations are systems of under-constraint so that they can not be solved automatically. Then, we present the algorithm of solving basis images using four images with two sun positions under two kinds of weather conditions. In the algorithm, the intensity of the skylight is estimated according to pixels in the shadow region, and the intensity of the sunlight is calculated based on the characteristic of the basis image decomposition. In addition, the hue consistency of pixels in the sun area between the basis images of the sunlight and the skylight is used to optimize the basis images and illumination parameters.2) On the basis of the global illumination model without any limitation on reflectance, we propose a linear decomposition equation for the images of static outdoor scenes. Then, we proved that each image of a static outdoor scene can be decomposed into a linear combination of basis images of the sunlight and skylight, which encapsulate the geometry and material reflectivity of the scene. As well as, it is proved that the resulted basis images are invariants of the scene, corresponding to the global illumination effects of the outdoor scene under a unit intensity of the sunlight and skylight. Based on the input images of an outdoor static scene, basis images can be obtained by minimizing a quadratic energy function.3) We investigate the constraints and priors of the basis images, and propose a novel decomposition method to solve for the sunlight and skylight basis images of static outdoor scenes from a time-lapse image sequence without any user interaction. During decomposition, we first detect shadowed pixels by analyzing the time-lapse curve of each pixel through k-means clustering, and then the basis images of sunlight and skylight are solved by an iterative procedure with the decomposition equation. The basis images are further optimized by exploiting their constraints and priors.The above approaches don’t request any information of scene geometry, avoiding the3D reconstruction of large-scale outdoor scenes. As well as, they are applicable to the general outdoor scene because any assumption on materials and textures of the scene is not required and there need not be a special object or special surface in the scene. And that, these approaches do not need any other input except video images of the outdoor scene and request no user interaction. So they can be easily used for the system of augmented reality.更多还原