【作者】 云挺；

【导师】 吴慧中；

【作者基本信息】 南京理工大学 ， 计算机应用， 2008， 博士

【摘要】 虚拟空间会议(Virtual Space Teleconferencing,VST)系统是虚拟现实技术与计算机网络和多媒体相结合的产物,它突破了传统的地域观念,利用虚拟现实技术,将不同地点与会终端的局部会场合成一个共同的虚拟空间,与会者以化身(avatar)的形式在这个虚拟空间中,通过化身在虚拟空间中定位、观察、操纵虚拟空间的实体、与其他用户进行交互等方式“共享同一空间”,具有身临其境之感。因而又被称为“终极会议系统(Ultimate Teleconferencing System)”。为了达到具有眼神接触、凝视感知的真实会议系统,构建具有实时性和逼真感的与会者的化身是VST的研究热点。本文着重研究了VST中与会者的化身建模、三维重建、立体图像匹配等演示技术,运用了超小波、随机数学和偏微分方程等理论,提出了多种立体图像匹配方法并构建一个虚拟空间会议的原型系统。主要研究内容如下:首先针对立体图像的区域匹配中,遮掩、区域变形及光照条件会对匹配算法造成很大的影响,传统的顺序性约束、惟一性约束、外极线约束和邻域约束并不能很好地解决这些问题,而近几年提出的相对位置约束却能解决其中大部分问题,但对于区域的遮掩情况依然效果不佳。并且当对多个候选区域进行最佳匹配选择时,通常依靠区域的单一参量进行判别,但受到视点的变换时效果并不理想。本文提出一种结合已匹配区域参量构造视差梯度函数的立体图像区域匹配方法,该方法摒弃了现有算法中单一参量不稳定的特性;此外提出了基于Zernike矩、中心环投影曲线相关、小波变换及分形维的立体图像区域匹配方法。实验结果证明,针对于区域的半遮掩、变形和细微差别等情况,本文的算法都具备更好的识别性能,是一种行之有效的区域匹配算法。其次基于曲波变换是继小波变换后的一种新型的多尺度分析方法,它能够更好的描述图像中曲线状和超平面的奇异性问题,本文把曲波变换系数作为图像匹配中的基元,并结合图像分割原理和马尔可夫随机场(MRF)模型,提出了一种新的基于Markov模型和曲波的立体图像匹配方法,该方法克服了基于图像灰度的匹配方法在平滑区域或细节匮乏处无法得到正确视差的弊端,并使得视差图在物体内部平滑并保持边缘处的不连续性。实验结果表明,本文提出的算法无论从视觉评价上还是视差图客观指标来看,都取得更优的结果。再次本文运用偏微分方程理论提出一种改进的立体图像匹配的正则化方法。先分析了匹配点对在不同相对位置下对匹配项产生的影响;接着提出了适用于视差图的各向异性的热扩散方程,它不仅继承了Alvarez定义的正则项对初始视差图内部平滑和保持边缘不连续的特性,还引入了图像的噪声屏蔽函数和二阶方向导数来分别控制对应视差图中不同区域的扩散速度和角点处的扩散方向;通过本文定义的正则项和匹配项构造新的能量函数,并把基于区域匹配算法得到的视差图作为初始值,数值求解相应的最小能量泛函。实验结果表明无论从视觉效果上还是重构深度图的判别上,本文的算法均为优秀。由于视差图反映了图像中各个景物的深度信息量,因此视差图的边缘并非是对应图像的边缘。而现有的正则化方法中能量函数的正则项都依照图像梯度场对初始视差图内部进行平滑并保持边缘的不连续性,这导致最终获取的密集视差图不仅不能准确分辨物体边缘且平滑区域具有较多的图像边界残留痕迹。光流反映了目标物体的运动状态且包含的运动边界与物体边界相吻合,但光流计算中的正则项使其在边界上是模糊不准确的,因此本文采用了多分辨率框架下由粗到精的分级策略来计算大基线时左右图像中的光流场,并结合图像边界和光流来确定准确的运动边界,提出了一种融合光流信息的正则化方法,该方法利用携带景物深度信息的光流场构造了融合光流信息的各向同性和异性的视差图正则项。通过与现有方法进行实验比较,提高了最终获得的视差图的质量。最后在前面理论和实验的基础上建立了一个虚拟空间会议的原型系统,实现了虚拟会场的构造、显示与漫游、与会者的视频对象提取与合成、基于视线的中间视生成等主要功能。更多还原

【Abstract】 Virtual Space Teleconferencing System (VST) is a synthesis of many kinds of techniques: Virtual Reality, Computer Network, Multimedia, etc. It combines different local conference spaces into one virtual space breaking through the traditional perspective about conference. Participants appear in computer-generated virtual spaces as avatars in VST, and these avatars can locate, view, manipulate virtual objects, communicate with others face to face, so participants could share ’the same space’ and do cooperative work. VST is also named Ultimate Teleconferencing System. This dissertation concentrates on the display technique such as: conventioneer avatar modeling, 3D reconstruction, stereo image matching. Super-wavelets, stochastic mathematic and partial differential equation theory are used, many stereo image matching methods are presented and a virtual space teleconference prototype system is designed. The main research points in this paper are as follows:Firstly, existing stereo images regional matching methods will be affected by the factors such as regions occlusion, regions warping and lighting condition, so the traditional constraint conditions, like order constraint, unique constraint , epipolar constraint and adjacent constraint, may be violated by these cases. In past few years a new algorithm based on the relative position constraint (RPC) between regions is proposed which can overcome most of problem mentioned above, but it has not satisfying performance in matching occluded objects in the stereo images. And only single parameter of region is usually used to judge during selection of a best region among multiple candidate regions, but its performance is not satisfying because of the view angle transform. In order to choose the best matching region, a method that combining with the parameter of matched region to construct disparity gradient function is presented, which casts away the instability of existing algorithms that judge by single parameter. Furthermore regional matching methods based on zernike moments, center ring-projection, wavelet transform and fractal dimension theory are proposed. Finally, the proposed region matching algorithm is illustrated with many synthesized stereo images, and the superior recognition performance to the case of regions deforming, regions occlusion and regions slight difference is experimentally verified.Secondly, curvelet transform is a new multi-scale analysis method based on wavelet transform, it is better suitable for describing of those images that have curve or super plane singularities. In this paper a new stereo matching algorithm based on markov random field and curvelet transform is proposed which takes curvelet transform coefficients as the image matching primitive and combines image segmentation principle with markov random field model. The approach not only overcomes these defects that image matching algorithm based on gray level can not get accurate disparity of areas where are smooth or lack of details, but also smoothes disparity inside the object boundaries and keeps the discontinuities across the boundaries. Finally, the superior visual effect and evaluation indexes of our algorithm are experimentally verified.Thirdly an improved regularization method of stereo matching is proposed. Above all, the effects of matching pairs at various relative positions to the attachment item are analyzed. then anisotropic heat diffusion equation adapts to disparity map is presented, which inherite from the ability of the Alvarez defining regularization item that keeping the discontinuities across the boundaries of the image and smoothing disparity inside the boundary, in addition image noise shielded function and second order direct ional derivative are introduced to separately control disparity diffusion velocity of different area and diffusion direction of edge position. At last, new energy function according to our approach is defined, adopting the output of the area stereo matching method as the initial value, steepest descent is exploited to solve the energy functional. Experiment results demonstrate the effectiveness of our approach, both in the visual effect and 3D depth retrieval.The edge of disparity map which reflect the sceneries’ depth information are not equivalent to corresponding image edge. But in the current regularization methods, regularization item of the energy function smoothes the interior domain and keeps the boundaries discontinuity of disparity map according to the gradient field of image, it leads the finial dense disparity map can not clearly distinguish the object edge and retain much traces of image edge in the smooth area. The motion state of target is mirrored by optical flow among which moving boundaries are identical with the target edge, but the regularization item of optical flow calculation causes the result inaccurate at the image edge, so the coarse-to-fine classifying strategy in the multi resolution frame is applied to calculate the optical flow for wide baseline stereo pairs, then precise moving boundaries are obtained through the union of image edge and optical flow, the regularization method fusing the optical flow is presented, isotropic and anisotropic regularization items of disparity map fusing optical flow information are constructed. At last, new energy function according to our approach is defined and compared with current methods. Experimental results demonstrate the better performance of our approach both in the visual effect and the disparity map evaluation indexes.Lastly, the VST prototype system is designed on the basis of textual theory and experiment, the function such as: virtual meeting hall creating and divagation, conventioneer video object extraction and synthesizing, intermediate view image creation are realized.更多还原