【作者】 廖赟；

【导师】 周华；

【作者基本信息】 云南大学 ， 系统分析与集成， 2012， 博士

【摘要】 基于计算机视觉的人机交互是自然人机交互技术领域的研究热点,它利用视觉信息从视频序列中捕获并理解人的肢体动作,使人与计算机之间的交互摆脱了鼠标及键盘的约束,人只需要通过肢体动作、手势、表情等自然信息就能与计算机进行交流,突破了从计算机诞生至今在人机交互过程中必须由人来适应机器的屏障,人与计算机之间的交流更加自然、流畅。人手在信息交互过程中蕴含大量信息,因此在人机交互系统中处于非常重要的地位。文献[30]指出,手指交互技术主要有四类应用：手语识别、手势识别、手指书写、虚拟触控,如何获得手指状态信息是其中的关键点。通过附着在手上的特殊设备如：数据手套或传感器可以准确的获得手部3D模型从而获得手部运动和形状的准确信息,然而却要求附加额外的设备,价格昂贵,不易普及应用。随着图像处理、机器视觉、人工智能等学科的不断发展,基于普通摄像头,并以一种机器视觉的方式获得手指状态信息成为可能。这是一种更加自然同时也更加廉价的人机交互方式。然而,由于人手是一种多关节非刚性物体,手指的状态在运动过程中不断发生变化(可高达27个自由度)以及视觉本身的不适定性,使得基于视觉的手指交互是一项极具挑战性的研究。国内外众多学者及相关研究机构在基于视觉的人机交互系统的研究上做出了大量富有成效的研究,然而,由于多数研究对应用环境都做出了苛刻的限制,与真实应用环境出入较大,难以适应真实的应用环境。针对此问题,本文分别提出复杂环境中手势分割算法、实时指尖定位算法和基于Mean Shift及粒子滤波的手部运动跟踪算法,并在以上算法的基础上设计并实现了基于视觉的人机交互系统。具体而言,本文贡献如下：1)复杂环境中的手势分割方面。针对复杂环境下,肤色识别效果较差的问题,本文提出了基于颜色及局部背景差分法的肤色检测算法。该算法首先在YCbCr颜色空间中检测背景中的类肤色区域,并通过基于码本(codebook)的背景建模方法对背景中的类肤色区域进行建模。然后应用亮度无关的静态椭圆肤色模型在YCbCr颜色空间中的CbCr平面上对肤色进行初次检测,如果初次检测结果与背景中的类肤色区域存在重叠部分,则在重叠部分应用局部背景差分法分割出准确的前景肤色区域。该算法能够在复杂环境中准确分割出肤色与非肤色,据有极强的适应能力及较高的执行效率。2)手指指尖定位方面。针对3D手部模型在基于视觉的人机交互系统中过于复杂难以进行实时计算的问题,本文提出了一种简化的手部2D模型,与以往的手部2D模型相比较,该模型的特点是强调手掌重心位置及手指指尖位置,根据该模型只要准确找到手掌重心就能根据指尖与手掌重心的距离关系定位指尖位置。其次,提出了基于距离变换的手掌重心查找算法,该算法能够稳定准确的定位手掌重心,并且查找到的重心位置与手指伸出的数量及状态无关。最后,本文提出了基于距离变换的手指指尖定位算法,本算法能够准确的查找到指尖位置,且运行效率较高,完全满足实时计算任务要求。3)手部运动跟踪方面。对目标进行快速准确的跟踪一直是基于视觉的人机交互系统的研究重点,针对人手的无规则快速运动难以进行有效跟踪的问题,本文提出了基于MeanShift及粒子滤波的四向预测跟踪算法,该算法结合了MeanShift算法及粒子滤波算法各自的优点,当目标发生突然变向运动而导致运动目标跟踪丢失的前景下将启动四向预测机制,对运动目标实施二次预测搜索,该算法能对快速变向运动目标进行有效跟踪,完全满足手部不规则变向运动的实时跟踪要求。4)手部跟踪目标区域初始化方面。在基于区域的跟踪方法中,被跟踪区域的初始化过程通常是人手工标记完成的,如何自动初始化被跟踪目标区域一直是个研究难点,本文提出了一种基于手部特征的跟踪区域自动识别算法,该算法能够有效利用肤色检测及手指指尖查找的结果,与本文提出的指尖查找算法及快速目标跟踪算法据有良好的集成性。5)实时手部运动跟踪及指尖定位方面。由于基于视觉的人机交互系统是一个涉及到计算机图形学、模式识别、人工智能等众多学科领域的复杂系统,因此,如何将处理不同问题的算法进行有效集成,使其成为一个有机的统-体,是任何一个人机交互系统的设计者所必须面对的重要问题。本文对复杂环境肤色检测算法、基于码本的背景建模方法、指尖定位算法及四向预测跟踪算法进行了有效集成,充分考虑了各个算法计算结果的复用性,保证了系统整体的运行效率,形成了基于手部运动及指尖定位的人机交互系统的基础,实验证明该系统能够有效应用于基于计算机视觉的人机交互环境。更多还原

【Abstract】 Vision-based human-computer interaction is a research focus in natural human-computer interaction field.It uses the visual information to capture from the video sequence, and makes the interaction between humans and computers out from the mouse and keyboard constraints. People can communicate with computer just through natural information such as body movements, gestures, and facial expressions.The exchange between humans and computers is more natural and smooth. Finger contains a large number of information in human-computer interaction process, so it is in a very important position in natural human-computer interaction system.Reference[31] indicate that finger interaction technology has four main applications:Sign language recognition;gesture recognition; Finger writing and virtual touch, how to get finger status information is one of the key points.It is possible to get finger status with the development of image processing,computer vision and artificial intelligence.However, due to hand is a multi-joint non-rigid object (Can be as high as27degrees of freedom [23]),so vision-based finger interaction is a challenging study. Domestic and foreign scholars and research institutions have made much effective research in the field of vision-based human-computer interaction systems.However, the majority of research has made harsh restrictions in application environment, and the environment is quite different with the real-world environments.So the vision-based human-computer interaction systems tend to only stay in the laboratory. According to this problem,this paper presented the complex environment skin detection algorithm,real-time fingertips localization algorithm and Tracking algorithm of random movement and high speed. Based on the above algorithms, we design and implement a vision-based human-computer interaction system. The contributions of this paper are as follows:1)Skin color detection in complex environments.According to the problem of poor color recognition in complex environment, this paper presented skin color detection based on color and part background subtraction algorithm.Firstly, the algorithm detects the skin color regions in background area with YCbCr color space.Secondly, it models the background in skin color regions with code book background modeling method.Thirdly, it initials detect skin color with static elliptical model in CbCr flat of YCbCr color space.If the initial detection skin color results have overlapping parts with skin color region in background, the algorithm uses part background subtraction to segment foreground skin color region in overlapping parts.The algorithm can effectively distinguish skin color and non-skin color in complex environment, and it possessed a very strong resilience and high efficiency2) Fingertip location.This paper presented fingertip location base on distance transform algorithm.Firstly, according to the problem that the3D hand models are too complex to real time calculate in vision-based human-computer interaction system, this paper presented a simplified2D hand model,and the characteristics of the model is emphasis on palm center of gravity position and fingertips position. Using this model,we can locate fingertips according to the distance between fingertips and palm of the center of gravity, if the palm center of gravity can be found accurately. Secondly, this paper presented palm of the center of gravity location algorithm based on distance transform.The palm of the center of gravity can be located accurately location by this algorithm.3)Fast moving target tracking.Tracking object which makes random and rapid movement is always a research focus.This paper presented a four-way prediction tracking algorithm based on particle filtering algorithm and Mean Shift algorithm. The algorithm combines respective advantages from both particle filtering algorithm and Mean Shift Algorithm.Firstly, it uses particle filtering algorithm to predict the possible region of target object. After that, we lock on the precise position of target object by using Mean Shift algorithm,which is proved to be efficient and speedy. Meanwhile, it uses the four-way prediction tracking algorithm to deal with the losing frames which lead by the random movement of target object, and makes a dramatically improvement for the possibility of tracking.Experimental results show the algorithm has high robust when tracking target with random and rapid movement.4) Tracking region initialization. Tracking regional initialization process is usually hand-finished in tracking method based on region. How to automatically initialize tracking regional has been a research difficulty. The algorithm can effectively use color detection and fingertip search results.5)Real-time hand motion tracking and fingertip location. The vision-based human-computer interaction system is a very complex, which related to a computer graphics, pattern recognition, artificial intelligence and many other fields.Therefore, how to effectively integrated differences algorithms is an important problem for any designer of human-computer interaction system. In this paper, the algorithms of complex environment skin color detection, part background subtraction, fingertip location and four-way prediction tracking are effective integration. We consider the reusability of the results of each algorithm;ensure the operational efficiency of the system.All of these formed the basis of human-computer interaction system. Experiments show that this system can be effectively used in vision-based human-computer interaction environment.更多还原