【作者】 孟春宁；

【导师】 常胜江；

【作者基本信息】 南开大学 ， 光学工程， 2013， 博士

【摘要】 眼睛是人类最主要的感觉器官,也是人脸最显著的特征。对眼睛及其运动的研究是了解人类视觉机制、理解人的情感和行为以及基于眼动的人机交互等问题的关键。基于计算机视觉的人眼检测与跟踪技术具有低侵入性、高精度等优势,现已成为眼动研究的主流方法。人眼检测与跟踪是人脸识别、表情识别、眼动分析、虹膜识别等技术的必要步骤,涉及图像处理、计算机视觉、模式识别以及认知心理学等多个学科,并在工业检测、智能机器人、人机交互、公共安全、智能交通、心理学、医学诊断以及军事侦察等领域均有广泛的应用。现有的人眼检测与跟踪算法存在的客观问题是：精确定位并跟踪人眼的算法都建立在高质量的成像设备与复杂的光学系统之上,缺乏低质量人脸图像中精确获取人眼位置的有效算法,导致精确记录眼动信息的系统价格昂贵且对用户束缚较多,应用范围受到很大的限制。针对这个问题,本文在现有成果的基础上,通过深入研究低质量图像中的眼睛特点,提出了一系列基于低质量图像的人眼检测与跟踪算法,通过设计和搭建低成本的图像采集系统,对本文提出的基于眼动分析的行为识别及注视估计算法的性能进行了系统分析。本文的主要创新点包括：1.为解决现有人眼区域分割方法难以剔除眉毛区域的问题,提出一种基于尺度不变梯度积分投影函数的人眼分割方法。该方法充分考虑到人眼区域图像局部灰度变化丰富的特点,并继承了投影方法计算量小的优点。对不同分辨率、不同光照条件及不同姿态的人脸图像的眼部分割结果表明,该算法能够在保证检测正确率和速度的同时,有效地去除眉毛的干扰。2.为提高低质量人脸图像中人眼检测的精度,提出了一种梯度积分投影结合最大期望算法的人眼检测算法。该方法通过引入最大期望算法进一步缩小了人眼窗口的范围,然后在精确的人眼窗口的基础上采用一种加权质心法获取人眼位置。对YaleB及自采数据库中图像的测试结果表明,该算法对光照、姿态、眼镜及不同的眼睛状态都有很好的鲁棒性,对比实验结果显示该方法的检测精度优于现有的四种投影方法。3.针对现有的虹膜精确定位算法难以用于低质量人脸图像的问题,提出了一种基于矩形积分方差算子的虹膜定位算法,该算法可以在低质量人脸图像中实现快速精确的虹膜定位。在YaleB及FERET人脸库上的实验结果证明该算法对眼睑遮挡、尺度变化、轻微头部转动及光照变化有较好的鲁棒性,在严格的评价标准下正确率达到了百分之九十以上,远高于经典虹膜定位算法的结果。4.为了对低质量视频图像中的眼睛运动进行跟踪,提出了一种基于差分指导的重要性采样虹膜跟踪算法,并根据梯度投影积分函数设计了一种眨眼检测方法以避免眨眼导致的跟踪漂移。采用本文提出的虹膜跟踪方法获取的眼动信号经小波去噪后用于视觉行为识别,取得了理想的效果。5.首次利用单个普通网络摄像头实现了基于眼动分析的行为识别。采用一个置于计算机监视器附近的网络摄像头采集测试者阅读电子文档、浏览网页、观看视频等行为的视频图像,针对低质量视频图像中不同行为的眼动特点,从眼动信号中提取了十种眼动特征,利用支持向量机实现了对不同注视行为的分类。针对不同的应用背景设计了三组验证实验,实验结果证明了基于普通视频图像的眼动分析在行为识别领域应用的可行性。6.为扩展眼动跟踪技术的应用范围,设计了一种低成本、低侵入性的耳麦式眼动记录系统,该系统主要由两个普通CMOS摄像机构成。针对该系统采集的图像特点,提出了一种分段加权环形Hough变换算法用于提取虹膜轮廓中心。最后设计了一种简便的注视点标记方法,利用支持向量回归机实现了对注视方向的估计,实验结果表明低成本眼动仪可以满足日常交互需求。更多还原

【Abstract】 The eye is not only the most important sensory organ of human body but also the most salient feature of human face. Research on the eye and its movement is essential for eye based human-computer interaction and understanding human visual mechanism, emotion and behavior. Eye detection and tracking based on computer vision have become dominant methods due to their harmlessness and high accuracy.Eye detection and tracking are the necessary steps of face recognition, facial expression recognition, eye movement analysis and iris recognition. Research of eye detection and tracking involve image processing, computer vision pattern recognition, cognitive psychology science and so on. Research results of eye detection and tracking have widly been used in the fields of industrial inspection, intelligent robots, human computer interaction, public safety, intelligent transportation, psychology, medical diagnostics and military reconaissance.Locating eye center precisely is difficult for existing eye detection and tracking algorithms based on low-quality images. In addition, accurate iris localization is usually carried out on high-quality and high-resolution eye images. Precise eye tracking device is usually very expensive and inconvenient, so its application field is very narrow. To solve these problems, the characteristics of the low quality eye images are studied, and a series of eye detection and tracking algorithms on the basis of previous research results are proposed. In addition, we evaluate the performances of our methods, which have been used in activity recognition and eye gaze estimation, by designing low-cost eye movement recorders.The major innovative researches in this paper are shown as follows:1. To overcome the weakness that eyebrow is difficult to be excluded in the eye windows by traditional methods, a Scale-invariant Gradient Integral Projection Function algorithm is proposed to segment eye region. The features of low-quality eye images are taken into full account in this method. Like other projection methods, its computational complexity is very low. Experiments on images with different scales, poses, and illumination condition demonstrate that our method can effectively remove the interference of the eyebrows with high detection rate.2. In order to improve the accuracy of eye detection in low quality face images, an eye detection method based on gradient integral projection and expectation maximization algorithm is proposed. More precise eye windows are abtained by use of expectation maximization algorithm. Then, the weighted barycenter algorithm is used to improve the detection accuracy. Comparison experimental results on our image database and YaleB face database demonstrate that this method is robust to illumination, head poses, glasses and blinks, and more accurate than four existing projection methods.3. In low-quality images, precise iris center is difficult to be abtained by existing iris localization algorithms, because accurate iris localization methods need high-quality and high resolution eye images. Considering this problem, a novel rectangular integro-variance operator is designed and used to precisely locate both of the irises. Experimental results on FERET and YaleB face database demonstrate that this method is robust to eyelid occlusion, different scale, mild head rotation and illumination changes. The detection rate is over90%under a harsh evaluation criterion, which is far higher than the detction rate of the classical iris localization methods.4. To acquire the eye movement data in low quality videos, a difference image based importance sampling iris tracking method is proposed, and a blink detection method is proposed based on the gradient integral projection algorithm to aviod drift. After denoising by wavelet, eye movement signals abtained by this method have a good performance in activity recognition based on eye movement analysis.5. Eye movement analysis for activity recognition under one Web camera is first proposed. A Web camera is seted on the computer monitor to record the activity videos of reading electronic document, browsing the web and watching video. First, ten novel features are extracted from the eye movement signals. Then, support vector machine is used to activity classification. Finally, we design three experiments based on different application situations, and the expermental results show that the eye movement analysis in common video for activity recognition is a promise sensing method. 6. To extand the application range of the eye tracking, a low-cost and less intrusive eye movement recording system is designed, which is composed of two common CMOS cameras. A segmented weighting-annular Hough transform localization algorithm is proposed on the basis of the eye images captured by this system. Then, a convenient calibration procedure is designed, and the gaze direction is estimated by support vector regression. The experiments demonstrate that the low-cost eye tracker can be used for everyday human computer interaction.更多还原