【作者】 李飞；

【导师】 钱剑敏；

【作者基本信息】 东华大学 ， 通信与信息系统， 2008， 硕士

【摘要】 普通鼠标是一种手动的传统交互方式,满足不了特殊人群的需求,譬如手臂残疾的人和游戏爱好者就期望采用多种交互方式来操作PC。随着图像、视觉核心算法的发展,语音识别的理论和应用研究的进展,再加上CCD、CMOS图像传感器制造工艺的成熟,为基于视觉与语音的鼠标实现提供了条件。本文主要的研究是基于视觉与语音的新型鼠标。首先对各种视觉跟踪算法的实现方法进行探讨。在充分了解各种算法优缺点的基础上,系统采用鼻子特征跟踪的方法来作为鼠标的光标移动,采用眼睛睁合判断的方法作为鼠标按键的操作,用HALCON视觉处理软件对各算法进行仿真,并在Visual Basic中调用HALCON库函数加以实现。另外,鼠标按键的操作还可选用语音识别的方式来实现,文中用MATLAB仿真实现了DTW(动态时间规整)算法,该算法训练方法简单,计算量小,适合于本系统的设计。采用Visual Basic与Matlab混编的方式在Visual Basic中调用创建的COM组件,实现语音识别的功能,进行性能测试。将视觉跟踪模块与语音识别模块结合起来,根据基于视觉与语音鼠标的体系结构进行设计,编写全部程序进行系统的整体调试,调试成功后进行预定的实验,记录实验结果。将视觉与语音技术应用于鼠标中,在国内还没有人开发研究,这无疑很有研究的价值,具有一定的市场潜力。从完成的设计表明,基于视觉与语音的方案可以完成鼠标的基本功能,而且可以适用于特殊人群。但本系统还处于研究设计阶段,还不够完善,后续工作将主要围绕算法性能的提高,程序的优化以及其他一些细节问题作进一步的改善、测试。更多还原

【Abstract】 Ordinarily mouse is the tradition alternation mode with hand but the people with hand deformity and game enthusiast wish to operate PC with manifold alternations. It is likely to realize mouse based on vision and word recognition with the development of CCD、CMOS image sensor’s manufacture technics and image、vision arithmetic.The paper most research and realize the new pattern mouse based on vision and voice. Firstly, the paper discuss all the methods of realizing vision track arithmetic, the system use top of nose character track as cursor motion, use eyes’ open and close as the operate of keystroke, use Halcon to simulate the arithmetic and realize with Visual Basic. Otherwise, the operation of keystroke can also realize with voice recognize. The paper simulate the DTW arithmetic with Matlab to create COM module, and mix-programme with Visual Basic and Matlab , then performance test. The training method of the arithmetic is simple , it’s fit for system’s design. Lastly, combine the vision track module and voice recognize module, and debug the whole system, note the experiment result. Application vision and voice to mouse, there is nobody research inland, and this is no doubt worth to studying. Full of marketing potential.The completed design indicated that the scheme based on vision and voice could fulfill the mouse’s function, and it is fit for special persons. However, this system has been designed and completed experimentally. The following work will mainly improve and develop the arithmetic and optimize the program as well as some detail problems.更多还原