【作者】 李云；

【导师】 周平； 陈香；

【作者基本信息】 中国科学技术大学 ， 生物医学工程， 2013， 博士

【摘要】 作为最常用的人体动作,手势动作以其丰富多变的含义和灵活方便的执行方式,被广泛用于人类生活的方方面面。手势识别是指利用计算机来检测、分析和解释人所执行的手势动作,从而用于判断动作意图并提供相应的服务。随着现代科技的发展和人们生活水平的提升,手势动作识别成为了人机交互、手语识别、康复训练、运动医学等领域的研究热点。神经系统通过命令和协调不同肌群的活动来完成不同的手势动作。人体肌肉活动的时候,从皮肤表面采集到的电信号即为表面肌电信号(Surface electromyogranphy, SEMG)。SEMG通过对不同肌群活动时电信号的捕获来反映肢体的伸屈状态、位置等信息,是一种重要的手势动作感知方式。同时,放置在上肢的加速计捕获到的加速度信号(Acceleration, ACC)也能反映手势的运动轨迹及姿态变化,是另一种感知手势动作的方式。本文对基于SEMG的手势动作检测与识别技术进行了深入研究,一方面融合SEMG和ACC信息提出词汇量可扩展的连续中国手语识别方案,旨在实现自然和谐的聋哑人与正常人之间的交流,提高聋哑人的生活质量。另一方面将基于SEMG的手势动作识别技术推广到康复工程领域,以手势动作的识别结果辅助神经肌肉疾病患者进行康复训练,具有重要的医学价值。本研究的主要工作内容和创新点包括：1.基于多通道表面肌电信号的中国手语字母手势动作识别研究。此研究的目标是提出基于SEMG的手语手势动作识别方案,并研究能提高手语字母语识别结果的动作执行规范方案。主要的研究内容有：1)利用SEMG对30类中国手语字母手势进行识别,提出一种包括信号采集、数据分割、特征提取和分类器设计的手势识别算法；2)通过对手势动作过程的分解和所涉及肌肉活动状态的分析,提出了对30类中国手语字母动作的执行规范方案。分别开展不同用户执行规范方案前后的动作识别实验,结果都表明动作规范后识别结果得到了较大的提高；3)提出了一种利用独立成分分析和自适应滤波相结合的去噪方法。利用该方法对掺杂了ECG的SEMG信号进行处理,结果显示信号中的ECG噪声得到了有效的去除。2.融合表面肌电信号和加速度信号的中国手语孤立词识别研究。初步探索了两类传感器信息融合的手势识别技术,分别利用SEMG捕获手指手腕等精细动作的肌肉活动信息以及利用ACC捕获不同运动轨迹的手臂挥划信息,提出了一种多流隐马尔科夫(Hidden Markov Model, HMM)分类器和多级决策树相结合的手语孤立词识别方案。以中国手语30类单手孤立词和121类双手孤立词为对象开展识别实验,结果表明：SEMG的瞬时能量可有效分割出两类信号的活动段数据,融合两类信息的识别结果较单一传感器信息的识别有大幅提升,同时多流HMM结合多级决策树的分类算法除了具有较好的识别效果还提升了计算速度。3.融合两类传感器信息词汇量可扩展的连续中国手语手势识别。此研究目的是希望利用有限种类的手语词数据得到对更多类别手语词的识别,并实现对连续中国手语手势的识别。依据两类传感器捕获的信息,将中国手语孤立词从执行构成角度拆分为手型、朝向和轨迹三类要素。提出利用要素的识别结果识别孤立词,再用孤立词识别结果识别手语词的连续手语识别方案,在一定程度上扩展可识别的词汇规模。实验结果表明,以120类中国手语双手孤立词组成的200个连续手语句子为对象,利用本方法可以得到很好的孤立词分类结果。同时以统计语言模型进行句子中前后孤立词的识别纠错,以句法模型进行句子整体架构检错,能实现连续句子的有效识别。4.探索基于SEMG的手势识别在康复领域的应用。此研究目标是以手势识别结果辅助神经肌肉系统疾病患者进行康复训练,是基于SEMG的手势识别技术在康复工程领域的初步探索。针对中风患者执行的20类手势动作,利用高密度肌电电极捕获患肢带有无意识肌肉痉挛干扰的SEMG信号,提出了一套包括特征提取、特征降维和分类器设计在内的有效的动作识别方案。同时,针对高密度电极资源占用多、佩戴繁琐的问题,通过选择空间滤波方式、电极通道的选取、SEMG信号分析窗长度、采样率和高通滤波截止频率,大大减少了数据的冗余,为实际的临床应用提供参考依据。本论文研究工作得到了国家863高科技研究发展计划“基于肌电传感器和加速计的手势交互设备研究”(2009AA01Z322)、国家自然科学基金项目“基于表面肌电的中国手语手势识别研究”(60703069)、NOKIA赫尔辛基研究中心及北京研究院合作项目,以及中国科学技术大学研究生创新基金的资助。更多还原

【Abstract】 Hand gesture, as the most commonly used body gesture, is widely used in all aspects of human life due to its rich and varied meanings and flexible and convenient way of execution. Hand gesture recognition is known as the process that computer automatically capture, analyze and understand various types of gestures to determine human intentions and to provide the corresponding services. As the advance of modern technology and the improvement of human living standard, gesture recognition becomes a research focus in the fields of human computer interaction, sign language recognition (SLR), rehabilitation training and sports medicine etc..Arbitrary hand movements are completed by groups of muscles which are coordinated and work closely together under the control of the nervous system. The surface electromyographic (SEMG) signal is regarded as one kind of important bioelectric signals caused by muscular activities. SEMG sensor can capture the information of muscular activities, which not only reflect the state and strength of flexion and extension of the joints, but also reflect the information of limb postures and positions. The SEMG processing technologies provide us with important opportunities to capture hand gestures. At the same time, the accelerometer sensor placed on the limb can record the acceleration (ACC) signal, which reflects the position and the trajectory of the hand. The ACC processing technologies provide us another opportunity to capture hand gestures.This work investigates the detection and recognition of various kinds of hand gestures based on SEMG signals. On the one hand, a combined SEMG and ACC approach for large vocabulary continuous Chinese SLR is proposed, which provides a practical solution for the deaf and the health communication and improves the deaf living standard. On the other hand, the gesture recognition technology based on SEMG can be extended to the field of rehabilitation engineering. The recognition result of hand gesture can aid individuals with neuromuscular diseases in rehabilitation training. The main work and achievements of the dissertation focuses on the following aspects:1. Chinese sign language (CSL) alphabet gesture recognition based on the multi-channel SEMG. This study aims at realizing a SEMG-based hand gesture recognition method and exploring a set of gesture normalization schemes, which could improve the gesture recognition result. The main work are as follows:1) A SEMG-based hand gesture recognition framework including signal measurement, active segmentation, feature extraction and classification were proposed and used to classify30kinds of CSL alphabet gestures.2) According to the analysis of sign language gesture process and activities of relevant muscles, a set of gesture definition improvement and action normalization schemes were proposed.3) The results of user testing experiments showed that the average recognition accuracies of30CSL alphabets were improved after user learned the action normalization schemes.4) A novel signal filtering method combined with independent component analysis and adaptive filtering was proposed. The experiments demonstrated that this method can remove the noise of electrocardiography (ECG) from SEMG signals.2. CSL isolated word recognition based on the information fusion of SEMG and ACC. This study aiming at investigating the hand gesture recognition technique based on the SEMG and ACC signals. Considering the complementary characteristics of the two kinds of signals, a hand gesture recognition framework with joint multi-stream hidden markov model (HMM) and decision tree based on the information fusion of SEMG and ACC was presented. Classification tasks were conducted on30CSL one-hand isolated words and121CSL two-hand isolated words, and the experimental results demonstrated that the proposed method can shorten the train and recognition time and improve the recognition accuracy.3. Vocabulary scalable and continuous CSL recognition based on the SEMG and ACC information. The purpose of this work is to propose a vocabulary scalable CSL recognition method and to realize the recognition of continuous CSL, based on the SEMG and ACC data, three basic components can be extracted from each hand gesture. The three components are the hand shape, orientation and trajectory. The CSL isolated word can be recognized by fusing the components recognition results and the CSL sentence can be recognized by assemble the CSL isolated words. Experimental results on the recognition of200CSL sentences composed by120frequently used CSL isolated words demonstrated the CSL gestures could be well recognized by this component-based method. Statistical language model and syntactic model were used to detect and correct continuous CSL error in this study and got a good result.4. Exploration on the SEMG-based gesture recognition application in rehabilitation engineering. Gesture recognition results were used to aid individuals with neuromuscular diseases in rehabilitation training. A novel framework including feature extraction, feature dimensionality reduction and classification were proposed and used to classify20different arm, hand, and finger movements performed by stroke survivors. High gesture recognition accuracies were obtained using high-density SEMG sensors. A series of practical issues were investigated in this study for practical application. The issues include SEMG electrode configuration, channel reduction, and the appropriate choice of some SEMG signal conditioning and preprocessing parameters such as window length, sampling rate and high-pass cut-off frequency. The outcomes reported in this study can be regarded as guidelines for developing myoelectric control systems toward stroke rehabilitation.The research is supported by the National High Technology Research and Development Program of China (The863Program)"Research on the Gesture Input Devices Based on the Accelerometers and Surface EMG sensors"(2009AA01Z322), National Natural Science Foundation of China "Chinese Sign Language Recognition based on Surface Electromyogram"(60703069), cooperation projects with NOKIA Research Center (Helsinki&Beijing) and Graduate Innovation Foundation of USTC.更多还原