【作者】 徐亚；

【导师】 刘光远；

【作者基本信息】 西南大学 ， 信号与信息处理， 2010， 硕士

【摘要】 情感计算是一个高度综合化的新兴研究领域,目的是通过赋予计算机识别、理解和适应人类的情感能力来建立和谐的人机环境,并使计算机具有更高的、全面的智能。情感识别是情感计算的一个重要部分,它研究的内容包括语音信号、身体姿态、面部表情和生理信号等方面。由于生理信号具有真实的、可靠和难以伪造的特点,用生理信号进行情感识别成为其中的一个热点方向。美国麻省理工学院媒体实验室情感计算研究小组首先用生理信号作为研究对象进行情感识别,且取得一定的硕果,这也为生理信号情感识别的研究提供了可靠的支撑。心电、心率信号蕴含丰富的情感特征,可以明显的反映出人类在不同情感状态下的变化。因此文中利用心电、心率信号来进行情感识别,并验证了在高兴、惊奇、厌恶、悲伤、愤怒和恐惧这六种情感状态下情感用户模型建立的可行性。其过程有四个主要步骤：1、情感数据采集；2、情感信号的特征提取；3、情感信号的特征子集选择；4、分类器的设计。文中设计了周密的方案保证用于研究的心电、心率信号包含着某种特定情感：选取有效的电影片段激发被试情感,且让被试记录当时看完后片段的感受,也通过隐藏的摄像头观察被试观看电影片段的情况,并在Superlab采集软件上做上相应的标记。采用美国Biopac公司提供的多导生理记录仪MP150,为了有效地激发被试情感,文中只对300位西南大学在校大一学生进行信号的采集,建立了情感心电、心率信号数据库。这个方案中对素材电影片段的有效选取、被试的要求,保证了采集的信号包含有某种特定情感,文中激发被试的情感状态有：高兴、惊奇、厌恶、悲伤、愤怒和恐惧六种情感；心电信号的特征提取关键在于P-QRS-T波位置检测,然而,采集的心电信号容易存在基线漂移等噪声干扰而难以准确进行P-QRS-T波的检测。小波变换具有良好的时、频局部化,在时频域都具有表征信号局部特征的能力,且在图像分析、去噪和压缩等方面得到广泛的应用。文中采用连续小波变换将原始心电信号进行5层分解,根据R波的频率范围,使用第一层小波系数准确检测到R波位置,然后检测Q,S,P和T波位置,且能自动检测出信噪比过小的信号段,对该信号段不进行特征提取以保证研究结果的正确性,然后通过几何平均法去除了心电的高频噪声,对心率信号的高频噪声也进行了平滑处理。由于提取到的大量冗余和无效的心电、心率的特征会影响情感识别的效果,需要使用特征选择筛选出有效的特征用于情感分类。特征选择问题是一个组合优化问题,其计算复杂度随着维数的加大成指数倍的增加,它需要使用有效的搜索算法来解决。离散二进制粒子群算法(BPSO)是一种智能的全局优化算法,它因具有计算速度快、算法参数简单和易于实现而被广泛应用,目前已应用于组合优化问题、函数优化、信号处理、神经网络训练、数据挖掘和数据聚类等应用领域；序列后向选择算法(SBS)也是一种有效的搜索算法。因此,论文研究将BPSO、SBS算法应用于心电、心率的特征选择问题上,以提高情感状态的识别率。针对BPSO易陷入局部导致早熟收敛现象,文中提出两种改进策略：一种改进算法是基于邻域搜索的方法(IBPSO),让粒子群有更多机会的跳出局部最优,向全局最优方向前进；另一种是将遗传操作(交叉和变异)引入到BPSO中,用来增加种群的多样性。同时,由于fisher分类器具有高效、准确率高的特点,文中采用fisher分类器,并且将其与BPSO、SBS算法结合共同解决特征选择问题。实验结果表明,心率特征用于情感识别优于心电特征,特别是在恐惧、惊奇情绪下,在两种改进的BPSO算法中,心率最佳特征组合的平均验证识别率都高于心电特征10个百分点左右。对两种改进的BPSO来说,对于完全相同的训练集、测试集和验证集却得到了不同的最佳特征组合,导致得到的验证识别率有较大的区别。一般来说,IBPSO算法得到的平均验证识别率都高于或和GBPSO算法相差不大(除了高兴情绪状态),而且IBPSO算法得到的平均特征维数也明显少于GBPSO算法,这表明IBPSO算法得到的最佳特征组合更适用于情感用户模型的建立。而就SBS算法而言,虽然选择的最佳特征组合的测试、验证结果都差于两种改进的BPSO算法结果,但是在进行高兴情感状态的识别中,获得了很少的特征,却达到了与BPSO算法一样的效果,而且此时选中的特征很大一部分在BPSO算法中也被选中,证明这个特征组合适合于高兴情感用户模型的建立。更多还原

【Abstract】 Affective Computing is a highly integrated research area that has rapidly evolved recently, aiming at giving the computer more emotional abilities in identifying, understanding and adapting for building a harmonious man-machine environment and further possessing a higher and comprehensive intelligence. As an important part of affective computing, emotion recognition mainly contains diverse studies including voice signals, body posture, facial expressions and physiological signals and so on. Owing to the characteristics of truth, credibility, and difficulties to counterfeit, it is becoming one of the hottest research fields based on physiological signals. Affective Computing Research Group from MIT Media Lab first has a research on emotion recognition from physiological signals, and had already reached some good achievement. Therefore, it also provides a reliable support for emotion recognition from physiological signals.As ECG and heart rate of abundant emotional features, they can clearly reflect human’s emotional states under the different changes. Therefore, this paper has a research for emotion recognition and validation of user model from the extracted data on ECG and heart rate (six emotion states: happy, anger, surprise, fear, sadness and disgust). Main contents are as follows: data acquisition, feature extraction, feature selection and classifier design.This paper has planed elaborate rules for guaranteeing ECG and heart rate containing some special characteristics of emotions. In the experiment, the program is just designed to ensure that our chose movie clips are enough effective to elicit subject’s emotion, which also need subjects to record their own feelings after watching over. Besides, operator will also carefully observe everything from subjects by the camera hidden in somewhere, and momentarily give the according marks on capture software Superlab. Through U.S. Multi-Polygraph MP150 from Biopac, this paper acquires emotion data from 300 freshman from Southwest University and then sets up emotion database on ECG and heart rate. For the effectively rigid selection on film clips and subjects, it meets ideal requirement at some extent that the collected signals must contain certain emotion which includes happy, surprise, disgust, sadness, anger and fear. Feature extraction of emotion signals on ECG and heart rate is mainly depending on position detection of P-QRS-T wave, but there are also some difficulties in accurately detecting P-QRS-T wave with noise interference such as ECG baseline drift, etc. Wavelet transform has a good time and frequency localization, and could reflect local characteristics of signals in the time-frequency domain, so it is widely be used in image analysis, denoising and compressing, etc. In this paper, continuous wavelet transform is suitably adopted to ECG signal for 5-layer decomposition. According to frequency range of R-wave based on the first level of wavelet coefficients, R-wave position can be exactly detected, and then Q, S, P and T-wave locations would be easily found. Through automatically filtering signal intervals of lower signal-noise ratio, it can finally remove the high-frequency noise and smoothen effective signal by the geometric mean..As effect of emotion recognition will be disturbed by the extracted features of redundant and ineffective characteristics in ECG and heart rate, then it is necessary to select effective feature for classifying emotion. Feature selection is a combinatorial optimization problem and its computational complexity would be exponentially enhanced with the increasingly amount of dimension, so it requires effective search algorithms to solve this troubles. Discrete binary particle swarm optimization (BPSO) is an intelligent global optimization algorithm, which has been widely applied at present to combinatorial optimization problems, function optimization, signal processing, neural network training, data mining and data clustering and other applications with more advantages in speed computing, simple algorithm parameters and easily implementing. Besides, sequence backward selection algorithm (SBS) is also an effective search algorithm. Therefore, this paper will satisfactorily introduce BPSO, SBS to feature selection on ECG and heart rate to improve recognition rate of emotion states. For BPSO can be easily lead to local search bringing into premature convergence, this paper proposes two kinds of improvement strategies: an improved algorithm is based on neighborhood search method (IBPSO), and it can conveniently supply particle swarm more opportunities to jump out of the local optimum to the global optimum; the other perfectly introduce the genetic operations (crossover and mutation) into the BPSO, and it mainly increase the diversity of evolution population. Moreover, as fisher classifier of high efficiency and accuracy, it is just imported to feature selection with BPSO and SBS.Ultimately, the experimental results show that the features of heart rate are superior to ECG for emotion recognition, especially in fear, surprise. In both improved BPSO algorithm, the average recognition rate attained by the best feature combination in heart rate is nearly respectively 10% higher than in ECG. Furthermore, for the same training, test and validation set, the best combinations are exactly different. So there are greater differences in recognition rate for validation among them. In general, the average recognition rate for validation by IBPSO is higher or as much as GBPSO (in addition to happy), and the dimension of the selected feature subset is obviously less accordingly, which shows that the best feature combination is more suitable for affective user modeling. And for SBS, although the best feature combinations for test and validation are worse than the two improved BPSO above, it gladly gained a few features that could achieve the same effect as BPSO during emotion recognition for happy. And surprisingly, some selected features also mostly existing in BPSO. So it does show that the feature combination is suitable for affective user modeling in happy更多还原