【作者】 朱海龙；

【导师】 唐降龙；

【作者基本信息】 哈尔滨工业大学 ， 人工智能与信息处理， 2012， 博士

【摘要】 基于计算机视觉的动态人群场景分析在智能视频监控、智能人群管理、公共空间设计、智能交通和智能环境、虚拟环境等领域有着巨大的应用潜力，特别是近年来大规模群体性公共安全事件频发对以智能视频监控为代表的人群行为分析提出了更高的要求。分析动态人群场景是分析以人群运动形式所表现出来的人群行为，即场景中所呈现出的人群运动状态。人群场景分析的内容在检测异常人群行为的基础上还应该包括跟踪人群运动状态的演进过程、分析运动状态变化规律和预报人群状态的未来发展趋势。目前通过视频进行人群运动状态分析面临的主要问题有两个：一是复杂天气造成的视频退化影响分析结果的准确性，二是缺乏利用计算机视觉理论与方法对人群运动状态的演变规律开展更为深入的研究。本课题研究了复杂气象条件对提取人群运动速度和人群密度等主要特征的干扰现象，并从局部和整体两个角度研究了人群运动状态演化规律，提出了复杂气象视频分类与动态天气视频复原算法，然后利用增强后的视频数据建立人群流量方程和人群运动状态无向图两种人群运动分析模型，解决了雨雪天气视频退化现象对人群运动状态分析的影响和如何对未来人群运动状态进行预测的问题。研究工作的主要贡献体现在以下四个方面：一、提出一种基于视频序列短时相关性和平均相关性特征的分级天气类型分类算法。在像素亮度值的时间序列上建立自相关函数提取场景特征，构造线性分类器并使用CART决策树将室外场景分类为静态、动态和不稳定三类天气。分类结果为下面的视频复原方法提供天气视频分类前提，即后续的去雨雪方法是基于正确的动态天气视频分类之上。二、提出一种滑动窗口序列在线PCA投影降噪和二次目标分割的视频去雨雪(动态天气)方法。室外监控视频常常受到各种复杂气象干扰（如雨、雪、雾、光照变化和低温等），造成监控视频发生退化，影响了场景分析的准确性。有必要在抑制天气干扰算法方面展开研究，并把它纳入群体系统模型中。针对动态天气类型分析室外视频的噪声特点、前景目标运动性质的差异，提出一种窗口序列在线PCA投影降噪的二次分割视频去雨雪方法。将具有不同运动性质的移动目标（如人、车、镜头前的雨滴和雪花等）精确分类，定位雨滴和雪花区域后使用对应的背景图像修饰来消除不利天气对视频的影响，为后续的人群运动分析提供增强后的视频数据。三、为避免复杂的个体目标分割和跟踪运算，提出一种无需前景分割的局部人群状态分析方法。把视频局部区域在时-空上视为一个独立的线性动态系统，通过混合动态纹理方法在空间上对局部人群分类得到人群密度属性；使用主路径跟从方法计算局部人群运动速度，然后依据人群密度和运动速度属性建立人群流量方程，用该方程可分析局部区域内人群密度、速度和流量之间的变化关系。该模型定量描述了局部人群运动状态、克服了群体运动的不确定性，可用于人群场景中重点监视区域的异常事件检测。四、针对用于人群场景分析的参数模型缺乏对未来人群运动状态的预测手段和泛化能力不强等问题，提出一种非参数聚类的人群异常状态检测与预测的图分析方法。首先，计算场景的速度场作为基本特征数据，使用改进的均值飘移(Mean Shift)聚类算法将速度特征聚类，得到不同运动类别的聚类中心。然后以聚类中心为顶点、各中心间距离(速度差)为边构成无向图，在时间域分析图边权重对称矩阵的变化，比较观测值和预测值之间的背离程度可以检测当前和预测未来一段时间内人群运动状态。由于采用非参数聚类，避免了更换人群场景时重新设置初值、重新挑选训练样本和参数估计等繁杂工作，克服了参数估计精度问题产生的检测和预报误差。前两项内容属于视频的预处理过程，完成天气分类并针对分类结果中的动态天气进行视频复原工作，为后续的人群行为分析提供较高质量的视频数据；后两个研究内容为动态人群场景建模过程，分别采用局部和整体方法描述了人群运动状态。本课题在人群运动场景分析方面的研究工作，较好地解决了受雨雪气象条件干扰的视频复原问题，为模型提供增强的输入视频；提出人群流量方程分析局部人群运动状态，实现了重点区域的差别化监控；非参数化的运动特征分类方法和图分析算法增强了人群运动状态分析方法的泛化能力，为基于计算机视觉的人群行为分析提供了一种在整体上监测和预报人群运动状态的方法。更多还原

【Abstract】 The analysis of dynamic crowd scene based on computer vision can be appliedbroadly to several of fields including intelligent surveillance, crowd management,public space design, intelligent transportation, virtual environment, etc. Thefrequent occurrences of large-scale mass incidents come to challenges tointelligent surveillance, in recent years, based on crowd motion analysis. Priormodels concerning crowd abnormity detection does not satisfy the public safesituation, more researches, such as tracking the population status andevolutionary process, analysis of variation of crowd motion, predicting futuretrends and so on, are required. Modeling crowd motion scene facing two majorchallenges, one is the negative impact on analysis result caused by visiondegradation of complicated weather conditions, and the other is the lack ofin-depth investigation for the law of crowd motion based on computer vision.In the thesis, the author discusses how the bad weathers interfere featuresextraction of crowd scene in videos and investigates the law of crowd motion inways of local and holistic aspects; and proposes a classification method ofweather types and a video (captured in dynamic weather environment) restorationmeans. Furthermore, performs a comprehensive analysis for crowd motion statesusing enhanced video dataset. Finally, the problem of video degradation byrain/snow is solved and a crowd scene model is built by using a crowd flowequation and graphic analysis. The proposed model can be used to detect andpredict feature crowd status.Aiming at building a complete crowd motion model in complicated weatherconditions, the main contributions of the thesis is reflected in the following fouraspects:1. A classification method of outdoor videos in complex weather conditionsbased on short time correlation and mean of video is proposed. The studyformulates an autocorrelation function in the time domain to extract pixelintensity feature, and classifying videos employing a linear classifier with aclassification and regression-tree (CART). As a result, the input video isclassified to three types: static, dynamic and non-stationary weather according to different weather conditions. It is worth noting that the dynamic weather video isused as input data in the following study.2. A video denoising and rain/snow removal method is proposed by a videowindow series on-line PCA projection algorithm on video window series.Outdoor video often suffer interferences from bad weathers (rain, snow, fog,illumination changing, high and low temperature, etc.), the degraded videosreduce the accuracy of the crowd scene analysis. Therefore, it is necessary todevelop algorithm about inhibition of weather disturbances as a part of wholecrowd model. Using the proposed model, the moving objects with differentmotion properties can be separated and those non-interesting regions are replacedby background. Consequently, the model outputs enhanced video for subsequentapplication of crowd motion analysis.3. To avoid complex calculation of object segmentation and tracking, a localanalysis method of crowd status without foreground segmentation is proposed.Each local area in a video is treated as an independent linear dynamic system(LDS) in temporal-spatial domain. Local crowd groups are classified employingthe mixture of dynamic texture to get crowd density, and employing a main pathfollowing method to obtain local crowd speed. Next, using the density and speedproperties a crowd flow equation, which holds the relations between crowddensity, speed and flow, is built. The equation overcomes the drawbacks of theuncertainty of crowd motion, describes the local scene state and can be used todetecting abnormal events in key areas.4. A non-parameter clustering graph analysis is put forward in crowdabnormity detecting and motion predicting to fit the lack of generalizationcapability in most existing crowd models. The velocity fields are extracted as abasic dataset in which cluster centers are obtained via Mean Shift (MS) clusteringalgorithm, and all centers and the Euclidean distances between them form anundirected graph that reveals the insight of crowd motion status. Thus, theabnormal events could be detected and future states of the crowd could bepredicted by computing distributions of all graph vertexes in feature space, aswell as testing the deviations between observations and estimates of the edgeweights matrix dynamic system. The proposed method need not reinitialize model,retrain new samples and estimate parameters. The first two items are video preprocessing that completes the weather videoclassification and dynamic weather video restoration task, their output providesmore quality video data for subsequent crowd behavior analysis. The latter twoitems involve crowd scene modeling by local and holistic approaches.The research involves video restoration, local crowd state analysis andpredicting future crowd behavior via a graph method. All parts form an integralmodel of dynamic crowd scene that has the capacity of video denoising, imageenhancement, crowd abnormity detection and state forecasting. It provides afeasible monitoring and forecasting way for crowd behaviors analysis based oncomputer vision techniques.更多还原