【作者】 陈文；

【导师】 黄圣国；

【作者基本信息】 南京航空航天大学 ， 交通信息工程及控制， 2013， 博士

【摘要】 随着社会经济的发展，科学技术水平的提高，迅速形成了各类生活社区、交通运输网、车站和码头等组成的视频监控系统网络。为确保系统安全和运行效益，满足人们的宜居要求，国内外都在积极地开展面向复杂运动目标的检测、跟踪和行为判断等技术的研究，以期形成高度智能化的视频监控系统网络。而立体视觉的监控技术又是其中的核心和热点技术。现有的立体视觉监控技术的研究主要是利用同源传感器，即用可见光摄像机实现立体视觉。虽然可以减少光线变化和阴影的影响，但算法实现相对复杂；而且在低能见度的复杂环境下，仅依靠可见光视觉系统实现运动目标的检测和跟踪有一定的难度。此外，还有学者和研究机构在试探将多元视觉传感器运用于视觉监控中，即可见光和-热红外视频运动目标融合检测。然而，却没有充分利用双目立体视觉能够获得三维信息的优势，未能够获取空间目标三维信息。结合红外热像仪和可见光摄像机组成立体监控系统，充分利用可见光系统提供的灰度信息，和红外热像仪提供的温度信息，提取目标的运动信息以及三维空间信息，对复杂环境下全天候的目标连续跟踪实现信息互补是非常有意义的。基于此，本文开展了利用红外热像仪和可见光摄像机组成的异源双目立体系统，实现运动目标检测和跟踪的研究，主要内容及取得的研究成果总结如下：（1）明确提出现有基于四象限分割思想的二维图像分割理论的不足，并经过系统性测试实验的验证。在此基础上，提出了基于阈值线的二维阈值图像分割技术方法。并以二维熵阈值分割方法为例，给出了一种分步骤二维熵阈值线的确定方法，即在被划分为边缘和噪声的象限中寻找第二个阈值点，进一步明确边缘和噪声的属性。本文方法不仅充实了二维图像分割理论，解决了传统算法中因忽略大量有效信息而导致的分割失败的问题，也具有较好的操作性。系统性的实验证明了该方法可以大大改善图像的分割结果。（2）对于二维直方图像素分布严重不均的图像，即使不忽略边缘和噪声象限中的像素信息，传统的四象限法仍然无法得到理想的分割结果。为此，本文提出了一种基于二维直方图质心的图像阈值分割方法。不仅利用了各个质点的质量，也充分考虑了各质点的位置信息，有效改善了分割结果。尤其是对于二维直方图中目标和背景的灰度分布差异极小的图像,本文方法可以得到更加理想的分割结果。（3）结合帧间差和背景差实现了运动目标检测的方法可以充分利用两种方法的优势，弥补了单一使用帧间差法或背景差法的不足。在此目标提取结果的基础上，利用角点信息完成了运动目标前后帧的匹配，提高了单目视觉下图像运动跟踪的精度。（4）考虑到异源图像视差匹配的特殊性，结合异源图像信息互补的特点，提出了一种基于目标区域的匹配方法。该方法融合了归一化转动惯量和归一化互相关特征，不仅避免了红外热图像和可见光图像成像机理不同带来的灰度差异缺点，也可以充分考虑到像素间的空间位置关系。此外，该方法利用红外热图像的视差匹配结果在可见光图像中提取的目标区域不易受拖影以及背景光线的影响的特性，大大改善了可见光图像中的目标提取结果。即使无法同时从可见光图像和红外热图像中提取出理想的前景目标，本文方法也能实现匹配。实际测试实验结果证明了方法的有效性。（5）设计了一种以云台旋转中心为基准点的空间点三维重建模型。在分析红外热像仪特性的基础上，根据立体监控实用性要求和双目立体视觉原理，以不考虑畸变的针孔模型为依据，实现了空间点的三维重建。同时，针对双目系统参数标定复杂的问题，给出了适合本文模型的简化参数标定方法，最多需要四个参考点，利用两个摄像机同时拍摄一幅图片即可完成所需参数的标定。实验证明，利用已标定的双目系统，在不同的场景，即使两台云台的相对位置发生旋转变化或是平移依然可以实现三维测距，非常适合野外环境的快速标定。（6）在设计的红外热像仪和可见光摄像机组成的双目立体视觉实验系统的基础上，获取运动物体对应特征点的三维信息，以此作为运动物体在序列图像不同帧的特征值，实现空间运动物体的跟踪定位，最终获得了目标运动信息以及三维空间信息。更多还原

【Abstract】 With the development of social economy and the improvement of science, the video monitoringsystem implemented in living communities, stations and terminals is rapidly established. To improvethe living quality, this system must be operated stably and effectively. Therefore, researches on thedetection, tracking and behavior judgment of complex moving objects are actively carried out all overthe world. Based on these researches, excellent intelligent video monitoring system can be built. Andstereo vision has attracted great concern as one of the key points in this system.Most of the existing stereo vision systems are consisted of two visible cameras, which arehomologous sensors. Although it could reduce the impact of illuminate change and shadow, thealgorithm is relatively complex. In addition, it is a huge challenge to detect and track the movingobject in poor visibility. On the other hand, a lot of researchers have made an attempt on movingobject detection with visual-thermal fusion. However, they did not make full use of the advantages ofstereo vision to obtain three-dimensional (3-D) information of the moving object. A stereo monitoringsystem consisted of a thermal infrared camera and a visible camera is able to make full use ofgrayscale and temperature to get the motion information and3-D information of a moving object.Therefore, it contributes to continuous tracking in all weather conditions. In this paper, we focus onthe research of moving object detection and tracking with visual-thermal fusion. The primary workand remarks are as follows:1) The shortcoming of the existing two-dimensional (2-D) thresholding methods based on fourquadrants is analyzed in this paper. And it is verified through a series of testing. Therefore, anovel2-D threshold line segmentation method is proposed. A2-phase strategy is specified todetermine the2-D thresholding line based on entropy. The second threshold point is determinedin the quadrants including the edge pixels and the noise pixels. Thus the attribution of the edgepixels and the noise pixels is refined. The proposed method improves the segmentation results bymaking full use of the ignored pixels. It not only improves the2-D thresholding method, but itcan be easily carried out as well. The experiments on typical images demonstrated that theproposed method achieves very competitive segmentation results in comparison with the existingrepresentative methods.2) When the2-D histogram of an image is extremely uneven in distribution, it can not get successfulsegmentation through2-D thresholding method based on four quadrants even if the ignored pixelsare fully utilized. To solve this problem, a threshoding method using the centre of mass of the2-Dhistogram is proposed to binarize this kind of images. It not only makes use of the mass of each pixel but also fully considers the location information of each pixel. Therefore, it couldeffectively improve the segmentation result, especially for the images in which2-D histogram ispoor in distinguishing the object from the background.3) A strategy takes advantage of frame difference method and background subtraction method isadopted to detect the moving object. Because there are shortcomings when any of them is usedalone in motion detection. And corner matching is implemented to match the moving object indifferent frames. Thus it makes the moving object tracking more accurate.4) Considering the complement effect came from different source images of the same scene, aregistration method of regions of interest is provided to solve the disparity correspondence inmultimodal stereo systems. The method takes advantage of NCC and NMI. It avoids the graydifference caused by the imaging mechanism of thermal image and visible image, and meanwhileit also makes use of the position information of the pixels. The fusion of the object in differentimages reduces the impact of light and shadow, and greatly improves the object detection resultsin visible images. Even if ideal objects could not be detected from the visible images or thethermal images at the same time, this method is available. And the experiments proved itsvalidity.5) According to the feature of multimodal stereo vision, a3-D reconstruction method of point isproposed on the basis of pinhole model. The rotation centers of the pan-tilt devices are used as thereference points in the method. Since the parameters identification is complicated in stereo vision,the procedure is shown to identify the parameters used in this method. A pair of images shot bytwo cameras at the same time and four reference points are required for calibration. Even if therelative position of two cameras is changed, it can still get3-D information of the object with thecalibrated multimodal stereo system. It is appropriate to quickly get the parameters of the stereovision system in field.6) Based on the multimodal stereo vision system consisted of a thermal camera and a visible camera,the3-D information of the feature point of the moving object can be obtained. It can provideposition information in the stereo tracking system. Therefore, the motion information and3-Dinformation of the moving object in all weather conditions can be gotten.更多还原