【作者】 黄俊；

【导师】 方向忠；

【作者基本信息】 上海交通大学 ， 通信与信息系统， 2011， 博士

【摘要】 随着网络技术的飞速发展,Web上拥有的图像资源已经越来越丰富。这个巨大的图像数据库中蕴藏着大量对用户有价值的信息。图像挖掘技术致力于对海量图像数据自动分析处理,以此获取有意义的模式和知识。基于Web的图像内容挖掘是近年来多媒体数据挖掘领域的热点研究方向之一。本文围绕Web图像的视觉模式发现和提取展开,重点研究了基于显著性和基于语义的视觉模式挖掘方法,并将这两种方法用于图像检索中的重排序。首先,本文通过研究视觉选择性注意机制的计算模型,提出了一种基于统计学习的多尺度视觉显著性模型建立方法。在此基础上,将该模型用于Web图像挖掘,提出了一种基于显著性的视觉模式提取方法。文中给出了显著图和非显著图的定义,把通常考虑显著性问题的视角从区域扩展到整幅图像。同时,进行了多尺度相关性的讨论,使用多尺度的表示方法能对图像进行更为精确地描述。根据构建的显著图和非显著图像数据库,分析了这两类图像所呈现的不同视觉特性,以此选取颜色、边缘、纹理和图像要旨(Gist)四个不同的底层特征来训练视觉显著性模型。实验部分,对模型进行了客观性的定量分析和主观性的眼动实验,验证了该模型理论假设的正确性及在实际应用中对显著图检测的有效性。其次,本文研究基于语义的视觉模式挖掘问题,提出了一种无监督的方法对来自于Web的图像自动进行聚类分析,从而提取出特定语义概念的主要视觉模式。该算法主要针对物体类概念,对于给定的查询关键词,充分利用了丰富的Web图像资源,无需人工干预自动地挖掘所包含的主要视觉模式。文中给出了具有视觉一致性图像的定义:当使用Web图像搜索引擎时,在返回结果的前几页中出现频率较高同时视觉上相似的图像往往与用户的查询主题相关。利用这些图像的一致性信息,挖掘特定语义概念的主要视觉模式。此外,Google图像搜索引擎提供了剪贴画检索功能。剪贴画不仅具有干净的背景,而且最大程度上反映了物体的基本形状。利用剪贴画的特有属性,可以方便有效地提取出有价值的图像底层特征信息。基于以上两点,本文提出一种基于语义的视觉模式挖掘的新方法。实验结果表明,该方法充分利用了图像集合中的一致性信息和剪贴画的特有属性,能有效挖掘出特定语义概念的主要视觉模式。利用该方法挖掘出的视觉模式,不仅能用于提升图像聚类、浏览和检索的性能,同时也能应用于物体分类、检测、识别等领域。最后,本文研究搜索引擎返回的原始图像的重排序问题,提出了一种基于视觉显著性和一致性的图像重排序算法。在Web图像搜索应用中,视觉一致性的图像在重排序时应给予更高的相关性分值。此外,从视觉角度出发,视觉显著的图像更能吸引人的注意。同时也观察到在搜索引擎返回结果的前几页中,视觉显著的图像更有可能与用户查询相关。从以上两点出发,本文提出一种新的基于随机游走的融合方法,将视觉显著性和一致性结合起来用于图像的重排序。实验结果表明,该算法能有效地提升搜索引擎的检索性能,将视觉上显著的且与查询主题密切相关的图像优先返回给用户。更多还原

【Abstract】 With the rapid development of network technology, web image resourceshave become a huge image database which provides users with abundant valu-able information. In order to obtain meaningful patterns and knowledge, theimage mining technology has been applied in the analyzing and processing oflarge scale image data. In the field of multimedia data mining, web-based im-age content mining has been one of the hottest research topics in resent years.The present study dues with web image visual pattern discovery and extraction.It mainly investigates the methods of saliency-based and concept-based visualpattern mining, and uses the methods to re-rank images in image retrieval.Firstly, the dissertation probes into the computational model of visual se-lective attention mechanism, and provide a method to build multiscale visualsaliency model based on statistical learning. By using such a model, the miningmethod of saliency-based visual pattern is proposed in the study. The authordefines salient image and cluttered image based on the saliency of a whole imagerather then a region. And, multiscale relevance is analyzed in order to give moreaccurate descriptions to images. Based on the databases of salient images andcluttered images, the author analyzes the perceptual di?erences of the two classesof images, and chooses color, edge, texture, Gist as the four features to train thevisual saliency model. In the experiments, quantitative analysis is conducted tothe model. Eye movement test is done to test the hypothesis about the modeltheory and the applicability of the model in saliency image detection.Secondly, the dissertation discusses the problem of concept-based visual pat-tern mining. A new method of automatic clustering analysis for web images isproposed to extract the main visual pattern according to specific concept. Devel- oped for the concept of object category and by making full use of rich web imageresources, the algorithm is able to mine the main visual pattern of images withouthuman intervention for a given query. The author defines the visual consistentimage: When using a web image search engine,the images closely related to thequery are often visually similar and occur most frequently in the first few webpages. Based on such a fact, the proposed method mines the main visual patternby using the visual consistency. Besides, Google image search engine providesclip art retrieval function. The clip arts often have clear backgrounds, and re?ectthe basic shapes of objects to the greatest extent. Such special attributes arehelpful for the extraction of valuable low-level features of objects. Grounded onthe two points, the dissertation proposes a new visual pattern mining methodbased on semantic concept. The experiment shows that the proposed methodcan be able to find the main visual pattern of specific concept e?ciently by uti-lizing the consistency of image and the special attributes of clip arts. This visualpattern can not only be used to improve the performance of image clustering,browsing and retrieval, but also be applied in some other fields such as objectclassification, detection and recognition.Lastly, we propose a new algorithm for image re-ranking in web image searchapplications. The proposed method focuses on investigating the following twomechanisms: visual consistency and visual saliency. In web image search cases,when re-ranking images, these visually consistent images would be given higherranks. Besides, from visual aspect, it is obvious that salient images would beeasier to catch users’eyes and it is observed that these visually salient images inthe front pages are often relevant to the user’s query. By integrating the above twomechanisms, our method can e?ciently re-rank the images from search enginesand obtain a more satisfactory search result. Experimental results on a real-world web image dataset demonstrate that our approach can e?ectively improvethe performance of image retrieval. These images which are salient and closelyrelated to the query would be given priority to return for users.更多还原