【作者】 范欣；

【导师】 李明镜； 俞能海；

【作者基本信息】 中国科学技术大学 ， 信号与信息处理， 2007， 博士

【摘要】 在信息时代，全世界有大量的数字化内容以多媒体(文本、图形、图像、视频、音频等)的形式生成、存储、传播和转换，并呈爆炸性增长。特别是互联网的迅速发展，更加推动了信息服务的多样化发展和多媒体数据的急剧膨胀。当前绝大多数网络信息服务和内容都是为传统的台式机设计的，采用固定单一的发布方式，向所有客户提供完全相同的内容，并没有考虑客户端主观和客观环境的差异。因此，多数情况下，它们都不适合那些屏幕、存储、计算和网络连接都有限的移动设备，如个人数字助理、手机等。目前，这些新兴的移动设备已经被被广泛使用。据统计，截止2006年底，中国的手机用户超过了三亿。借助互联网的威力，这些设备给用户带来了前所未有和无所不在的信息体验。移动搜索作为互联网搜索技术与移动通信技术相结合的产物，打破了利用台式机机进行搜索获取信息的终端局限性，使得移动设备用户可以随时随地地获取所需的信息。但在移动设备上，由于种种原因，信息并不象在台式计算机上那样易于访问。现有的信息获取方法并不能很好地适用于移动设备。移动搜索绝对不是将互联网搜索照搬到手机上来。比起互联网搜索，移动搜索对于搜索的准确性、搜索结果的提示信息、满足个性化搜索需求等方面，有更高的要求。移动搜索引擎需要考虑分析用户的使用习惯，具有一定的智能性和便利的用户交互方式。另外，通过移动设备来方便地获取信息所面临的另一大难题是如何在低带宽的网络连接和小面积的显示屏幕的限制之下浏览信息。在这样的背景下，如何给移动设备用户提供有效便捷的信息检索方式和适合不同设备的自适应多媒体结构化内容就逐渐成为人们研究的重点。本论文重点讨论利用跨媒体检索的技术，满足移动设备用户的多媒体信息检索需求，并改善移动搜索的体验。另外，通过智能化的多媒体内容分析和建立多媒体内容表示模型，实现自适应的多媒体信息显示，给移动设备用户提供访问多媒体信息的易用的交互方式。论文主要研究内容与创新成果如下：1．介绍了移动搜索产生的背景、技术动力和市场前景；总结了移动搜索和网络多媒体内容自适应领域相关的工作；探讨了当前移动搜索和网络多媒体自适应研究包含的内容以及所面临的问题。2．从移动互联网的发展和现状的介绍入手，根据目前正在进行的相关研究和相关应用，较全面地总结了目前移动搜索技术的发展现状。对移动搜索的特征、技术等作了详细的梳理和归纳，并对移动搜索的未来可能的研究方向进行了探讨。3．阐述了跨媒体检索的概念和相关技术；讨论了跨媒体检索的概念及其和单一媒体检索的比较；分析了跨媒体检索中常采用的技术和发展现状。4．通过一个具体的实例Photo2Search移动搜索系统来说明如何把跨媒体检索的技术运用到移动搜索中。提出了一套在大数据量下进行可视化搜索的算法和针对有内置摄像头的移动设备的移动搜索方案，支持用户进行多模态的查询，实现了移动设备上直观的便捷的可视化搜索。在海量真实网络数据集上的实验结果表明，所提出的解决方案和算法是高效实用的。5．充分利用电视节目中含有多模态信息的特点，提出并实现了一种网络化的电视节目浏览方案。使用多模态的信息对电视节目进行结构化，并利用跨媒体检索的技术获取和节目相关的网络内容和生成超链接。提供一种集成的可自适应内容缩放的网络化浏览界面，可以满足不同设备访问的需要。6．针对多样化的移动设备获取所需网络多媒体信息的问题，提出了一种有扩展性自适应的图像的视觉注意力内容模型。在这个模型的基础上提出了几种图像自适应显示的方案，并结合基于H.264的空间转码技术对视频的自适应显示加以研究。更多还原

【Abstract】 In the current information age, a large amount of digital contents are generated, stored, spreading and transformed in the form of multimedia (text, graphic/image, video/audio etc.) and the increase is explosive. Especially the galloping development of Internet promotes the endless emergence of application service and the rapid expansion of multimedia data. The present multimedia information service and content are mostly specifically designed for desktop PCs. In many cases, they are not suitable for devices like Personal Digital Assistants and Smartphones with relatively limited display capability, storage, processing power, and network access.Nowadays, the rising mobile devices are more and more popular in our lives. With the capability to access to Internet, these devices provide users a brand new experience for pervasive information. As a combination of Internet search and mobile communication technologies, mobile search overcomes the limitation that the information can only be retrieved and browsed by a desktop PC. With mobile search technology, users can freely acquire the needed information on-the-go.However, the information access on mobile devices is still not as easy as desktop PCs due to some difficulties. The current information retrieval methods are not equally applied on mobile devices. Mobile search is not a simple duplicate of Internet search on mobiles. Further considerations are required such as accuracy of retrieved results, assistant information for displayed entries and personalized information needs. Usability is also a necessary consideration for a mobile search engine, which should be with an intelligent and friendly user interface.In addition, the other difficulty for the information acquisition on mobiles is how to browse the retrieved information under the low band-width connection and small display. Currently, most of Internet information services publish the media data in a unique form -totally same content for the different users. The objective and context differences among variant terminals are still not taken into account. Therefore, more and more research attention has been drawn on the topics how to make mobile users obtain the Web information in an effective and easy way and how to make the multimedia content structuralized and more adaptive for different devices.The dissertation focuses on facilitating the multimedia information search on mobiles and improving the user experience of mobile search. In addition, we research on the intelligent multimedia content analysis and modeling to achieve adaptive multimedia display and to provide mobile users a friendly interface to access multimedia data. The main content and innovations in this dissertation are as follows: 1. The background, motivations and promising prospect of mobile search are introduced. Researches on mobile search and multimedia content adaptation are also reviewed. We also discuss the main topics and problems on mobile search and multimedia content adaptation.2. We introduce the development and status of mobile Web. The literature of mobile search technology is reviewed according to related ongoing researches. The characteristics, techniques of mobile search are specified and the possible breakthrough is discussed.3. We elaborate on the concept and related techniques of cross-media information retrieval, which is compared with the information retrieval for the single kind of media. The common technical solutions cross-media information retrieval in are also discussed.4. We illustrate the mobile search by the cross-media information retrieval techniques in an example - Photo2search mobile search system. A set of visual search algorithms in large-scale data are proposed in a mobile search scheme for the devices with the embedded camera. The system supports multimodal queries and provides an easy way to carry out visual search on mobiles. The experimental results in large-scale Web data show the effectiveness and efficiency.5. Considering the advantage of multimodal information in the TV program, we propose and implement a Web-like browsing scheme for TV programs. Multimodal information is used to structurize the TV program and to produce the hyperlinks to Web content. An integrated Web-like display interface is provided in an adaptive and zoomable manner, which can be applied to different display sizes.6. A scalable and adaptive image attention model is proposed for heterogeneous devices and universal access. Based on this model, several solutions of adaptive image display are designed. Moreover, we combine the image attention model with video transcoding based on H.264 encoding standard to carry out an effective video adaptation scheme for mobile devices.更多还原