【摘要】 目前，主题爬虫是从海量异构网络中获取有效领域知识的重要方法。目前大多数主题爬虫技术难以获得高质量爬行结果。主要难点包括主题基准模型的建立、超链接主题相关性的评估和爬行策略的设计。本文采用领域本体为特定主题构建主题基准模型，并提出一种新的基于局部本体和全局本体的多重筛选策略（MFSLG）。为提高待访问超链接主题相关性计算精度，提出一种基于网页文本和链接结构的综合优先度评估方法（CPEM），同时，采用模拟退火（SA）算法避免主题爬虫陷入局部最优搜索。本文首次设计融合SA算法、MFSLG策略和CPEM策略实现主题爬虫，提出两种新的基于本体和SA主题爬虫策略（FCOSA），包括基于全局本体的FCOSA策略（FCOSA＿G）和基于局部本体和全局本体的FCOSA策略（FCOSA＿LG），以从网络中获取与暴雨灾害主题相关的网页。实验结果表明，针对不同性能指标，所提爬虫策略优于其他主题爬虫策略。更多还原

【Abstract】 At present, focused crawler is a crucial method for obtaining effective domain knowledge from massive heterogeneous networks. For most current focused crawling technologies, there are some difficulties in obtaining high-quality crawling results.The main difficulties are the establishment of topic benchmark models, the assessment of topic relevance of hyperlinks, and the design of crawling strategies. In this paper, we use domain ontology to build a topic benchmark model for a specific topic, and propose a novel multiple-filtering strategy based on local ontology and global ontology(MFSLG). A comprehensive priority evaluation method(CPEM) based on the web text and link structure is introduced to improve the computation precision of topic relevance for unvisited hyperlinks, and a simulated annealing(SA) method is used to avoid the focused crawler falling into local optima of the search. By incorporating SA into the focused crawler with MFSLG and CPEM for the first time, two novel focused crawler strategies based on ontology and SA(FCOSA), including FCOSA with only global ontology(FCOSA＿G) and FCOSA with both local ontology and global ontology(FCOSA＿LG), are proposed to obtain topic-relevant webpages about rainstorm disasters from the network. Experimental results show that the proposed crawlers outperform the other focused crawling strategies on different performance metric indices.更多还原