【作者】 刘云霞；

【导师】 胡彧；

【作者基本信息】 太原理工大学 ， 计算机软件与理论， 2007， 硕士

【摘要】 随着信息技术的迅速发展，网络信息不断膨胀。如何让网络信息更好地为人类服务，已成为未来几年的一个研究热点。一方面是人们对快速、准确而全面获取信息的渴望，而另一方面却是网络信息的纷繁芜杂，在这两者之间架设一座桥梁的确是一个巨大的挑战。网页自动分类技术正为解决这个问题提供了一种合理有效地组织信息的方法。为了有效地组织和分析网页信息，帮助用户迅速地获取所需要的信息，论文针对不同用户对网络信息的不同需求来提取对应的规则，同时根据知识中规则与例外相互补充的学习理论对存在的例外进行分析，从而对中文网页文本进行精确分类。本文从理论和应用的角度对中文网页文本信息的分类技术进行了深入的研究，提出了将粗糙集与面向自然语言处理的规则与例外学习理论应用到中文网页分类中，并实现了一个基于粗糙集的“规则+例外”中文网页分类系统。论文对中文网页分类的关键技术、粗糙集理论的主要内容、规则归纳以及例外分析进行了系统的研究和详细的介绍，并在这些理论知识的指导下设计了一个解决用户需求的中文网页文本分类器。论文主要做了以下研究工作：网页文本分类首先需要收集WEB文本，对WEB文本进行预处理，保存其中的文本信息。在这部分，文章首先实现了抢先式多线程中文网页收集器，采用深度优先的算法获取特定类型的网页，接着根据HTML Tag文本的特点，实现了基于非递归方式匹配的WEB文本预处理器，它用于提取网页中的文本信息以及定义的网页标记集。其次，本文在研究文本信息表示和网页信息特点的基础上，改进了中文网页文本表示的权重计算方法，设计了面向用户需求的属性约简算法，该算法在文本分类系统中取得了较好的效果。此外，本文结合粗糙集理论中的研究内容分析了规则与例外的形成过程，并提出基于reduct的例外鉴别方法。论文最后设计了中文网页文本分类系统的总体方案，并根据方案实现了基于粗糙集的“规则+例外”中文网页文本分类系统。为了进行实验评估，论文进行了两组实验进行结果比较。实验数据表明本文设计的网页文本分类器提高了网页文本分类的效率，有一定的实际意义。更多还原

【Abstract】 Along with the rapid development of information technology, network information increases explosively. It’s a real researching hotspot to make network information easier and more efficient to be used. The information in Internet is in short of organization and full of a mass of pages. On the other hand, people want to retrieve information quickly and accurately. The technique of automatic web pages classification seemed as a good approach to solve such problems.To effectively organize and analyze massive web information resource and help users to promptly get knowledge and information they need, this thesis extracts diverse rules according to users’ different requirements and analyses the existing exceptions to reach the aim of accurate classification on the basis of the learning theory that rules and exception are complementary. This paper studies the Chinese web text mining techniques deeply in the aspects of theory and application, puts forward applying rough sets and the learning theory of "rule + exception" in natural language processing to Chinese web text mining and realizes a classifier of the Chinese web page text. The key techniques of Chinese web pages classification and the main theory of rough sets, rule induction and exception analyzing have been introduced systematically in this thesis. At last, a Chinese web pages classifier has been designed under the guidance of the theory. The achievements of this thesis are:Unlike the general text classification, we need to collect Chinese web pages, preprocess these web pages and save the weight of the text information. First, a preemptive multi-thread web text collector which is used to collect web pages of special catalog using Depth First Algorithm is realized. Besides, a web text preprocessor which is used to erase the meaningless HTML tag and extract web text by recursive match method is implemented.Furthermore, a weight computing algorithm is improved taking into account of the characters of text information and web pages information. To be important, an attributes reducing algorithm oriented users’ requirements is proposed, which is proved to be highly effective in the text classification system and a Reduct exception analysis method is proposed based on the theory of rough sets by analyzing the reasons that rules and exception appear in the web pages text classification.At last, the designing process of Chinese web pages text classification is listed and the Chinese web pages text classifier based on the theory of rough set and rule plus exception is realized according to the process. To evaluate the performance of the classifier, we did two experiments and compared the results. The results show both the efficiency and the correctness of the web pages text classification system are higher and these researches are worthy to be referenced in the field of text classification.更多还原