【摘要】 Python平台开发了网络新闻爬虫,通过实验对比常用的网页数据获取方法,本文提出一种适合新闻网页的解析方式,克服了获取动态网页时源码不完整、单一方法解析网页效率低下等弊端,满足了新闻网页抓取、解析、结构化和入库存储等需求。并将新浪新闻中心作为网络新闻数据采集的目标,分析发现新浪新闻近几年的发展情况。此外,利用词频统计等手段对十九大会议新闻进行分析,直观地反映了十九大的核心人物、关键党派、热点话题变化等重要信息。更多还原

【Abstract】 This paper develops a web news crawler on Python platform,and presents a web page analysis method suitable for news web pages by comparing common web data acquisition methods. The crawler overcomes the disadvantages such as incomplete source code and low efficiency of web page,meets the needs of news web page grabbing,parsing,structuring and warehousing.This paper takes Sina news center as the target of network news data collection,and analyzes the development of Sina news in recent years.In addition,using the word frequency statistics and other means to analyze the news of the 19 h congress,it intuitively reflects the important information such as the core figures,key parties and the change of hot topics.更多还原