python百度爬虫关键词python
时间: 2023-12-26 13:28:36 浏览: 94
以下是一个使用Python编写的爬取百度新闻中包含关键词"python"的爬虫示例:
```python
import requests
import re
def baidu_news_spider(keyword):
url = 'https://www.baidu.com/s?wd=' + keyword + '&tn=news&ie=utf-8'
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'}
response = requests.get(url, headers=headers)
content = response.text
pattern = re.compile('<h3 class="c-title">.*?<a.*?href="(.*?)".*?>(.*?)</a>', re.S)
results = re.findall(pattern, content)
for result in results:
news_title = result[1]
news_url = result[0]
print(news_title, news_url)
# 调用爬虫函数,传入关键词"python"
baidu_news_spider('python')
```
这个爬虫示例使用requests库发送HTTP请求,通过正则表达式解析爬取的数据,然后打印出新闻标题和链接。
阅读全文