python爬取热门微博
时间: 2023-11-17 18:05:26 浏览: 61
以下是使用Python爬取热门微博的一些步骤和代码示例:
1. 导入必要的库
```python
import requests
from lxml import etree
```
2. 构造请求头和URL
```python
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'}
url = 'https://weibo.cn/search/mblog?hideSearchFrame=&keyword=%E7%83%AD%E9%97%A8%E5%BE%AE%E5%8D%9A&advancedfilter=1&hasori=1&starttime=20220101&endtime=20220131&sort=time&page=1'
```
3. 发送请求并解析HTML
```python
response = requests.get(url, headers=headers)
html = response.content
selector = etree.HTML(html)
```
4. 提取热门微博信息
```python
# 提取微博内容
content = selector.xpath('//span[@class="ctt"]')
for each in content:
text = each.xpath('string(.)')
print(text)
# 提取微博发布时间
times = selector.xpath('//span[@class="ct"]')
for each in times:
text = each.xpath('string(.)')
print(text)
# 提取微博转发、评论、点赞数
nums = selector.xpath('//div[@class="card-act"]/ul/li')
for each in nums:
text = each.xpath('string(.)')
print(text)
```