写一个能够爬取每日微博热搜的代码
时间: 2023-09-24 10:11:51 浏览: 69
python爬取微博热搜榜
好的,以下是一个简单的Python代码,可以爬取每日微博热搜:
```python
import requests
from bs4 import BeautifulSoup
url = "https://s.weibo.com/top/summary?cate=realtimehot"
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36 Edge/16.16299",
"Referer": "https://s.weibo.com/top/summary?cate=realtimehot",
"X-Requested-With": "XMLHttpRequest"
}
response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.content, "html.parser")
hot_list = soup.find_all("td", class_="td-02")
for hot in hot_list:
hot_title = hot.find("a").get_text(strip=True)
hot_rank = hot.find_previous_sibling().get_text(strip=True)
print(f"{hot_rank} - {hot_title}")
```
这段代码使用了requests库向微博热搜页面发送请求,使用BeautifulSoup库解析页面内容,然后提取出热搜榜单并输出。注意,为了模拟浏览器请求,我们需要设置请求头信息。
阅读全文