爬取hhttps://api.bilibili.com/x/player/online/total?aid=1354896847&cid=1552641271&bvid=BV1wz421e7wS&ts=57690298网页的所有评论并保存CSV的代码
时间: 2024-11-16 17:03:29 浏览: 10
爬取B站视频评论并保存到CSV文件通常需要使用网络请求库,如Python的requests和BeautifulSoup解析HTML内容,然后利用pandas处理数据并导出。这里提供一个简化的示例,假设视频URL、API路径和其他参数不变:
```python
import requests
from bs4 import BeautifulSoup
import pandas as pd
# B站视频相关信息
video_id = 'BV1wz421e7wS'
aid = 1354896847
cid = 1552641271
url = f'https://api.bilibili.com/x/player/online/total?aid={aid}&cid={cid}&bvid={video_id}&ts=57690298'
# 发起GET请求获取页面内容
response = requests.get(url)
content = response.text
# 使用BeautifulSoup解析HTML
soup = BeautifulSoup(content, 'lxml')
comments_container = soup.find('div', class_='comment-item-list') # 假设评论在某个class为'comment-item-list'的容器里
# 解析并收集评论数据
comments = []
if comments_container:
for comment in comments_container.find_all('li'): # 遍历每个评论项
username = comment.find('span', class_='author').text.strip()
content = comment.find('span', class_='content').text.strip()
timestamp = comment.find('time')['datetime']
comments.append({'用户名': username, '评论内容': content, '时间戳': timestamp})
# 将评论数据存储到pandas DataFrame
df_comments = pd.DataFrame(comments)
# 导出到CSV文件
filename = 'bilibili_comments.csv'
df_comments.to_csv(filename, index=False)
阅读全文