帮我用python爬虫爬取携程网广州塔的评论并输入到csv文件
时间: 2024-06-10 21:03:54 浏览: 23
好的,我可以为您编写Python爬虫程序来爬取携程网广州塔的评论,并将其输入到CSV文件中。以下是代码示例:
```python
import requests
import csv
from bs4 import BeautifulSoup
url = "https://piao.ctrip.com/thingstodo-booking-bookingwebapi/api/comment"
# 设置请求参数
data = {
"poiID": "13122", # 广州塔的poi ID
"districtId": "100", # 广州市的区域ID
"pagenum": "1", # 第一页评论
"pageSize": "10", # 每页10条评论
"contentType": "json",
"sortType": "3",
"isPageScrolling": "false",
"head": {
"locale": "cn",
"cid": "09031191312047955705",
"ctok": "",
"cver": "1.0",
"lang": "01",
"sid": "8888",
"syscode": "09",
"auth": "",
"extension": []
}
}
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3"
}
response = requests.post(url, json=data, headers=headers)
content = response.json()
# 解析评论内容
comments = []
if content["data"]["comments"]:
for comment in content["data"]["comments"]:
comment_data = {
"user": comment["authorName"], # 用户名
"content": comment["content"], # 评论内容
"score": comment["score"], # 评分
"time": comment["date"], # 评论时间
}
comments.append(comment_data)
# 将评论数据写入CSV文件
with open("comments.csv", "w", newline="", encoding="utf-8") as f:
writer = csv.DictWriter(f, fieldnames=["user", "content", "score", "time"])
writer.writeheader()
writer.writerows(comments)
print("爬取完成")
```
请将以上代码复制到您的Python文件中,然后执行即可爬取并保存评论数据。请注意,代码中的`poiID`和`districtId`分别为广州塔和广州市的ID,您需要根据实际情况进行修改。另外,请确保您已经安装了`requests`、`beautifulsoup4`和`pandas`等必要的Python库。