python爬取携程景点评论
时间: 2023-08-14 09:07:01 浏览: 172
你可以使用Python来爬取携程的景点评论。这里给出一个简单的示例代码,使用requests和BeautifulSoup库:
```python
import requests
from bs4 import BeautifulSoup
def crawl_ctrip_comments(url):
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36'
}
response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.text, 'html.parser')
comments = []
comment_elements = soup.select('.comment_detail')
for element in comment_elements:
comment = element.get_text().strip()
comments.append(comment)
return comments
# 指定要爬取的携程景点评论页面链接
url = 'https://you.ctrip.com/sight/beijing1/1.html'
comments = crawl_ctrip_comments(url)
for comment in comments:
print(comment)
```
这个示例代码会爬取携程指定页面的评论内容,并打印出来。你可以根据自己的需要进行进一步处理或保存。记得在代码中设置合适的请求头(User-Agent),以避免被网站拦截或限制访问。
阅读全文