python爬虫天猫评论
时间: 2024-02-03 20:03:00 浏览: 109
以下是一个示例的Python爬虫代码,用于爬取天猫商品的评论数据:
```python
import requests
from bs4 import BeautifulSoup
def get_tmall_comments(url):
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'
}
response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.text, 'html.parser')
comments = []
comment_items = soup.find_all('div', class_='comment-item')
for item in comment_items:
username = item.find('div', class_='user-name').text.strip()
time = item.find('div', class_='date').text.strip()
product_info = item.find('p', class_='product-info').text.strip()
content = item.find('div', class_='comment-content').text.strip()
comment = {
'username': username,
'time': time,
'product_info': product_info,
'content': content
}
comments.append(comment)
return comments
# 示例使用
url = 'https://rate.tmall.com/list_detail_rate.htm?itemId=1234567890&page=1'
comments = get_tmall_comments(url)
for comment in comments:
print(comment)
```
请注意,这只是一个示例代码,具体的爬取逻辑和网页结构可能会有所不同。你需要根据实际情况进行调整和修改。
阅读全文