大众点评上海美食top榜爬虫
时间: 2023-12-19 20:27:33 浏览: 160
Python版大众点评爬虫(下载即用).zip
5星 · 资源好评率100%
大众点评上海美食top榜爬虫可以使用Python编程语言和爬虫框架(如Scrapy)来实现。以下是一个示例代码,可以帮助你开始编写爬虫:
```
import requests
from bs4 import BeautifulSoup
def get_top_restaurants():
url = 'https://www.dianping.com/shanghai/ch10/g110'
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'}
response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.text, 'html.parser')
restaurants = []
for item in soup.select('.content-wrap .content'):
name = item.select('.tit h4 a')[0].text.strip()
rating = item.select('.ranking-info .star_score')[0].text.strip()
reviews = item.select('.review-num b')[0].text.strip()
address = item.select('.tag-addr .addr')[0].text.strip()
restaurants.append({'name': name, 'rating': rating, 'reviews': reviews, 'address': address})
return restaurants
top_restaurants = get_top_restaurants()
for restaurant in top_restaurants:
print(restaurant)
```
这个示例代码通过向大众点评的上海美食top榜的URL发送请求,并使用BeautifulSoup解析HTML响应来获取餐厅的名称、评分、评论数和地址。你可以根据自己的需求对代码进行修改和扩展,例如提取更多的餐厅信息或实现翻页功能。
阅读全文