帮我写一个能爬取豆瓣电影评分的脚本
时间: 2023-12-18 09:04:04 浏览: 32
好的,以下是简单的Python脚本,可以用于爬取豆瓣电影的评分:
```python
import requests
from bs4 import BeautifulSoup
url = 'https://movie.douban.com/top250'
headers = {
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36'
}
def get_movies(url):
movie_list = []
r = requests.get(url, headers=headers)
soup = BeautifulSoup(r.text, 'html.parser')
movies = soup.find_all('div', {'class': 'info'})
for movie in movies:
title = movie.find('span', {'class': 'title'}).text
rating = movie.find('span', {'class': 'rating_num'}).text
movie_list.append((title, rating))
next_page = soup.find('span', {'class': 'next'}).find('a')
if next_page:
return movie_list, url + next_page['href']
return movie_list, None
while url:
movies, url = get_movies(url)
for movie in movies:
print(movie)
```
这个脚本使用了Python的requests库和BeautifulSoup库,它会从豆瓣电影top250页面中获取电影名称和评分,并将它们打印出来。您可以将打印的结果保存到一个文件中,或将其存储到数据库中以供进一步分析。请注意,豆瓣电影网站有反爬虫机制,使用该脚本可能会触发反爬虫机制,因此请谨慎使用。