写一个python代码,爬取豆瓣电影排行榜
时间: 2024-06-10 13:03:46 浏览: 182
好的,以下是Python爬取豆瓣电影排行榜的代码:
```python
import requests
from bs4 import BeautifulSoup
url = 'https://movie.douban.com/chart'
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'}
response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.text, 'html.parser')
list_top = soup.find('div', {'class': 'indent'})
list_movies = list_top.find_all('div', {'class': 'item'})
for movie in list_movies:
name = movie.find('div', {'class': 'pic'}).find('img')['alt']
rating = movie.find('span', {'class': 'rating_num'}).get_text()
print(name + ' ' + rating)
```
上述代码首先发送一个GET请求到豆瓣电影排行榜的页面,然后用BeautifulSoup库解析HTML文本。接着,通过查找特定的标签和类名,从页面中获取电影名称和评分,最后将它们打印出来。注意需要设置User-Agent,以防止请求被豆瓣屏蔽。
阅读全文