python爬取电影数据代码
时间: 2024-11-05 11:31:36 浏览: 18
Python爬取电影数据通常会涉及网络请求库如requests、BeautifulSoup或更现代的Scrapy框架,以及JSON处理库如json或pandas。下面是一个简单的例子,展示如何使用requests和BeautifulSoup从IMDb获取电影信息:
```python
import requests
from bs4 import BeautifulSoup
def get_movie_data(url):
# 发送GET请求
response = requests.get(url)
# 检查请求是否成功
if response.status_code == 200:
# 解析HTML内容
soup = BeautifulSoup(response.text, 'html.parser')
# 找到电影标题和链接元素
movie_title = soup.find('h1', class_='title_wrapper').text
movie_link = soup.find('a', {'href': True})['href']
return {
'title': movie_title,
'link': 'https://www.imdb.com' + movie_link,
}
else:
print(f"Failed to fetch data. Status code: {response.status_code}")
return None
# 示例URL
url = "https://www.imdb.com/chart/top"
movie_info = get_movie_data(url)
if movie_info:
print(f"Title: {movie_info['title']}")
print(f"Link: {movie_info['link']}")
阅读全文