写出python requests库爬取豆瓣电影和电视剧的信息并以CSV文件形式保存到本地的代码
时间: 2024-01-08 20:04:53 浏览: 153
下面是爬取豆瓣电影和电视剧信息并以CSV文件形式保存到本地的Python代码:
```python
import requests
import csv
# 电影url
movie_url = 'https://movie.douban.com/j/search_subjects?type=movie&tag=热门&sort=recommend&page_limit=20&page_start=0'
# 电视剧url
tv_url = 'https://movie.douban.com/j/search_subjects?type=tv&tag=热门&sort=recommend&page_limit=20&page_start=0'
# 请求头部信息
header = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'
}
# 爬取电影信息
movie_response = requests.get(movie_url, headers=header)
movie_json = movie_response.json()
movie_list = movie_json['subjects']
# 爬取电视剧信息
tv_response = requests.get(tv_url, headers=header)
tv_json = tv_response.json()
tv_list = tv_json['subjects']
# 将电影和电视剧信息保存到CSV文件中
with open('douban.csv', mode='w', encoding='utf-8', newline='') as f:
writer = csv.writer(f)
writer.writerow(['名称', '评分', '导演', '演员'])
for movie in movie_list:
writer.writerow([movie['title'], movie['rate'], movie['directors'][0], ','.join(movie['casts'])])
for tv in tv_list:
writer.writerow([tv['title'], tv['rate'], tv['directors'][0], ','.join(tv['casts'])])
```
这份代码通过requests库对豆瓣电影和电视剧的API接口进行请求,获取到电影和电视剧的信息,并使用csv库将信息保存到本地douban.csv文件中。
阅读全文