python爬虫爬取豆瓣读书top250书评功能实现
时间: 2023-11-06 19:09:25 浏览: 122
python爬虫实现豆瓣按书类获取优质书单
5星 · 资源好评率100%
Python爬虫可以通过requests和BeautifulSoup库来实现对豆瓣读书Top250的爬取。具体步骤如下:
1. 导入requests和BeautifulSoup库
2. 使用requests库发送请求,获取网页源代码
3. 使用BeautifulSoup库解析网页源代码,获取需要的数据
4. 将获取到的数据存储到本地文件或数据库中
下面是一个简单的示例代码,可以爬取豆瓣读书Top250的书名、作者和评分:
```python
import requests
from bs4 import BeautifulSoup
url = 'https://book.douban.com/top250'
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'}
response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.text, 'html.parser')
books = soup.find_all('tr', class_='item')
for book in books:
title = book.find('div', class_='pl2').a['title']
author = book.find('p', class_='pl').get_text().strip().split('/')[0]
rating = book.find('span', class_='rating_nums').get_text()
print(title, author, rating)
```
相关问题:
阅读全文