scrapy爬取豆瓣电影top250代码
时间: 2023-09-02 21:05:50 浏览: 58
爬取豆瓣电影Top250数据
5星 · 资源好评率100%
可以直接回答,以下是 Python 代码示例:
```
import scrapy
class DoubanTop250Spider(scrapy.Spider):
name = 'douban_top250'
allowed_domains = ['movie.douban.com']
start_urls = ['https://movie.douban.com/top250']
def parse(self, response):
for movie in response.xpath('//*[@class="grid_view"]/li'):
yield {
'title': movie.xpath('.//span[@class="title"]/text()').extract_first(),
'rating': movie.xpath('.//span[@class="rating_num"]/text()').extract_first(),
'link': movie.xpath('.//@href').extract_first()
}
next_page = response.xpath('//*[@class="next"]/a/@href')
if next_page:
yield response.follow(next_page, self.parse)
```
希望能对你有所帮助。
阅读全文