Scrapy爬虫实战—虎扑

为了实现Scrapy爬虫实战虎扑，您需要按照以下步骤进行操作：

1.创建一个Scrapy项目并定义Item类

scrapy startproject hupu
cd hupu
scrapy genspider hupu_spider tieba.hupu.com

在items.py文件中定义Item类：

import scrapy

class HupuImageItem(scrapy.Item):
    image_urls = scrapy.Field()
    image_path = scrapy.Field()

2.编写爬虫代码
在spiders/hupu_spider.py文件中编写爬虫代码：

import scrapy
from hupu.items import HupuImageItem

class HupuSpider(scrapy.Spider):
    name = 'hupu'
    allowed_domains = ['tieba.hupu.com']
    start_urls = ['http://tieba.hupu.com/picture']

    def parse(self, response):
        item = HupuImageItem()
        img_urls = response.css('.textPic img::attr(src)').extract()
        item['image_urls'] = img_urls
        yield item

3.配置settings.py文件
在settings.py文件中添加以下配置：

ITEM_PIPELINES = {'scrapy.pipelines.images.ImagesPipeline': 1}
IMAGES_STORE = 'images'

4.运行爬虫
在命令行中运行以下命令启动爬虫：

scrapy crawl hupu

5.查看结果
爬取的图片将保存在项目根目录下的images文件夹中。