爬取图片保存图片多线程代码
时间: 2023-05-22 18:06:05 浏览: 116
好的,以下是一个 Python 爬取图片并多线程保存的代码示例:
```
import requests
from lxml import html
import os
import urllib
from multiprocessing.pool import ThreadPool
# 创建一个线程池
pool = ThreadPool(10)
# 爬取图片并保存的函数
def download_image(url):
response = requests.get(url)
filename = os.path.basename(url)
with open(filename, 'wb') as f:
f.write(response.content)
print(f'{url} downloaded.')
# 爬取图片的函数
def crawl_images(url):
response = requests.get(url)
tree = html.fromstring(response.content)
links = tree.xpath('//img/@src')
for link in links:
if 'http' not in link:
link = urllib.parse.urljoin(response.url, link)
pool.apply_async(download_image, args=[link])
# 调用爬取图片的函数
crawl_images('https://unsplash.com/search/photos')
```
这段代码可以在一个网页中爬取所有图片的链接,并使用 10 个线程同时下载保存图片。请注意,下载的图片将保存在当前目录下,并以链接中的文件名作为文件名。
阅读全文