Python多线程爬取图片
时间: 2023-10-18 11:11:01 浏览: 100
以下是Python多线程爬取图片的代码示例:
```python
import requests
from bs4 import BeautifulSoup
import threading
def download_image(url, filename):
response = requests.get(url)
with open(filename, "wb") as f:
f.write(response.content)
def crawl_images(url):
response = requests.get(url)
soup = BeautifulSoup(response.content, "html.parser")
images = soup.find_all("img")
for i, image in enumerate(images):
image_url = image["src"]
filename = "image{}.jpg".format(i)
t = threading.Thread(target=download_image, args=(image_url, filename))
t.start()
if __name__ == "__main__":
url = "https://www.example.com"
crawl_images(url)
```
该代码首先定义了一个`download_image`函数用于下载图片,然后定义了`crawl_images`函数用于爬取图片链接,并创建多线程下载图片。最后,使用`if __name__ == "__main__":`语句调用`crawl_images`函数,传入需要爬取的网站URL。
阅读全文