full code python for image crawler
时间: 2023-12-16 08:03:43 浏览: 102
Certainly! Here is a complete Python code example for an image crawler:
```python
import requests
from bs4 import BeautifulSoup
import os
def download_images(url):
# Send a GET request to the website
response = requests.get(url)
# Check if the request was successful
if response.status_code == 200:
# Use BeautifulSoup to parse the HTML
soup = BeautifulSoup(response.content, 'html.parser')
# Find all image tags on the website
img_tags = soup.find_all('img')
# Create a directory to store the images
os.makedirs('images', exist_ok=True)
# Loop through the list of image tags
for img in img_tags:
# Check if the image tag has a 'src' attribute
if 'src' in img.attrs:
# Get the image URL
img_url = img.attrs['src']
# Create a file name from the image URL
filename = img_url.split('/')[-1]
# Create the file path to store the image
filepath = os.path.join('images', filename)
# Download and store the image
with open(filepath, 'wb') as file:
# Send a GET request to download the image
img_response = requests.get(img_url)
# Write the data into the file
file.write(img_response.content)
print(f"Downloaded: {filename}")
# Example usage
url = 'https://example.com'
download_images(url)
```
In this code, you can replace `'https://example.com'` with the URL of the website you want to crawl images from. The script will create an `images` directory (if it doesn't exist) and download all the images found on the webpage into that directory.
Please note that crawling and downloading images from websites should be done responsibly and in compliance with the website's terms of service.
阅读全文