requests库爬虫实例

Requests库提供了丰富的功能，可以用于编写爬虫程序。以下是一些requests库的爬虫实例：
1. 发送GET请求并获取响应内容：

import requests

url = "http://example.com"
response = requests.get(url)
content = response.text

2. 发送POST请求并携带参数：

import requests

url = "http://example.com"
data = {"key": "value"}
response = requests.post(url, data=data)
content = response.text

3. 设置请求头信息：

import requests

url = "http://example.com"
headers = {"User-Agent": "Mozilla/5.0"}
response = requests.get(url, headers=headers)
content = response.text

4. 下载文件：

import requests

url = "http://example.com/file.jpg"
response = requests.get(url)
with open("file.jpg", "wb") as file:
    file.write(response.content)

5. 使用代理IP发送请求：

import requests

url = "http://example.com"
proxy = {"http": "http://127.0.0.1:8080"}
response = requests.get(url, proxies=proxy)
content = response.text

6. 处理Cookie：

import requests

url = "http://example.com"
cookies = {"key": "value"}
response = requests.get(url, cookies=cookies)
content = response.text