【Basic】Detailed Explanation of HTTP Request Library Requests: Sending GET and POST Requests

# 2.1 GET Requests

### 2.1.1 Basic Usage of GET Requests

GET requests are the most commonly used HTTP method for retrieving resources from a server. The `get()` method is used in the Requests library to send GET requests, with the syntax as follows:

requests.get(url, params=None, **kwargs)

Where:

* `url`: The URL address of the request
* `params`: The request parameters, passed in as a dictionary
* `**kwargs`: Other optional parameters, such as `headers`, `timeout`, etc.

For example, sending a simple GET request to get the content of Baidu's homepage:

import requests

url = '***'
response = requests.get(url)

# 2. Using the Requests Library

### 2.1 GET Requests

#### 2.1.1 Basic Usage of GET Requests

GET requests are the most basic HTTP method, used for retrieving resources from a server. The `get()` method in the Requests library is used to send GET requests.

import requests

url = '***'
response = requests.get(url)

The above code sends a GET request to the `/api/v1/users` endpoint of `***`. The `response` object contains the server's response.

#### 2.1.2 Passing Parameters in GET Requests

GET request parameters can be passed via the `params` parameter. `params` is a dictionary, where the keys are parameter names and the values are parameter values.

params = {'page': 1, 'limit': 10}
response = requests.get(url, params=params)

The above code sends a GET request to the `/api/v1/users` endpoint of `***`, and passes `page` and `limit` as parameters.

### 2.2 POST Requests

#### 2.2.1 Basic Usage of POST Requests

POST requests are used to submit data to the server. The `post()` method in the Requests library is used to send POST requests.

data = {'username': 'admin', 'password': 'password'}
response = requests.post(url, data=data)

The above code sends a POST request to the `/api/v1/users` endpoint of `***`, and submits `username` and `password` as data.

#### 2.2.2 Data Formats for POST Requests

Data for POST requests can be in various formats, including JSON, form data, and binary data. The Requests library provides `json` and `data` parameters to specify the data format.

# JSON format
data = {'username': 'admin', 'password': 'password'}
response = requests.post(url, json=data)

# Form data format
data = {'username': 'admin', 'password': 'password'}
response = requests.post(url, data=data)

# Binary data format
data = b'...'
response = requests.post(url, data=data)

### 2.3 Other Request Methods

The Requests library also supports other HTTP methods, including HEAD, PUT, and DELETE. The usage of these methods is similar to GET and POST.

#### 2.3.1 HEAD Requests

HEAD requests are used to retrieve header information of a resource, without getting the content itself.

response = requests.head(url)

#### 2.3.2 PUT Requests

PUT requests are used to update resources.

data = {'username': 'admin', 'password': 'password'}
response = requests.put(url, data=data)

#### 2.3.3 DELETE Requests

DELETE requests are used to delete resources.

response = requests.delete(url)

# 3.1 Response Status Codes

#### 3.1.1 Meanings of Common Status Codes

HTTP response status codes are three-digit codes that indicate the server'***mon HTTP status codes and their meanings are as follows:

| Status Code | Meaning |
|---|---|
| 200 | OK |
| 201 | Created |
| 204 | No Content |
| 301 | Moved Permanently |
| 302 | Found |
| 400 | Bad Request |
| 401 | Unauthorized |
| 403 | Forbidden |
| 404 | Not Found |
| 500 | Internal Server Error |
| 502 | Bad Gateway |
| 503 | Service Unavailable |

#### 3.1.2 Handling Responses with Different Status Codes

When using the Requests library to send requests, we can obtain the response status code through the `response.status_code` attribute. Based on different status codes, we can adopt different handling strategies:

- **200 OK:** Indicates a successful request, we can directly parse the response content.
- **201 Created:** Indicates the request successfully created a resource, we can obtain the `Location` header field to get the URI of the new resource.
- **204 No Content:** Indicates a successful request, but no content is returned.
- **301 Moved Permanently:** Indicates that the requested resource has been permanently moved to a new URI, we can obtain the new URI through the `Location` header field.
- **302 Found:** Indicates that the requested resource has been temporarily moved to a new URI, we can obtain the new URI through the `Location` header field.
- **400 Bad Request:** Indicates a request syntax error, we can check if the request parameters are correct.
- **401 Unauthorized:** Indicates that the request is unauthorized, we can check if the request headers contain correct authorization information.
- **403 Forbidden:** Indicates that the request is forbidden, we can check if the request permissions are correct.
- **404 Not Found:** Indicates that the requested resource does not exist, we can check if the requested URI is correct.
- **500 Internal Server Error:** Indicates a server internal error, we can contact the server administrator to resolve the issue.
- **502 Bad Gateway:** Indicates that the server received an invalid response as a gateway or proxy, we can check if the server configuration is correct.
- **503 Service Unavailable:** Indicates that the server is temporarily unavailable, we can try again later.

### 3.2 Response Content

#### 3.2.1 Obtaining Response Content

The Requests library provides several methods to obtain response content, including:

- **`response.text`**: Gets the text form of the response content.
- **`response.json()`**: Gets the JSON format of the response content.
- **`response.content`**: Gets the binary form of the response content.
- **`response.raw`**: Gets the raw byte stream of the response content.

#### 3.2.2 Parsing Response Content

Depending on the format of the response content, we can use different methods to parse:

- **Text format:** Can be parsed using regular expressions, HTML parsers, or XML parsers.
- **JSON format:** Can be parsed using the `json` module.
- **Binary format:** Can be parsed using the `struct` module or other binary data parsing libraries.
- **Raw byte stream:** Can be parsed using the `io` module or other byte stream processing libraries.

# 4. Session Management in the Requests Library

### 4.1 Session Object

#### 4.1.1 Creating and Using Session Objects

Session objects are used to manage HTTP sessions; they can maintain session state across multiple requests. The Requests library provides the `requests.Session()` function to create Session objects.

import requests

# Create a Session object
session = requests.Session()

# Send requests using the Session object
response = session.get("***")

#### 4.1.2 Persistence of Session Objects

Session objects can be persisted to disk so that they can still be used after the program restarts. The Requests library provides `session.save()` and `session.load()` methods to achieve persistence.

# Persist the Session object
session.save("session.pkl")

# Load the persisted Session object
session = requests.Session()
session.load("session.pkl")

### 4.2 Cookie Management

#### 4.2.1 Obtaining and Setting Cookies

Cookies are small chunks of data sent by the server to the client to recognize the client in subsequent requests. The Requests library provides the `session.cookies` attribute to obtain and set Cookies.

# Obtain Cookies
cookies = session.cookies

# Set a Cookie
session.cookies["name"] = "value"

#### 4.2.2 Persistence of Cookies

Cookies can also be persisted to disk for later use after the program restarts. The Requests library provides `session.cookies.save()` and `session.cookies.load()` methods to achieve this.

# Persist Cookies
session.cookies.save("cookies.pkl")

# Load persisted Cookies
session.cookies.load("cookies.pkl")

### 4.3 Interaction between Session and Cookie

There is a close interaction between Session objects and Cookies. Session objects can manage Cookies, while Cookies can help Session objects maintain session state.

When sending requests using a Session object, the Session object will automatically send the Cookies associated with the session to the server. Once the server receives the Cookies, it can recognize the client and provide the appropriate response.

### 4.4 Application Scenarios for Session and Cookie

Session and Cookie are particularly useful in the following scenarios:

- **User Authentication:** Cookies can store user login status to enable automatic login in subsequent requests.
- **Shopping Cart Management:** Cookies can store shopping cart information to maintain the state of the shopping cart across different pages.
- **Personalized Recommendations:** Cookies can store user browsing history and preferences to provide personalized recommendations.

# 5. Exception Handling in the Requests Library

When using the Requests library to make HTTP requests, various exceptions may occur. These exceptions may be caused by network connection issues, server response issues, or coding errors. It is crucial to properly handle these exceptions to ensure the robustness and reliability of the application.

### 5.1 Common Exception Types

Common exception types thrown by the Requests library include:

- **ConnectionError:** Thrown when a connection to the server cannot be established. This could be due to network connection issues or the server being unavailable.
- **TimeoutError:** Thrown when a request times out. This could be due to slow server responses or network delays.
- **HTTPError:** Thrown when the server returns a non-200 status code. This indicates that the server has encountered an error or cannot process the request.

### 5.2 Strategies for Handling Exceptions

To handle exceptions thrown by the Requests library, the following strategies can be adopted:

#### 5.2.1 Retry Mechanism

For connection errors and timeout errors, a retry mechanism can be implemented. The retry mechanism allows the application to automatically retry requests when errors occur, until successful or the maximum number of retries is reached.

#### 5.2.2 Logging

For all exceptions, detailed information should be logged, including the exception type, error message, and stack trace. This helps diagnose problems and track the status of the application.

#### 5.2.3 Example of Exception Handling Code

The following code example demonstrates how to use a try-except block to handle exceptions thrown by the Requests library:

try:
    response = requests.get(url)
except ConnectionError as e:
    # Handle connection errors
    print("Unable to connect to the server:", e)
except TimeoutError as e:
    # Handle timeout errors
    print("Request timed out:", e)
except HTTPError as e:
    # Handle HTTP errors
    print("Server returned an error:", e)

In practical applications, the logic for exception handling can be customized according to specific needs. For instance, for connection errors, an exponential backoff retry strategy can be implemented to avoid putting too much pressure on the server. For HTTP errors, different actions can be taken based on the status code, such as redirection or displaying error messages.

# 6.1 Proxy Settings

### 6.1.1 Configuration and Usage of Proxies

In some cases, a proxy server is needed to access network resources, for example:

- To circumvent network restrictions
- To hide the real IP address
- To increase access speed

The Requests library supports proxies, which can be configured using the `proxies` parameter, as follows:

proxies = {
    "http": "***<proxy server address>:<port>",
    "https": "***<proxy server address>:<port>"
}

For example, accessing Baidu using an HTTP proxy server:

import requests

proxies = {
    "http": "***"
}

response = requests.get("***", proxies=proxies)

### 6.1.2 Verification of Proxies

Some proxy servers require authentication, and the Requests library supports providing authentication information through the `auth` parameter, as follows:

auth = (
    "<username>",
     "<password>"
)

For example, accessing Baidu using an authenticated HTTP proxy server:

import requests

proxies = {
    "http": "***"
}

auth = ("username", "password")

response = requests.get("***", proxies=proxies, auth=auth)