Time Series Data Preprocessing: Experts Teach Standardization and Normalization Techniques

# Machine Learning Approaches in Time Series Forecasting

Time series data is a sequence of observations recorded over time, widely used in various fields such as finance, meteorology, retail, etc. Preprocessing this data is a critical step in ensuring the accuracy of analysis, involving data cleaning, formatting, and transformation. Understanding the purpose of preprocessing and its position in the overall data analysis workflow is crucial for enhancing the accuracy of model predictions. This chapter will outline the necessity of time series data preprocessing and its main components: standardization, normalization, and outlier handling, among others. This lays the foundation for an in-depth exploration of various preprocessing techniques in subsequent chapters.

# Basic Principles and Methods of Standardization

## Theoretical Basis of Standardization

### Definition and Purpose of Standardization

Standardization is a statistical method that aims to unify the variables in a dataset to a common scale, usually in the form of a normal distribution with a mean of 0 and a standard deviation of 1. The goal is to eliminate the influence of different dimensions, making the data comparable. In machine learning and statistical analysis, standardization is typically used in the following scenarios:

- When data distributions are extremely skewed or variable ranges differ significantly, standardization can adjust them to improve the convergence speed and stability of the model.
- It is a necessary step in algorithms that require the computation of distances or similarities between variables, such as K-Nearest Neighbors (K-NN) and Principal Component Analysis (PCA).
- When the application relies on data distribution, such as the normal distribution, standardization helps the model better understand and process the data.

### Applications of Standardization

In practical applications, standardization is widely used. Here are some common cases:

- In multivariate analysis, such as multiple linear regression, cluster analysis, artificial neural networks, etc., standardization ensures each feature has equal influence.
- When using gradient descent algorithms to solve optimization problems, standardization can accelerate convergence because the scales of features are consistent, preventing one feature's gradient from being much larger than another, which would cause偏差 in gradient updates.
- When comparing data with different dimensions and units, such as comparing height and weight, data needs to be standardized first.

## Practical Operations of Standardization

### Z-Score Method

The Z-Score method is one of the most commonly used standardization methods. It subtracts the mean of the data from each data point and then divides by the standard deviation of the data. The formula is as follows:

\[ Z = \frac{(X - \mu)}{\sigma} \]

Where \( X \) is the original data point, \( \mu \) is the mean of the data, and \( \sigma \) is the standard deviation of the data.

#### Python Code Demonstration

import numpy as np

# Example dataset
data = np.array([10, 12, 23, 23, 16, 23, 21, 16])
# Calculate mean and standard deviation
mean = np.mean(data)
std_dev = np.std(data)
# Apply Z-Score standardization
z_scores = (data - mean) / std_dev

print(z_scores)

In the code above, we first imported the NumPy library and defined a one-dimensional array containing the original data. Then we calculated the mean and standard deviation of the data and used these statistics to standardize the data.

### Min-Max Standardization

Min-Max standardization scales the original data to a specified range (usually between 0 and 1), thereby eliminating the dimensional impact of the original data. The formula is:

\[ X_{\text{new}} = \frac{(X - X_{\text{min}})}{(X_{\text{max}} - X_{\text{min}})} \]

Where \( X \) is the original data, \( X_{\text{min}} \) and \( X_{\text{max}} \) are the minimum and maximum values in the dataset, respectively.

#### Python Code Demonstration

# Apply Min-Max standardization
min_max_scaled = (data - np.min(data)) / (np.max(data) - np.min(data))
print(min_max_scaled)

In the code above, we used the `np.min()` and `np.max()` functions from the NumPy library to find the minimum and maximum values in the dataset and used the Min-Max formula to transform the data.

### Other Standardization Techniques

In addition to Z-Score and Min-Max standardization, there are other standardization techniques, such as Robust standardization. Robust standardization does not use the standard deviation but uses 1.5 times the interquartile range (IQR) as the boundary for outliers. This method is not sensitive to outliers and is suitable for situations where there are outliers in the data.

## Evaluation of Standardization Effects and Case Analysis

### Comparison of Data Before and After Standardization

One simple method to evaluate the effects of standardization is to observe the changes in the distribution of data before and after standardization. Histograms or box plots can visually show how standardization unifies data into a standard normal distribution.

### The Impact of Standardization on Model Performance

In practical applications, by modeling the data before and after preprocessing and comparing the model performance indicators (such as accuracy, mean squared error (MSE), etc.), the impact of standardization on model performance can be assessed. Typically, properly preprocessed data can improve the accuracy and robustness of the model.

# Normalization Strategies and Techniques

## Theoretical Discussion of Normalization

### Concept of Normalization and Its Importance

Normalization, also known as scaling or min-max normalization, ***monly, data is scaled to the range [0, 1], primarily to eliminate differences between different dimensions and reduce the computational impact of data differences.

In time series analysis, normalization is particularly important because data often has different dimensions and scales. Through normalization, different variables can have the same scale, making algorithm models focus more on the patterns between data rather than absolute values. Additionally, normalization can accelerate the learning process of models, increasing convergence speed, especially when using gradient-based optimization algorithms. Normalization can avoid problems such as gradient vanishing or gradient explosion.

### Comparison of Normalization with Other Preprocessing Methods

Compared with other preprocessing techniques like standardization, normalization differs in its scope and objectives. Normalization usually pays more attention to maintaining the data distribution rather than the statistical characteristics of the data. Standardization, by subtracting the mean and dividing by the standard deviation, gives the data unit variance, which to some extent preserves the statistical characteristics of the data but not necessarily within the range of 0 to 1.

In certain cases, normalization may be more suitable than standardization for neural network models, as the activation functions in neural networks often have restrictions on the range of input values. For example, Sigmoid and Tanh activation functions require input values to be within [-1, 1] or [0, 1]. Although standardization can scale the data, the results may still fall outside these ranges. Therefore, normalization may be more convenient and direct in practice.

## Practical Guide to Normalization

### Maximum-Minimum Normalization

Maximum-minimum normalization linearly transforms the original data to a specified range, usually [0, 1]. The transformation formula is:

X' = (X - X_min) / (X_max - X_min)

Where `X` is the original data, `X_min` and `X_max` are the minimum and maximum values in the dataset, respectively, and `X'` is the normalized data.

This method is simple and easy to implement but is very sensitive to outliers. If the minimum and maximum values in the dataset change, the normalized results of all data will also change. Here is a Python code example:

import numpy as np

# Original dataset
X = np.array([1, 2, 3, 4, 5])

# Calculate minimum and maximum
X_min = X.min()
X_max = X.max()

# Apply maximum-minimum normalization
X_prime = (X - X_min) / (X_max - X_min)
print(X_prime)

### Decimal Scaling Normalization

Decimal scaling normalization is achieved by dividing the data by a constant value. Typically, the chosen constant is a representative value in the data, such as 10, 100, etc. This method can quickly reduce the data to a smaller range, for example, reducing the data range to less than 1. The formula is:

X' = X / k

Where `k` is a pre-set constant, and `X'` is the normalized data.

### Vector Normalization

When dealing with multi-dimensional data, vector normalization, also known as unitization, is often used. For any vector X, its normalized vector is calculated using the following formula:

X' = X / ||X||

Where `||X||` represents the norm of vector X (usually the Euclidean norm), and `X'` is the normalized vector with a magnitude of 1. Vector normalization ensures that each co