MATLAB Matrix Singular Value Decomposition (SVD) Application Guide: From Dimensionality Reduction to Image Processing, 5 Practical Cases

# Guide to Singular Value Decomposition (SVD) Applications in MATLAB: From Dimensionality Reduction to Image Processing, 5 Practical Cases

## 1. Introduction to Singular Value Decomposition (SVD)

Singular Value Decomposition (SVD) is a powerful linear algebra technique used to factorize a matrix into the product of three matrices: an orthogonal matrix U, a diagonal matrix Σ, and another orthogonal matrix V. The form of SVD is as follows:

A = UΣV^T

where:

* A is the original matrix
* U is the matrix of left singular vectors
* Σ is the singular value matrix
* V is the matrix of right singular vectors

The singular values in SVD represent a measure of importance of the matrix A, i.e., the larger the singular value, the more important the corresponding column or row is in the matrix. SVD is widely applied in dimensionality reduction, image processing, signal processing, and many other fields.

## 2. Application of SVD in Dimensionality Reduction

### 2.1 Principal Component Analysis (PCA)

**Definition:**

Principal Component Analysis (PCA) is a dimensionality reduction technique that projects high-dimensional data into a lower-dimensional space through a linear transformation while preserving the maximum amount of data variance.

**Principle:**

The principle of PCA is to find the eigenvectors of the data covariance matrix, which represent the main directions of the data. The larger the sum of the eigenvalues corresponding to the first k eigenvectors as a proportion of the total eigenvalues, the more information is retained in the reduced data.

**Steps:**

1. Center the data by subtracting the mean of each column.
2. Calculate the data covariance matrix.
3. Perform eigendecomposition on the covariance matrix to obtain eigenvalues and eigenvectors.
4. Select the eigenvectors corresponding to the first k eigenvalues as the basis vectors for the reduced data.
5. Project the data onto the subspace spanned by the basis vectors to obtain the reduced data.

**Code Example:**

% Data
data = randn(100, 10);

% Centering
data = data - mean(data);

% Covariance matrix
cov_matrix = cov(data);

% Eigendecomposition
[eigenvectors, eigenvalues] = eig(cov_matrix);

% Dimensionality reduction
reduced_data = data * eigenvectors(:, 1:2);

**Logical Analysis:**

* `randn(100, 10)` generates a random matrix with 100 rows and 10 columns.
* `mean(data)` calculates the mean of each column.
* `cov(data)` calculates the covariance matrix.
* `eig(cov_matrix)` performs eigendecomposition to obtain eigenvalues and eigenvectors.
* `eigenvectors(:, 1:2)` selects the first two eigenvectors.
* `data * eigenvectors(:, 1:2)` projects the data onto the subspace spanned by the basis vectors.

### 2.2 Linear Discriminant Analysis (LDA)

**Definition:**

Linear Discriminant Analysis (LDA) is a supervised dimensionality reduction technique that projects high-dimensional data into a lower-dimensional space through a linear transformation, maximizing the distance between classes and minimizing the distance within classes.

**Principle:**

The principle of LDA is to find a linear projection matrix that maximizes the distance between the centroids of different classes and minimizes the distance between the centroids within the same class after projection.

**Steps:**

1. Calculate the within-class scatter matrix and the between-class scatter matrix.
2. Perform eigendecomposition on the between-class scatter matrix to obtain eigenvalues and eigenvectors.
3. Select the eigenvectors corresponding to the first k eigenvalues as the basis vectors for the reduced data.
4. Project the data onto the subspace spanned by the basis vectors to obtain the reduced data.

**Code Example:**

% Data
data = [randn(50, 10); randn(50, 10) + 5];
labels = [ones(50, 1); ones(50, 1) * 2];

% Within-class scatter matrix
Sw = zeros(size(data, 2));
for i = 1:max(labels)
     Sw = Sw + cov(data(labels == i, :));
end

% Between-class scatter matrix
Sb = zeros(size(data, 2));
for i = 1:max(labels)
     Sb = Sb + (mean(data(labels == i, :)) - mean(data))' * (mean(data(labels == i, :)) - mean(data));
end

% Eigendecomposition
[eigenvectors, eigenvalues] = eig(Sb, Sw);

% Dimensionality reduction
reduced_data = data * eigenvectors(:, 1:2);

**Logical Analysis:**

* `randn(50, 10)` generates two random matrices with 50 rows and 10 columns each, representing two classes of data.
* `ones(50, 1)` generates a matrix with 50 rows and 1 column, with all elements set to 1, as labels for the first class.
* `ones(50, 1) * 2` generates a matrix with 50 rows and 1 column, with all elements set to 2, as labels for the second class.
* `cov(data(labels == i, :))` calculates the within-class scatter matrix for each class.
* `mean(data(labels == i, :))` calculates the centroid for each class.
* `eig(Sb, Sw)` performs eigendecomposition on the between-class scatter matrix and the within-class scatter matrix.
* `eigenvectors(:, 1:2)` selects the first two eigenvectors.
* `data * eigenvectors(:, 1:2)` projects the data onto the subspace spanned by the basis vectors.

### 2.3 Non-negative Matrix Factorization (NMF)

**Definition:**

Non-negative Matrix Factorization (NMF) is a technique f