Advanced Section: Image Optical Flow Estimation in MATLAB: Using Optical Flow Methods for Image Optical Flow Estimation

# 2.1 Optical Flow Equation

The optical flow equation describes the relationship between the variation of pixel intensity over time and the pixel movement within the image. Assuming the pixel intensity I(x, y, t) is at the position (x, y) at time t, and it moves to (x+dx, y+dy) at time t+dt, the optical flow equation is:

I(x+dx, y+dy, t+dt) = I(x, y, t)

Expanding this equation using Taylor series and ignoring higher order terms, we get:

I(x, y, t) + ∂I/∂x * dx + ∂I/∂y * dy + ∂I/∂t * dt = I(x, y, t)

After rearranging, the optical flow equation is obtained:

∂I/∂x * dx + ∂I/∂y * dy + ∂I/∂t = 0

Here, (dx, dy) represents the displacement of the pixel within the time interval dt.

# 2. Foundations of Optical Flow Methods

### 2.1 Optical Flow Equation

The optical flow equation describes the constancy of brightness when image pixels move between adjacent frames. It assumes that the brightness of pixels in the image remains constant over a short period, i.e.:

I(x, y, t) = I(x + dx, y + dy, t + dt)

Where:

- `I(x, y, t)` represents the brightness of pixel `(x, y)` in the image at time `t`
- `dx` and `dy` are the horizontal and vertical displacements of pixel `(x, y)` from time `t` to `t + dt`
- `dt` denotes the time interval

Taking partial derivatives, the optical flow equation is obtained:

∂I/∂t + ∂I/∂x * dx + ∂I/∂y * dy = 0

### 2.2 Common Optical Flow Algorithms

#### 2.2.1 Lucas-Kanade Optical Flow Method

The Lucas-Kanade optical flow method is an iterative algorithm based on gradient descent. It estimates optical flow by minimizing the sum of squared differences in pixel brightness between adjacent frames. The specific steps are as follows:

1. **Initialization:** Assume the optical flow to be `(0, 0)`.
2. **Compute Error:** Calculate the sum of squared differences in pixel brightness between adjacent frames.
3. **Compute Gradients:** Calculate the gradient `(∂I/∂x, ∂I/∂y)` of the current pixel.
4. **Update Optical Flow:** Update the optical flow using gradient descent: `Δu = -H^-1 * b`, where `H` is the Hessian matrix, and `b` is the derivative of the error.
5. **Repeat Steps 2-4:** Repeat the above steps until the error reaches a minimum value or the maximum number of iterations is reached.

#### 2.2.2 Pyramid Optical Flow Method

The pyramid optical flow method is a hierarchical optical flow estimation algorithm. It constructs the image into a pyramid structure, estimating optical flow from low to high resolution. The specific steps are as follows:

1. **Build the Pyramid:** Construct the image into a pyramid with successively lower resolution layers.
2. **Estimate Layer by Layer:** Start estimating optical flow from the lowest resolution layer and move upwards.
3. **Interpolation and Fusion:** Interpolate the optical flow from the high-resolution layer to the low-resolution layer and fuse the estimation results from different layers.

#### 2.2.3 Variational Optical Flow Method

The variational optical flow method is an algorithm that estimates optical flow by minimizing an energy functional. It minimizes an energy functional that includes a data term and a regularization term. The specific steps are as follows:

1. **Define Energy Functional:** Define an energy functional that includes a data term (measuring the difference in pixel brightness between adjacent frames) and a regularization term (measuring the smoothness of optical flow).
2. **Solve the Energy Functional:** Use variational methods to solve the energy functional and obtain optical flow estimation.

| Algorithm | Advantages | Disadvantages |
|---|---|---|
| Lucas-Kanade Optical Flow Method | Simple computation, high efficiency | Only suitable for small displacements |
| Pyramid Optical Flow Method | Suitable for large displacements | High computational cost |
| Variational Optical Flow Method | Robust性强、适用性强 | High computational cost, complex parameter settings |

# 3. Practical Optical Flow Estimation in MATLAB

### 3.1 Image Reading and Preprocessing in MATLAB

Before starting optical flow estimation, it is necessary to read and preprocess the images. MATLAB provides various image processing functions that can easily accomplish these tasks.

**Image Reading**

Use the `imread` function to read images:

image = imread('image.jpg');

**Image Preprocessing**

Image preprocessing includes operations such as grayscale conversion, noise removal, and image scaling.

***Grayscale Conversion:** Convert the color image to a grayscale image to reduce the computational load:

image_gray = rgb2gray(image);

***Noise Removal:** Use a median filter to remove noise:

image_denoised = medfilt2(image_gray);

***Image Scaling:** Re