informer复现

### 复现 Informer 模型 PyTorch 时间序列预测实验代码

为了复现 Informer 模型的时间序列预测功能，可以遵循以下结构化的方法。此过程涉及环境配置、数据准备以及模型构建等多个方面。

#### 1. 安装依赖库
确保安装必要的Python包，这些包对于处理时间序列数据和训练神经网络至关重要。

pip install numpy pandas matplotlib torch torchvision torchaudio

#### 2. 导入所需模块并设置全局变量
导入所需的Python库，并定义一些全局使用的超参数。

import os
import time
from datetime import timedelta
import numpy as np
import pandas as pd
import torch
import torch.nn.functional as F
from torch.utils.data import DataLoader, Dataset
import matplotlib.pyplot as plt

device = 'cuda' if torch.cuda.is_available() else 'cpu'
batch_size = 64
seq_len = 96  # 输入长度
pred_len = 24  # 预测长度
learning_rate = 0.0001
epochs = 10

#### 3. 数据预处理函数
创建自定义的数据加载器类`TimeSeriesDataset`以便于后续批量读取数据。

class TimeSeriesDataset(Dataset):
    def __init__(self, df, seq_len=seq_len, pred_len=pred_len):
        self.seq_len = seq_len
        self.pred_len = pred_len
        data = df.values.astype(np.float32)
        max_seq_len = min(len(data), int(seq_len * 1.5))
        
        mean = data[:max_seq_len].mean(axis=0)
        std = data[:max_seq_len].std(axis=0)+1e-8
        
        self.mean = mean
        self.std = std
        self.data = (data-mean)/std
    
    def __len__(self):
        return len(self.data)-self.seq_len-self.pred_len
    
    def __getitem__(self, idx):
        x = self.data[idx:idx+self.seq_len]
        y = self.data[idx+self.seq_len : idx+self.seq_len+self.pred_len][:, 0]

        return torch.tensor(x).float(), torch.tensor(y).float()

#### 4. 构建Informer模型架构
基于论文中的描述，在PyTorch框架下实现Informer的核心组件——ProbSparse Self Attention机制和其他辅助层。

class ProbAttention(torch.nn.Module):
    """Probability Sparse Self-attention Layer"""
    
    ...

class EncoderLayer(torch.nn.Module):
    """Encoder layer of the Transformer model."""
    
    ...
    
class DecoderLayer(torch.nn.Module):
    """Decoder layer of the Transformer model."""

    ...

class Informer(torch.nn.Module):
    """
    The main architecture of the Informer Model.
    It consists of an encoder and a decoder with multiple layers each.
    """

    def __init__(...):
        super(Informer, self).__init__()
        # Initialize components here...
        
    def forward(...):
        # Define forward pass logic here...

def get_model():
    enc_in = ...  # Number of input features
    dec_in = ...  # Same or different from `enc_in`
    c_out = ...   # Output feature count
    d_model = ... # Dimensionality of embeddings
    n_heads = ... # Number of attention heads
    e_layers = ...# Number of encoder layers
    d_layers = ...# Number of decoder layers
    d_ff = ...     # Feedforward dimension
    dropout = ...  # Dropout rate
    factor = ...   # Factor for prob-sparse attn
    distil = True  # Whether to use RBF kernels in attention mechanism
    activation = "gelu"
    
    model = Informer(
        enc_in,
        dec_in,
        c_out,
        d_model,
        n_heads,
        e_layers,
        d_layers,
        d_ff,
        dropout,
        factor,
        distil,
        activation
    ).to(device)

    return model

注意：上述代码片段仅展示了部分核心逻辑；实际应用时需参照原始文献补充完整细节[^1]。

#### 5. 训练与评估流程
编写脚本来执行模型的训练循环，并定期保存最佳性能版本。

train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)
val_loader = DataLoader(val_dataset, batch_size=batch_size*2, shuffle=False)

model = get_model()

optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)
criterion = nn.MSELoss().to(device)

for epoch in range(epochs):
    train_loss = []
    start_time = time.time()
    
    model.train()
    for i, (inputs, targets) in enumerate(train_loader):
        inputs = inputs.to(device)
        targets = targets.to(device)
        
        optimizer.zero_grad()
        outputs = model(inputs)[..., :-1]
        loss = criterion(outputs.view(-1), targets.view(-1))
        loss.backward()
        optimizer.step()
        
        train_loss.append(loss.item())
        
    avg_train_loss = sum(train_loss)/len(train_loss)
    elapsed_time = str(timedelta(seconds=time.time()-start_time)).split('.')[0]
    print(f"Epoch {epoch+1}/{epochs}, Loss={avg_train_loss:.4f} ({elapsed_time})")

print("Training completed.")

通过以上步骤即可完成Informers模型的基础搭建工作。当然，具体实施过程中还需要针对特定应用场景调整相应参数设定及优化策略[^2]。