dropout层代码

### PyTorch 中实现 Dropout 层

在 PyTorch 中，`nn.Dropout` 是一种常用的正则化技术，可以防止模型过拟合。下面是一个简单的例子来展示如何在一个全连接神经网络中加入 `nn.Dropout`：

import torch
from torch import nn, optim

class SimpleNet(nn.Module):
    def __init__(self):
        super(SimpleNet, self).__init__()
        self.fc1 = nn.Linear(784, 500)
        self.fc2 = nn.Linear(500, 300)
        self.fc3 = nn.Linear(300, 10)
        self.dropout = nn.Dropout(p=0.5)

    def forward(self, x):
        x = x.view(-1, 784)  # Flatten the input image to a vector of size 784.
        x = torch.relu(self.fc1(x))
        x = self.dropout(x)  # Apply dropout after activation function.
        x = torch.relu(self.fc2(x))
        x = self.dropout(x)  # Apply dropout again before final layer.
        x = self.fc3(x)
        return x

model = SimpleNet()
print(model)

这段代码定义了一个三层的前馈神经网络，并在网络的不同层次之间加入了两个 Dropout 层[^2]。

对于特定维度的数据，比如卷积层后的张量，则应使用相应的 Dropout 实现方式如 `nn.Dropout1d` 或者 `nn.Dropout2d` 来匹配输入数据结构:

# Example with Convolutional Neural Network and Dropout2D
class CNNWithDropout(nn.Module):
    def __init__(self):
        super(CNNWithDropout, self).__init__()
        self.conv1 = nn.Conv2d(in_channels=1, out_channels=6, kernel_size=5)
        self.pool = nn.MaxPool2d(kernel_size=2, stride=2)
        self.conv2 = nn.Conv2d(in_channels=6, out_channels=16, kernel_size=5)
        self.fc1 = nn.Linear(16 * 4 * 4, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)
        
        # Applying spatial dropout on convolution outputs (spatial dimensions HxW).
        self.spatial_dropout = nn.Dropout2d(p=0.2)

    def forward(self, x):
        batch_size = x.size(0)
        x = self.pool(torch.relu(self.conv1(x)))
        x = self.spatial_dropout(x)  # Spatial dropout applied here.
        x = self.pool(torch.relu(self.conv2(x)))
        x = x.view(batch_size, -1)  # Reshape tensor for fully connected layers.
        x = torch.relu(self.fc1(x))
        x = torch.relu(self.fc2(x))
        x = self.fc3(x)
        return x

此部分展示了带有空间dropout的应用实例，适用于二维特征映射的情况，例如来自卷积操作的结果。

### TensorFlow/Keras 中实现 Dropout 层

同样地，在 TensorFlow 和 Keras API 下面也有类似的机制用于添加 dropout 正则项到模型架构之中：

import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Flatten, Conv2D, MaxPooling2D

def create_model():
    model = Sequential([
        Conv2D(filters=32, kernel_size=(3, 3), padding='same', activation='relu', input_shape=(28, 28, 1)),
        MaxPooling2D(pool_size=(2, 2)),
        Dropout(rate=0.25),
        Flatten(),
        Dense(units=128, activation='relu'),
        Dropout(rate=0.5),
        Dense(units=10, activation='softmax')
    ])
    
    optimizer = 'adam'
    loss_function = 'sparse_categorical_crossentropy'

    model.compile(optimizer=optimizer,
                  loss=loss_function,
                  metrics=['accuracy'])
                  
    return model

tf_model = create_model()
tf_model.summary()

上述代码片段构建了一个小型卷积神经网络并包含了两次不同的 dropout 应用——一次是在最大池化之后，另一次是在第一个完全连通层之后。这有助于减少过度拟合的风险，特别是在训练大型复杂的深层网络时[^1]。