带池化层和全连接层的卷积神经层后拼接dueling DQN的价值函数、优势函数网络的pytorch代码

以下是带池化层和全连接层的卷积神经层后拼接dueling DQN的价值函数、优势函数网络的PyTorch代码：

import torch.nn as nn
import torch.nn.functional as F

class DuelingDQN(nn.Module):
    def __init__(self, input_shape, num_actions):
        super(DuelingDQN, self).__init__()
        self.input_shape = input_shape
        self.num_actions = num_actions
        
        self.conv1 = nn.Conv2d(input_shape[0], 32, kernel_size=8, stride=4)
        self.conv2 = nn.Conv2d(32, 64, kernel_size=4, stride=2)
        self.conv3 = nn.Conv2d(64, 64, kernel_size=3, stride=1)
        
        self.fc_adv1 = nn.Linear(self.feature_size(), 128)
        self.fc_adv2 = nn.Linear(128, num_actions)
        
        self.fc_val1 = nn.Linear(self.feature_size(), 128)
        self.fc_val2 = nn.Linear(128, 1)
        
    def forward(self, x):
        x = x / 255.0
        x = F.relu(self.conv1(x))
        x = F.relu(self.conv2(x))
        x = F.relu(self.conv3(x))
        
        x = x.view(x.size(0), -1) # flatten
        
        adv = F.relu(self.fc_adv1(x))
        adv = self.fc_adv2(adv)
        
        val = F.relu(self.fc_val1(x))
        val = self.fc_val2(val)
        
        return val + adv - adv.mean(dim=1, keepdim=True)
        
    def feature_size(self):
        return self.conv3(self.conv2(self.conv1(torch.zeros(1, *self.input_shape)))).view(1, -1).size(1)

这个代码定义了一个名为DuelingDQN的类，它继承自`nn.Module`。类中定义了卷积神经网络层、池化层、全连接层，并且拼接了dueling DQN的价值函数和优势函数。在`forward`函数中，首先将输入的图像数据进行归一化处理，然后通过卷积神经网络层得到特征图，将特征图展平，并分别通过两个全连接层得到价值函数和优势函数。最后，将价值函数和优势函数拼接起来，得到最终的Q值。`feature_size`函数用于计算展平后的特征图大小。