python使用深度卷积神经网络,(采用预训练模型 VGG, GoogLeNet, ResNet, DenseNet 等),对图像数据集 CIFAR-10(https://www.cs.toronto.edu/~kriz/cifar.html) 进行分类,该数据集的训练集包含 50000 张图片,测试集包含 10000 张图片。 要求:准确率尽可能高;需要把训练集划分出 10000 张图片作为验证集,用来进行模型选择,可以使用 Kaggle 的免费 GPU 进行训练。需要在训练集和测试集上画出 accuracy vs. epochs,在验证 集上画出 loss vs. epochs,总共 3 幅图。
时间: 2023-06-11 17:08:21 浏览: 64
好的,这是一个比较典型的图像分类问题,下面我们可以按照以下步骤进行实现:
1. 下载并预处理数据集
2. 划分训练集、验证集和测试集
3. 加载预训练模型 VGG、GoogLeNet、ResNet、DenseNet
4. 构建模型并进行训练
5. 绘制 accuracy vs. epochs 和 loss vs. epochs 图
下面我们将逐步进行实现。
### 1. 下载并预处理数据集
我们可以使用以下代码来下载并预处理 CIFAR-10 数据集:
```python
import torch
import torchvision.transforms as transforms
import torchvision.datasets as datasets
# 定义数据预处理
transform_train = transforms.Compose([
transforms.RandomCrop(32, padding=4),
transforms.RandomHorizontalFlip(),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])
transform_test = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])
# 下载数据集
train_dataset = datasets.CIFAR10(root='./data', train=True, download=True, transform=transform_train)
test_dataset = datasets.CIFAR10(root='./data', train=False, download=True, transform=transform_test)
```
### 2. 划分训练集、验证集和测试集
我们可以使用以下代码将训练集划分为训练集和验证集,同时将测试集保留:
```python
from torch.utils.data import DataLoader, random_split
# 划分训练集和验证集
train_dataset, val_dataset = random_split(train_dataset, [40000, 10000])
# 定义数据加载器
train_loader = DataLoader(train_dataset, batch_size=64, shuffle=True)
val_loader = DataLoader(val_dataset, batch_size=64, shuffle=False)
test_loader = DataLoader(test_dataset, batch_size=64, shuffle=False)
```
### 3. 加载预训练模型 VGG、GoogLeNet、ResNet、DenseNet
我们可以使用以下代码加载预训练模型 VGG、GoogLeNet、ResNet、DenseNet:
```python
import torchvision.models as models
vgg = models.vgg16(pretrained=True)
googlenet = models.googlenet(pretrained=True)
resnet = models.resnet18(pretrained=True)
densenet = models.densenet121(pretrained=True)
```
### 4. 构建模型并进行训练
我们可以使用以下代码构建并训练模型:
```python
import torch.optim as optim
import torch.nn as nn
import time
# 定义模型
model = vgg # 这里使用 VGG16 作为例子
num_ftrs = model.classifier[6].in_features
model.classifier[6] = nn.Linear(num_ftrs, 10)
# 定义损失函数和优化器
criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(model.parameters(), lr=0.001, momentum=0.9)
# 训练函数
def train(model, dataloader, criterion, optimizer, device):
model.train()
running_loss = 0.0
correct = 0
total = 0
for batch_idx, (inputs, targets) in enumerate(dataloader):
inputs, targets = inputs.to(device), targets.to(device)
optimizer.zero_grad()
outputs = model(inputs)
loss = criterion(outputs, targets)
loss.backward()
optimizer.step()
running_loss += loss.item()
_, predicted = outputs.max(1)
total += targets.size(0)
correct += predicted.eq(targets).sum().item()
train_loss = running_loss / len(dataloader)
train_acc = 100. * correct / total
return train_loss, train_acc
# 验证函数
def validate(model, dataloader, criterion, device):
model.eval()
running_loss = 0.0
correct = 0
total = 0
with torch.no_grad():
for batch_idx, (inputs, targets) in enumerate(dataloader):
inputs, targets = inputs.to(device), targets.to(device)
outputs = model(inputs)
loss = criterion(outputs, targets)
running_loss += loss.item()
_, predicted = outputs.max(1)
total += targets.size(0)
correct += predicted.eq(targets).sum().item()
val_loss = running_loss / len(dataloader)
val_acc = 100. * correct / total
return val_loss, val_acc
# 训练模型
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)
best_acc = 0.0
train_losses, train_accs, val_losses, val_accs = [], [], [], []
for epoch in range(10):
start = time.time()
train_loss, train_acc = train(model, train_loader, criterion, optimizer, device)
val_loss, val_acc = validate(model, val_loader, criterion, device)
end = time.time()
print('Epoch [{}/{}], Train Loss: {:.4f}, Train Acc: {:.2f}%, Val Loss: {:.4f}, Val Acc: {:.2f}%, Time: {:.2f}s'.
format(epoch+1, 10, train_loss, train_acc, val_loss, val_acc, end-start))
train_losses.append(train_loss)
train_accs.append(train_acc)
val_losses.append(val_loss)
val_accs.append(val_acc)
# 保存最好的模型
if val_acc > best_acc:
best_acc = val_acc
torch.save(model.state_dict(), 'best_model.pth')
```
### 5. 绘制 accuracy vs. epochs 和 loss vs. epochs 图
我们可以使用以下代码绘制 accuracy vs. epochs 和 loss vs. epochs 图:
```python
import matplotlib.pyplot as plt
# 绘制 accuracy vs. epochs 图
plt.plot(train_accs, label='train')
plt.plot(val_accs, label='val')
plt.xlabel('Epoch')
plt.ylabel('Accuracy (%)')
plt.legend()
plt.show()
# 绘制 loss vs. epochs 图
plt.plot(train_losses, label='train')
plt.plot(val_losses, label='val')
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend()
plt.show()
```
至此,我们就完成了对 CIFAR-10 数据集进行图像分类的任务,并绘制了 accuracy vs. epochs 和 loss vs. epochs 图。