写一个用pytorch,VGG16处理jaffe数据集的代码,用cpu,划分训练集,验证集和测试集,给出准确率图,损失图和混淆矩阵
时间: 2024-05-15 10:17:54 浏览: 214
pytorch人脸表情识别数据集(2w8训练集+7k测试集)
好的,我来回答你的问题。以下是用pytorch和VGG16处理jaffe数据集的代码,给出了准确率图,损失图和混淆矩阵。
首先,导入所需的库:
```
import torch
import torch.nn as nn
import torch.optim as optim
from torch.optim import lr_scheduler
import torchvision
from torchvision import datasets, models, transforms
import numpy as np
import time
import os
import copy
from sklearn.metrics import confusion_matrix
```
然后,定义一些参数:
```
data_dir = "/path/to/jaffe/dataset"
batch_size = 4
num_workers = 4
num_epochs = 25
input_size = 224
learning_rate = 0.001
```
接着,定义数据预处理:
```
data_transforms = {
'train': transforms.Compose([
transforms.Resize(input_size),
transforms.RandomHorizontalFlip(),
transforms.RandomRotation(10),
transforms.ToTensor(),
transforms.Normalize([0.5, 0.5, 0.5], [0.5, 0.5, 0.5])
]),
'val': transforms.Compose([
transforms.Resize(input_size),
transforms.ToTensor(),
transforms.Normalize([0.5, 0.5, 0.5], [0.5, 0.5, 0.5])
]),
'test': transforms.Compose([
transforms.Resize(input_size),
transforms.ToTensor(),
transforms.Normalize([0.5, 0.5, 0.5], [0.5, 0.5, 0.5])
])
}
```
然后,加载数据集:
```
image_datasets = {x: datasets.ImageFolder(os.path.join(data_dir, x), data_transforms[x]) for x in ['train', 'val', 'test']}
dataloaders = {x: torch.utils.data.DataLoader(image_datasets[x], batch_size=batch_size, shuffle=True, num_workers=num_workers) for x in ['train', 'val', 'test']}
dataset_sizes = {x: len(image_datasets[x]) for x in ['train', 'val', 'test']}
class_names = image_datasets['train'].classes
```
接下来,定义VGG16模型:
```
model_ft = models.vgg16(pretrained=True)
num_ftrs = model_ft.classifier[6].in_features
model_ft.classifier[6] = nn.Linear(num_ftrs, len(class_names))
```
然后,定义损失函数和优化器:
```
criterion = nn.CrossEntropyLoss()
optimizer_ft = optim.SGD(model_ft.parameters(), lr=learning_rate, momentum=0.9)
```
接着,定义学习率调度器:(可选)
```
exp_lr_scheduler = lr_scheduler.StepLR(optimizer_ft, step_size=7, gamma=0.1)
```
然后,定义训练和测试函数:
```
def train_model(model, criterion, optimizer, scheduler, num_epochs=25):
since = time.time()
best_model_wts = copy.deepcopy(model.state_dict())
best_acc = 0.0
train_losses = []
val_losses = []
train_acc = []
val_acc = []
for epoch in range(num_epochs):
print('Epoch {}/{}'.format(epoch, num_epochs - 1))
print('-' * 10)
for phase in ['train', 'val']:
if phase == 'train':
model.train() # Set model to training mode
else:
model.eval() # Set model to evaluate mode
running_loss = 0.0
running_corrects = 0
for inputs, labels in dataloaders[phase]:
inputs = inputs.to(device)
labels = labels.to(device)
optimizer.zero_grad()
with torch.set_grad_enabled(phase == 'train'):
outputs = model(inputs)
_, preds = torch.max(outputs, 1)
loss = criterion(outputs, labels)
if phase == 'train':
loss.backward()
optimizer.step()
running_loss += loss.item() * inputs.size(0)
running_corrects += torch.sum(preds == labels.data)
if phase == 'train':
scheduler.step()
epoch_loss = running_loss / dataset_sizes[phase]
epoch_acc = running_corrects.double() / dataset_sizes[phase]
if phase == 'train':
train_losses.append(epoch_loss)
train_acc.append(epoch_acc)
else:
val_losses.append(epoch_loss)
val_acc.append(epoch_acc)
print('{} Loss: {:.4f} Acc: {:.4f}'.format(phase, epoch_loss, epoch_acc))
if phase == 'val' and epoch_acc > best_acc:
best_acc = epoch_acc
best_model_wts = copy.deepcopy(model.state_dict())
print()
time_elapsed = time.time() - since
print('Training complete in {:.0f}m {:.0f}s'.format(time_elapsed // 60, time_elapsed % 60))
print('Best val Acc: {:.4f}'.format(best_acc))
model.load_state_dict(best_model_wts)
return model, train_losses, val_losses, train_acc, val_acc
def test_model(model):
model.eval()
test_losses = []
test_acc = []
y_true = []
y_pred = []
with torch.no_grad():
for inputs, labels in dataloaders['test']:
inputs = inputs.to(device)
labels = labels.to(device)
outputs = model(inputs)
_, preds = torch.max(outputs, 1)
loss = criterion(outputs, labels)
y_true += labels.cpu().numpy().tolist()
y_pred += preds.cpu().numpy().tolist()
test_losses.append(loss.item() * inputs.size(0))
test_acc.append(torch.sum(preds == labels.data))
epoch_loss = sum(test_losses) / dataset_sizes['test']
epoch_acc = sum(test_acc).double() / dataset_sizes['test']
conf_mat = confusion_matrix(y_true, y_pred)
print('Test Loss: {:.4f} Acc: {:.4f}'.format(epoch_loss, epoch_acc))
print('Confusion Matrix:')
print(conf_mat)
return epoch_loss, epoch_acc, conf_mat
```
最后,运行代码,得到准确率图,损失图和混淆矩阵:
```
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
model_ft = model_ft.to(device)
criterion = criterion.to(device)
model_ft, train_losses, val_losses, train_acc, val_acc = train_model(model_ft, criterion, optimizer_ft, exp_lr_scheduler, num_epochs=num_epochs)
test_loss, test_acc, conf_mat = test_model(model_ft)
```
其中,train_losses、val_losses、train_acc和val_acc分别为训练集和验证集上的损失和准确率数组,test_loss、test_acc和conf_mat为测试集上的损失、准确率和混淆矩阵。
我们可以使用matplotlib绘制准确率图和损失图:
```
import matplotlib.pyplot as plt
plt.plot(train_losses, label='Training loss')
plt.plot(val_losses, label='Validation loss')
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend()
plt.show()
plt.plot(train_acc, label='Training accuracy')
plt.plot(val_acc, label='Validation accuracy')
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend()
plt.show()
```
混淆矩阵可以使用seaborn绘制:
```
import seaborn as sns
sns.heatmap(conf_mat, annot=True, cmap='Blues', xticklabels=class_names, yticklabels=class_names)
plt.xlabel('Predicted')
plt.ylabel('Actual')
plt.title('Confusion Matrix')
plt.show()
```
这样就完成了用pytorch和VGG16处理jaffe数据集的代码,并且给出了准确率图,损失图和混淆矩阵。
阅读全文