针对两个图片文件夹进行以下操作的代码编写 1. 编写数据加载函数; 2. 根据任务要求对数据集进行划分; 3. 数据增强; 4. 构建深度学习模型和损失函数; 5. 编写模型训练相关代码,完成模型训练 6. 使用可视化库 Matplotlib 对训练过程进行可视化展示,如损失值变化、准确率变化、召回率、ROC曲线等
时间: 2024-02-03 17:15:51 浏览: 65
以下是基于Python和PyTorch框架的代码实现,用于两个图片文件夹的分类任务。
### 1. 数据加载函数
```python
import os
import torch
from torchvision import transforms, datasets
def load_data(data_dir):
train_transforms = transforms.Compose([transforms.RandomRotation(30),
transforms.RandomResizedCrop(224),
transforms.RandomHorizontalFlip(),
transforms.ToTensor(),
transforms.Normalize([0.485, 0.456, 0.406],
[0.229, 0.224, 0.225])])
test_transforms = transforms.Compose([transforms.Resize(256),
transforms.CenterCrop(224),
transforms.ToTensor(),
transforms.Normalize([0.485, 0.456, 0.406],
[0.229, 0.224, 0.225])])
train_dir = os.path.join(data_dir, 'train')
test_dir = os.path.join(data_dir, 'test')
train_data = datasets.ImageFolder(train_dir, transform=train_transforms)
test_data = datasets.ImageFolder(test_dir, transform=test_transforms)
trainloader = torch.utils.data.DataLoader(train_data, batch_size=64, shuffle=True)
testloader = torch.utils.data.DataLoader(test_data, batch_size=64)
return trainloader, testloader
```
### 2. 数据集划分
```python
data_dir = 'path/to/data'
trainloader, testloader = load_data(data_dir)
```
### 3. 数据增强
数据增强在数据加载函数中已经完成,包括了随机旋转、裁剪、水平翻转等操作。
### 4. 构建深度学习模型和损失函数
```python
import torch.nn as nn
import torch.nn.functional as F
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
self.conv1 = nn.Conv2d(3, 16, 3, padding=1)
self.conv2 = nn.Conv2d(16, 32, 3, padding=1)
self.conv3 = nn.Conv2d(32, 64, 3, padding=1)
self.pool = nn.MaxPool2d(2, 2)
self.fc1 = nn.Linear(64 * 28 * 28, 500)
self.fc2 = nn.Linear(500, 2)
self.dropout = nn.Dropout(0.25)
def forward(self, x):
x = self.pool(F.relu(self.conv1(x)))
x = self.pool(F.relu(self.conv2(x)))
x = self.pool(F.relu(self.conv3(x)))
x = x.view(-1, 64 * 28 * 28)
x = self.dropout(x)
x = F.relu(self.fc1(x))
x = self.dropout(x)
x = self.fc2(x)
return x
model = Net()
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
```
### 5. 模型训练
```python
import time
start = time.time()
epochs = 20
steps = 0
train_losses, test_losses = [], []
for e in range(epochs):
running_loss = 0
for images, labels in trainloader:
steps += 1
optimizer.zero_grad()
output = model(images)
loss = criterion(output, labels)
loss.backward()
optimizer.step()
running_loss += loss.item()
else:
test_loss = 0
accuracy = 0
with torch.no_grad():
model.eval()
for images, labels in testloader:
log_ps = model(images)
test_loss += criterion(log_ps, labels)
ps = torch.exp(log_ps)
top_p, top_class = ps.topk(1, dim=1)
equals = top_class == labels.view(*top_class.shape)
accuracy += torch.mean(equals.type(torch.FloatTensor))
model.train()
train_losses.append(running_loss / len(trainloader))
test_losses.append(test_loss / len(testloader))
print("Epoch: {}/{}.. ".format(e+1, epochs),
"Training Loss: {:.3f}.. ".format(train_losses[-1]),
"Test Loss: {:.3f}.. ".format(test_losses[-1]),
"Test Accuracy: {:.3f}".format(accuracy/len(testloader)))
print(f"Training time: {(time.time() - start)/60:.3f} minutes")
```
### 6. 可视化展示
```python
import matplotlib.pyplot as plt
%matplotlib inline
plt.plot(train_losses, label='Training loss')
plt.plot(test_losses, label='Validation loss')
plt.legend(frameon=False)
plt.show()
```
以上是一个简单的图片分类任务的实现,其中包括了数据加载函数、数据集划分、数据增强、模型构建、损失函数定义、模型训练和可视化展示等步骤。
阅读全文