基于pytorch框架对五十五列excel数据进行预处理,包括数据导入,数据标准化,数据集划分等,然后建立多尺度1DCNN神经网络进行训练和验证,并绘制ACC、loss曲线混淆矩阵
时间: 2024-05-07 14:21:02 浏览: 129
首先,需要安装以下依赖库:
- pandas
- numpy
- sklearn
- matplotlib
- torch
然后,可以按照以下步骤进行预处理和建立多尺度1DCNN神经网络进行训练和验证,并绘制ACC、loss曲线混淆矩阵:
1. 数据导入
使用pandas库中的read_excel()函数读取excel文件中的数据,将其转换为DataFrame格式,如下所示:
``` python
import pandas as pd
data = pd.read_excel('data.xlsx')
```
2. 数据标准化
将数据标准化可以提高模型的性能和训练速度。可以使用sklearn库中的StandardScaler进行标准化,如下所示:
``` python
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
data = scaler.fit_transform(data)
```
3. 数据集划分
将数据集划分为训练集、验证集和测试集,可以使用sklearn库中的train_test_split()函数,如下所示:
``` python
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(data[:, :-1], data[:, -1], test_size=0.2, random_state=42)
X_train, X_val, y_train, y_val = train_test_split(X_train, y_train, test_size=0.2, random_state=42)
```
其中,X_train、X_val和X_test分别是训练集、验证集和测试集的特征数据,y_train、y_val和y_test分别是训练集、验证集和测试集的标签数据。
4. 建立多尺度1DCNN神经网络
建立多尺度1DCNN神经网络可以使用PyTorch框架,如下所示:
``` python
import torch
import torch.nn as nn
import torch.optim as optim
class MultiScale1DCNN(nn.Module):
def __init__(self):
super(MultiScale1DCNN, self).__init__()
self.conv1 = nn.Conv1d(in_channels=55, out_channels=32, kernel_size=3, stride=1, padding=1)
self.pool1 = nn.MaxPool1d(kernel_size=2)
self.conv2 = nn.Conv1d(in_channels=32, out_channels=64, kernel_size=3, stride=1, padding=1)
self.pool2 = nn.MaxPool1d(kernel_size=2)
self.conv3 = nn.Conv1d(in_channels=64, out_channels=128, kernel_size=3, stride=1, padding=1)
self.pool3 = nn.MaxPool1d(kernel_size=2)
self.fc1 = nn.Linear(in_features=128*6, out_features=256)
self.fc2 = nn.Linear(in_features=256, out_features=1)
self.dropout = nn.Dropout(p=0.5)
def forward(self, x):
x1 = self.pool1(torch.relu(self.conv1(x)))
x2 = self.pool2(torch.relu(self.conv2(x1)))
x3 = self.pool3(torch.relu(self.conv3(x2)))
x4 = x3.view(-1, 128*6)
x5 = self.dropout(torch.relu(self.fc1(x4)))
x6 = torch.sigmoid(self.fc2(x5))
return x6
model = MultiScale1DCNN()
```
其中,MultiScale1DCNN是一个继承自nn.Module的类,用于定义多尺度1DCNN神经网络的结构。该网络包括三个卷积层和两个全连接层,其中每个卷积层后面都接一个最大池化层,每个全连接层后面都接一个dropout层,最后一个全连接层的输出通过sigmoid函数进行激活,得到二分类概率值。在每个卷积层和全连接层之间都使用relu激活函数。
5. 训练和验证
使用PyTorch框架对多尺度1DCNN神经网络进行训练和验证,如下所示:
``` python
criterion = nn.BCELoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)
train_losses, val_losses, train_accs, val_accs = [], [], [], []
for epoch in range(100):
# 训练
train_loss, train_acc = 0, 0
model.train()
for i in range(len(X_train)):
inputs = torch.tensor([X_train[i]], dtype=torch.float32)
labels = torch.tensor([y_train[i]], dtype=torch.float32)
optimizer.zero_grad()
outputs = model(inputs)
loss = criterion(outputs, labels)
loss.backward()
optimizer.step()
train_loss += loss.item()
train_acc += (outputs.round() == labels).sum().item()
train_loss /= len(X_train)
train_acc /= len(X_train)
train_losses.append(train_loss)
train_accs.append(train_acc)
# 验证
val_loss, val_acc = 0, 0
model.eval()
with torch.no_grad():
for i in range(len(X_val)):
inputs = torch.tensor([X_val[i]], dtype=torch.float32)
labels = torch.tensor([y_val[i]], dtype=torch.float32)
outputs = model(inputs)
loss = criterion(outputs, labels)
val_loss += loss.item()
val_acc += (outputs.round() == labels).sum().item()
val_loss /= len(X_val)
val_acc /= len(X_val)
val_losses.append(val_loss)
val_accs.append(val_acc)
print('Epoch [{}/{}], train_loss: {:.4f}, train_acc: {:.4f}, val_loss: {:.4f}, val_acc: {:.4f}'
.format(epoch+1, 100, train_loss, train_acc, val_loss, val_acc))
```
其中,使用BCELoss作为损失函数,Adam作为优化器。每个epoch训练完后,需要计算训练集和验证集的loss和accuracy,并将它们保存在train_losses、val_losses、train_accs和val_accs列表中。
6. 绘制ACC、loss曲线和混淆矩阵
使用matplotlib库可以绘制训练集和验证集的ACC、loss曲线,如下所示:
``` python
import matplotlib.pyplot as plt
plt.figure(figsize=(10, 4))
plt.subplot(1, 2, 1)
plt.plot(train_losses, label='Training loss')
plt.plot(val_losses, label='Validation loss')
plt.legend(frameon=False)
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.subplot(1, 2, 2)
plt.plot(train_accs, label='Training acc')
plt.plot(val_accs, label='Validation acc')
plt.legend(frameon=False)
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.show()
```
使用sklearn库可以绘制混淆矩阵,如下所示:
``` python
from sklearn.metrics import confusion_matrix
model.eval()
with torch.no_grad():
y_pred = []
for i in range(len(X_test)):
inputs = torch.tensor([X_test[i]], dtype=torch.float32)
outputs = model(inputs)
y_pred.append(outputs.round().item())
cm = confusion_matrix(y_test, y_pred)
print(cm)
```
其中,y_pred是模型在测试集上的预测结果,使用confusion_matrix()函数可以计算混淆矩阵,并将其打印出来。
阅读全文