autoencoder异常检测python实战代码
时间: 2023-08-06 08:11:05 浏览: 65
以下是一个简单的 autoencoder 异常检测的 Python 实战代码,用于检测 MNIST 数据集中的异常样本:
```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from keras.layers import Input, Dense
from keras.models import Model
# 加载 MNIST 数据集
from keras.datasets import mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
# 数据预处理
x_train = x_train.astype('float32') / 255.
x_test = x_test.astype('float32') / 255.
x_train = x_train.reshape((len(x_train), np.prod(x_train.shape[1:])))
x_test = x_test.reshape((len(x_test), np.prod(x_test.shape[1:])))
# 构建 autoencoder 模型
input_img = Input(shape=(784,))
encoded = Dense(32, activation='relu')(input_img)
decoded = Dense(784, activation='sigmoid')(encoded)
autoencoder = Model(input_img, decoded)
autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')
# 训练 autoencoder 模型
autoencoder.fit(x_train, x_train,
epochs=50,
batch_size=256,
shuffle=True,
validation_data=(x_test, x_test))
# 获取编码器模型
encoder = Model(input_img, encoded)
# 获取测试集的重构误差
x_test_encoded = encoder.predict(x_test)
x_test_decoded = autoencoder.predict(x_test)
mse = np.mean(np.power(x_test - x_test_decoded, 2), axis=1)
# 设置阈值
threshold = np.max(mse)
# 检测异常样本
x_test_normal = x_test[mse <= threshold]
x_test_abnormal = x_test[mse > threshold]
print('Number of normal samples:', len(x_test_normal))
print('Number of abnormal samples:', len(x_test_abnormal))
# 展示一些异常样本
n = 10
plt.figure(figsize=(20, 4))
for i in range(n):
# 原始图像
ax = plt.subplot(2, n, i + 1)
plt.imshow(x_test_abnormal[i].reshape(28, 28))
plt.gray()
ax.get_xaxis().set_visible(False)
ax.get_yaxis().set_visible(False)
# 重构图像
ax = plt.subplot(2, n, i + 1 + n)
plt.imshow(x_test_decoded[mse > threshold][i].reshape(28, 28))
plt.gray()
ax.get_xaxis().set_visible(False)
ax.get_yaxis().set_visible(False)
plt.show()
```
在上面的代码中,首先加载了 MNIST 数据集并进行了数据预处理。然后构建了一个简单的 autoencoder 模型,并用训练集进行训练。接着,获取编码器模型和测试集的重构误差。最后,根据重构误差的阈值检测异常样本,并展示一些异常样本的原始图像和重构图像。