PyTorch C++ Frontend 深度学习教程：构建 DCGAN 生成 MNIST 图像

需积分: 10 72 浏览量更新于2024-07-16 收藏 4.85MB PDF 举报

"PyTorch 1.5 的官方英文教程，包含了C++前端的使用，提供了C++11 API，支持神经网络模型构建、优化算法、数据加载等机器学习核心功能。教程通过训练一个用于生成MNIST数字图像的DCGAN模型，介绍C++前端的完整流程。" PyTorch 是一个广泛使用的深度学习框架，其主要接口是基于Python的。然而，PyTorch的C++前端提供了一个纯C++11的接口，使得开发者可以在不依赖Python的情况下利用PyTorch的强大功能。这个C++前端建立在PyTorch的底层C++代码库之上，提供了基础的数据结构如张量（Tensor）和自动微分功能。 PyTorch C++前端的关键特性包括： 1. **神经网络模型组件**：内置了一套常见的神经网络模块，用于构建各种模型。 2. **自定义模块扩展**：用户可以创建自己的模块来扩展内置组件，以满足特定需求。 3. **优化算法库**：支持如随机梯度下降（Stochastic Gradient Descent, SGD）等流行的学习算法，便于模型训练。 4. **并行数据加载器**：提供API来定义和加载数据集，提高训练效率，尤其在处理大型数据集时。 5. **序列化工具**：能够保存和加载模型权重，便于模型的持久化和继续训练。 6. **更多功能**：还包括其他辅助功能，如损失函数、梯度操作等。本教程通过一个具体的例子——训练一个Discrete Convolutional Generative Adversarial Network (DCGAN)模型，来演示如何使用PyTorch C++前端进行端到端的模型训练。DCGAN是一种生成模型，它能学习到数据的分布，然后生成新的、类似训练数据的样本。在这个例子中，我们将训练DCGAN生成MNIST手写数字的图像。尽管这是一个相对简单的示例，但它足以展示PyTorch C++前端的基本用法，帮助初学者快速理解和掌握其核心概念。教程会逐步指导你完成以下步骤： 1. **环境设置**：如何配置和引入必要的库和依赖。 2. **模型定义**：如何使用C++ API构建DCGAN的生成器和判别器网络结构。 3. **数据预处理**：如何处理和加载MNIST数据集，将其转换为适合训练的格式。 4. **损失函数和优化器**：如何设置损失函数（如对抗性损失）和选择优化器（如Adam）。 5. **训练循环**：如何编写训练代码，执行前向传播、计算损失、反向传播和更新权重。 6. **模型保存与加载**：如何保存训练好的模型，以及在需要时恢复训练。 7. **生成新样本**：训练完成后，如何使用生成器生成新的MNIST样例。通过这个教程，你可以深入了解PyTorch C++前端的工作原理，以及如何在C++环境中进行深度学习模型的开发和训练。这将对那些希望在非Python环境中利用PyTorch功能的开发者非常有价值。

modules in the C++ frontend have. The simplest way to achieve both is to create an instance of torch::Device at the top

level of our training script, and then pass that device to tensor factory functions like torch::zeros as well as the to() method.

We can start by doing this with a CPU device:

// Place this somewhere at the top of your training script.

torch::Device device(torch::kCPU);

New tensor allocations like

torch::Tensor fake_labels = torch::zeros(batch.data.size(0));

should be updated to take the device as the last argument:

torch::Tensor fake_labels = torch::zeros(batch.data.size(0), device);

For tensors whose creation is not in our hands, like those coming from the MNIST dataset, we must insert explicit to() calls.

This means

torch::Tensor real_images = batch.data;

becomes

torch::Tensor real_images = batch.data.to(device);

and also our model parameters should be moved to the correct device:

generator->to(device);

discriminator->to(device);

NOTE

If a tensor already lives on the device supplied to to(), the call is a no-op. No extra copy is made.

At this point, we’ve just made our previous CPU-residing code more explicit. However, it is now also very easy to change

the device to a CUDA device:

torch::Device device(torch::kCUDA)

And now all tensors will live on the GPU, calling into fast CUDA kernels for all operations, without us having to change any

downstream code. If we wanted to specify a particular device index, it could be passed as the second argument to the

Device constructor. If we wanted different tensors to live on different devices, we could pass separate device instances (for

example one on CUDA device 0 and the other on CUDA device 1). We can even do this configuration dynamically, which is

often useful to make our training scripts more portable:

torch::Device device = torch::kCPU;

if (torch::cuda::is_available()) {

std::cout <<"CUDA is available! Training on GPU."<< std::endl;

device = torch::kCUDA;

}

or even

torch::Device device(torch::cuda::is_available() ? torch::kCUDA : torch::kCPU);

Checkpointing and Recovering the Training State

The last augmentation we should make to our training script is to periodically save the state of our model parameters, the

state of our optimizers as well as a few generated image samples. If our computer were to crash in the middle of the

training procedure, the first two will allow us to restore the training state. For long-lasting training sessions, this is absolutely

essential. Fortunately, the C++ frontend provides an API to serialize and deserialize both model and optimizer state, as well

as individual tensors.

The core API for this is torch::save(thing,filename) and torch::load(thing,filename), where thing could be a

torch::nn::Module subclass or an optimizer instance like the Adam object we have in our training script. Let’s update our

training loop to checkpoint the model and optimizer state at a certain interval:

if (batch_index % kCheckpointEvery ==0) {

// Checkpoint the model and optimizer state.

torch::save(generator, "generator-checkpoint.pt");

torch::save(generator_optimizer, "generator-optimizer-checkpoint.pt");

torch::save(discriminator, "discriminator-checkpoint.pt");

torch::save(discriminator_optimizer, "discriminator-optimizer-checkpoint.pt");

// Sample the generator and save the images.

torch::Tensor samples = generator->forward(torch::randn({8, kNoiseSize, 1, 1}, device));

torch::save((samples +1.0) /2.0, torch::str("dcgan-sample-", checkpoint_counter, ".pt"));

std::cout <<"\n-> checkpoint "<<++checkpoint_counter <<'\n';

}

where kCheckpointEvery is an integer set to something like 100 to checkpoint every 100 batches, and checkpoint_counter is a

counter bumped every time we make a checkpoint.

To restore the training state, you can add lines like these after all models and optimizers are created, but before the training

loop:

torch::optim::Adam generator_optimizer(

generator->parameters(), torch::optim::AdamOptions(2e-4).beta1(0.5));

torch::optim::Adam discriminator_optimizer(

discriminator->parameters(), torch::optim::AdamOptions(2e-4).beta1(0.5));

if (kRestoreFromCheckpoint) {

torch::load(generator, "generator-checkpoint.pt");

torch::load(generator_optimizer, "generator-optimizer-checkpoint.pt");

torch::load(discriminator, "discriminator-checkpoint.pt");

torch::load(

discriminator_optimizer, "discriminator-optimizer-checkpoint.pt");

}

int64_t checkpoint_counter =0;

for (int64_t epoch =1; epoch <= kNumberOfEpochs; ++epoch) {

int64_t batch_index =0;

for (torch::data::Example<>&batch : *data_loader) {

Inspecting Generated Images

Our training script is now complete. We are ready to train our GAN, whether on CPU or GPU. To inspect the intermediary

output of our training procedure, for which we added code to periodically save image samples to the "dcgan-sample-xxx.pt"

file, we can write a tiny Python script to load the tensors and display them with matplotlib:

from__future__import print_function

from__future__import unicode_literals

importargparseimportmatplotlib.pyplotaspltimporttorch

This tutorial has hopefully given you a digestible digest of the PyTorch C++ frontend. A machine learning library like

PyTorch by necessity has a very broad and extensive API. As such, there are many concepts we did not have time or space

to discuss here. However, I encourage you to try out the API, and consult our documentation and in particular the Library API

section when you get stuck. Also, remember that you can expect the C++ frontend to follow the design and semantics of the

Python frontend whenever we could make this possible, so you can leverage this fact to increase your learning rate.

TIP

You can find the full source code presented in this tutorial in this repository.

As always, if you run into any problems or have questions, you can use our forum or GitHub issues to get in touch.

DCGAN TUTORIAL

Author: Nathan Inkawhich

Introduction

This tutorial will give an introduction to DCGANs through an example. We will train a generative adversarial network (GAN)

to generate new celebrities after showing it pictures of many real celebrities. Most of the code here is from the dcgan

implementation in pytorch/examples, and this document will give a thorough explanation of the implementation and shed

light on how and why this model works. But don’t worry, no prior knowledge of GANs is required, but it may require a first-

timer to spend some time reasoning about what is actually happening under the hood. Also, for the sake of time it will help

to have a GPU, or two. Lets start from the beginning.

Generative Adversarial Networks

What is a GAN?

GANs are a framework for teaching a DL model to capture the training data’s distribution so we can generate new data

from that same distribution. GANs were invented by Ian Goodfellow in 2014 and first described in the paper Generative

Adversarial Nets. They are made of two distinct models, a generator and a discriminator. The job of the generator is to

spawn ‘fake’ images that look like the training images. The job of the discriminator is to look at an image and output

whether or not it is a real training image or a fake image from the generator. During training, the generator is constantly

trying to outsmart the discriminator by generating better and better fakes, while the discriminator is working to become a

better detective and correctly classify the real and fake images. The equilibrium of this game is when the generator is

generating perfect fakes that look as if they came directly from the training data, and the discriminator is left to always

guess at 50% confidence that the generator output is real or fake.

Now, lets define some notation to be used throughout tutorial starting with the discriminator. Let

x be data representing an image.

D(x)

D(x) is the discriminator network which outputs the (scalar) probability that

x came from training data rather than the generator. Here, since we are dealing with images the input to

D(x)

D(x) is an image of CHW size 3x64x64. Intuitively,

D(x)

D(x) should be HIGH when

剩余295页未读，继续阅读

qianqing13579

粉丝: 1000
资源: 86

PyTorch C++ Frontend 深度学习教程：构建 DCGAN 生成 MNIST 图像

pytorch学习视频百度网盘链接.txt

pytorch tutorials v0.4.0 官方文档PDF版

PyTorch Tutorial.pdf

PytorchTutorials

pytorch1.5官方英文文档PythonAPI(包含书签)

pytorch.rar

PyTorch介绍及入门pdf

torch-1.5.0+cpu-cp37-cp37m-win_amd64.whl

pytorch1.5.1官方英文文档PythonAPI和Library(包含书签，吐血整理)

Pytorch视频教程（基于Pytorch1.5版本，2020年录制）.rar

最新资源