Lasagne 0.2dev教程：快速入门与API详解

需积分: 12 104 浏览量更新于2024-07-21 收藏 561KB PDF 举报

Lasagne参考教程是针对Lasagne库的一个详尽指南，它是一个基于Python中的numpy、scipy和Theano构建的轻量级工具，专用于在Theano环境中设计和训练神经网络。该教程适用于Lasagne 0.2dev版本，发布日期为2016年2月2日，适合初学者和开发者使用。在本教程中，用户将学到如何： 1. **安装** - Lasagne依赖于一些基本库，包括numpy、scipy和Theano，因此首先需要确保这些环境已经安装并配置好。此外，由于是工作中的项目，可能需要访问GitHub获取最新更新和贡献指南。 2. **入门与教程** - 指南从安装开始，逐步引导读者通过实际例子理解如何使用Lasagne创建、配置和训练神经网络模型。这包括了基础架构搭建、数据预处理和训练过程。 3. **层（Layers）** - 介绍了Lasagne的核心组件，即各种类型的层（如全连接层、卷积层、池化层等），它们在构建神经网络时扮演关键角色。学习如何组合这些层以形成复杂的网络结构。 4. **自定义层（Creating custom layers）** - 鼓励用户扩展库的功能，通过继承Lasagne的基本层类来创建自己特有的网络层，适应特定的应用场景。 5. **开发支持** - 提供了关于如何参与项目开发的信息，包括代码贡献、问题追踪和协作流程，这对于希望改进或扩展库功能的开发者来说非常重要。 6. **API参考** - 整理了拉斯agne库的关键模块，如lasagne.layers（包含各种层的详细API）、lasagne.updates（优化算法）、lasagne.init（初始化函数）、lasagne.nonlinearities（激活函数）、lasagne.objectives（损失函数）和lasagne.regularization（正则化方法）等。每个部分都详细列出了函数和类的使用方法和参数。 7. **索引与表格** - 提供了目录索引和模块索引，便于查找特定功能和信息。 8. **附录** - 包括参考文献和Python模块索引，为深入研究和进一步学习提供额外资源。这个Lasagne参考教程是一份全面且实用的指南，无论你是想要快速入门神经网络开发还是深入理解Lasagne库的工作原理，都将是一个重要的学习资源。

lasagne Documentation, Release 0.2.dev1

The ﬁrst line loads the inputs and targets of the MNIST dataset as numpy arrays, split into training, validation and test

data. The next two statements deﬁne symbolic Theano variables that will represent a mini-batch of inputs and targets

in all the Theano expressions we will generate for network training and inference. They are not tied to any data yet,

but their dimensionality and data type is ﬁxed already and matches the actual inputs and targets we will process later.

Finally, we call one of the three functions for building the Lasagne network, depending on the ﬁrst command line

argument – we’ve just removed command line handling here for clarity. Note that we hand the symbolic input variable

to build_mlp() so it will be linked to the network’s input layer.

Loss and update expressions

Continuing, we create a loss expression to be minimized in training:

prediction = lasagne.layers.get_output(network)

loss = lasagne.objectives.categorical_crossentropy(prediction, target_var)

loss = loss.mean()

The ﬁrst step generates a Theano expression for the network output given the input variable linked to the network’s

input layer(s). The second step deﬁnes a Theano expression for the categorical cross-entropy loss between said network

output and the targets. Finally, as we need a scalar loss, we simply take the mean over the mini-batch. Depending on

the problem you are solving, you will need different loss functions, see lasagne.objectives for more.

Having the model and the loss function deﬁned, we create update expressions for training the network. An update

expression describes how to change the trainable parameters of the network at each presented mini-batch. We will

use Stochastic Gradient Descent (SGD) with Nesterov momentum here, but the lasagne.updates module offers

several others you can plug in instead:

params = lasagne.layers.get_all_params(network, trainable=True)

updates = lasagne.updates.nesterov_momentum(

loss, params, learning_rate=0.01, momentum=0.9)

The ﬁrst step collects all Theano SharedVariable instances making up the trainable parameters of the layer, and

the second step generates an update expression for each parameter.

For monitoring progress during training, after each epoch, we evaluate the network on the validation set. We need a

slightly different loss expression for that:

test_prediction = lasagne.layers.get_output(network, deterministic=True)

test_loss = lasagne.objectives.categorical_crossentropy(test_prediction,

target_var)

test_loss = test_loss.mean()

The crucial difference is that we pass deterministic=True to the get_output call. This causes all nonde-

terministic layers to switch to a deterministic implementation, so in our case, it disables the dropout layers. As an

additional monitoring quantity, we create an expression for the classiﬁcation accuracy:

test_acc = T.mean(T.eq(T.argmax(test_prediction, axis=1), target_var),

dtype=theano.config.floatX)

It also builds on the deterministic test_prediction expression.

Compilation

Equipped with all the necessary Theano expressions, we’re now ready to compile a function performing a training

step:

12 Chapter 1. User Guide

lasagne Documentation, Release 0.2.dev1

train_fn = theano.function([input_var, target_var], loss, updates=updates)

This tells Theano to generate and compile a function taking two inputs – a mini-batch of images and a vector of

corresponding targets – and returning a single output: the training loss. Additionally, each time it is invoked, it applies

all parameter updates in the updates dictionary, thus performing a gradient descent step with Nesterov momentum.

For validation, we compile a second function:

val_fn = theano.function([input_var, target_var], [test_loss, test_acc])

This one also takes a mini-batch of images and targets, then returns the (deterministic) loss and classiﬁcation accuracy,

not performing any updates.

Training loop

We’re ﬁnally ready to write the training loop. In essence, we just need to do the following:

for epoch in range(num_epochs):

for batch in iterate_minibatches(X_train, y_train, 500, shuffle=True):

inputs, targets = batch

train_fn(inputs, targets)

This uses our dataset iteration helper function to iterate over the training data in random order, in mini-batches of 500

items each, for num_epochs epochs, and calls the training function we compiled to perform an update step of the

network parameters.

But to be able to monitor the training progress, we capture the training loss, compute the validation loss and print some

information to the console every time an epoch ﬁnishes:

for epoch in range(num_epochs):

# In each epoch, we do a full pass over the training data:

train_err = 0

train_batches = 0

start_time = time.time()

for batch in iterate_minibatches(X_train, y_train, 500, shuffle=True):

inputs, targets = batch

train_err += train_fn(inputs, targets)

train_batches += 1

# And a full pass over the validation data:

val_err = 0

val_acc = 0

val_batches = 0

for batch in iterate_minibatches(X_val, y_val, 500, shuffle=False):

inputs, targets = batch

err, acc = val_fn(inputs, targets)

val_err += err

val_acc += acc

val_batches += 1

# Then we print the results for this epoch:

print("Epoch {} of {} took {:.3f}s".format(

epoch + 1, num_epochs, time.time() - start_time))

print(" training loss:\t\t{:.6f}".format(train_err / train_batches))

print(" validation loss:\t\t{:.6f}".format(val_err / val_batches))

print(" validation accuracy:\t\t{:.2f} %".format(

val_acc / val_batches

100))

1.2. Tutorial 13

lasagne Documentation, Release 0.2.dev1

At the very end, we re-use the val_fn() function to compute the loss and accuracy on the test set, ﬁnishing the

script.

1.2.4 Where to go from here

This ﬁnishes our introductory tutorial. For more information on what you can do with Lasagne’s layers,

just continue reading through Layers and Creating custom layers. More tutorials, examples and code snip-

pets can be found in the Lasagne Recipes repository. Finally, the reference lists and explains all layers

(lasagne.layers), weight initializers (lasagne.init), nonlinearities (lasagne.nonlinearities),

loss expressions (lasagne.objectives), training methods (lasagne.updates) and regularizers

(lasagne.regularization) included in the library, and should also make it simple to create your own.

1.3 Layers

The lasagne.layers module provides various classes representing the layers of a neural network. All of them are

subclasses of the lasagne.layers.Layer base class.

1.3.1 Creating a layer

A layer can be created as an instance of a Layer subclass. For example, a dense layer can be created as follows:

>>> import lasagne

>>> l = lasagne.layers.DenseLayer(l_in, num_units=100)

This will create a dense layer with 100 units, connected to another layer l_in.

1.3.2 Creating a network

Note that for almost all types of layers, you will have to specify one or more other layers that the layer you are creating

gets its input from. The main exception is InputLayer, which can be used to represent the input of a network.

Chaining layer instances together like this will allow you to specify your desired network structure. Note that the

same layer can be used as input to multiple other layers, allowing for arbitrary tree and directed acyclic graph (DAG)

structures.

Here is an example of an MLP with a single hidden layer:

>>> import theano.tensor as T

>>> l_in = lasagne.layers.InputLayer((100, 50))

>>> l_hidden = lasagne.layers.DenseLayer(l_in, num_units=200)

>>> l_out = lasagne.layers.DenseLayer(l_hidden, num_units=10,

... nonlinearity=T.nnet.softmax)

The ﬁrst layer of the network is an InputLayer, which represents the input. When creating an input layer, you should

specify the shape of the input data. In this example, the input is a matrix with shape (100, 50), representing a batch

of 100 data points, where each data point is a vector of length 50. The ﬁrst dimension of a tensor is usually the batch

dimension, following the established Theano and scikit-learn conventions.

The hidden layer of the network is a dense layer with 200 units, taking its input from the input layer. Note that we did

not specify the nonlinearity of the hidden layer. A layer with rectiﬁed linear units will be created by default.

The output layer of the network is a dense layer with 10 units and a softmax nonlinearity, allowing for 10-way classi-

ﬁcation of the input vectors.

14 Chapter 1. User Guide

lasagne Documentation, Release 0.2.dev1

Note also that we did not create any object representing the entire network. Instead, the output layer instance l_out is

also used to refer to the entire network in Lasagne.

1.3.3 Naming layers

For convenience, you can name a layer by specifying the name keyword argument:

>>> l_hidden = lasagne.layers.DenseLayer(l_in, num_units=200,

... name="hidden_layer")

1.3.4 Initializing parameters

Many types of layers, such as DenseLayer, have trainable parameters. These are referred to by short names that

match the conventions used in modern deep learning literature. For example, a weight matrix will usually be called W,

and a bias vector will usually be b.

When creating a layer with trainable parameters, Theano shared variables will be created for them and initialized

automatically. You can optionally specify your own initialization strategy by using keyword arguments that match the

parameter variable names. For example:

>>> l = lasagne.layers.DenseLayer(l_in, num_units=100,

... W=lasagne.init.Normal(0.01))

The weight matrix W of this dense layer will be initialized using samples from a normal distribution with standard

deviation 0.01 (see lasagne.init for more information).

There are several ways to manually initialize parameters:

• Theano shared variable If a shared variable instance is provided, this is used unchanged as the parameter

variable. For example:

>>> import theano

>>> import numpy as np

>>> W = theano.shared(np.random.normal(0, 0.01, (50, 100)))

>>> l = lasagne.layers.DenseLayer(l_in, num_units=100, W=W)

• numpy array If a numpy array is provided, a shared variable is created and initialized using the array. For

example:

>>> W_init = np.random.normal(0, 0.01, (50, 100))

>>> l = lasagne.layers.DenseLayer(l_in, num_units=100, W=W_init)

• callable If a callable is provided (e.g. a function or a lasagne.init.Initializer instance), a shared

variable is created and the callable is called with the desired shape to generate suitable initial parameter

values. The variable is then initialized with those values. For example:

>>> l = lasagne.layers.DenseLayer(l_in, num_units=100,

... W=lasagne.init.Normal(0.01))

Or, using a custom initialization function:

>>> def init_W(shape):

... return np.random.normal(0, 0.01, shape)

>>> l = lasagne.layers.DenseLayer(l_in, num_units=100, W=init_W)

Some types of parameter variables can also be set to None at initialization (e.g. biases). In that case, the parameter

variable will be omitted. For example, creating a dense layer without biases is done as follows:

1.3. Layers 15

lasagne Documentation, Release 0.2.dev1

>>> l = lasagne.layers.DenseLayer(l_in, num_units=100, b=None)

1.3.5 Parameter sharing

Parameter sharing between multiple layers can be achieved by using the same Theano shared variable instance for

their parameters. For example:

>>> l1 = lasagne.layers.DenseLayer(l_in, num_units=100)

>>> l2 = lasagne.layers.DenseLayer(l_in, num_units=100, W=l1.W)

These two layers will now share weights (but have separate biases).

1.3.6 Propagating data through layers

To compute an expression for the output of a single layer given its input, the get_output_for() method can be used.

To compute the output of a network, you should instead call lasagne.layers.get_output() on it. This will

traverse the network graph.

You can call this function with the layer you want to compute the output expression for:

>>> y = lasagne.layers.get_output(l_out)

In that case, a Theano expression will be returned that represents the output in function of the input variables associated

with the lasagne.layers.InputLayer instance (or instances) in the network, so given the example network

from before, you could compile a Theano function to compute its output given an input as follows:

>>> f = theano.function([l_in.input_var], lasagne.layers.get_output(l_out))

You can also specify a Theano expression to use as input as a second argument to

lasagne.layers.get_output():

>>> x = T.matrix('x')

>>> y = lasagne.layers.get_output(l_out, x)

>>> f = theano.function([x], y)

This only works when there is only a single InputLayer in the network. If there is more than one, you can specify

input expressions in a dictionary. For example, in a network with two input layers l_in1 and l_in2 and an output layer

l_out:

>>> x1 = T.matrix('x1')

>>> x2 = T.matrix('x2')

>>> y = lasagne.layers.get_output(l_out, { l_in1: x1, l_in2: x2 })

Any keyword arguments passed to get_output() are propagated to all layers. This makes it possible to control the

behavior of the entire network. The main use case for this is the deterministic keyword argument, which disables

stochastic behaviour such as dropout when set to True. This is useful because a deterministic output is desirable at

evaluation time.

>>> y = lasagne.layers.get_output(l_out, deterministic=True)

Some networks may have multiple output layers - or you may just want to compute output expressions for intermediate

layers in the network. In that case, you can pass a list of layers. For example, in a network with two output layers

l_out1 and l_out2:

>>> y1, y2 = lasagne.layers.get_output([l_out1, l_out2])

16 Chapter 1. User Guide

剩余120页未读，继续阅读

thesby

粉丝: 102

Lasagne 0.2dev教程：快速入门与API详解

lasagne：JavaScript库的压缩技巧

使用Lasagne在Theano中构建训练神经网络

深度学习：Lasagne框架下的CTC时序分类

lasagne

lasagne-vae:Lasagne重新实现可变自动编码器

Lasagne PCB layout-开源

Lasagne-CTC:千层面的CTC

ntm-lasagne:千层面的Theano神经图灵机库

Lasagne PCB布局工具：多层PCB设计开源解决方案

lasagne的adaptive dropout方法

最新资源