18
Step 1: Generate training set
The first step is to generate a training set. To get a single training example x, randomly pick
one of the 10 images, then randomly sample an 8×8 image patch from the selected image,
and convert the image patch (either in row-major order or column-major order; it doesn't matter)
into a 64-dimensional vector to get a training example
Complete the code in sampleIMAGES.m. Your code should sample 10000 image patches and
concatenate them into a 64×10000 matrix.
To make sure your implementation is working, run the code in "Step 1" of train.m. This should
result in a plot of a random sample of 200 patches from the dataset.
Implementational tip: When we run our implemented sampleImages(), it takes under 5
seconds. If your implementation takes over 30 seconds, it may be because you are
accidentally making a copy of an entire 512×512 image each time you're picking a random
image. By copying a 512×512 image 10000 times, this can make your implementation much
less efficient. While this doesn't slow down your code significantly for this exercise (because
we have only 10000 examples), when we scale to much larger problems later this quarter
with 10
6
or more examples, this will significantly slow down your code. Please
implement sampleIMAGES so that you aren't making a copy of an entire 512×512 image each
time you need to cut out an 8x8 image patch.
Step 2: Sparse autoencoder objective
Implement code to compute the sparse autoencoder cost function J
sparse
(W,b) (Section 3 of the
lecture notes) and the corresponding derivatives of J
sparse
with respect to the different
parameters. Use the sigmoid function for the activation function, . In
particular, complete the code insparseAutoencoderCost.m.
The sparse autoencoder is parameterized by
matrices , vectors , .
However, for subsequent notational convenience, we will "unroll" all of these parameters into a
very long parameter vector θ with s
1
s
2
+ s
2
s
3
+ s
2
+ s
3
elements. The code for converting
between the(W
(1)
,W
(2)
,b
(1)
,b
(2)
) and the θ parameterization is already provided in the starter
code.
Implementational tip: The objective J
sparse
(W,b) contains 3 terms, corresponding to the
squared error term, the weight decay term, and the sparsity penalty. You're welcome to
implement this however you want, but for ease of debugging, you might implement the cost
function and derivative computation (backpropagation) only for the squared error term first
我的内容管理
展开
