Exercise:Vectorization
From Ufldl
for
Exercise:Vectorization
Jump to:
navigation
,
search
== Vectorization == In the previous problem set, we implemented a sparse autoencoder for patches taken from natural images. In this problem set, you will adapt your sparse autoencoder to work on images of handwritten digits. === Support Code/Data === The following additional files are required for this exercise: * [http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz MNIST Dataset (Training Images)] * [[Using the MNIST Dataset | Support functions for loading MNIST in Matlab ]] === Step 1: Vectorize your Sparse Autoencoder Implementation === Using the suggestions from [[Vectorization]] and [[Neural Network Vectorization]], vectorize your implementation of <tt>sparseAutoencoderCost.m</tt>. In our implementation, we were able to remove all for-loops with the use of matrix operations, <tt>repmat</tt> (and/or <tt>bsxfun</tt>). A vectorized version of our code ran in under one minute on a fast computer (for learning 25 features from 10000 8x8 image patches). (Note that you do not need to vectorize the code in the other files.) === Step 2: Learn features for handwritten digits === Now that you have vectorized the code, it is easy to learn larger sets of features on medium sized images. In this part of the exercise, you will use your sparse autoencoder to learn features for handwritten digits from the MNIST dataset. The MNIST data is available at [http://yann.lecun.com/exdb/mnist/]. Download the file <tt>train-images-idx3-ubyte.gz</tt> and decompress it. After obtaining the source images, we have [[Using the MNIST Dataset | provided functions ]] help you load them up as Matlab matrices. While the provided functions allow you to load up both the labels and data, for this assignment, you will only need the data since the training is ''unsupervised''. The following set of parameters worked well for us to learn good features on the MNIST dataset: visibleSize = 28*28 sparsityParam = 0.1 lambda = 3e-3 beta = 3 patches = first 10000 images from the MNIST dataset After 400 iterations of updates using minFunc, your autoencoder should have learned features that resemble pen strokes. Our implementation takes around 25-30 minutes on a fast machine. Visualized, the features should look like in the following image: [[File:mnistVectorizationEx.png|thumb]] If your parameters are improperly tuned, or if your implementation of the autoencoder is buggy, you may get one of the following images instead: <table> <tr><td>[[File:MNIST-false-bad-1.png|240px]]</td><td>[[File:MNIST-false-bad-2.png|240px]]</td></tr> </table> If your image looks like one of the above images, check your code and parameters again. Learning these features are a prelude to the later exercises, where we shall see how they will be useful for classification. <!-- === Natural images === Use the following parameters for the natural images dataset: visibleSize = 14*14; hiddenSize = 196; sparsityParam = 0.035; lambda = 0.0003; beta = 5; As with the first problem, the autoencoder should learn edge features. Your code should run in under 10 minutes on a reasonably fast machine. If it takes significantly longer, check your code and ensure that it is vectorized. [[Category:Exercises]]
Template:Vectorized Implementation
(
view source
)
Return to
Exercise:Vectorization
.
Views
Page
Discussion
View source
History
Personal tools
Log in
ufldl resources
UFLDL Tutorial
Recommended Readings
wiki
Main page
Recent changes
Random page
Help
Search
Toolbox
What links here
Related changes
Special pages