Exercise:Vectorization

From Ufldl

Jump to: navigation, search
Line 1: Line 1:
== Vectorization ==
== Vectorization ==
-
In the previous problem set, we implemented a sparse autoencoder for patches taken from natural images. In this problem set, you will adapt the sparse autoencoder to work on images of handwritten digits. You will be given a working but unvectorized implementation, and your task will be to vectorize a key step to improve its performance.
+
In the previous problem set, we implemented a sparse autoencoder for patches taken from natural images. In this problem set, you will adapt your sparse autoencoder to work on images of handwritten digits.
-
In the file <tt>vec_assign.zip</tt>, you will find MATLAB code implementing a sparse autoencoder. To run the code, you will need to download an additional data set from the [http://yann.lecun.com/exdb/mnist/ MNIST handwritten digit database]. Download the file <tt>train-images-idx3-ubyte.gz</tt> and decompress it to the <tt>MNIST/</tt> folder in the project path. After obtaining the source images, we have [[Using the MNIST Dataset | provided functions ]] help you load them up as Matlab matrices.
+
=== Support Code/Data ===
-
<!-- You should then be able to run <tt>train.m</tt> and obtain some preliminary results. -->
+
The following additional files are required for this exercise:
 +
* [http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz MNIST Dataset (Training Images)]
 +
* [[Using the MNIST Dataset | Support function for using MNIST ]]
-
=== MNIST ===
+
=== Step 1: Vectorize your Sparse Autoencoder Implementation ===
-
Use the following parameters for the MNIST dataset:
+
Using the suggestions from [[Vectorization]] and [[Neural_Network_Vectorization]], vectorize your implementation of <tt>sparseAutoencoderCost.m</tt>. In our implementation, we were able to remove all for-loops with the use of matrix operations, <tt>repmat</tt> (and/or <tt>bsxfun</tt>). A vectorized version of our code ran in under one minute on a fast computer. (Note that you do not need to vectorize the code in the other files.)
 +
 
 +
=== Step 2: Learn features for handwritten digits ===
 +
 
 +
Now that you have vectorized the code, it is easy to learn larger sets of features on medium sized images. In this part of the exercise, you will use your sparse autoencoder to learn features for handwritten digits from the MNIST dataset.
 +
 
 +
The MNIST data is available at [http://yann.lecun.com/exdb/mnist/]. Download the file <tt>train-images-idx3-ubyte.gz</tt> and decompress it to the <tt>MNIST/</tt> folder in the project path. After obtaining the source images, we have [[Using the MNIST Dataset | provided functions ]] help you load them up as Matlab matrices.
 +
 
 +
The following set of parameters worked well for us to learn good features on the MNIST dataset:
  patchSize: 28x28 patches
  patchSize: 28x28 patches
Line 15: Line 25:
  lambda = 3e-3
  lambda = 3e-3
  beta = 3
  beta = 3
-
normalizeData: linear scaling (patches = patches / 255)
 
-
The autoencoder should learn pen strokes as features. These features should start to become obvious after 400 iterations of minFunc, which takes around 20 - 25 minutes on the Corn cluster. Visualised, the features should look like in the following image:
+
After 500 iterations of updates using minFunc, your autoencoder should have learned features that resemble pen strokes. Our implementation takes around 20-25 minutes on a fast machine. Visualized, the features should look like in the following image:
[[File:MNIST-false-0.1-3e-3-3-linear.png]]
[[File:MNIST-false-0.1-3e-3-3-linear.png]]
Line 29: Line 38:
If your image looks like one of the above images, check your code and parameters again. In particular, templates of digits are not very useful as features, since they do not generalise very well to digits written differently.
If your image looks like one of the above images, check your code and parameters again. In particular, templates of digits are not very useful as features, since they do not generalise very well to digits written differently.
-
=== Natural images ===
+
<!-- === Natural images ===
Use the following parameters for the natural images dataset:
Use the following parameters for the natural images dataset:
Line 41: Line 50:
As with the first problem, the autoencoder should learn edge features. Your code should run in under 10 minutes on a reasonably fast machine. If it takes significantly longer, check your code and ensure that it is vectorized.
As with the first problem, the autoencoder should learn edge features. Your code should run in under 10 minutes on a reasonably fast machine. If it takes significantly longer, check your code and ensure that it is vectorized.
-
[[TODO]]
+
[[Category:Exercises]]
-
 
+
-
[[Category:Exercises]] [[Category:TODO]]
+

Revision as of 05:47, 28 April 2011

Personal tools