Exercise:Self-Taught Learning

From Ufldl

Jump to: navigation, search
(Support Code/Data)
 
Line 5: Line 5:
You will be building upon your code from the earlier exercises. First, you will train your sparse autoencoder on an "unlabeled" training dataset of handwritten digits. This produces feature that are penstroke-like. We then extract these learned features from a labeled dataset of handwritten digits. These features will then be used as inputs to the softmax classifier that you wrote in the previous exercise.  
You will be building upon your code from the earlier exercises. First, you will train your sparse autoencoder on an "unlabeled" training dataset of handwritten digits. This produces feature that are penstroke-like. We then extract these learned features from a labeled dataset of handwritten digits. These features will then be used as inputs to the softmax classifier that you wrote in the previous exercise.  
-
Concretely, for each example in the the labeled training dataset, <math>\textstyle x^{(k)}</math>, we forward propagate the example to obtain the activation of the hidden units <math>\textstyle a^{(2)}</math>. The data is now represented in term of <math>\textstyle a^{(2)}</math> used to train the softmax classifier.  
+
Concretely, for each example in the the labeled training dataset <math>\textstyle x_l</math>, we forward propagate the example to obtain the activation of the hidden units <math>\textstyle a^{(2)}</math>. We now represent this example using <math>\textstyle a^{(2)}</math> (the "replacement" representation), and use this to as the new feature representation with which to train the softmax classifier.  
-
Finally, we also extract the same features from the test dataset to obtain predictions.
+
Finally, we also extract the same features from the test data to obtain predictions.
-
We will use the  the digits 5 to 9 as an "unlabeled" dataset. while the digits 0 to 4 are used as the labeled training set.
+
In this exercise, our goal is to distinguish between the digits from 0 to 4.  We will use the digits 5 to 9 as our
 +
"unlabeled" dataset which which to learn the features; we will then use a labeled dataset with the digits 0 to 4 with
 +
which to train the softmax classifier.  
-
In the starter code, we have also provided '''<tt>stlExercise.m</tt>''' that will help walk you through the steps in this exercise.
+
In the starter code, we have provided a file '''<tt>stlExercise.m</tt>''' that will help walk you through the steps in this exercise.
=== Dependencies ===
=== Dependencies ===
Line 33: Line 35:
===Step 2: Train the sparse autoencoder===
===Step 2: Train the sparse autoencoder===
-
Next, we will train the unlabeled dataset on the sparse autoencoder, using the same <tt>sparseAutoencoderCost.m</tt> function from the previous assignments. (Use the frameworks from previous assignments to ensure that your code is working and vectorized.) The training step should take less than 25 minutes (on a reasonably fast computer). When it is completed, a visualization of pen strokes should be displayed.
+
Next, use the unlabeled data (the digits from 5 to 9) to train a sparse autoencoder, using the same <tt>sparseAutoencoderCost.m</tt> function as you had written in  the previous exercise. (From the earlier exercise, you should have a working and vectorized implementation of the sparse autoencoder.) For us, the training step took less than 25 minutes on a fast desktop. When training is complete, you should get a visualization of pen strokes like the image shown below:
[[File:selfTaughtFeatures.png]]
[[File:selfTaughtFeatures.png]]
-
The features learned by the sparse autoencoder should correspond to penstrokes.
+
Informally, the features learned by the sparse autoencoder should correspond to penstrokes.
===Step 3: Extracting features===
===Step 3: Extracting features===
-
After the sparse autoencoder is trained, we can use it to extract features from the handwritten digit images.  
+
After the sparse autoencoder is trained, you will use it to extract features from the handwritten digit images.  
-
Complete <tt>feedForwardAutoencoder.m</tt> to produce a matrix whose columns correspond to activation of the hidden layer for each example i.e. the vector <math>a^{(2)}</math> corresponding to activation of layer 2.
+
Complete <tt>feedForwardAutoencoder.m</tt> to produce a matrix whose columns correspond to activations of the hidden layer for each example, i.e., the vector <math>a^{(2)}</math> corresponding to activation of layer 2.  (Recall that we treat the inputs as layer 1).
-
After doing so, this step will use your modified function to convert the raw image data to feature unit activations.  
+
After completing this step, calling <tt>feedForwardAutoencoder.m</tt> should convert the raw image data to hidden unit activations <math>a^{(2)}</math>.
===Step 4: Training and testing the logistic regression model===
===Step 4: Training and testing the logistic regression model===
-
In this step, you should use your code from the softmax exercise (<tt>softmaxTrain.m</tt>) to train the softmax classifier using the training features (<tt>trainFeatures</tt>) and labels (<tt>trainLabels</tt>).
+
Use your code from the softmax exercise (<tt>softmaxTrain.m</tt>) to train a softmax classifier using the training set features (<tt>trainFeatures</tt>) and labels (<tt>trainLabels</tt>).
===Step 5: Classifying on the test set===
===Step 5: Classifying on the test set===
-
Finally, complete the code to make predictions on the test set (<tt>testFeatures</tt>) and see how your learned features perform! If you've done all the steps correctly, you should get an accuracy of about '''98%''' percent.
+
Finally, complete the code to make predictions on the test set (<tt>testFeatures</tt>) and see how your learned features perform! If you've done all the steps correctly, you should get an accuracy of about '''98%''' percent.
 +
 
 +
As a comparison, when ''raw pixels'' are used (instead of the learned features), we obtained a test accuracy of only around 96% (for the same train and test sets).
[[Category:Exercises]]
[[Category:Exercises]]
 +
 +
 +
{{STL}}

Latest revision as of 11:02, 26 May 2011

Personal tools