Exercise:Learning color features with Sparse Autoencoders
From Ufldl
(→Step 0: Initialization) |
(→Step 1: Modify your sparse autoencoder to use a linear decoder) |
||
Line 9: | Line 9: | ||
=== Dependencies === | === Dependencies === | ||
- | + | You will need: | |
- | + | ||
- | + | ||
- | + | ||
- | You will | + | |
* <tt>sparseAutoencoderCost.m</tt> (and related functions) from [[Exercise:Sparse Autoencoder]] | * <tt>sparseAutoencoderCost.m</tt> (and related functions) from [[Exercise:Sparse Autoencoder]] | ||
+ | |||
+ | The following additional file is also required for this exercise: | ||
+ | * [http://ufldl.stanford.edu/wiki/resources/stl10_patches_100k.zip Sampled 8x8 patches from the STL-10 dataset (stl10_patches_100k.zip)] | ||
''If you have not completed the exercise listed above, we strongly suggest you complete it first.'' | ''If you have not completed the exercise listed above, we strongly suggest you complete it first.'' | ||
Line 30: | Line 29: | ||
=== Step 1: Modify your sparse autoencoder to use a linear decoder === | === Step 1: Modify your sparse autoencoder to use a linear decoder === | ||
- | Copy <tt> | + | Copy <tt>sparseAutoencoderCost.m</tt> to the directory for this exercise and rename it to <tt>sparseAutoencoderLinearCost.m</tt>. Rename the function <tt>sparseAutoencoderCost</tt> in the file to <tt>sparseAutoencoderLinearCost</tt>, and modify it to use a [[Linear Decoders | linear decoder]]. In particular, you should change the cost and gradients returned to reflect the change from a sigmoid to a linear decoder. After making this change, check your gradients to ensure that they are correct. |
=== Step 2: Learn features on small patches === | === Step 2: Learn features on small patches === | ||
- | You will now use your sparse autoencoder to learn features on a set of 100 000 small 8x8 patches sampled from the larger 96x96 | + | You will now use your sparse autoencoder to learn features on a set of 100,000 small 8x8 patches sampled from the larger 96x96 STL-10 images (The [http://www.stanford.edu/~acoates//stl10/ STL-10 dataset] comprises 5000 training and 8000 test examples, with each example being a 96x96 labelled color image belonging to one of ten classes: airplane, bird, car, cat, deer, dog, horse, monkey, ship, truck.) |
- | The code provided in this step trains your sparse autoencoder for 400 iterations with the default parameters initialized in step 0. This should take around 45 minutes. Your sparse autoencoder should learn features which when visualized, look like edges and opponent colors, as in the figure below. | + | The code provided in this step trains your sparse autoencoder for 400 iterations with the default parameters initialized in step 0. This should take around 45 minutes. Your sparse autoencoder should learn features which when visualized, look like edges and "opponent colors," as in the figure below. |
[[File:CNN_Features_Good.png|480px]] | [[File:CNN_Features_Good.png|480px]] | ||
- | If your parameters are improperly tuned (the default parameters should work), or if your implementation of the autoencoder is buggy, you might get one of the following | + | If your parameters are improperly tuned (the default parameters should work), or if your implementation of the autoencoder is buggy, you might instead get images that look like one of the following: |
<table cellpadding=5px> | <table cellpadding=5px> |