UFLDL Recommended Readings

From Ufldl

Jump to: navigation, search
Line 6: Line 6:
* Olshausen and Field Sparse Coding paper (1996)  
* Olshausen and Field Sparse Coding paper (1996)  
* [http://www.cs.stanford.edu/~ang/papers/icml07-selftaughtlearning.pdf]  Rajat Raina, Alexis Battle, Honglak Lee, Benjamin Packer and Andrew Y. Ng. Self-taught learning: Transfer learning from unlabeled data. ICML 2007
* [http://www.cs.stanford.edu/~ang/papers/icml07-selftaughtlearning.pdf]  Rajat Raina, Alexis Battle, Honglak Lee, Benjamin Packer and Andrew Y. Ng. Self-taught learning: Transfer learning from unlabeled data. ICML 2007
-
* [http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf] Yoshua Bengio. Learning Deep Architectures for AI. FTML 2009. (Broad landscape description of the field, but technical details there are hard to follow so ignore that.)
+
 
Line 12: Line 12:
* [http://www.cs.toronto.edu/~hinton/science.pdf]  Hinton, G. E. and Salakhutdinov, R. R. Reducing the dimensionality of data with neural networks. Science 2006.  If you want to play with the code, you can also find it at [http://www.cs.toronto.edu/~hinton/MatlabForSciencePaper.html].  
* [http://www.cs.toronto.edu/~hinton/science.pdf]  Hinton, G. E. and Salakhutdinov, R. R. Reducing the dimensionality of data with neural networks. Science 2006.  If you want to play with the code, you can also find it at [http://www.cs.toronto.edu/~hinton/MatlabForSciencePaper.html].  
* [http://www-etud.iro.umontreal.ca/~larocheh/publications/greedy-deep-nets-nips-06.pdf] Bengio, Y., Lamblin, P., Popovici, P., Larochelle, H. Greedy Layer-Wise Training of Deep Networks. NIPS 2006  
* [http://www-etud.iro.umontreal.ca/~larocheh/publications/greedy-deep-nets-nips-06.pdf] Bengio, Y., Lamblin, P., Popovici, P., Larochelle, H. Greedy Layer-Wise Training of Deep Networks. NIPS 2006  
-
* [http://www.cs.toronto.edu/~larocheh/publications/icml-2008-denoising-autoencoders.pdf] Pascal Vincent, Hugo Larochelle, Yoshua Bengio and Pierre-Antoine Manzagol. Extracting and Composing Robust Features with Denoising Autoencoders. ICML 2008.
+
* [http://www.cs.toronto.edu/~larocheh/publications/icml-2008-denoising-autoencoders.pdf] Pascal Vincent, Hugo Larochelle, Yoshua Bengio and Pierre-Antoine Manzagol. Extracting and Composing Robust Features with Denoising Autoencoders. ICML 2008. (They have a nice model, but then backwards rationalize it into a probabilistic model.  Ignore the backwards rationalized probabilistic model.) (Someone please clarify eactly which section of the paper this is.)
Analyzing deep learning/why does deep learning work:  
Analyzing deep learning/why does deep learning work:  
Line 20: Line 20:
RBMs:
RBMs:
-
* [http://deeplearning.net/tutorial/rbm.html] Tutorial on RBMs. But ignore the Theano code examples.  
+
* [http://deeplearning.net/tutorial/rbm.html] Tutorial on RBMs. But ignore the Theano code examples. (Someone tell us if this should be moved later.  Useful for understanding some of DL literature, but not needed for many of the later papers?)
-
* A practical guide (read if you're trying to implement and RBM; but otherwise skip since this is not really a tutorial). [http://www.cs.toronto.edu/~hinton/absps/guideTR.pdf] Geoff Hinton. A practical guide to training restricted Boltzmann machines. UTML TR 2010–003.
+
Applications:
Applications:
Line 50: Line 49:
* Karlin & Lewicki Nature paper.  (someone tell us if this should be here.  Interesting algorithm + nice visualizations, though maybe slightly hard to understand.)  
* Karlin & Lewicki Nature paper.  (someone tell us if this should be here.  Interesting algorithm + nice visualizations, though maybe slightly hard to understand.)  
 +
Overview
 +
* [http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf] Yoshua Bengio. Learning Deep Architectures for AI. FTML 2009. (Broad landscape description of the field, but technical details there are hard to follow so ignore that.  This is also easier to read after you've gone over some of literature of the field.)
 +
 +
 +
Practical guides:
 +
* A practical guide (read if you're trying to implement and RBM; but otherwise skip since this is not really a tutorial). [http://www.cs.toronto.edu/~hinton/absps/guideTR.pdf] Geoff Hinton. A practical guide to training restricted Boltzmann machines. UTML TR 2010–003.
 +
* Efficient backprop by LeCun.  Read if you're trying to run backprop; but otherwise skip since very low-level engineering/hackery tricks and not that satisfying to read.

Revision as of 02:32, 1 March 2011

Personal tools