UFLDL Recommended Readings
From Ufldl
| Line 5: | Line 5: | ||
* Natural Image Statistics book, Hyvarinen et al. | * Natural Image Statistics book, Hyvarinen et al. | ||
* Olshausen and Field Sparse Coding paper (1996) | * Olshausen and Field Sparse Coding paper (1996) | ||
| - | |||
* [http://www.cs.stanford.edu/~ang/papers/icml07-selftaughtlearning.pdf] Rajat Raina, Alexis Battle, Honglak Lee, Benjamin Packer and Andrew Y. Ng. Self-taught learning: Transfer learning from unlabeled data. ICML 2007 | * [http://www.cs.stanford.edu/~ang/papers/icml07-selftaughtlearning.pdf] Rajat Raina, Alexis Battle, Honglak Lee, Benjamin Packer and Andrew Y. Ng. Self-taught learning: Transfer learning from unlabeled data. ICML 2007 | ||
| + | * [http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf] Yoshua Bengio. Learning Deep Architectures for AI. FTML 2009. (Broad landscape description of the field, but technical details there are hard to follow so ignore that.) | ||
| + | |||
Autoencoders: | Autoencoders: | ||
| + | * [http://www.cs.toronto.edu/~hinton/science.pdf] Hinton, G. E. and Salakhutdinov, R. R. Reducing the dimensionality of data with neural networks. Science 2006. If you want to play with the code, you can also find it at [http://www.cs.toronto.edu/~hinton/MatlabForSciencePaper.html]. | ||
* [http://www-etud.iro.umontreal.ca/~larocheh/publications/greedy-deep-nets-nips-06.pdf] Bengio, Y., Lamblin, P., Popovici, P., Larochelle, H. Greedy Layer-Wise Training of Deep Networks. NIPS 2006 | * [http://www-etud.iro.umontreal.ca/~larocheh/publications/greedy-deep-nets-nips-06.pdf] Bengio, Y., Lamblin, P., Popovici, P., Larochelle, H. Greedy Layer-Wise Training of Deep Networks. NIPS 2006 | ||
| - | |||
| - | |||
| - | |||
* [http://www.cs.toronto.edu/~larocheh/publications/icml-2008-denoising-autoencoders.pdf] Pascal Vincent, Hugo Larochelle, Yoshua Bengio and Pierre-Antoine Manzagol. Extracting and Composing Robust Features with Denoising Autoencoders. ICML 2008. | * [http://www.cs.toronto.edu/~larocheh/publications/icml-2008-denoising-autoencoders.pdf] Pascal Vincent, Hugo Larochelle, Yoshua Bengio and Pierre-Antoine Manzagol. Extracting and Composing Robust Features with Denoising Autoencoders. ICML 2008. | ||
| - | Deep | + | Analyzing deep learning/why does deep learning work: |
| + | * Larochelle, Erhan, Courville, Bergstra, Bengio, ICML 2007. (Someone read this and let us know if this is worth keeping,.) | ||
| + | * [http://www.jmlr.org/papers/volume11/erhan10a/erhan10a.pdf] Dumitru Erhan, Yoshua Bengio, Aaron Courville, Pierre-Antoine Manzagol, Pascal Vincent, and Samy Bengio. Why Does Unsupervised Pre-training Help Deep Learning? JMLR 2010 | ||
| + | * Goodfellow et al.'s invariance test. (Not sure if this should be included--someone let us know.) | ||
| + | |||
| + | RBMs: | ||
* [http://deeplearning.net/tutorial/rbm.html] Tutorial on RBMs. But ignore the Theano code examples. | * [http://deeplearning.net/tutorial/rbm.html] Tutorial on RBMs. But ignore the Theano code examples. | ||
| + | * A practical guide (read if you're trying to implement and RBM; but otherwise skip since this is not really a tutorial). [http://www.cs.toronto.edu/~hinton/absps/guideTR.pdf] Geoff Hinton. A practical guide to training restricted Boltzmann machines. UTML TR 2010–003. | ||
| + | |||
| + | Applications: | ||
| + | * Object Recognition | ||
| + | ** [http://www.ifp.illinois.edu/~jyang29/ScSPM.htm] Jianchao Yang, Kai Yu, Yihong Gong, Thomas Huang. Linear Spatial Pyramid Matching using Sparse Coding for Image Classification, CVPR 2009 | ||
| + | * Audio Recognition | ||
| + | ** [http://www.cs.stanford.edu/people/ang/papers/nips09-AudioConvolutionalDBN.pdf] Unsupervised feature learning for audio classification using convolutional deep belief networks, Honglak Lee, Yan Largman, Peter Pham and Andrew Y. Ng. In NIPS*2009. | ||
| - | |||
| - | |||
Natural Language Processing: | Natural Language Processing: | ||
| Line 33: | Line 42: | ||
** [http://cs.nyu.edu/~koray/publis/jarrett-iccv-09.pdf] Kevin Jarrett, Koray Kavukcuoglu, Marc'Aurelio Ranzato, and Yann LeCun, "What is the Best Multi-Stage Architecture for Object Recognition?", In ICCV 2009 | ** [http://cs.nyu.edu/~koray/publis/jarrett-iccv-09.pdf] Kevin Jarrett, Koray Kavukcuoglu, Marc'Aurelio Ranzato, and Yann LeCun, "What is the Best Multi-Stage Architecture for Object Recognition?", In ICCV 2009 | ||
| - | + | Mean-Covariance models | |
| - | * | + | * 3-way RBM |
| - | * | + | * mcRBM (someone and tell us if you need to read the 3-way RBM paper before the mcRBM one) |
| - | + | * [http://www.cs.toronto.edu/~hinton/absps/mcphone.pdf] Dahl, G., Ranzato, M., Mohamed, A. and Hinton, G. E. Phone Recognition with the Mean-Covariance Restricted Boltzmann Machine. NIPS 2010. | |
| - | + | * Karlin & Lewicki Nature paper. (someone tell us if this should be here. Interesting algorithm + nice visualizations, though maybe slightly hard to understand.) | |
| - | + | ||
| + | |||
Also, for other lists of papers: | Also, for other lists of papers: | ||
* [http://www.eecs.umich.edu/~honglak/teaching/eecs598/schedule.html] Honglak Lee's Course | * [http://www.eecs.umich.edu/~honglak/teaching/eecs598/schedule.html] Honglak Lee's Course | ||
| + | * [http://www.cs.toronto.edu/~hinton/deeprefs.html] from Geoff's tutorial | ||
