UFLDL Recommended Readings
From Ufldl
for
UFLDL Recommended Readings
Jump to:
navigation
,
search
If you're learning about UFLDL (Unsupervised Feature Learning and Deep Learning), here is a list of papers to consider reading. The basics: * [[http://cs294a.stanford.edu CS294A]] neural network/sparse autoencoder tutorial. (Most of this is now in the [[UFLDL Tutorial]], but the exercise is still on the CS294A website.) * Natural Image Statistics book, Hyvarinen et al. * Olshausen and Field Sparse Coding paper (1996) * [http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf] Yoshua Bengio. Learning Deep Architectures for AI. FTML 2009. (Broad landscape description of the field, but technical details there are hard to follow so ignore that.) * [http://www.cs.stanford.edu/~ang/papers/icml07-selftaughtlearning.pdf] Rajat Raina, Alexis Battle, Honglak Lee, Benjamin Packer and Andrew Y. Ng. Self-taught learning: Transfer learning from unlabeled data. ICML 2007 Autoencoders: * [http://www-etud.iro.umontreal.ca/~larocheh/publications/greedy-deep-nets-nips-06.pdf] Bengio, Y., Lamblin, P., Popovici, P., Larochelle, H. Greedy Layer-Wise Training of Deep Networks. NIPS 2006 * [http://www.jmlr.org/papers/volume11/erhan10a/erhan10a.pdf] Dumitru Erhan, Yoshua Bengio, Aaron Courville, Pierre-Antoine Manzagol, Pascal Vincent, and Samy Bengio. Why Does Unsupervised Pre-training Help Deep Learning? JMLR 2010 * Larochelle, Erhan, Courville, Bergstra, BBengio, 2007. (Someone read this and let us know if this is worth keeping,.) * [http://www.cs.toronto.edu/~hinton/science.pdf] [http://www.cs.toronto.edu/~hinton/MatlabForSciencePaper.html] Hinton, G. E. and Salakhutdinov, R. R. Reducing the dimensionality of data with neural networks. Science 2006 * [http://www.cs.toronto.edu/~larocheh/publications/icml-2008-denoising-autoencoders.pdf] Pascal Vincent, Hugo Larochelle, Yoshua Bengio and Pierre-Antoine Manzagol. Extracting and Composing Robust Features with Denoising Autoencoders. ICML 2008. Deep Belief Networks: * [http://deeplearning.net/tutorial/rbm.html] Tutorial on RBMs. But ignore the Theano code examples. Practical Guides: * [http://www.cs.toronto.edu/~hinton/absps/guideTR.pdf] Geoff Hinton. A practical guide to training restricted Boltzmann machines. UTML TR 2010–003 Natural Language Processing: * [http://www.iro.umontreal.ca/~lisa/publications2/index.php/attachments/single/57] Yoshua Bengio, Réjean Ducharme, Pascal Vincent and Christian Jauvin, A Neural Probabilistic Language Model. JMLR 2003. * [http://ronan.collobert.com/pub/matos/2008_nlp_icml.pdf] R. Collobert and J. Weston. A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning. ICML 2008. * [http://www.cs.toronto.edu/~hinton/absps/threenew.pdf] Mnih, A. and Hinton, G. E. Three New Graphical Models for Statistical Language Modelling. ICML 2007 Advanced stuff: * Slow Feature Analysis: ** [http://itb.biologie.hu-berlin.de/~wiskott/Publications/BerkWisk2005c-SFAComplexCells-JoV.pdf] Slow feature analysis yields a rich repertoire of complex cell properties. Journal of Vision, 2005. * Predictive Sparse Decomposition ** [http://cs.nyu.edu/~koray/publis/koray-psd-08.pdf] Koray Kavukcuoglu, Marc'Aurelio Ranzato, and Yann LeCun, "Fast Inference in Sparse Coding Algorithms with Applications to Object Recognition", Computational and Biological Learning Lab, Courant Institute, NYU, 2008. ** [http://cs.nyu.edu/~koray/publis/jarrett-iccv-09.pdf] Kevin Jarrett, Koray Kavukcuoglu, Marc'Aurelio Ranzato, and Yann LeCun, "What is the Best Multi-Stage Architecture for Object Recognition?", In ICCV 2009 Applications: * Object Recognition ** [http://www.ifp.illinois.edu/~jyang29/ScSPM.htm] Jianchao Yang, Kai Yu, Yihong Gong, Thomas Huang. Linear Spatial Pyramid Matching using Sparse Coding for Image Classification, CVPR 2009 * Audio Recognition ** [http://www.cs.stanford.edu/people/ang/papers/nips09-AudioConvolutionalDBN.pdf] Unsupervised feature learning for audio classification using convolutional deep belief networks, Honglak Lee, Yan Largman, Peter Pham and Andrew Y. Ng. In NIPS*2009. ** [http://www.cs.toronto.edu/~hinton/absps/mcphone.pdf] Dahl, G., Ranzato, M., Mohamed, A. and Hinton, G. E. Phone Recognition with the Mean-Covariance Restricted Boltzmann Machine. NIPS 2010. Also, for other lists of papers: * [http://www.eecs.umich.edu/~honglak/teaching/eecs598/schedule.html] Honglak Lee's Course
Return to
UFLDL Recommended Readings
.
Views
Page
Discussion
View source
History
Personal tools
Log in
ufldl resources
UFLDL Tutorial
Recommended Readings
wiki
Main page
Recent changes
Random page
Help
Search
Toolbox
What links here
Related changes
Special pages