UFLDL Recommended Readings
From Ufldl
Line 8: | Line 8: | ||
* Rajat Raina's Self-Taught Learning paper from ICML 2009. (2008?) | * Rajat Raina's Self-Taught Learning paper from ICML 2009. (2008?) | ||
- | + | Autoencoders: | |
* Greedy layerwise training of autoencoders | * Greedy layerwise training of autoencoders | ||
* Dumitru Erhan, Yoshua Bengio, Aaron Courville, Pierre-Antoine Manzagol, Pascal Vincent, and Samy Bengio. Why Does Unsupervised Pre-training Help Deep Learning? Journal of Machine Learning Research, 11(Feb):625−660, 2010 | * Dumitru Erhan, Yoshua Bengio, Aaron Courville, Pierre-Antoine Manzagol, Pascal Vincent, and Samy Bengio. Why Does Unsupervised Pre-training Help Deep Learning? Journal of Machine Learning Research, 11(Feb):625−660, 2010 | ||
Line 17: | Line 17: | ||
* | * | ||
- | |||
- | |||
- | + | Practical Guides | |
- | + | * [http://deeplearning.net/tutorial/rbm.html] Tutorial on RBMs. But ignore the Theano code examples. | |
- | * | + | |
* If you're implemeting RBMs, then read Hinton's "practical guide to RBMs." But otherwise not great as a tutorial. | * If you're implemeting RBMs, then read Hinton's "practical guide to RBMs." But otherwise not great as a tutorial. | ||
- | + | Natural Language Processing: | |
+ | * [http://www.iro.umontreal.ca/~lisa/publications2/index.php/attachments/single/57] Yoshua Bengio, Réjean Ducharme, Pascal Vincent and Christian Jauvin, A Neural Probabilistic Language Model. JMLR 2003. | ||
+ | * [http://ronan.collobert.com/pub/matos/2008_nlp_icml.pdf] R. Collobert and J. Weston. A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning. ICML 2008. | ||
+ | * [http://www.cs.toronto.edu/~hinton/absps/threenew.pdf] Mnih, A. and Hinton, G. E. | ||
+ | Three New Graphical Models for Statistical Language Modelling. ICML 2007 | ||
Advanced stuff: | Advanced stuff: | ||
+ | * Slow Feature Analysis: | ||
+ | ** [http://itb.biologie.hu-berlin.de/~wiskott/Publications/BerkWisk2005c-SFAComplexCells-JoV.pdf] Slow feature analysis yields a rich repertoire of complex cell properties. Journal of Vision, 2005. | ||
* Predictive Sparse Decomposition | * Predictive Sparse Decomposition | ||
** [http://cs.nyu.edu/~koray/publis/koray-psd-08.pdf] Koray Kavukcuoglu, Marc'Aurelio Ranzato, and Yann LeCun, "Fast Inference in Sparse Coding Algorithms with Applications to Object Recognition", Computational and Biological Learning Lab, Courant Institute, NYU, 2008. | ** [http://cs.nyu.edu/~koray/publis/koray-psd-08.pdf] Koray Kavukcuoglu, Marc'Aurelio Ranzato, and Yann LeCun, "Fast Inference in Sparse Coding Algorithms with Applications to Object Recognition", Computational and Biological Learning Lab, Courant Institute, NYU, 2008. | ||
Line 34: | Line 37: | ||
Applications: | Applications: | ||
* Object Recognition | * Object Recognition | ||
- | ** [ | + | ** [http://www.ifp.illinois.edu/~jyang29/ScSPM.htm] Jianchao Yang, Kai Yu, Yihong Gong, Thomas Huang. Linear Spatial Pyramid Matching using Sparse Coding for Image Classification, CVPR 2009 |
* Audio Recognition | * Audio Recognition | ||
** [http://www.cs.stanford.edu/people/ang/papers/nips09-AudioConvolutionalDBN.pdf] Unsupervised feature learning for audio classification using convolutional deep belief networks, Honglak Lee, Yan Largman, Peter Pham and Andrew Y. Ng. In NIPS*2009. | ** [http://www.cs.stanford.edu/people/ang/papers/nips09-AudioConvolutionalDBN.pdf] Unsupervised feature learning for audio classification using convolutional deep belief networks, Honglak Lee, Yan Largman, Peter Pham and Andrew Y. Ng. In NIPS*2009. | ||
- | + | ** [http://www.cs.toronto.edu/~hinton/absps/mcphone.pdf] Dahl, G., Ranzato, M., Mohamed, A. and Hinton, G. E. Phone Recognition with the Mean-Covariance Restricted Boltzmann Machine. NIPS 2010. | |
- | + | ||
- | + | ||
- | * | + | |
- | + | ||
- | + | ||
- | + | ||
Also, for other lists of papers: | Also, for other lists of papers: | ||
- | * Honglak Lee's | + | * [http://www.eecs.umich.edu/~honglak/teaching/eecs598/schedule.html] Honglak Lee's Course |