UFLDL Recommended Readings

From Ufldl

Jump to: navigation, search
Line 8: Line 8:
* Rajat Raina's Self-Taught Learning paper from ICML 2009. (2008?)  
* Rajat Raina's Self-Taught Learning paper from ICML 2009. (2008?)  
-
Papers on autoencoders:  
+
Autoencoders:  
* Greedy layerwise training of autoencoders
* Greedy layerwise training of autoencoders
* Dumitru Erhan, Yoshua Bengio, Aaron Courville, Pierre-Antoine Manzagol, Pascal Vincent, and Samy Bengio. Why Does Unsupervised Pre-training Help Deep Learning? Journal of Machine Learning Research, 11(Feb):625−660, 2010   
* Dumitru Erhan, Yoshua Bengio, Aaron Courville, Pierre-Antoine Manzagol, Pascal Vincent, and Samy Bengio. Why Does Unsupervised Pre-training Help Deep Learning? Journal of Machine Learning Research, 11(Feb):625−660, 2010   
Line 17: Line 17:
*  
*  
-
Papers on SFA:
 
-
* Wiskott, L and Sejnowski, T (2002). Slow feature analysis: Unsupervised learning of invariances. Neural Computation 14(4): 715-770.
 
-
 
+
Practical Guides
-
Papers on RBMs/DBNs:
+
* [http://deeplearning.net/tutorial/rbm.html] Tutorial on RBMs. But ignore the Theano code examples.
-
* Tutorial on RBMs.  But ignore the Theonu (or whatever the language is called) code examples. http://deeplearning.net/tutorial/rbm.html
+
* If you're implemeting RBMs, then read Hinton's "practical guide to RBMs."  But otherwise not great as a tutorial.  
* If you're implemeting RBMs, then read Hinton's "practical guide to RBMs."  But otherwise not great as a tutorial.  
-
 
+
Natural Language Processing:
 +
* [http://www.iro.umontreal.ca/~lisa/publications2/index.php/attachments/single/57] Yoshua Bengio, Réjean Ducharme, Pascal Vincent and Christian Jauvin, A Neural Probabilistic Language Model. JMLR 2003.
 +
* [http://ronan.collobert.com/pub/matos/2008_nlp_icml.pdf] R. Collobert and J. Weston. A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning. ICML 2008.
 +
* [http://www.cs.toronto.edu/~hinton/absps/threenew.pdf] Mnih, A. and Hinton, G. E.
 +
Three New Graphical Models for Statistical Language Modelling. ICML 2007
Advanced stuff:
Advanced stuff:
 +
* Slow Feature Analysis:
 +
** [http://itb.biologie.hu-berlin.de/~wiskott/Publications/BerkWisk2005c-SFAComplexCells-JoV.pdf] Slow feature analysis yields a rich repertoire of complex cell properties. Journal of Vision, 2005.
* Predictive Sparse Decomposition
* Predictive Sparse Decomposition
** [http://cs.nyu.edu/~koray/publis/koray-psd-08.pdf] Koray Kavukcuoglu, Marc'Aurelio Ranzato, and Yann LeCun, "Fast Inference in Sparse Coding Algorithms with Applications to Object Recognition", Computational and Biological Learning Lab, Courant Institute, NYU, 2008.  
** [http://cs.nyu.edu/~koray/publis/koray-psd-08.pdf] Koray Kavukcuoglu, Marc'Aurelio Ranzato, and Yann LeCun, "Fast Inference in Sparse Coding Algorithms with Applications to Object Recognition", Computational and Biological Learning Lab, Courant Institute, NYU, 2008.  
Line 34: Line 37:
Applications:
Applications:
* Object Recognition
* Object Recognition
-
** [link] Linear Spatial Pyramid Matching using Sparse Coding for Image Classification, CVPR 2009  
+
** [http://www.ifp.illinois.edu/~jyang29/ScSPM.htm] Jianchao Yang, Kai Yu, Yihong Gong, Thomas Huang. Linear Spatial Pyramid Matching using Sparse Coding for Image Classification, CVPR 2009  
* Audio Recognition
* Audio Recognition
** [http://www.cs.stanford.edu/people/ang/papers/nips09-AudioConvolutionalDBN.pdf] Unsupervised feature learning for audio classification using convolutional deep belief networks, Honglak Lee, Yan Largman, Peter Pham and Andrew Y. Ng. In NIPS*2009.
** [http://www.cs.stanford.edu/people/ang/papers/nips09-AudioConvolutionalDBN.pdf] Unsupervised feature learning for audio classification using convolutional deep belief networks, Honglak Lee, Yan Largman, Peter Pham and Andrew Y. Ng. In NIPS*2009.
-
 
+
** [http://www.cs.toronto.edu/~hinton/absps/mcphone.pdf] Dahl, G., Ranzato, M., Mohamed, A. and Hinton, G. E. Phone Recognition with the Mean-Covariance Restricted Boltzmann Machine. NIPS 2010.
-
 
+
-
Natural Language Processing:
+
-
* Neural Language Model (Bengio, ~2003?)  Don't remember how readable this is. If it's not,someone please take this out.
+
-
* Collobert and Weston, 2008
+
-
* Mnih and Hinton "3 new models for something or other", in ICML
+
-
 
+
Also, for other lists of papers:
Also, for other lists of papers:
-
* Honglak Lee's course
+
* [http://www.eecs.umich.edu/~honglak/teaching/eecs598/schedule.html] Honglak Lee's Course

Revision as of 01:39, 1 March 2011

Personal tools