UFLDL Recommended Readings
From Ufldl
| Line 8: | Line 8: | ||
* [http://redwood.psych.cornell.edu/papers/olshausen_field_nature_1996.pdf] Olshausen and Field. Emergence of simple-cell receptive field properties by learning a sparse code for natural images Nature 1996. (Sparse Coding) | * [http://redwood.psych.cornell.edu/papers/olshausen_field_nature_1996.pdf] Olshausen and Field. Emergence of simple-cell receptive field properties by learning a sparse code for natural images Nature 1996. (Sparse Coding) | ||
* [http://www.cs.stanford.edu/~ang/papers/icml07-selftaughtlearning.pdf] Rajat Raina, Alexis Battle, Honglak Lee, Benjamin Packer and Andrew Y. Ng. Self-taught learning: Transfer learning from unlabeled data. ICML 2007 | * [http://www.cs.stanford.edu/~ang/papers/icml07-selftaughtlearning.pdf] Rajat Raina, Alexis Battle, Honglak Lee, Benjamin Packer and Andrew Y. Ng. Self-taught learning: Transfer learning from unlabeled data. ICML 2007 | ||
| - | |||
| Line 15: | Line 14: | ||
* [http://www-etud.iro.umontreal.ca/~larocheh/publications/greedy-deep-nets-nips-06.pdf] Bengio, Y., Lamblin, P., Popovici, P., Larochelle, H. Greedy Layer-Wise Training of Deep Networks. NIPS 2006 | * [http://www-etud.iro.umontreal.ca/~larocheh/publications/greedy-deep-nets-nips-06.pdf] Bengio, Y., Lamblin, P., Popovici, P., Larochelle, H. Greedy Layer-Wise Training of Deep Networks. NIPS 2006 | ||
* [http://www.cs.toronto.edu/~larocheh/publications/icml-2008-denoising-autoencoders.pdf] Pascal Vincent, Hugo Larochelle, Yoshua Bengio and Pierre-Antoine Manzagol. Extracting and Composing Robust Features with Denoising Autoencoders. ICML 2008. (They have a nice model, but then backwards rationalize it into a probabilistic model. Ignore the backwards rationalized probabilistic model.) (Someone please clarify eactly which section of the paper this is.) | * [http://www.cs.toronto.edu/~larocheh/publications/icml-2008-denoising-autoencoders.pdf] Pascal Vincent, Hugo Larochelle, Yoshua Bengio and Pierre-Antoine Manzagol. Extracting and Composing Robust Features with Denoising Autoencoders. ICML 2008. (They have a nice model, but then backwards rationalize it into a probabilistic model. Ignore the backwards rationalized probabilistic model.) (Someone please clarify eactly which section of the paper this is.) | ||
| + | |||
Analyzing deep learning/why does deep learning work: | Analyzing deep learning/why does deep learning work: | ||
| Line 20: | Line 20: | ||
* [http://www.jmlr.org/papers/volume11/erhan10a/erhan10a.pdf] Dumitru Erhan, Yoshua Bengio, Aaron Courville, Pierre-Antoine Manzagol, Pascal Vincent, and Samy Bengio. Why Does Unsupervised Pre-training Help Deep Learning? JMLR 2010 | * [http://www.jmlr.org/papers/volume11/erhan10a/erhan10a.pdf] Dumitru Erhan, Yoshua Bengio, Aaron Courville, Pierre-Antoine Manzagol, Pascal Vincent, and Samy Bengio. Why Does Unsupervised Pre-training Help Deep Learning? JMLR 2010 | ||
* [http://cs.stanford.edu/~ang/papers/nips09-MeasuringInvariancesDeepNetworks.pdf] Ian J. Goodfellow, Quoc V. Le, Andrew M. Saxe, Honglak Lee and Andrew Y. Ng.Measuring invariances in deep networks. NIPS 2009. | * [http://cs.stanford.edu/~ang/papers/nips09-MeasuringInvariancesDeepNetworks.pdf] Ian J. Goodfellow, Quoc V. Le, Andrew M. Saxe, Honglak Lee and Andrew Y. Ng.Measuring invariances in deep networks. NIPS 2009. | ||
| + | |||
RBMs: | RBMs: | ||
* [http://deeplearning.net/tutorial/rbm.html] Tutorial on RBMs. But ignore the Theano code examples. (Someone tell us if this should be moved later. Useful for understanding some of DL literature, but not needed for many of the later papers?) | * [http://deeplearning.net/tutorial/rbm.html] Tutorial on RBMs. But ignore the Theano code examples. (Someone tell us if this should be moved later. Useful for understanding some of DL literature, but not needed for many of the later papers?) | ||
| + | |||
Applications: | Applications: | ||
| Line 44: | Line 46: | ||
** [http://cs.nyu.edu/~koray/publis/koray-psd-08.pdf] Koray Kavukcuoglu, Marc'Aurelio Ranzato, and Yann LeCun, "Fast Inference in Sparse Coding Algorithms with Applications to Object Recognition", Computational and Biological Learning Lab, Courant Institute, NYU, 2008. | ** [http://cs.nyu.edu/~koray/publis/koray-psd-08.pdf] Koray Kavukcuoglu, Marc'Aurelio Ranzato, and Yann LeCun, "Fast Inference in Sparse Coding Algorithms with Applications to Object Recognition", Computational and Biological Learning Lab, Courant Institute, NYU, 2008. | ||
** [http://cs.nyu.edu/~koray/publis/jarrett-iccv-09.pdf] Kevin Jarrett, Koray Kavukcuoglu, Marc'Aurelio Ranzato, and Yann LeCun, "What is the Best Multi-Stage Architecture for Object Recognition?", In ICCV 2009 | ** [http://cs.nyu.edu/~koray/publis/jarrett-iccv-09.pdf] Kevin Jarrett, Koray Kavukcuoglu, Marc'Aurelio Ranzato, and Yann LeCun, "What is the Best Multi-Stage Architecture for Object Recognition?", In ICCV 2009 | ||
| + | |||
Mean-Covariance models | Mean-Covariance models | ||
| Line 52: | Line 55: | ||
* [http://www.nature.com/nature/journal/v457/n7225/pdf/nature07481.pdf] Y. Karklin and M. S. Lewicki, Emergence of complex cell properties by learning to generalize in natural scenes, Nature, 2008. | * [http://www.nature.com/nature/journal/v457/n7225/pdf/nature07481.pdf] Y. Karklin and M. S. Lewicki, Emergence of complex cell properties by learning to generalize in natural scenes, Nature, 2008. | ||
** (someone tell us if this should be here. Interesting algorithm + nice visualizations, though maybe slightly hard to understand.) | ** (someone tell us if this should be here. Interesting algorithm + nice visualizations, though maybe slightly hard to understand.) | ||
| + | |||
Overview | Overview | ||
* [http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf] Yoshua Bengio. Learning Deep Architectures for AI. FTML 2009. (Broad landscape description of the field, but technical details there are hard to follow so ignore that. This is also easier to read after you've gone over some of literature of the field.) | * [http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf] Yoshua Bengio. Learning Deep Architectures for AI. FTML 2009. (Broad landscape description of the field, but technical details there are hard to follow so ignore that. This is also easier to read after you've gone over some of literature of the field.) | ||
| + | |||
Practical guides: | Practical guides: | ||
