UFLDL Tutorial

From Ufldl

Jump to: navigation, search
 
Line 1: Line 1:
-
Sparse Autoencoder
+
'''Description:''' This tutorial will teach you the main ideas of Unsupervised Feature Learning and Deep Learning.  By working through it, you will also get to implement several feature learning/deep learning algorithms, get to see them work for yourself, and learn how to apply/adapt these ideas to new problems.
 +
 
 +
This tutorial assumes a basic knowledge of machine learning (specifically, familiarity with the ideas of supervised learning, logistic regression, gradient descent).  If you are not familiar with these ideas, we suggest you go to this [http://openclassroom.stanford.edu/MainFolder/CoursePage.php?course=MachineLearning Machine Learning course] and complete
 +
sections II, III, IV (up to Logistic Regression) first.
 +
 
 +
 
 +
'''Sparse Autoencoder'''
* [[Neural Networks]]
* [[Neural Networks]]
* [[Backpropagation Algorithm]]
* [[Backpropagation Algorithm]]
Line 6: Line 12:
* [[Visualizing a Trained Autoencoder]]
* [[Visualizing a Trained Autoencoder]]
* [[Sparse Autoencoder Notation Summary]]  
* [[Sparse Autoencoder Notation Summary]]  
-
* [[Exercise:Sparse_Autoencoder]]
+
* [[Exercise:Sparse Autoencoder]]
-
Vectorized implementation
+
'''Vectorized implementation'''
* [[Vectorization]]
* [[Vectorization]]
* [[Logistic Regression Vectorization Example]]
* [[Logistic Regression Vectorization Example]]
* [[Neural Network Vectorization]]
* [[Neural Network Vectorization]]
-
* [[Using the MNIST Dataset]]
 
* [[Exercise:Vectorization]]
* [[Exercise:Vectorization]]
-
Preprocessing: PCA and Whitening
+
'''Preprocessing: PCA and Whitening'''
* [[PCA]]
* [[PCA]]
* [[Whitening]]
* [[Whitening]]
* [[Implementing PCA/Whitening]]
* [[Implementing PCA/Whitening]]
-
* [[Exercise: PCA in 2D]]
+
* [[Exercise:PCA in 2D]]
* [[Exercise:PCA and Whitening]]
* [[Exercise:PCA and Whitening]]
-
Self-Taught Learning and Unsupervised Feature Learning
+
 
-
* [[Unsupervised Feature Learning]]
+
'''Softmax Regression'''
-
* [[Self-Taught Learning]]
+
-
* [[Image Classification]]  (talk about how to extract features from a large image)
+
* [[Softmax Regression]]
* [[Softmax Regression]]
* [[Exercise:Softmax Regression]]
* [[Exercise:Softmax Regression]]
 +
 +
 +
'''Self-Taught Learning and Unsupervised Feature Learning'''
 +
* [[Self-Taught Learning]]
* [[Exercise:Self-Taught Learning]]
* [[Exercise:Self-Taught Learning]]
-
Fine-tuning
 
-
- Exercise: Experiment with and without pre-training
 
-
Stacked Autoencoders
+
'''Building Deep Networks for Classification'''
 +
* [[Self-Taught Learning to Deep Networks | From Self-Taught Learning to Deep Networks]]
 +
* [[Deep Networks: Overview]]
 +
* [[Stacked Autoencoders]]
 +
* [[Fine-tuning Stacked AEs]]
 +
* [[Exercise: Implement deep networks for digit classification]]
 +
 
 +
 
 +
'''Linear Decoders with Autoencoders'''
 +
* [[Linear Decoders]]
 +
* [[Exercise:Learning color features with Sparse Autoencoders]]
 +
 
-
Fine-tuning Stacked AEs
+
'''Working with Large Images'''
 +
* [[Feature extraction using convolution]]
 +
* [[Pooling]]
 +
* [[Exercise:Convolution and Pooling]]
-
Convolutional models (1 layer)
+
----
 +
'''Note''': The sections above this line are stable.  The sections below are still under construction, and may change without notice.  Feel free to browse around however, and feedback/suggestions are welcome.
-
Pooling
+
'''Miscellaneous'''
 +
* [[MATLAB Modules]]
 +
* [[Style Guide]]
 +
* [[Useful Links]]
-
Multiple layers of convolution and pooling
+
'''Miscellaneous Topics'''
 +
* [[Data Preprocessing]]
 +
* [[Deriving gradients using the backpropagation idea]]
'''Advanced Topics''':
'''Advanced Topics''':
-
RBM
+
'''Sparse Coding'''
 +
* [[Sparse Coding]]
 +
* [[Sparse Coding: Autoencoder Interpretation]]
 +
* [[Exercise:Sparse Coding]]
-
DBN
+
'''ICA Style Models'''
 +
* [[Independent Component Analysis]]
 +
* [[Exercise:Independent Component Analysis]]
-
[[Sparse Coding]]
+
'''Others'''
 +
* [[Convolutional training]]
 +
* [[Restricted Boltzmann Machines]]
 +
* [[Deep Belief Networks]]
 +
* [[Denoising Autoencoders]]
 +
* [[K-means]]
 +
* [[Spatial pyramids / Multiscale]]
 +
* [[Slow Feature Analysis]]
 +
* [[Tiled Convolution Networks]]
-
K-means
+
----
-
Spatial pyramids??
+
Material contributed by: Andrew Ng, Jiquan Ngiam, Chuan Yu Foo, Yifan Mai, Caroline Suen
-
SFA
 
-
ICA/TICA/TCNN
+
{{Languages|UFLDL教程|中文}}

Latest revision as of 18:22, 7 April 2013

Personal tools