栈式自编码算法
From Ufldl
(→Training) |
(→Concrete example) |
||
| Line 122: | Line 122: | ||
First, you would train a sparse autoencoder on the raw inputs <math>x^{(k)}</math> to learn primary features <math>h^{(1)(k)}</math> on the raw input. | First, you would train a sparse autoencoder on the raw inputs <math>x^{(k)}</math> to learn primary features <math>h^{(1)(k)}</math> on the raw input. | ||
| + | |||
| + | 【初译】 | ||
| + | 真实的故事.. | ||
| + | 咱们来看个具体例子，假设你想要训练一个有两个隐含层的栈式网络，用来对MNIST数字集分类。别急，这是你的下一个练习。 | ||
| + | 首先，你需要训练一个自编码网络，它用输入<math>x^{(k)}</math> 来学到最基本的特征表示 <math>h^{(1)(k)}</math>。 | ||
| + | |||
| + | 【一审】 | ||
| + | 具体实例 | ||
| + | 让我们来看个具体的例子，假设你想要训练一个包含两个隐含层的栈式自编码网络用于MNIST数字分类（这将会是你的下一个练习）。 | ||
| + | 首先，你需要用原始输入<math>x^{(k)}</math> 训练第一个自编码器，它能够学习得到原始输入的一阶特征表示<math>h^{(1)(k)}</math>（如下图所示）。 | ||
| + | |||
[[File:Stacked_SparseAE_Features1.png|400px]] | [[File:Stacked_SparseAE_Features1.png|400px]] | ||
Next, you would feed the raw input into this trained sparse autoencoder, obtaining the primary feature activations <math>h^{(1)(k)}</math> for each of the inputs <math>x^{(k)}</math>. You would then use these primary features as the "raw input" to another sparse autoencoder to learn secondary features <math>h^{(2)(k)}</math> on these primary features. | Next, you would feed the raw input into this trained sparse autoencoder, obtaining the primary feature activations <math>h^{(1)(k)}</math> for each of the inputs <math>x^{(k)}</math>. You would then use these primary features as the "raw input" to another sparse autoencoder to learn secondary features <math>h^{(2)(k)}</math> on these primary features. | ||
| + | |||
| + | 【初译】 | ||
| + | 接着，你需要把原始数据输入到训练好的稀疏网络中，对于每一个<math>x^{(k)}</math>，得到它对应的基础特征表示<math>h^{(2)(k)}</math>。然后你再用这些特征作为另一个稀疏网络的输入，在它们的基础上，来学习第二层特征。 | ||
| + | |||
| + | |||
| + | 【一审】 | ||
| + | 接着，你需要把原始数据输入到上述训练好的稀疏自编码器中，对于每一个输入<math>x^{(k)}</math>，都可以得到它对应的一阶特征表示<math>h^{(2)(k)}</math>。然后你再用这些一阶特征作为另一个稀疏自编码器的输入，使用它们来学习二阶特征。（如下图所示） | ||
[[File:Stacked_SparseAE_Features2.png|400px]] | [[File:Stacked_SparseAE_Features2.png|400px]] | ||
Following this, you would feed the primary features into the second sparse autoencoder to obtain the secondary feature activations <math>h^{(2)(k)}</math> for each of the primary features <math>h^{(1)(k)}</math> (which correspond to the primary features of the corresponding inputs <math>x^{(k)}</math>). You would then treat these secondary features as "raw input" to a softmax classifier, training it to map secondary features to digit labels. | Following this, you would feed the primary features into the second sparse autoencoder to obtain the secondary feature activations <math>h^{(2)(k)}</math> for each of the primary features <math>h^{(1)(k)}</math> (which correspond to the primary features of the corresponding inputs <math>x^{(k)}</math>). You would then treat these secondary features as "raw input" to a softmax classifier, training it to map secondary features to digit labels. | ||
| + | |||
| + | 【初译】 | ||
| + | 照这样，再把基础特征输入到刚训练好的第二层稀疏网络中，得到对应每个<math>h^{(1)(k)}</math>的更高层表示<math>h^{(2)(k)}</math>。现在你可以把这些二级特征作为softmax分类器的输入了，训练之，完成从特征到数字标签的匹配。 | ||
| + | |||
| + | 【一审】 | ||
| + | 同样，再把一阶特征输入到刚训练好的第二层稀疏自编码器中，得到每个<math>h^{(1)(k)}</math>对应的二阶特征响应<math>h^{(2)(k)}</math>。接下来，你可以把这些二阶特征作为softmax分类器的输入，训练得到一个将二阶特征映射到数字标签的模型。 | ||
[[File:Stacked_Softmax_Classifier.png|400px]] | [[File:Stacked_Softmax_Classifier.png|400px]] | ||
Finally, you would combine all three layers together to form a stacked autoencoder with 2 hidden layers and a final softmax classifier layer capable of classifying the MNIST digits as desired. | Finally, you would combine all three layers together to form a stacked autoencoder with 2 hidden layers and a final softmax classifier layer capable of classifying the MNIST digits as desired. | ||
| + | |||
| + | 【初译】 | ||
| + | 最终，你需要把这三层一起组成一个栈式网络，它有两个隐藏层，一个最终的分类器。这个网络会如你所愿的把MNIST数字集分类。 | ||
| + | |||
| + | 【一审】 | ||
| + | 如下图所示，最终，你可以将这三层结合起来构建一个包含两个隐藏层和一个最终softmax分类器层的栈式自编码网络，这个网络能够如你所愿的对MNIST数字进行分类。 | ||
[[File:Stacked_Combined.png|500px]] | [[File:Stacked_Combined.png|500px]] | ||
