Visualizing a Trained Autoencoder
From Ufldl
Line 6: | Line 6: | ||
a^{(2)}_i = f\left(\sum_{j=1}^{100} W^{(1)}_{ij} x_j + b^{(1)}_i \right). | a^{(2)}_i = f\left(\sum_{j=1}^{100} W^{(1)}_{ij} x_j + b^{(1)}_i \right). | ||
\end{align}</math> | \end{align}</math> | ||
- | + | <!-- This is the activation function <math>\textstyle g(\cdot)</math> applied to an affine function of the input.!--> | |
We will visualize the function computed by hidden unit <math>\textstyle i</math>---which depends on the | We will visualize the function computed by hidden unit <math>\textstyle i</math>---which depends on the | ||
parameters <math>\textstyle W^{(1)}_{ij}</math> (ignoring | parameters <math>\textstyle W^{(1)}_{ij}</math> (ignoring | ||
- | the bias term for now) using a 2D image. In particular, we think of | + | the bias term for now)---using a 2D image. In particular, we think of |
- | <math>\textstyle a^{( | + | <math>\textstyle a^{(2)}_i</math> as some non-linear feature of the input <math>\textstyle x</math>. |
We ask: | We ask: | ||
What input image <math>\textstyle x</math> would cause | What input image <math>\textstyle x</math> would cause | ||
- | <math>\textstyle a^{( | + | <math>\textstyle a^{(2)}_i</math> to be maximally activated? |
- | + | (Less formally, what is the feature that hidden unit <math>\textstyle i</math> is looking for?) | |
For this question to have a non-trivial answer, | For this question to have a non-trivial answer, | ||
we must impose some constraints on <math>\textstyle x</math>. If we suppose that | we must impose some constraints on <math>\textstyle x</math>. If we suppose that | ||
Line 34: | Line 34: | ||
When we do this for a sparse autoencoder (trained with 100 hidden units on | When we do this for a sparse autoencoder (trained with 100 hidden units on | ||
- | 10x10 pixel inputs we get the following result: | + | 10x10 pixel inputs<sup>1</sup> we get the following result: |
[[Image:ExampleSparseAutoencoderWeights.png|thumb|400px|center]] | [[Image:ExampleSparseAutoencoderWeights.png|thumb|400px|center]] | ||
Line 48: | Line 48: | ||
domains too. | domains too. | ||
- | ''The learned features were obtained by training on '''whitened''' natural images. Whitening is a preprocessing step which removes redundancy in the input, by causing adjacent pixels to become less correlated.'' | + | ---- |
+ | |||
+ | <sup>1</sup> ''The learned features were obtained by training on '''whitened''' natural images. Whitening is a preprocessing step which removes redundancy in the input, by causing adjacent pixels to become less correlated.'' | ||
+ | |||
+ | |||
+ | {{Sparse_Autoencoder}} | ||
+ | |||
+ | |||
+ | {{Languages|可视化自编码器训练结果|中文}} |