Visualizing a Trained Autoencoder
From Ufldl
Line 6: | Line 6: | ||
a^{(2)}_i = f\left(\sum_{j=1}^{100} W^{(1)}_{ij} x_j + b^{(1)}_i \right). | a^{(2)}_i = f\left(\sum_{j=1}^{100} W^{(1)}_{ij} x_j + b^{(1)}_i \right). | ||
\end{align}</math> | \end{align}</math> | ||
- | + | <!-- This is the activation function <math>\textstyle g(\cdot)</math> applied to an affine function of the input.!--> | |
We will visualize the function computed by hidden unit <math>\textstyle i</math>---which depends on the | We will visualize the function computed by hidden unit <math>\textstyle i</math>---which depends on the | ||
parameters <math>\textstyle W^{(1)}_{ij}</math> (ignoring | parameters <math>\textstyle W^{(1)}_{ij}</math> (ignoring | ||
- | the bias term for now) using a 2D image. In particular, we think of | + | the bias term for now)---using a 2D image. In particular, we think of |
- | <math>\textstyle a^{( | + | <math>\textstyle a^{(2)}_i</math> as some non-linear feature of the input <math>\textstyle x</math>. |
We ask: | We ask: | ||
What input image <math>\textstyle x</math> would cause | What input image <math>\textstyle x</math> would cause | ||
- | <math>\textstyle a^{( | + | <math>\textstyle a^{(2)}_i</math> to be maximally activated? |
- | + | (Less formally, what is the feature that hidden unit <math>\textstyle i</math> is looking for?) | |
For this question to have a non-trivial answer, | For this question to have a non-trivial answer, | ||
we must impose some constraints on <math>\textstyle x</math>. If we suppose that | we must impose some constraints on <math>\textstyle x</math>. If we suppose that | ||
Line 51: | Line 51: | ||
<sup>1</sup> ''The learned features were obtained by training on '''whitened''' natural images. Whitening is a preprocessing step which removes redundancy in the input, by causing adjacent pixels to become less correlated.'' | <sup>1</sup> ''The learned features were obtained by training on '''whitened''' natural images. Whitening is a preprocessing step which removes redundancy in the input, by causing adjacent pixels to become less correlated.'' | ||
+ | |||
+ | |||
+ | {{Sparse_Autoencoder}} | ||
+ | |||
+ | |||
+ | {{Languages|可视化自编码器训练结果|中文}} |