Exercise:PCA and Whitening

Revision as of 01:37, 30 April 2011 (view source)

Ang (Talk | contribs)

(→Step 5: ZCA whitening)

← Older edit

Revision as of 01:39, 30 April 2011 (view source)

Ang (Talk | contribs)

(→Step 3: PCA with dimension reduction)

Newer edit →

Line 38:

Now that you have found <math>k</math>, compute <math>\tilde{x}</math>, the reduced-dimension representation of the data. This gives you a representation of each image patch as a <math>k</math> dimensional vector instead of a 144 dimensional vector. If you are training a sparse autoencoder or other algorithm on this reduced-dimensional data, it will run faster than if you were training on the original 144 dimensional data.

-

To see the effect of dimension reduction, go back from <math>\tilde{x}</math> to produce the matrix <math>\hat{x}</math>, the dimension-reduced data but expressed in the original 144 dimensional space of image patches. Visualise <math>\hat{x}</math> and compare it to the raw data, <math>x</math>. You will observe that there is little loss due to throwing away the principal components that correspond to dimensions with low variation. For comparison, you may also wish to generate and visualise <math>\hat{x}</math> for when only 50% of the variance is retained.

+

To see the effect of dimension reduction, go back from <math>\tilde{x}</math> to produce the matrix <math>\hat{x}</math>, the dimension-reduced data but expressed in the original 144 dimensional space of image patches. Visualise <math>\hat{x}</math> and compare it to the raw data, <math>x</math>. You will observe that there is little loss due to throwing away the principal components that correspond to dimensions with low variation. For comparison, you may also wish to generate and visualise <math>\hat{x}</math> for when only 90% of the variance is retained.

<table>

Exercise:PCA and Whitening

From Ufldl

Revision as of 01:39, 30 April 2011

Views

Personal tools

ufldl resources

wiki

Search

Toolbox

@@ Line 38: / Line 38: @@
 Now that you have found <math>k</math>, compute <math>\tilde{x}</math>, the reduced-dimension representation of the data.  This gives you a representation of each image patch as a <math>k</math> dimensional vector instead of a 144 dimensional vector.  If you are training a sparse autoencoder or other algorithm on this reduced-dimensional data, it will run faster than if you were training on the original 144 dimensional data.
-To see the effect of dimension reduction, go back from <math>\tilde{x}</math> to produce the matrix <math>\hat{x}</math>, the dimension-reduced data but expressed in the original 144 dimensional space of image patches. Visualise <math>\hat{x}</math> and compare it to the raw data, <math>x</math>. You will observe that there is little loss due to throwing away the principal components that correspond to dimensions with low variation. For comparison, you may also wish to generate and visualise <math>\hat{x}</math> for when only 50% of the variance is retained.
+To see the effect of dimension reduction, go back from <math>\tilde{x}</math> to produce the matrix <math>\hat{x}</math>, the dimension-reduced data but expressed in the original 144 dimensional space of image patches. Visualise <math>\hat{x}</math> and compare it to the raw data, <math>x</math>. You will observe that there is little loss due to throwing away the principal components that correspond to dimensions with low variation. For comparison, you may also wish to generate and visualise <math>\hat{x}</math> for when only 90% of the variance is retained.
 <table>