Exercise:PCA and Whitening

From Ufldl

Jump to: navigation, search
(Step 4a: Implement PCA with whitening and regularisation)
Line 63: Line 63:
==== Step 4b: Check covariance ====
==== Step 4b: Check covariance ====
-
As with PCA alone, PCA with whitening results in processed data that has a diagonal covariance matrix. However, unlike PCA alone, whitening additionally ensures that the diagonal entries are (approximately) equal to 1, i.e. that the covariance matrix is (approximately) the identity matrix. To verify that your implementation of PCA with whitening is correct, you can check this property.Implement code to compute the covariance matrix and verify this property. As earlier, you can visualise the covariance matrix with <tt>imagesc</tt>. When visualised as an image, you should see a red line across the diagonal (corresponding to the one entries) against a blue background (corresponding to the zero entries).
+
As with PCA alone, PCA with whitening results in processed data that has a diagonal covariance matrix. However, unlike PCA alone, whitening additionally ensures that the diagonal entries are equal to 1, i.e. that the covariance matrix is the identity matrix.  
 +
 
 +
That would be the case if you were doing whitening alone with no regularisation. However, in this case you are whitening with regularisation, in order to avoid amplifying high-frequency noise in the data. As such, the variance of the high-frequency noise components, which can be seen from the appropriate diagonal entries in the covariance matrix, will be much less than 1.
 +
 
 +
To verify that your implementation of PCA whitening with and without regularisation is correct, you can check these properties. Implement code to compute the covariance matrix and verify this property. (How should you check your PCA whitening code without regularisation? Simply set epsilon to 0, or close to 0). As earlier, you can visualise the covariance matrix with <tt>imagesc</tt>. When visualised as an image, for PCA whitening without regularisation you should see a red line across the diagonal (corresponding to the one entries) against a blue background (corresponding to the zero entries); for PCA whitening with regularisation you should see a red line that slowly turns blue across the diagonal (corresponding to the 1 entries slowly becoming smaller).
[[File:pca_whitened_covar.png|360px]]
[[File:pca_whitened_covar.png|360px]]

Revision as of 18:39, 29 April 2011

Personal tools