Exercise:PCA and Whitening
From Ufldl
(→Step 3: PCA with dimension reduction) |
(→Step 4b: Check covariance) |
||
| Line 67: | Line 67: | ||
That would be the case if you were doing whitening alone with no regularization. However, in this case you are whitening with regularization, to avoid numerical/etc. problems associated with small eigenvalues. As a result of this, some of the diagonal entries of the covariance of your <math>x_{\rm PCAwhite}</math> will be smaller than 1. | That would be the case if you were doing whitening alone with no regularization. However, in this case you are whitening with regularization, to avoid numerical/etc. problems associated with small eigenvalues. As a result of this, some of the diagonal entries of the covariance of your <math>x_{\rm PCAwhite}</math> will be smaller than 1. | ||
| - | To verify that your implementation of PCA whitening with and without regularization is correct, you can check these properties. Implement code to compute the covariance matrix and verify this property. (To check the result of PCA without whitening, simply set epsilon to 0, or close to 0, say <tt>1e-10</tt>). As earlier, you can visualise the covariance matrix with <tt>imagesc</tt>. When visualised as an image, for PCA whitening without regularization you should see a red line across the diagonal (corresponding to the one entries) against a blue background (corresponding to the zero entries); for PCA whitening with | + | To verify that your implementation of PCA whitening with and without regularization is correct, you can check these properties. Implement code to compute the covariance matrix and verify this property. (To check the result of PCA without whitening, simply set epsilon to 0, or close to 0, say <tt>1e-10</tt>). As earlier, you can visualise the covariance matrix with <tt>imagesc</tt>. When visualised as an image, for PCA whitening without regularization you should see a red line across the diagonal (corresponding to the one entries) against a blue background (corresponding to the zero entries); for PCA whitening with regularization you should see a red line that slowly turns blue across the diagonal (corresponding to the 1 entries slowly becoming smaller). |
<table> | <table> | ||
<tr> | <tr> | ||
| - | <td>[[File:pca_whitened_covar.png|360px|alt=Covariance for PCA whitening with | + | <td>[[File:pca_whitened_covar.png|360px|alt=Covariance for PCA whitening with regularization|Covariance for PCA whitening with regularization]]</td> |
| - | <td>[[File:pca_whitened_unregularised_covar.png|360px|alt=Covariance for PCA whitening with | + | <td>[[File:pca_whitened_unregularised_covar.png|360px|alt=Covariance for PCA whitening with regularization|Covariance for PCA whitening without regularization]]</td> |
</tr> | </tr> | ||
<tr> | <tr> | ||
| - | <td><center>Covariance for PCA whitening with | + | <td><center>Covariance for PCA whitening with regularization</center></td> |
| - | <td><center>Covariance for PCA whitening without | + | <td><center>Covariance for PCA whitening without regularization</center></td> |
</tr> | </tr> | ||
</table> | </table> | ||
