Whitening
From Ufldl
(→ZCE Whitening) |
|||
| Line 90: | Line 90: | ||
of the data, and do not try to reduce its dimension. | of the data, and do not try to reduce its dimension. | ||
| - | + | == Regularizaton == | |
| - | + | When implementing PCA whitening or ZCA whitening in practice, sometimes some | |
| - | + | ||
of the eigenvalues <math>\textstyle \lambda_i</math> will be numerically close to 0, and thus the scaling | of the eigenvalues <math>\textstyle \lambda_i</math> will be numerically close to 0, and thus the scaling | ||
step in Equation~(\ref{eqn-sqrtlambda}) above would involve dividing by a value close to zero, and may cause | step in Equation~(\ref{eqn-sqrtlambda}) above would involve dividing by a value close to zero, and may cause | ||
| Line 104: | Line 103: | ||
When <math>\textstyle x</math> takes values around <math>\textstyle [-1,1]</math>, a value of <math>\textstyle \epsilon \approx 10^{-5}</math> | When <math>\textstyle x</math> takes values around <math>\textstyle [-1,1]</math>, a value of <math>\textstyle \epsilon \approx 10^{-5}</math> | ||
might be typical. With this form of regularization, the features won't all | might be typical. With this form of regularization, the features won't all | ||
| - | |||
| - | |||
| - | |||
| - | |||
For the case of images, adding <math>\textstyle \epsilon</math> here also has the effect of slightly smoothing (or low-pass | For the case of images, adding <math>\textstyle \epsilon</math> here also has the effect of slightly smoothing (or low-pass | ||
