Implementing PCA/Whitening
From Ufldl
| Line 2: | Line 2: | ||
and also describe how you can implement them using efficient linear algebra libraries. | and also describe how you can implement them using efficient linear algebra libraries. | ||
| - | First, we need to ensure that the data has zero-mean, | + | First, we need to ensure that the data has (approximately) zero-mean. For natural images, we achieve this (approximately) by subtracting the mean value of each image patch. |
| - | avg = mean(x, | + | We achieve this by computing the mean for each patch and subtracting it for each patch. In Matlab, we can do this by using |
| - | x = x - repmat(avg | + | |
| + | avg = mean(x, 1); | ||
| + | x = x - repmat(avg, size(x, 1), 1); | ||
Next, we need to compute <math>\textstyle \Sigma = \frac{1}{m} \sum_{i=1}^m (x^{(i)})(x^{(i)})^T</math>. If you're implementing this in Matlab (or even if you're implementing this in C++, Java, etc., but have access to an efficient linear algebra library), doing it as an explicit sum is inefficient. Instead, we can instead compute this in one fell swoop as | Next, we need to compute <math>\textstyle \Sigma = \frac{1}{m} \sum_{i=1}^m (x^{(i)})(x^{(i)})^T</math>. If you're implementing this in Matlab (or even if you're implementing this in C++, Java, etc., but have access to an efficient linear algebra library), doing it as an explicit sum is inefficient. Instead, we can instead compute this in one fell swoop as | ||
| Line 24: | Line 26: | ||
Finally, you can compute <math>\textstyle x_{\rm rot}</math> and <math>\textstyle \tilde{x}</math> as follows: | Finally, you can compute <math>\textstyle x_{\rm rot}</math> and <math>\textstyle \tilde{x}</math> as follows: | ||
| - | xRot = U(:,1:k)' * x; | + | xRot = U(:,1:k)' * x; % k is number of eigenvectors to keep |
| - | xTilde = U(:,1:k) | + | xTilde = U(:,1:k) * xRot; % which corresponds to the # dimensions after reduction |
| - | + | % set k = size(x, 1) to keep all the eigenvectors | |
This gives your PCA representation of the data in terms of <math>\textstyle \tilde{x} \in \Re^k</math>. | This gives your PCA representation of the data in terms of <math>\textstyle \tilde{x} \in \Re^k</math>. | ||
