Exercise:Convolution and Pooling
From Ufldl
(→Step 2: Learn features on small patches) |
(→Step 3a: Convolution) |
||
Line 62: | Line 62: | ||
Observe that the convolution above can be broken down into the following three small steps. First, compute <math>Wx_{(r,c)}</math> for all <math>(r, c)</math>. Next, add b to all the computed values. Finally, apply the sigmoid function to the resultant values. This doesn't seem to buy you anything, since the first step still requires a loop. However, you can replace the loop in the first step with one of MATLAB's optimized convolution functions, <tt>conv2</tt>, speeding up the process slightly. | Observe that the convolution above can be broken down into the following three small steps. First, compute <math>Wx_{(r,c)}</math> for all <math>(r, c)</math>. Next, add b to all the computed values. Finally, apply the sigmoid function to the resultant values. This doesn't seem to buy you anything, since the first step still requires a loop. However, you can replace the loop in the first step with one of MATLAB's optimized convolution functions, <tt>conv2</tt>, speeding up the process slightly. | ||
- | + | However, there are two complications in using <tt>conv2</tt>. | |
- | + | First, <tt>conv2</tt> performs a 2-D convolution, but you have 5 "dimensions" - image number, feature number, row of image, column of image, and channel of image - that you want to convolve over. Because of this, you will have to convolve each image, feature and image channel separately for each image, using the row and column of the image as the 2 dimensions you convolve over. This means that you will need three outer loops over the image number <tt>imageNum</tt>, feature number <tt>featureNum</tt>, and the channel number of the image <tt>channel</tt>, with the 2-D convolution of the weight matrix for the <tt>featureNum</tt>-th feature and <tt>channel</tt>-th channel with the image matrix for the <tt>imageNum</tt>-th image going inside. | |
+ | |||
+ | Second, because of the mathematical definition of convolution, the feature matrix must be "flipped" before passing it to <tt>conv2</tt>. This is explained in greater detail in the implementation tip section following the code. | ||
+ | |||
+ | Concretely, the code to do the convolution using <tt>conv2</tt> will look something like the following: | ||
<syntaxhighlight lang="matlab"> | <syntaxhighlight lang="matlab"> | ||
Line 89: | Line 93: | ||
</syntaxhighlight> | </syntaxhighlight> | ||
- | |||
- | |||
<div style="border:1px solid black; padding: 5px"> | <div style="border:1px solid black; padding: 5px"> | ||