Exercise:Convolution and Pooling

From Ufldl

Jump to: navigation, search
(Step 2: Learn features on small patches)
(Step 3a: Convolution)
Line 62: Line 62:
Observe that the convolution above can be broken down into the following three small steps. First, compute <math>Wx_{(r,c)}</math> for all <math>(r, c)</math>. Next, add b to all the computed values. Finally, apply the sigmoid function to the resultant values. This doesn't seem to buy you anything, since the first step still requires a loop. However, you can replace the loop in the first step with one of MATLAB's optimized convolution functions, <tt>conv2</tt>, speeding up the process slightly.
Observe that the convolution above can be broken down into the following three small steps. First, compute <math>Wx_{(r,c)}</math> for all <math>(r, c)</math>. Next, add b to all the computed values. Finally, apply the sigmoid function to the resultant values. This doesn't seem to buy you anything, since the first step still requires a loop. However, you can replace the loop in the first step with one of MATLAB's optimized convolution functions, <tt>conv2</tt>, speeding up the process slightly.
-
There are a few complications in using <tt>conv2</tt>. First,  <tt>conv2</tt> performs a 2-D convolution, but you have 5 "dimensions" - image number, feature number, row of image, column of image, and channel of image - that you want to convolve over. Because of this, you will have to convolve each image, feature and image channel separately for each image, using the row and column of the image as the 2 dimensions you convolve over. This means that you will need three outer loops over the image number <tt>imageNum</tt>, feature number <tt>featureNum</tt>, and the channel number of the image <tt>channel</tt>, with the 2-D convolution of the weight matrix for the <tt>featureNum</tt>-th feature and <tt>channel</tt>-th channel with the image matrix for the <tt>imageNum</tt>-th image going inside.  
+
However, there are two complications in using <tt>conv2</tt>.  
-
More concretely, your code will look something like the following:
+
First, <tt>conv2</tt> performs a 2-D convolution, but you have 5 "dimensions" - image number, feature number, row of image, column of image, and channel of image - that you want to convolve over. Because of this, you will have to convolve each image, feature and image channel separately for each image, using the row and column of the image as the 2 dimensions you convolve over. This means that you will need three outer loops over the image number <tt>imageNum</tt>, feature number <tt>featureNum</tt>, and the channel number of the image <tt>channel</tt>, with the 2-D convolution of the weight matrix for the <tt>featureNum</tt>-th feature and <tt>channel</tt>-th channel with the image matrix for the <tt>imageNum</tt>-th image going inside.
 +
 
 +
Second, because of the mathematical definition of convolution, the feature matrix must be "flipped" before passing it to <tt>conv2</tt>. This is explained in greater detail in the implementation tip section following the code.
 +
 
 +
Concretely, the code to do the convolution using <tt>conv2</tt> will look something like the following:
<syntaxhighlight lang="matlab">
<syntaxhighlight lang="matlab">
Line 89: Line 93:
</syntaxhighlight>
</syntaxhighlight>
   
   
-
One detail in the above code needs to be explained - observe that the we "flip" the feature matrix about its rows and columns before passing it into <tt>conv2</tt>. This is necessary because the mathematical definition of convolution involves "flipping" the matrix that is convolved with, as explained in more detail in the implementation tip section below.
 
-
 
<div style="border:1px solid black; padding: 5px">
<div style="border:1px solid black; padding: 5px">

Revision as of 07:26, 20 May 2011

Personal tools