Exercise:Softmax Regression

From Ufldl

Jump to: navigation, search
(Step 0: Initialize constants and parameters)
(Step 2: Implement softmaxCost)
Line 25: Line 25:
In <tt>softmaxCost.m</tt>, implement code to compute the softmax cost function. Since minFunc minimizes this cost, we consider the '''negative''' of the log-likelihood <math>-\ell(\theta)</math>, in order to maximize <math>\ell(\theta)</math>. Remember also to include the weight decay term in the cost as well. Your code should also compute the appropriate gradients, as well as the predictions for the input data (which will be used in the cross-validation step later).  
In <tt>softmaxCost.m</tt>, implement code to compute the softmax cost function. Since minFunc minimizes this cost, we consider the '''negative''' of the log-likelihood <math>-\ell(\theta)</math>, in order to maximize <math>\ell(\theta)</math>. Remember also to include the weight decay term in the cost as well. Your code should also compute the appropriate gradients, as well as the predictions for the input data (which will be used in the cross-validation step later).  
-
 
It is important to vectorize your code so that it runs quickly. We also provide several implementation tips below:
It is important to vectorize your code so that it runs quickly. We also provide several implementation tips below:
-
 
'''Implementation Tip''': Computing the ground truth matrix - In your code, you may need to compute the ground truth matrix <tt>M</tt>, such that <tt>M(r, c)</tt> is 1 if <math>y^{(c)} = r</math> and 0 otherwise. This can be done quickly, without a loop, using the MATLAB functions <tt>sparse</tt> and <tt>full</tt>. <tt>sparse(r, c, v)</tt> creates a sparse matrix such that <tt>M(r(i), c(i)) = v(i)</tt> for all i. That is, the vectors <tt>r</tt> and <tt>c</tt> give the position of the elements whose values we wish to set, and <tt>v</tt> the corresponding values of the elements. Running <tt>full</tt> on a sparse matrix gives the full representation of the matrix for use. Note that the code for using <tt>sparse</tt> and <tt>full</tt> to compute the ground truth matrix has already been included in softmaxCost.m.
'''Implementation Tip''': Computing the ground truth matrix - In your code, you may need to compute the ground truth matrix <tt>M</tt>, such that <tt>M(r, c)</tt> is 1 if <math>y^{(c)} = r</math> and 0 otherwise. This can be done quickly, without a loop, using the MATLAB functions <tt>sparse</tt> and <tt>full</tt>. <tt>sparse(r, c, v)</tt> creates a sparse matrix such that <tt>M(r(i), c(i)) = v(i)</tt> for all i. That is, the vectors <tt>r</tt> and <tt>c</tt> give the position of the elements whose values we wish to set, and <tt>v</tt> the corresponding values of the elements. Running <tt>full</tt> on a sparse matrix gives the full representation of the matrix for use. Note that the code for using <tt>sparse</tt> and <tt>full</tt> to compute the ground truth matrix has already been included in softmaxCost.m.

Revision as of 02:17, 7 May 2011

Personal tools