Neural Network Vectorization

From Ufldl

Jump to: navigation, search
Line 130: Line 130:
% Sparsity Penalty Delta
% Sparsity Penalty Delta
sparsity_delta = - rho / rho_hat + (1 - rho) / rho_hat;
sparsity_delta = - rho / rho_hat + (1 - rho) / rho_hat;
-
delta2 = (W2'*delta3(:,i) + beta*sparsity_delta).* fprime(z2(:,i));
+
for i=1:m,
 +
  ...
 +
  delta2 = (W2'*delta3(:,i) + beta*sparsity_delta).* fprime(z2(:,i));
 +
  ...
 +
end;
</syntaxhighlight>  
</syntaxhighlight>  
-
* [TODO] Clean up above and add repmat instructions
+
Notice that the sparsity_delta is the same regardless of the example we are processing.
 +
 
 +
Recall that when we vectorizing the gradient computations, <tt>delta2</tt> is replaced with matrices. Thus, to vectorize the <tt>sparsity_delta</tt>, we can simply add it using <tt>repmat</tt> while computing the <tt>delta2</tt> matrices.

Revision as of 00:05, 5 April 2011

Personal tools