反向传导算法
From Ufldl
Line 139: | Line 139: | ||
</math> | </math> | ||
[译者注:由于原作者简化了推导过程,会影响理解,我将推导过程补全为以下公式: | [译者注:由于原作者简化了推导过程,会影响理解,我将推导过程补全为以下公式: | ||
- | + | :<math> | |
\delta^{(n_l)}_i = \frac{\partial}{\partial z^{n_l}_i}J(W,b;x,y) | \delta^{(n_l)}_i = \frac{\partial}{\partial z^{n_l}_i}J(W,b;x,y) | ||
= \frac{\partial}{\partial z^{n_l}_i}\frac{1}{2} \left\|y - h_{W,b}(x)\right\|^2 | = \frac{\partial}{\partial z^{n_l}_i}\frac{1}{2} \left\|y - h_{W,b}(x)\right\|^2 | ||
Line 150: | Line 150: | ||
</math> | </math> | ||
[译者注:由于原作者简化了推导过程,使我本人看着十分费解,于是就自己推导了一遍,将过程写在这里: | [译者注:由于原作者简化了推导过程,使我本人看着十分费解,于是就自己推导了一遍,将过程写在这里: | ||
- | + | :<math> | |
\delta^{(n_{l-1})}_i = \frac{\partial}{\partial z^{n_l-1}_i}J(W,b;x,y) = \frac{\partial}{\partial z^{n_l}_i}J(W,b;x,y)\cdot\frac{\partial z^{n_l}_i}{\partial z^{n_{l-1}}_i} | \delta^{(n_{l-1})}_i = \frac{\partial}{\partial z^{n_l-1}_i}J(W,b;x,y) = \frac{\partial}{\partial z^{n_l}_i}J(W,b;x,y)\cdot\frac{\partial z^{n_l}_i}{\partial z^{n_{l-1}}_i} | ||
</math> | </math> | ||
- | + | ||
+ | :<math> | ||
= \delta^{(n_l)}_i\cdot\frac{\partial z^{n_l}_i}{\partial z^{n_{l-1}}_i} = \delta^{(n_l)}_i\cdot\frac{\partial}{\partial z^{n_{l-1}}_i}\sum_{j=1}^{s_{l-1}} W^{n_l-1}_{ji} f(z^{n_l-1}_i) | = \delta^{(n_l)}_i\cdot\frac{\partial z^{n_l}_i}{\partial z^{n_{l-1}}_i} = \delta^{(n_l)}_i\cdot\frac{\partial}{\partial z^{n_{l-1}}_i}\sum_{j=1}^{s_{l-1}} W^{n_l-1}_{ji} f(z^{n_l-1}_i) | ||
= \left( \sum_{j=1}^{s_{l-1}} W^{n_l-1}_{ji} \delta^{(n_l)}_i \right) f(z^{n_l-1}_i) | = \left( \sum_{j=1}^{s_{l-1}} W^{n_l-1}_{ji} \delta^{(n_l)}_i \right) f(z^{n_l-1}_i) |