用反向传导思想求导
From Ufldl
| Line 1: | Line 1: | ||
[原文] | [原文] | ||
| - | == Introduction == | + | == Introduction [简介]== |
In the section on the [[Backpropagation Algorithm | backpropagation algorithm]], you were briefly introduced to backpropagation as a means of deriving gradients for learning in the sparse autoencoder. It turns out that together with matrix calculus, this provides a powerful method and intuition for deriving gradients for more complex matrix functions (functions from matrices to the reals, or symbolically, from <math>\mathbb{R}^{r \times c} \rightarrow \mathbb{R}</math>). | In the section on the [[Backpropagation Algorithm | backpropagation algorithm]], you were briefly introduced to backpropagation as a means of deriving gradients for learning in the sparse autoencoder. It turns out that together with matrix calculus, this provides a powerful method and intuition for deriving gradients for more complex matrix functions (functions from matrices to the reals, or symbolically, from <math>\mathbb{R}^{r \times c} \rightarrow \mathbb{R}</math>). | ||
| Line 166: | Line 166: | ||
[原文] | [原文] | ||
| - | == Examples == | + | == Examples[示例] == |
To illustrate the use of the backpropagation idea to compute derivatives with respect to the inputs, we will use two functions from the section on [[Sparse Coding: Autoencoder Interpretation | sparse coding]], in examples 1 and 2. In example 3, we use a function from [[Independent Component Analysis | independent component analysis]] to illustrate the use of this idea to compute derivates with respect to weights, and in this specific case, what to do in the case of tied or repeated weights. | To illustrate the use of the backpropagation idea to compute derivatives with respect to the inputs, we will use two functions from the section on [[Sparse Coding: Autoencoder Interpretation | sparse coding]], in examples 1 and 2. In example 3, we use a function from [[Independent Component Analysis | independent component analysis]] to illustrate the use of this idea to compute derivates with respect to weights, and in this specific case, what to do in the case of tied or repeated weights. | ||
| Line 177: | Line 177: | ||
[一审] | [一审] | ||
| + | |||
| + | 示例 | ||
为了描述如何使用反向传导思想计算对于输入的导数，我们在例1,例2中用到稀疏编码一节中的两个函数。在例3中，我们使用一个独立成分分析一节中的函数来描述如何使用此思想计算对于权重的偏导，以及在这个特定例子中，如何处理相同或重复的权重的情况。 | 为了描述如何使用反向传导思想计算对于输入的导数，我们在例1,例2中用到稀疏编码一节中的两个函数。在例3中，我们使用一个独立成分分析一节中的函数来描述如何使用此思想计算对于权重的偏导，以及在这个特定例子中，如何处理相同或重复的权重的情况。 | ||
| Line 182: | Line 184: | ||
[原文] | [原文] | ||
| - | === Example 1: Objective for weight matrix in sparse coding === | + | === Example 1: Objective for weight matrix in sparse coding [例1:对稀疏编码中权重矩阵的目标函数]=== |
Recall for [[Sparse Coding: Autoencoder Interpretation | sparse coding]], the objective function for the weight matrix <math>A</math>, given the feature matrix <math>s</math>: | Recall for [[Sparse Coding: Autoencoder Interpretation | sparse coding]], the objective function for the weight matrix <math>A</math>, given the feature matrix <math>s</math>: | ||
| Line 415: | Line 417: | ||
[原文] | [原文] | ||
| - | === Example 2: Smoothed topographic L1 sparsity penalty in sparse coding | + | === Example 2: Smoothed topographic L1 sparsity penalty in sparse coding [例2：稀疏编码中的平滑地形L1稀疏罚函数] === |
Recall the smoothed topographic L1 sparsity penalty on <math>s</math> in [[Sparse Coding: Autoencoder Interpretation | sparse coding]]: | Recall the smoothed topographic L1 sparsity penalty on <math>s</math> in [[Sparse Coding: Autoencoder Interpretation | sparse coding]]: | ||
| Line 660: | Line 662: | ||
[原文] | [原文] | ||
| - | === Example 3: ICA reconstruction cost === | + | === Example 3: ICA reconstruction cost [例3：ICA重建成本] === |
Recall the [[Independent Component Analysis | independent component analysis (ICA)]] reconstruction cost term: | Recall the [[Independent Component Analysis | independent component analysis (ICA)]] reconstruction cost term: | ||
