Deep Networks: Overview
From Ufldl
(→Advantages of deep networks) |
(→Difficulty of training deep architectures) |
||
Line 69: | Line 69: | ||
researchers had little success training deep architectures. | researchers had little success training deep architectures. | ||
- | The main | + | The main learning algorithm that researchers were using was to randomly initialize |
- | the weights of | + | the weights of a deep network, and then train it using a labeled |
training set <math>\{ (x^{(1)}_l, y^{(1}), \ldots, (x^{(m_l)}_l, y^{(m_l}) \}</math> | training set <math>\{ (x^{(1)}_l, y^{(1}), \ldots, (x^{(m_l)}_l, y^{(m_l}) \}</math> | ||
using a supervised learning objective, using gradient descent to try to | using a supervised learning objective, using gradient descent to try to |