Feature extraction using convolution
From Ufldl
(→Fully Connected Networks) |
(→Locally Connected Networks) |
||
Line 9: | Line 9: | ||
== Locally Connected Networks == | == Locally Connected Networks == | ||
- | One simple solution to | + | One simple solution to this problem is to restrict the connections between the hidden units and the input units, allowing each hidden unit to connect to only a small subset of the input units. Specifically, each hidden unit will connect to only a small contiguous region of pixels in the input. (For input modalities other than vision, there is often a natural way to select "contiguous groups" of inputs to connect to a single hidden units as well; for example, for audio, each hidden unit might be connected to only a certain time span of the input audio clip.) |
- | This idea of having locally connected networks also draws inspiration from how the early visual system is wired up. Specifically, neurons in the visual cortex | + | This idea of having locally connected networks also draws inspiration from how the early visual system is wired up in biology. Specifically, neurons in the visual cortex have localized receptive fields (i.e., they respond only to stimuli in a certain location). |
== Weight Sharing (Convolution) == | == Weight Sharing (Convolution) == |