Feature extraction using convolution

From Ufldl

Jump to: navigation, search
(Fully Connected Networks)
(Locally Connected Networks)
Line 9: Line 9:
== Locally Connected Networks ==
== Locally Connected Networks ==
-
One simple solution to the problem is to restrict the connections between the hidden units and the input units, allowing each hidden unit to connect to only a select number of input units. The selection of connections between the hidden and input units can often be determined based on the input modality -- e.g., for images, we will have hidden units that connect to local contiguous regions of pixels.  
+
One simple solution to this problem is to restrict the connections between the hidden units and the input units, allowing each hidden unit to connect to only a small subset of the input units. Specifically, each hidden unit will connect to only a small contiguous region of pixels in the input. (For input modalities other than vision, there is often a natural way to select "contiguous groups" of inputs to connect to a single hidden units as well; for example, for audio, each hidden unit might be connected to only a certain time span of the input audio clip.)
-
This idea of having locally connected networks also draws inspiration from how the early visual system is wired up. Specifically, neurons in the visual cortex are found to have localized receptive fields (i.e., they respond only to stimuli in a certain location).  
+
This idea of having locally connected networks also draws inspiration from how the early visual system is wired up in biology. Specifically, neurons in the visual cortex have localized receptive fields (i.e., they respond only to stimuli in a certain location).
== Weight Sharing (Convolution) ==
== Weight Sharing (Convolution) ==

Revision as of 18:08, 27 May 2011

Personal tools