Previous | Next --- Slide 6 of 41
Back to Lecture Thumbnails

Depending on the application, one can either add more hidden layers or add more units to one of the hidden layers, either of which may improve accuracy, although often with diminishing returns.


What do hidden layers do?


@grarawr Hidden layers do some transforms on the input that the output can use. Intuitively you can think hidden layers like components in a face detector. Some hidden layers detect eyes, some detect ears, so detect nose.


@grarawr The hidden layers allow the network to model non-linear functions. The number of hidden layers and nodes per layer depends on what is being computed and what tradeoffs you are willing to make. For instance any boolean function of the inputs can be computed with a single hidden layer with an exponential number of hidden units. However, there are more efficient representations with more hidden layers with fewer hidden units.