How Neural Circuitry is Laid Down

In thinking about hidden layers, it’s important to distinguish between the routine efficiency and power of a good network, once that network has been set up, and the difficult issue of how to set it up in the first place. That difference is reflected in the difference between playing the piano (or, say, riding a bicycle, or swimming) once you’ve learned (easy) and learning to do it in the first place (hard). Understanding exactly how new hidden layers get laid down in neural circuitry is ...