How Neural Circuitry is Laid Down

In thinking about hidden layers, it’s important to distinguish between the routine efficiency and power of a good network, once that network has been set up, and the difficult issue of how to set it up in the first place. That difference is reflected in the difference between playing the piano (or, say, riding a bicycle, or swimming) once you’ve learned (easy) and learning to do it in the first place (hard). Understanding exactly how new hidden layers get laid down in neural circuitry is a great unsolved problem of science. I’m tempted to say it’s the greatest unsolved problem.

Liberated from its origin in neural networks, the concept of hidden layers becomes a versatile metaphor with genuine explanatory power. For example, in my work in physics I’ve noticed many times the impact of inventing names for things. When Murray Gell-Mann invented “quarks,” he was giving a name to a paradoxical pattern of facts. Once that pattern was recognized, physicists faced the challenge of refining it into something mathematically precise and consistent, but identifying the problem was the crucial step toward solving it! Similarly, when I invented “anyons,” for theoretical particles existing in only two dimensions, I knew I had put my finger on a coherent set of ideas, but I hardly anticipated how wonderfully those ideas would evolve and be embodied in reality. In cases like this, names create new nodes in hidden layers of thought.


Frank Wilczek describes one of the great questions of science, how the difficult taks of learning something leads to the learned easy of later doing it.

Folksonomies: cognition neurology

/science/physics (0.468391)
/technology and computing/consumer electronics/camera and photo equipment/cameras and camcorders/cameras (0.389954)
/society/crime/personal offense/homicide (0.177863)

neural circuitry (0.935934 (positive:0.412013)), hidden layers (0.880760 (positive:0.531826)), unsolved problem (0.814173 (positive:0.015952)), greatest unsolved problem (0.765458 (negative:-0.515875)), great unsolved problem (0.739666 (positive:0.531826)), genuine explanatory power (0.728961 (positive:0.718116)), difficult taks (0.563616 (neutral:0.000000)), Frank Wilczek (0.553694 (positive:0.292200)), difficult issue (0.504432 (neutral:0.000000)), routine efficiency (0.503218 (positive:0.580455)), great questions (0.486237 (positive:0.292200)), good network (0.477177 (positive:0.580455)), Murray Gell-Mann (0.475049 (neutral:0.000000)), neural networks (0.471477 (neutral:0.000000)), versatile metaphor (0.461364 (positive:0.718116)), paradoxical pattern (0.453762 (positive:0.502812)), theoretical particles (0.442549 (neutral:0.000000)), crucial step (0.440206 (neutral:0.000000)), coherent set (0.433384 (negative:-0.420248)), new nodes (0.429256 (neutral:0.000000)), difference (0.247918 (neutral:0.000000)), science (0.243588 (positive:0.412013)), place (0.242446 (positive:0.208976)), names (0.226442 (negative:-0.232596)), ideas (0.211192 (negative:-0.420248))

Frank Wilczek:Person (0.957758 (positive:0.292200)), coherent:OperatingSystem (0.736494 (negative:-0.420248)), Murray:Person (0.660166 (neutral:0.000000))

Quantum chromodynamics (0.983380): dbpedia | freebase | opencyc
Mathematics (0.972078): dbpedia | freebase | opencyc
Theory (0.825371): dbpedia | freebase
Neural network (0.697452): dbpedia | freebase | opencyc
Cognitive science (0.665694): dbpedia | freebase | opencyc
Psychology (0.656115): dbpedia | freebase | opencyc
Quark (0.654344): dbpedia | freebase
Neuroscience (0.607276): dbpedia | freebase

 This Will Make You Smarter
Books, Brochures, and Chapters>Book:  Brockman , John (2012-02-14), This Will Make You Smarter, HarperCollins, Retrieved on 2013-12-19
  • Source Material []
  • Folksonomies: science