27 JUL 2018 by ideonexus

 Shannon's Learning Mouse Theseus

Theseus was propelled by a pair of magnets, one embedded in its hollow core, and one moving freely beneath the maze. The mouse would begin its course, bump into a wall, sense that it had hit an obstacle with its “whiskers,” activate the right relay to attempt a new path, and then repeat the process until it hit its goal, a metallic piece of cheese. The relays stored the directions of the right path in “memory”: once the mouse had successfully navigated the maze by trial and error, it ...
  1  notes
 
27 JUL 2018 by ideonexus

 Redundancy of English language is a Goldilocks zone for C...

In Shannon’s terms, the feature of messages that makes codecracking possible is redundancy. A historian of cryptography, David Kahn, explained it like this: “Roughly, redundancy means that more symbols are transmitted in a message than are actually needed to bear the information.” Information resolves our uncertainty; redundancy is every part of a message that tells us nothing new. Whenever we can guess what comes next, we’re in the presence of redundancy. Letters can be redundant: be...
  1  notes
 
27 JUL 2018 by ideonexus

 Redundancy in the English Language

Whenever we communicate, rules everywhere restrict our freedom to choose the next letter and the next pineapple.I Because these rules render certain patterns more likely and certain patterns almost impossible, languages like English come well short of complete uncertainty and maximal information: the sequence “th” has already occurred 6,431 times in this book, the sequence “tk” just this once. From the perspective of the information theorist, our languages are hugely predictable— al...
  1  notes

Monte Carlo method for building words and sentences.

05 JUN 2012 by ideonexus

 Rapid Growth of Physics

As a result of the phenomenally rapid change and growth of physics, the men and women who did their great work one or two generations ago may be our distant predecessors in terms of the state of the field, but they are our close neighbors in terms of time and tastes. This may be an unprecedented state of affairs among professionals; one can perhaps be forgiven if one characterizes it epigrammatically with a disastrously mixed metaphor; in the sciences, we are now uniquely privileged to sit si...
  1  notes

Reminds me of the rapid growth of IT, where working with people just 10 years older marks a phenomenal difference in technological understanding.

01 JAN 2012 by ideonexus

 Measuring Cultural Information

But there may be more significant ways to characterize civilizations than by the energy they use for communications purposes. An important criterion of a civilization is the total amount of information that it stores. This information can be described in terms of bits, the number of yes-no statements concerning itself and the universe that such a civilization knows. An example of this concept is the popular game of "Twenty Questions," as played on Earth. One player imagines an object or conc...
  1  notes

The number of bits communicated in our radio broadcasts is quite enormous, conveying a great deal of information about our culture.

03 JAN 2011 by ideonexus

 Number of Bits for a Set of Encyclopedias are Minuscule C...

I have estimaged how many letters there are in a the Enclyclopaedia, and I have assumed that each of my 24 million books is as big as an Encyclopaedia volume, and have calculated, then, how many bits of information there are (10^15). For each bit I allow 100 atoms. And it turns out that all of the information that man has carefully accumulated in all the books in the world can be written in this form in a cube of material one two-hundredths of an inch wide--which is the barest piece of dust t...
  1  notes

Feynman estimates the number of atoms neccessary for storing a set of encyclopedias, and then compares that to the amount of data included in a DNA string.