Redundancy in the English Language

Whenever we communicate, rules everywhere restrict our freedom to choose the next letter and the next pineapple.I Because these rules render certain patterns more likely and certain patterns almost impossible, languages like English come well short of complete uncertainty and maximal information: the sequence “th” has already occurred 6,431 times in this book, the sequence “tk” just this once. From the perspective of the information theorist, our languages are hugely predictable— al...

 Redundancy of English language is a Goldilocks zone for C...

In Shannon’s terms, the feature of messages that makes codecracking possible is redundancy. A historian of cryptography, David Kahn, explained it like this: “Roughly, redundancy means that more symbols are transmitted in a message than are actually needed to bear the information.” Information resolves our uncertainty; redundancy is every part of a message that tells us nothing new. Whenever we can guess what comes next, we’re in the presence of redundancy. Letters can be redundant: be...

 Shannon's Learning Mouse Theseus

Theseus was propelled by a pair of magnets, one embedded in its hollow core, and one moving freely beneath the maze. The mouse would begin its course, bump into a wall, sense that it had hit an obstacle with its “whiskers,” activate the right relay to attempt a new path, and then repeat the process until it hit its goal, a metallic piece of cheese. The relays stored the directions of the right path in “memory”: once the mouse had successfully navigated the maze by trial and error, it ...

 Claude Shannon's "Entropy House"

Built in 1858, the house was constructed for Ellen Dwight, a great-granddaughter of a genius tinkerer of an earlier era, Thomas Jefferson. Originally seated on twelve acres, its design was inspired by Monticello. Encircled by “a three-sided verandah with segmental openings and chamfered posts,” the house was a stately three stories at the crest of a “broad expanse of lawn reaching down to the wooded shore of Upper Mystic Lake.” Toward the end of Shannon’s life, it was added to the N...

 Shannon and Thorp Hack the Roulette Wheel

It was in this tinkerer’s laboratory that they set out to understand how roulette could be gamed, ordering “a regulation roulette wheel from Reno for $1,500,” a strobe light, and a clock whose hand revolved once per second. Thorp was given inside access to Shannon in all his tinkering glory: Gadgets . . . were everywhere. He had a mechanical coin tosser which could be set to flip the coin through a set number of revolutions, producing a head or tail according to the setting. As a joke...