Redundancy in the English Language

Whenever we communicate, rules everywhere restrict our freedom to choose the next letter and the next pineapple.I Because these rules render certain patterns more likely and certain patterns almost impossible, languages like English come well short of complete uncertainty and maximal information: the sequence “th” has already occurred 6,431 times in this book, the sequence “tk” just this once. From the perspective of the information theorist, our languages are hugely predictable— al...

 Measuring Cultural Information

But there may be more significant ways to characterize civilizations than by the energy they use for communications purposes. An important criterion of a civilization is the total amount of information that it stores. This information can be described in terms of bits, the number of yes-no statements concerning itself and the universe that such a civilization knows. An example of this concept is the popular game of "Twenty Questions," as played on Earth. One player imagines an object or conc...