Aftab, Cheung, Kim, Thakkar, Yeddanapudi I
NFORMATION
T
HEORY
& T
HE
D
IGITAL
R
EVOLUTION
6
6.933 Project History, Massachusetts Institute of Technology
SNAPES@MIT.EDU
The traditional solution was to use narrow-band radios, which would focus all their power into a
small range of frequencies. The problem was that as the number of users increased, the number
of channels began to be used up. Additionally, such radios were highly susceptible to
interference: so much power was confined to a small portion of the spectrum that a single
interfering signal in the frequency range could disrupt communication
Shannon offered a solution to this problem by redefining the relationship between information,
noise and power. Shannon quantified the amount of information in a signal, stating that is the
amount of unexpected data the message contains. He called this information content of a
message ‘entropy’. In digital communication a stream of unexpected bits is just random noise.
Shannon showed that the more a transmission resembles random noise, the more information it
can hold, as long as it is modulated to an appropriate carrier: one needs a low entropy carrier to
carry a high entropy message. Thus Shannon stated that an alternative to narrow-band radios was
sending a message with low power, spread over a wide bandwidth.
Spread spectrum is just such a technique: it takes a narrow band signal and spreads its power
over a wide band of frequencies. This makes it incredibly resistant to interference. However it
does use additional frequency ranges, and thus the FCC until recently had confined the technique
to the military. It is now widely used in CDMA cellular phones.
Now that we’ve discussed some of the fundamental concepts in Shannon’s work, let’s take a step
back and see how the formalization of these concepts started a chain of research that eventually
became known as the field of Information Theory.
TRAJECTORY OF INFORMATION THEORY - I
We begin by exploring the history of Information Theory, how the field evolved and weathered
various influences to become what it is today. In essence, we chart the trajectory of a new
science.
Creating the Field
Information Theory grew out of the concepts introduced in "A Mathematical Theory of
Communication." Although, the phrase "information theory" was never used in the paper,
Shannon's emphasis on the word "information" probably helped coin the term. The idea that
something as nebulous as "information" could be quantified, analyzed, and reduced to a
mathematical formula attracted tremendous attention.
This initial excitement gave life to the field. But what were the forces that enabled this process?
According to Latour, one of the tasks in creating a new field is gathering the support and
enthusiasm of the masses
4
. Although Shannon had intended his audience to be confined to
communication engineering, his concepts and methodology of thinking quickly moved into the
popular press. 1953’s Fortune magazine gushingly describes the field as more crucial to
‘
man's
progress in peace, and security in war’ than Einstein’s nuclear physics.
4
Latour. B,
Science in Action
, pg. 150