Communications-centric Motivation: Communication over noisy channels.

Noise

Source Encoder Channel Decoder Destination

Examples: Email over internet, Voice over phone network, Data over USB storage.
In these examples, the source would be the voice, email text, and data, the channel would be the
(wired
or wireless) phone network, internet, and storage device.
Purpose: Reproduce source data at the destination (with presence of noise in channel)
Channel can introduce noise (static), interference (multiple copies at destination arriving at
different times from radio waves bouncing on buildings, etc), or distortion (amplitude, phase,
modulation)
Sources have redundancy, e.g. the english language, can reconstruct words from parts: th_t
that, and images, where individual pixel differences are not noticed by the human eye. Some
sources can tolerate distortion (images).
A typical example of an encoder is broken into three components:
From Source Source Encoder Channel Encoder Modulator to Channel
The source encoder outputs a source codeword, an efficient representation of the source,
removing redundancies. The output is taken to be a sequence of bits (binary)
The channel encoder outputs a channel codeword, introducing controlled redundancy to
tolerate errors that may be introduced by the channel. The controlled redundancy is taken into
account while decoding. Note the distinction from redundancies present in the source, where the
redundancies may be arbitrary (many pixels in an image). The codeword is designed by examining
channel properties. The simplest channel encoder involves just repeating the source codeword.
The modulator converts the bits of the channel codeword into waveforms for transmission over
the channel. They may be modulated in amplitude or frequency, e.g. radio broadcasts.
The corresponding decoder can also be broken into three components that invert the operations of
the
source encoder above:

From Channel Demodulator Channel Decoder Source Decoder To Destination

We call the output of the demodulator the received word and the output of the channel decoder
the estimated source codeword.
Remark. Later we will see that separating the source and channel encoder/decoder as above does
not
cause a loss in performance under certain conditions. This allows us to concentrate on each
component
independently of the other components. Called the Joint Source Channel Coding Theorem.