Source coding Vs channel coding

Source codes compress naturally redundant messages for efficient

storage/transmission. Channel codes systematically add redundancy to
enable high rate transmissions with as few errors as possible.

Source Coding and Channel Coding are the children of Digital

Communication which itself revolutionized the field of communication and
has made everything that we see now possible. These ideas owe their
current form to Claude Shannon in his landmark paper, “A mathematical
theory of Communication”.

Shannon imagined a “source” as a generator of discrete symbols (for

example, bits) (and also as emitting real numbers in another idealistic
setting). This viewpoint is made possible because of sampling and further
quantization of the samples, which are the first steps for digital
communications. Furthermore he also assumed that these symbols coming
from a discrete set (like binary {0,1}) or a continuous set (like real
numbers) come with a probability distribution associated with them.

One of the key advantages of digital communication is that in order for the
communication to happen successfully, it is not important to convey the
actual source signals/messages themselves. It is sufficient to convey just a
lossless representation of the source.

For example, if the source had only two possible strings as outputs in any
given turn (for example, “YES” or “NO”), then it is sufficient to simply
transmit a 0(mapped to “YES”, say) and a 1 (for “NO”). The receiver can
easily reconstruct (assuming noiseless reception of the transmitted bit,) the
actual source message provided it knows this mapping between {YES, NO}
and {0,1}.
He then looked at answering the question - “Given a noiseless channel,
what is the most efficient way of communicating the source message to the
receiver?”. Asource code is something which attempts to answer this
mathematically (The rough idea is, highly probable symbols are assigned
the shortest coded transmissions). There have been many variations,
improvements, etc since Shannon’s time. But is the goal of all source coding
- using the channel (which can also be a ‘time’ channel like a storage
medium) most efficiently (transmit less, convey maximum).

Communication Channels however are not noise free. Noise essentially

means randomness or unpredictability. Shannon thus looked at the channel
using a “conditional probability distribution”, that is, he modeled the
channel using a set of chances of getting particular output symbols given
particular input symbols.

Given this random nature of the channel (we typically know something
about it, like the type of randomness, the degree of it etc.), Shannon asked
the question - “What is the maximum rate at which we can convey
information with small chance of error through this probabilistic channel?”,
and thus was born channel coding, which is the art of designing mapping
the (source-coded) information symbols to the noisy-channel such that
communication happens at the maximum possible rate as well as low error
probabilities. Again this has grown leaps and bounds over the years, but
Shannon gave us the foundation for it all.

1. Actually the correct terms are source encoding and channel

encoding. Source (en)coding: is a process of where the message is
coded/encoded at the source, much like the sender creating a
message like: "This is a message" and then coding the message with
a set of coding rules e.g: Have a=1, b=2, c=3 ... etc. and then
transmitting this message as numbers instead of letters. Now at the
receiving end, the receiver should also have such a decoding rules
such that the message is clearly understood. Extending on this you
can use compression etc which encapsulates the term source
(en)coding. Channel (en)coding : is a process of trying to correct
errors within transmitted message over the channel. A simple
example can be that of a checksum being sent across the message in
the above example. The checksum allows the channel to correct
some of the errors in the message.

2. Source coding is much about removing the redundant data or the data
which is not that important for the data to be conveyed to the
receiver, I mean the extra data. It is more like quantizing and run
length coding, where in you can send a code like '05'(zero five)which
symbolizes five zeros at a stretch instead of sending 00000. So in
source coding we remove more of a redundant data which is not.
Channel coding: Channel coding is more about adding some extra bits
in the form of parity bits so that you can protect the data from being
becoming corrupt. Or these data will also serve as the mathematical
encryption. And also even helps in correction the corrupt data.

3. Source coding is much about removing the redundant data or the data
which is not that important for the data to be conveyed to the
 receiver, I mean the extra data. It is more like quantizing and run
length coding, where in you can send a code like '05'(zero five)which
symbolizes five zeros at a stretch instead of sending 00000. So in
source coding we remove more of a redundant data which is not.
Channel coding: Channel coding is more about adding some extra bits
in the form of parity bits so that you can protect the data from being
becoming corrupt. Or these data will also serve as the mathematical
encryption. And also even helps in correction the corrupt data.

4. Source coding is the process that is used to encrypt information

removing unnecessary data so that bandwidth of the signal is
adjusted for effective transmission. Popular source coding methods-
Huffman coding, Shannon-fano coding etc. Channel coding is the
process that is used to protect the source code from being corrupt or
error detect at the receiver by adding some extra bits in the form of
parity bits. Popular channel coding methods - block codes,
convoloution codes etc.

5. Source Coding is the efficient representation of data generated by a

discrete source. This means we convert our generated source data to
binary form and it is uniquely decodable at the receiver. Suppose our
source generates four different voltages and we need to transmit that
information to the receiver. Now to do that we need to represent that
data which we can do by using two bits (00,01,10,11) that is four
different possibilities. To make source coding efficient we use
statistics of the source data. The source symbols which are more
probable are represented with short code and symbols which are least
 probable are represented with code longer than that of more probable
symbols. Examples of source coding are Huffman Coding , Lempel-
Ziv coding etc. Channel coding is done to control errors occurred due
to noise in the channel. The channel encoder in the transmitter
accepts the message bits and adds redundancy according to a
prescribed rule. The receiver exploits the redundancy and to decide
which message bits were transmitted. For more details about channel
encoding refer linear block codes, cyclic codes and convolutional
codes

6. I guess it's related to data transmission.

Source coding is done to encrypt the data so that it's not redundant
and precise and also takes less space. In short compressing it.
While channel coding is done by adding redundancy to the data so
that noise in the channel doest effect or corrupt the data.

7. Source coding is all about studying the structure of the information

that is to be stored or sent over a channel and representing the data

using minimum number of bits while channel coding involves error
detecting and error correcting codes. Some examples of sourde coders
may be gzip etc while error codes can be found in almost every
communication system to minimize error in data transfer.