~

Page No... 7 EC63

[
,

NEWSCHE~E J
Reg. No.

Sixth Semester B.E. Degree Examination, January/February 2006

Electronics& Communication/TelecommunicationEngineering
Information Theory and Coding
Time: 3 hrs.) (Max. Marks : 100
Note: Answer any FIVEfull questions.
1. (a) Define:
i) Amount of information
ii) Average information and
....
iii) Rate of information as applied to a discrete memoryless source. (6 Marks)

(b) Prove that the entropy of a discrete memory less source is maximum, when all
the symbols are equally probable and hence plot entropy versus probability of a
source with two symbol outputs 0 and 1. (6+3=9Marks)

(c) In conventional telegraphy we use dots and dashes to transmit messages. A dash
is thrice as long as a dot and one third as probable as a dot. Find
i) Information in a dot and dash
ii) average information in dot-dash-code
iii) If a dot takes 2 msec and same time is aHowed between symbols the
information rate of this code. (2+1+2 Marks)

2. (a) The state diagram of a Marleov source is given below:

-
~ (~tPt- d ~.I
--kJ
A.~ ~' t (1-1j

~/~

i) Find the entropy of each state Hi(i = 1,2,3).

ii) Find entropy of the source H.
iii) Find G1, G2 and G3 and show that G1 ~ G2 ~ G3 ~ A (3+2+7=12 Marks),

Contd 2
Page No... 2 EC6
(b) Consider a source Sgiving out symbols51,52 and 53 with probabilities of!, t and
respectively
i) Find the entropy of the source
ii) Write all the symbols of the 2nd extension of S and find its entropy
iii) Hence find relation between H( 82) & H( 8) (2+4+2 Mark

3. (a) What are the properties of codes? State Shannon's source encoding theorem.
. (3+3=6 Mark

(b) Use Shannon's encoding procedure for the following symbols and find codin
efficiency:

Symbol 51 52 53 54 55
Probability 0:4 0.2 0.2 0.1 0.1
(4+2=6 Mar~

(c) A source has an a1phabet of 7 symbols with probabilities as given below:

Symbol 51 52 53 54 55 56 5-I
Probability "1/4 1/4 1/8 1/8 1/8 1/16 1/16

Construct Huffman binary code and find its efficiency. (6+2 Marl

4. (a) A non-symmetric binary channel is given below:

'0
.1)

~! 7' It>0\) } '<t:

'J.. -
4-
~

i) Find H(x), H(y), H(xy) and H(y/x) given P[x = 0] = t and P[x = 1] =
Also a = 0.75 and j3 = 0.9
ii) Find the capacity of the binary symmetric channel given a = j3 = 0.9
(7+2 Mar'
(b) Define mutual information and list its properties. (5 Mar
(c) For a channel whose matrix is given below:
Y
0.6 0.2 0.2
P[Y/ X] = X 0.2 0.6 0.2
. [ 0.2 0.2 0.6 ]
Find I(XY) and channel capacity given the I/P symbols occur with equc
probability. (4+2=6Mar

5. (a) State and explain Shannon, Hartley law. Find the upper limit of the channel capac
with increasing bandwidth. Also explain the concept of SIN ratio and bandwic
tradeoff. (3+3+3Mar
 Page No... 3 EC63
(b) An analog signal is band limited as 4 kHz. It is sampled at 2.5 times an Nyquist
rate and each sample is quantized to 256 levels. These levels are equally likely to
occur. The samples are assumed to be statistically independent. Find
i) Information rate of the sampled signal
Ii) can yon transmit the signal without errors on a gaussan channel with 50 kHz
bandwidth and SIN rate of 23 dB?
iii) What bandwidth is needed to transmit the signa without errors if SIN ratio is
10 dB? (3+3+2Marks)
(c) Write the channel diagram and matrix of binary erasure channel. (2+1 Marks)
6. (a) Why do we need error control coding? What are the types of errors and types of
coding to combat them? (2+2+2Marks)
(b) The parity check bits of a (8,4) linear block code Is given by
05 = d1 + d2 + d4
06 = d1 + d2 + d3
07 = d1 + d3 + d4
08 = d2 + d3 +
d4
where d1, d2, d3 and d4 are data bits.
i) Findthe generator and parity check matrix of this code
ii) Findthe code vectors for all data inputs using the above generator matrix
Hi) Findthe minimumweight of this coding scheme..(4+S+2 Marks)
7. A(15,5) algebraic code is generated using the generator polynomial
g(X) =1 + X + X2 + X4 + X5 + X8 + XlO
(a) Draw the block diagram of encoder and syndrome calculptor
(b) Find the code polynomial for the message polynomial 1 + X2 + X4 using the
encoder diagram
(c) Is v( x) = 1 + x4 + x6 + x8 + x14 a code polynomial or not? (5+5+7+3=20 Marks)

" 8. Consider the convolutional encoder given below:

tJp
,f~ ( ~rJ 1-

1{f
Page No... 4 EC~

(a) Write the impulse response of this encoder

(b) Find the output for the message (10011) using time domain approach
(c) Find the output for the message in (b) using transform domain approach.
(tl) Draw the code tree for the encoder. (3+5+5+7 Marl

** * **