STUDENT ID NO MULTIMEDIA eo UNIVERSITY MULTIMEDIA UNIVERSITY FINAL EXAMINATION TRIMESTER 2, 2022/2023 (TERM ID 2220) TIT3131 —- INFORMATION THEORY (All sections / Groups) 6 JULY 2023 9.00 am ~ 11.00 am (2 Hours). INSTRUCTIONS TO STUDENTS 1. This question paper consists of 6 pages with Four questions only. 2. Attempt All questions. All questions carry equal marks and the distribution of the marks for each question is given. 3, Please print all your answers in the Answer Booklet provided.QUESTION 1 Question 1 contains 10 MCQ questions, please choose the best answer and write it clearly in the answer booklet. @ Information theory applies the laws of probability, and mathematics in general, to study the collection and manipulation of information. {1 mark] Ru: Po Lacture 01.02. prt! (B] FALSE. [ii we 0) ‘The amount of information, I(S) is related to the inverse of the probability pfeccurrence. UL mari (a) TRUE, B] FALSE, © H(S™), the entropy of the extended source, is equal to 200*H(S), the entropy ofthe original source, [1 mark} Al TRUE. (B)) FALSE. @ Given an DMS, $= {A,B, C, D} and the probability of occurrence for these symbols are showed in the Table 1. Calculate the information gained for symbol B. {1 marlg Table 1 = x B c DY Be Or 02 a3 04, MSs) I A] (Ss) =3.3219 divsymbol ({B) 18s) =23219 bitisymbot IC] MSs)= 13219 bivsymbol [D] None of the above. © A prefix code is defined as the code in which no code word isthe prefix of any other code word. [1 mar] TRUE. |] FALSE. © Block codes in which message bits are transmitted in unaltered form are called systematic codes. [1 mari fa) TRUE. [B] FALSE. Continued. 6cc (a) @ oO Ter The mutual information of a channel is symmetric, that is the mutual information of 10,¥) equal to 1(¥,X). [mark] (a) TRUE. B] FALSE. From the Table 2 below, given that the Code Tis a prefix code, estimate a suitable code for.x. {mark} Table 2 Symbol Code A 000 B 001 c x D 1000 [A] x=000. * [B] -x=100.~ x= 0010. None of the above. Given that message m = (10110). The message polynomial MD) is equal tol+D?+DS 1 oe {1 mark] TRUE @ FALSE Consider a generator matrix (6, 3) linear block code below. The generator matrix is expressed in non-systematic form, {1 mark] 011100 G=11 11010 Systema tie 110001 y % ef Tq, ie Continued... 316wmmat____ ron mpony tan QUESTION 2 (a) Consider the following source with discrete memoryless source with source alphabet S = {So, Si, S2} with po = 0.5, pi = 0.25, p2 = 0.25. Prove that H(S?) = 2 H(S). [3 marks] (b) Construct the encoder circuit for the generator polynomial for a (6, 3) cyclic code generated by g(D)=1+D*+D*. [4 marks] (©) Given convolutional code of path 1, path 2 and path 3 in Figure 1. Find gl, g2 id g3. Cc {3 marks} tn 1 Output o Pats Figure 1 Pot | = Mo gu) = C1,0,9) path 2: tm, COM) Gg (2) a C 6, 0 I ! ) Path 3: Mo, ™ ga)= (1, 1,0) Continued.smoryst_____________infowatiowtipory ___e eis 2023 QUESTION 3 (a) From the Figure 2 below, given that P(X) = {0.58, 0.42}. Show that I(¥,X) is equal fo 1(X,F). You must show all the necessary steps. [8 marks} POND (b) From the Table 3 below, use Kraft inequality to determine whether the following code are unique decodable. You must show all the necessary steps. {2 marks] Table 3 Symbol Code I Code IT A 000, 00. B Or HW c 010 10 D Hi id Continued... Te 316QUESTION 3 (a) From the Figure 2 below, given that P(X) = {0.58, 0.42}. Show that I(¥,X) is equal to 1(X,¥). You must show all the necessary steps. [8 marks} POLO Pxlc) = pov) fro) P&L,y) = PCY|X) * PLD Conditional Provability mci Yo uy | Ye %> | 0.689% | o.re8e| o-0sran |—> havivontul dead =| | 0.2881 | oles | %-289F] > gastnar bozo Add H SF | Font Provawitity Tavle Ply) Ye 4, 4 Pcx) , | 2 0444 # 0-58 Ob x OSE |o0pIt ax OSE 0.6f Pur, yw Ss 04 = 0.15 = 0.03 . . Gy [esate Tarra fo | 2% Ak ih aR POD ‘feo. = 0.2 2 0.42 OD 2 2 PLY Poy} 9-5 0.35 0.15 1 4 Gea ) Conditronul — Probability 9exl@ he a, 04 10.5 D1 [05 i 70.8 20.2 > VBA. + | 015] 0.35 0-2/ 0.35 Yo fe odse | cosy |’ oT oS OT oy 4 YW f= oe zoe 2HO) = 0.68 log, Ci/oes) + 0-49 Ing, Life. 4a) = 0981G HOY) = 0.5 log, C1108) + 0.96 og, Cilo3) +015 log, Li/0-ts) 2 1.4lob w HOKE) = 04 logy Lis0.689F) + DAS Ing CY #28) + 0. 05 tog, C1/ 4.05142) + O-1 dogs C1 0-238) 4 o> fog, C1) 0-476) + 0-19 log, Calo 2st) * 1.3733 HOY) = 0b tog, C082 # 0-1 lags (110-294 0-15 log, Cilo.gade) + 69 logy C/o BHI) + 0-93 Ing, Cio. 2) + 0-2 bag, (i109) > 0-84) LU, = THY) Huy) - HCY) = HOD — HOY) 1.440, ~ 1.9723 = o.98iS - 0 GUI 0- | 6t3 = 0-\b+4(b) From the Table 3 below, use Kraft inequality to determine whether the following code are unique decodable. You must show all the necessary steps. {2 marks] Table 3 Symbol Code I ‘Code IL A 000 "00, ¢ af0 Si D iii Gib - 4 Kor 7 C222 %3 + 9 = 0.4345 <| -2 “3 Kot 7 (277 ¥3 + 2 = p-8ts «| Codd LT & & prefr code and it — Sutdey the Kat maquality. Thoredore, Code TS wry decodable Code TB ot ag pHeX lt, oMkough it sutisty he KvoFt quality, it Bnet tt B owt a prefix cle. wnruoly — decod able beacauseTT INFORMATION THEOR July 03 QUESTION 4 (@ A discrete memoryless source produces one of the five outputs {4, B, C, D, E} ) randomly. The corresponding output probabilities are 0.3, 0.25, 0.2, 0.15 and 0.1. i Create a Huffiman tree and encoding table for the source. You must show all the necessary steps. (4 marks} ii) Calculate the efficiency of this code. (3 marks} Consider the generator polynomial for a (6,3) cyclic code generated by g(D)=1+D+D’ . Construct all the valid code words for the following messages in “non-systematic" form. Show your steps. [3 marks] ‘Message ‘Code word ‘001 010) oil End of Page Tec 6QUESTION 4 (a) A discrete memoryless source produces one of the five outputs {4, B, C, D, E} randomly. The corresponding output probabilities are 0.3, 0.25, 0.2, 0.15 and 0.1. id Create a Huffman tree and encoding table for the source. You must show all the necessary steps. [4 marks] ii) Calculate the efficiency of this code. [3 marks] Huffman TRL Nyro | Stage Lo | Sage 1 Guy | Sage tv A 03 03 >o-ls > 65S £0] 6 0uK tS 0K No 2 oo | Sous TQ c 02 0% ta, | Sous ce D 1s Toh } PSo2 YH —& oy) od Ending tube Sym | codeword want probabil fy A 00 2 0.3 2 10 or’ Cc \ ) 2 0-2 D ol 0 2 6.18 E O\ \ 3 0. }Gi) HS) = 0 log, 1/03) # 928 Ing, C10 u) + OL togy (1/02) + 0-15 leg, Cio 1S) + 0 log, (10.159 = 9.2262 C= 300-2) + 900 VC) £1C0-2) + 3LO{S) + 20-1) UK EMriony = Hs) 1 CL = 9.2242 [ 2.6 X 1004. = 99 03%.(») Consider the generator polynomial for a (6,3) cyclic code generated by g(D)=1+ D+D* , Construct all the valid code words for the following messages in "non-systematic" form. Show your steps. [3 marks] [ Message QD) _ [ EO) Code word D |b? +0? +DF 001101 E oi rot ee Dept lat mt0) = o? let mp) = D clp) = mtv) gd) ctD) = mW) gcd) =) Cl4b+p?) = 6 (¢ I4p+D*) =p+p 404 ~ Dp +d? + d® (et mo) = 0+ p? CLD) = m0) yld) = (0+0*) Cit o+0*) = (0 + +t) + cp +0 + oS) = Pt po? + p* to