Scribd
Upload
12 views4 pages

Infomation Theory

The document contains MATLAB Live Script solutions for Homework #2, addressing questions on Huffman encoding, wine tasting strategies, and Shannon code construction. It includes calculations for Huffman coding, expected lengths, symbol probabilities, and optimal tasting strategies, as well as the generation of Shannon codes with their respective lengths. Key results include average code lengths and expected tastings for different strategies.

Uploaded by

coolchessplayer001
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
12 views4 pages

Infomation Theory

The document contains MATLAB Live Script solutions for Homework #2, addressing questions on Huffman encoding, wine tasting strategies, and Shannon code construction. It includes calculations for Huffman coding, expected lengths, symbol probabilities, and optimal tasting strategies, as well as the generation of Shannon codes with their respective lengths. Key results include average code lengths and expected tastings for different strategies.

Uploaded by

coolchessplayer001
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

MATLAB Live Script for Labs: Questions 3, 5, and 7

This live script implements solutions for Homework #2 questions:

Lab for Question 3: Huffman via MATLAB

% (a) Huffman encoding for given pmf


p = [1/2, 1/8, 1/8, 1/16, 1/16, 1/16, 1/16];
symbols = 0:6;
[dictionary, avglen] = huffmandict(symbols, p);
disp('Huffman dictionary (symbol : codeword):');

Huffman dictionary (symbol : codeword):

for i = 1:length(symbols)
code = dictionary{i,2};
fprintf('%d : [%s]\n', symbols(i), num2str(code));
end

0 : [0]
1 : [1 1 1]
2 : [1 1 0]
3 : [1 0 0 1]
4 : [1 0 0 0]
5 : [1 0 1 1]
6 : [1 0 1 0]

fprintf('Expected code length per symbol: %.4f bits\n', avglen);

Expected code length per symbol: 2.2500 bits

% (b) PMF of compressed file and expected length for 1000 symbols
expected_len = avglen * 1000;
% Compute probability of 0 and 1 in compressed bits
total0 = sum(p .* cellfun(@(c) sum(c==0), dictionary(:,2)));
total1 = sum(p .* cellfun(@(c) sum(c==1), dictionary(:,2)));
prob0 = total0 / avglen;
prob1 = total1 / avglen;
fprintf('Probability of 0: %.4f, Probability of 1: %.4f\n', prob0, prob1);

Probability of 0: 2.2222, Probability of 1: 0.5556


Probability of 0: 0.5556, Probability of 1: 0.2778
Probability of 0: 0.2778, Probability of 1: 0.2778
Probability of 0: 0.2778, Probability of 1: 2.8889
Probability of 0: 0.7222, Probability of 1: 0.7222
Probability of 0: 0.3611, Probability of 1: 0.3611
Probability of 0: 0.3611, Probability of 1: 0.3611

fprintf('Expected total length for 1000 symbols: %.0f bits\n', expected_len);

Expected total length for 1000 symbols: 2250 bits

% (c) Generate sequence of 10,000 symbols

1
t = 10000;
seq = randsample(symbols, t, true, p);

% (d) Percentage of each symbol


disp('Percentage of each symbol:');

Percentage of each symbol:

percentages = histcounts(seq, 0:7)/t*100;


for i = 1:length(symbols)
fprintf('Symbol %d: %.2f%%\n', symbols(i), percentages(i));
end

Symbol 0: 50.54%
Symbol 1: 12.35%
Symbol 2: 12.41%
Symbol 3: 6.18%
Symbol 4: 6.01%
Symbol 5: 6.31%
Symbol 6: 6.20%

% (e) Simple binary representation for 3 bits per symbol


simpleCodes = arrayfun(@(x) dec2bin(x,3), symbols, 'UniformOutput', false);

% (f) Length and bit percentages for simple representation


bitStrings = simpleCodes(seq+1);
concatenated = [bitStrings{:}];
len_simple = numel(concatenated);
prob0_simple = sum(concatenated=='0')/len_simple;
prob1_simple = sum(concatenated=='1')/len_simple;
fprintf('Simple representation length: %d bits\n', len_simple);

Simple representation length: 30000 bits

fprintf('Simple 0%%: %.4f, 1%%: %.4f\n', prob0_simple, prob1_simple);

Simple 0%: 0.7728, 1%: 0.2272

% (g) Compress sequence with Huffman code


compressed = huffmanenco(seq, dictionary);
len_compressed = numel(compressed);
prob0_comp = sum(compressed==0)/len_compressed;
prob1_comp = sum(compressed==1)/len_compressed;
fprintf('Compressed length: %d bits\n', len_compressed);

Compressed length: 22362 bits

fprintf('Compressed 0%%: %.4f, 1%%: %.4f\n', prob0_comp, prob1_comp);

Compressed 0%: 0.5011, 1%: 0.4989

% (i) Explanation:

2
% The Huffman code reduces average code length below the fixed-length of 3
bits,
% resulting in fewer total bits and a change in bit probability due to
% variable-length assignments favoring more likely symbols.

Lab for Question 5: Bad Wine Tasting Strategies

% Probabilities for each of the 6 bottles


p = [7, 5, 4, 4, 3, 3] / 26;
n = length(p);

% (a) Optimal sequential tasting (test bottles in descending probability)


[psorted, order] = sort(p, 'descend');
E_seq = sum(psorted .* (1:n));
fprintf('Sequential tasting expected number of tastings: %.4f\n', E_seq);

Sequential tasting expected number of tastings: 3.0000

fprintf('Taste first bottle: %d\n', order(1));

Taste first bottle: 1

% (c) Optimal mixture strategy by testing all subset mixtures


bestE = inf;
bestS = [];
for k = 1:(2^n - 2)
S = find(bitget(k,1:n));
C = setdiff(1:n, S);
P_S = sum(p(S));
P_C = 1 - P_S;
if P_S > 0 && P_C > 0
% Order tests within groups by descending probability
[~, idxS] = sort(p(S), 'descend');
[~, idxC] = sort(p(C), 'descend');
% Expected number of extra tests in each group
E_rank_S = sum((1:length(S)) .* (p(S(idxS)) / P_S));
E_rank_C = sum((1:length(C)) .* (p(C(idxC)) / P_C));
% Total expected tests including mixture taste
E = 1 + P_S * E_rank_S + P_C * E_rank_C;
if E < bestE
bestE = E;
bestS = S;
end
end
end
fprintf('Optimal mixture strategy expected tastings: %.4f\n', bestE);

Optimal mixture strategy expected tastings: 2.7692

fprintf('Bottles to mix in first test: %s\n', mat2str(bestS));

3
Bottles to mix in first test: [1 3 5]

Lab for Question 7: Shannon Code Construction

% Distribution for symbols 1 to 4


p = [0.5, 0.25, 0.125, 0.125];
m = length(p);

% (a) Compute cumulative distribution F and code lengths


F = [0, cumsum(p(1:end-1))];
L = ceil(-log2(p));

% Generate Shannon codes


disp('Shannon codewords (symbol : codeword):');

Shannon codewords (symbol : codeword):

avgL = sum(p .* L);


for i = 1:m
Fi = F(i);
li = L(i);
k = floor(Fi * 2^li);
code = dec2bin(k, li);
fprintf('Symbol %d: %s (length = %d)\n', i, code, li);
end

Symbol 1: 0 (length = 1)
Symbol 2: 10 (length = 2)
Symbol 3: 110 (length = 3)
Symbol 4: 111 (length = 3)

fprintf('Average code length: %.4f bits/symbol\n', avgL);

Average code length: 1.7500 bits/symbol

You might also like