Scribd
Upload
89 views23 pages

Arithmetic Lempel and Ziv Coding Chapter 2 Part 2 EH

The document discusses problems with Huffman coding and introduces arithmetic coding and Lempel-Ziv coding as universal source coding techniques that do not require knowledge of source statistics. It explains that arithmetic coding assigns a unique interval to each possible sequence and encodes the source by selecting the proper interval, achieving an average code length equal to entropy. It also describes the Lempel-Ziv coding algorithm which replaces substrings with dictionary codes and updates the dictionary after each match.

Uploaded by

dipu ghosh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
89 views23 pages

Arithmetic Lempel and Ziv Coding Chapter 2 Part 2 EH

The document discusses problems with Huffman coding and introduces arithmetic coding and Lempel-Ziv coding as universal source coding techniques that do not require knowledge of source statistics. It explains that arithmetic coding assigns a unique interval to each possible sequence and encodes the source by selecting the proper interval, achieving an average code length equal to entropy. It also describes the Lempel-Ziv coding algorithm which replaces substrings with dictionary codes and updates the dictionary after each match.

Uploaded by

dipu ghosh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 23

Universal Codes and Channel Capacity

Arithmetic Coding

Problems of Huffman coding:


• The size of Huffman coding table is qn, hence exponential
memory and computation is required.
• Code table needs to transmit to receiver.
• The source statistics are assumed stationary. If there are changes
an adaptive scheme is required which re-estimates the
probabilities, and recalculates the Huffman code table.
• Encoding and decoding is performed on a per block basis; the
code is not produced until a block of n symbols is received.

1
Universal Codes and Channel Capacity
Arithmetic Coding

2
Universal Codes and Channel Capacity
Arithmetic Coding

3
Universal Codes and Channel Capacity
Arithmetic Coding

4
Universal Codes and Channel Capacity
Arithmetic Coding

5
Universal Codes and Channel Capacity
Arithmetic Coding

6
Universal Codes and Channel Capacity
Arithmetic Coding

The average codeword length is equal to the entropy

7
Universal Codes and Channel Capacity
Arithmetic Coding

8
Universal Codes and Channel Capacity
Arithmetic Coding

9
Universal Codes and Channel Capacity
Arithmetic Coding

Fig. 3.15: Arithmetic coding for Example 3.20. 10


Universal Codes and Channel Capacity
Arithmetic Coding

11
Universal Codes and Channel Capacity
Arithmetic Coding

12
Universal Codes and Channel Capacity
Lempel Ziv Coding

Universal Source Coding:

Other coding algorithms required the knowledge of


statistics of source

But the universal source coding does not need knowledge


of statistics of source.

Example: Lempel Ziv Coding Algorithm

13
Universal Codes and Channel Capacity
Lempel Ziv Coding

The algorithm is very efficient.

• Variant of LZ77 algorithm used in gzip, png


• Variant of LZW algorithm used in zip, pkzip, compress,
gif, tiff

14
Universal Codes and Channel Capacity
Lempel Ziv Coding

The algorithm

• It is dictionary based source coding technique


• A substring in the source is replaced by a codeword that
identifies the substring in the dictionary
• Each codeword CI except the zeroth codeword in the
concatenation of the two parameters LI and mI.
• mI is related to the position in the dictionary where
the substring match starts.
• Length of the text to be copied from the past is given
by LI .
• The string coded is used as dictionary
• The parameter nW denotes the window size 15
Universal Codes and Channel Capacity
Lempel Ziv Coding

The algorithm

• Both encoder and decoder maintain a dictionary


containing the most recent nW encoded source symbols.
• When encoding a substring starting with source symbol
wk, the dictionary contains the source symbols .
• Initially dictionary contains the first nW symbols of .
• First codeword C0 is the initial dictionary, .
• Second codeword C1is found by searching for the largest
substring Y1, beginning with such that Y1 starts in the
dictionary and ends L1  1 time units later,
• The index of the dictionary symbol where the copy of Y1
begins is denoted by p. m = n – p. 16
Universal Codes and Channel Capacity
Lempel Ziv Coding
The algorithm

• If multiple choices for m1 exist, choose the smallest m1.


• After determining C1, the dictionary is updated by deleting the L1
oldest entities and adding the L1 most recently encoded symbols.
• This is similar to sliding window of the dictionary to the right by
L1 shifts.
• The third codeword C2 is found by searching for largest substring
Y3, beginning with such that Y2 starts in the dictionary and ends L2
 1 time units later. m2 is calculated
• Dictionary then updated.
• The process is repeated until we have encoded the full source
sequence . 17
Universal Codes and Channel Capacity
Lempel Ziv Coding
I YI LI mI CI Dictionary and phase locations
0 {00101001} {00101001}
1 {00101001} {00} 2 2 [2, 2] {:00101001:0000011011011000}
2 {10100100} {000} 3 0 [3, 0] {00:10100100:00011011011000}
3 {00100000} {1} 1 5 [1, 5] {00101:00100000:11011011000}
4 {01000001} {10} 2 6 [2, 6] {001010:01000001:1011011000}
5 {00000110} {110110} 6 2 [6, 2] {00101001:00000110:11011000}
6 {10110110} {00} 2 0 [2, 0] {00101001000001:10110110:00}

18
Universal Codes and Channel Capacity

Encoding of mI and LI:

A comma free is a binary encoding technique for LI.

Wyner-Ziv binary encoding: For a positive integer k, we


denote the binary representation of k by b(k) and to be the
number of bis in the representation

We denote sequence of k-1 zeros followed by 1 as u(k) = 0k-1


1
The comma free encoding of LI is then

19
Universal Codes and Channel Capacity

Encoding of mI and LI:

Example :
The comma free encoding of LI is then

LI = 4, binary representation of 4 is 100


e(4) = .100
binary representation of 3 is 11

e(4) = 0111100

20
Universal Codes and Channel Capacity
Lempel Ziv Coding

• Decoder receives the encoded sequences and knows the


window size nW.
• First bits are directly stored.
• To determine the L1, the decoder reads a sequence of 0.s
followed by the first 1, to determine the length of
encoding of p1, which in terms can be used to determine
the number of bits to represent the length of L1 and
subsequently the L1.
• Having determined , the decoder know that the next binay
bits are for representing m1.

21
Universal Codes and Channel Capacity
Lempel Ziv Coding

22
Universal Codes and Channel Capacity
Lempel Ziv Coding

23

You might also like