GROUP ID: 18 SMART SHEET BASED ATTENDANCE SYSTEM 1
Date: 9th November, 2017
UNIVERSITY OF SINDH, JAMSHORO
Title: HUFFMAN CODING ALGORITHM
Presented To:
Miss Syeda Hira Fatima Naqvi.
Presented By:
Sadaf Rasheed (2K15-CSE-72)
Department of Computer Science
INTRODUCTION
Huffman codes are an effective technique of lossless data compression which means no
information is lost.
Huffman coding could perform effective data compression by reducing the
amount of redundancy in the coding of symbols.
Huffman code is method for the compression for standard text documents.
It makes use of a binary tree to develop codes of varying lengths for the letters used in the
original message.
The algorithm was introduced by David Huffman in 1952 as part of a course assignment at
MIT.
HUFFMAN CODING SCHEME 1
CONTD:
Huffman codes can be used to compress information
Like WinZip although WinZip doesnt use the Huffman algorithm
JPEGs do use Huffman as part of their compression process
The basic idea is that instead of storing each character in a file as an 8-bit
ASCII value, we will instead store the more frequently occurring
characters using fewer bits and less frequently occurring characters using
more bits
On average this should decrease the file size (usually )
2
HUFFMAN CODING SCHEME
EXAMPLE
Consider a file of 100,000 characters from af, with these frequencies:
o a = 45,000
o b = 13,000
o c = 12,000
o d = 16,000
o e = 9,000
o f = 5,000
HUFFMAN CODING SCHEME 3
CONTD
(FIXED-LENGTH CODE )
Typically each character in a file is stored as a single byte (8 bits)
If we know we only have six characters, we can use a 3 bit code for the characters instead:
a = 000, b = 001, c = 010, d = 011, e = 100, f = 101
This is called a fixed-length code (If every word in the code has the same length, the code is called a
fixed-length code, or a block code.)
With this scheme, we can encode the whole file with 300,000 bits
(45000*3+13000*3+12000*3+16000*3+9000*3+5000*3)
We can do better
Better compression
More flexibility
HUFFMAN CODING SCHEME 4
Variable length codes (If every word has the different length code, the code is called a fixed-length
code, or a block code) can perform significantly better
Frequent characters are given short code words, while infrequent characters get longer code words
o Consider this scheme:
a = 0; b = 101; c = 100; d = 111; e = 1101; f = 1100
How many bits are now required to encode our file?
45,000*1 + 13,000*3 + 12,000*3 + 16,000*3 + 9,000*4 + 5,000*4 = 224,000 bits
This is in fact an optimal character code for this file
HUFFMAN CODING SCHEME 5
PROBLEMS:
Suppose that we want to encode a message constructed from the symbols A, B, C,
D, and E using a fixed-length code
How many bits are required to encode each symbol?
at least 3 bits are required
2 bits are not enough (can only encode four symbols)
How many bits are required to encode the message DEAACAAAAABA?
there are twelve symbols, each requires 3 bits
12*3 = 36 bits are required
6
HUFFMAN CODING SCHEME
DRAWBACKS OF FIXED-LENGTH CODES:
Wasted space
Unicode uses twice as much space as ASCII
inefficient for plain-text messages containing only ASCII characters
Same number of bits used to represent all characters
a and e occur more frequently than q and z
Potential solution: use variable-length codes
variable number of bits to represent characters when frequency of occurrence is known
short codes for characters that occur frequently
7
HUFFMAN CODING SCHEME
ADVANTAGES OF VARIABLE-LENGTH CODES:
The advantage of variable-length codes over fixed-length is short codes can be given to characters that
occur frequently
on average, the length of the encoded message is less than fixed-length encoding
Potential problem: how do we know where one character ends and another begins?
not a problem if number of bits is fixed!
8
HUFFMAN CODING SCHEME
PREFIX PROPERTY:
Prefix codes
Huffman codes are constructed in such a way that they can be unambiguously
translated back to the original data, yet still be an optimal character code
Huffman codes are really considered prefix codes
A code has the prefix property if no character code is the prefix (start of the
code) for another character code
9
HUFFMAN CODING SCHEME
EXAMPLE (PREFIX):
000 is not a prefix of 11, 01, 001, or 10
11 is not a prefix of 000, 01, 001, or 10
10
HUFFMAN CODING SCHEME
CODE WITHOUT PREFIX PROPERTY:
The following code does not have prefix property
The pattern 1110 can be decoded as QQQP, QTP, QQS, or TS
13
11
HUFFMAN CODING SCHEME
CONTD:
A prefix code is a type of code system (typically a variable-length code) distinguished
by its possession of the "prefix property", which requires that there is no code word in
the system that is a prefix (initial segment) of any other code word in the system.
A prefix code is a uniquely decodable code: a receiver can identify each word
without requiring a special marker between words.
12
HUFFMAN CODING SCHEME
CONTD:
Suppose we have two binary code words a and b, where a is k bits long,
b is n bits long, and k < n. If the first k bits of b are identical to a, then a is
called a prefix of b. The last n k bits of b are called the dangling suffix.
For example, if
a = 010 and b = 01011,
then a is a prefix of b and the dangling suffix is 11.
13
HUFFMAN CODING SCHEME
GROUP ID: 18 SMART SHEET BASED ATTENDANCE SYSTEM 16
PURPOSE OF HUFFMAN CODING:
Proposed by Dr. David A. Huffman in 1952
A Method for the Construction of Minimum Redundancy Codes
Applicable to many forms of data transmission
Our example: text files
14
HUFFMAN CODING SCHEME
THE BASIC ALGORITHM:
Huffman coding is a form of statistical coding
Not all characters occur with the same frequency!
Yet all characters are allocated the same amount of space
1 char = 1 byte, be it e or x
15
HUFFMAN CODING SCHEME
THE BASIC ALGORITHM:
Any savings in tailoring codes to frequency of character?
Code word lengths are no longer fixed like ASCII.
Code word lengths vary and will be shorter for the more
frequently used characters.
16
HUFFMAN CODING SCHEME
THE (REAL) BASIC ALGORITHM:
1. Scan text to be compressed and tally occurrence of all characters.
2. Sort or prioritize characters based on number of occurrences in text.
3. Build Huffman code tree based on prioritized list.
4. Perform a traversal of tree to determine all code words.
5. Scan text again and create new file using the Huffman codes.
17
HUFFMAN CODING SCHEME
ALGORITHM:
n <- |C
Q <- C
for i <- 1 to n-1
do allocate a new node z
left [ z ] <- x <- EXTRACT-MIN (Q)
right [ z ] <- y <- EXTRACT-MIN (Q)
f [ z ] <- f [ x ] + f [ y ]
INSERT(Q, Z)
return EXTRACT-MIN (Q)
18
HUFFMAN CODING SCHEME
ANALYSIS :
Time Complexity
Time complexity of Huffman algorithm is O(nlogn) where each
iteration requires O(logn) time to determine the cheapest
weight and there would be O(n) iterations.
19
HUFFMAN CODING SCHEME
BUILDING A TREE
SCAN THE ORIGINAL TEXT
Consider the following short text:
Eerie eyes seen near lake.
Count up the occurrences of all characters in the text
20
HUFFMAN CODING SCHEME
BUILDING A TREE
SCAN THE ORIGINAL TEXT
Eerie eyes seen near lake.
Q. What characters are present?
E e r i space
ysnarlk.
21
HUFFMAN CODING SCHEME
BUILDING A TREE
SCAN THE ORIGINAL TEXT
Eerie eyes seen near lake.
What is the frequency of each character in the text?
22
HUFFMAN CODING SCHEME
BUILDING A TREE
PRIORITIZE CHARACTERS
Create binary tree nodes with character and frequency
of each character
Place nodes in a priority queue
The lower the occurrence, the higher the priority in the queue
23
HUFFMAN CODING SCHEME
BUILDING A TREE
The queue after inserting all nodes
E i y l k . r s n a sp e
1 1 1 1 1 1 2 2 2 2 4 8
24
HUFFMAN CODING SCHEME
BUILDING A TREE
While priority queue contains two or more nodes
Create new node
Dequeue node and make it left subtree
Dequeue next node and make it right subtree
Frequency of new node equals sum of frequency of left and right
children
Enqueue new node back into queue
25
HUFFMAN CODING SCHEME
BUILDING A TREE
E i y l k . r s n a sp e
1 1 1 1 1 1 2 2 2 2 4 8
26
HUFFMAN CODING SCHEME
BUILDING A TREE
27
HUFFMAN CODING SCHEME
BUILDING A TREE
28
HUFFMAN CODING SCHEME
BUILDING A TREE
29
HUFFMAN CODING SCHEME
BUILDING A TREE
30
HUFFMAN CODING SCHEME
BUILDING A TREE
31
HUFFMAN CODING SCHEME
BUILDING A TREE
32
HUFFMAN CODING SCHEME
BUILDING A TREE
33
HUFFMAN CODING SCHEME
BUILDING A TREE
34
HUFFMAN CODING SCHEME
BUILDING A TREE
35
HUFFMAN CODING SCHEME
BUILDING A TREE
36
BUILDING A TREE
37
HUFFMAN CODING SCHEME
BUILDING A TREE
38
HUFFMAN CODING SCHEME
BUILDING A TREE
39
HUFFMAN CODING SCHEME
BUILDING A TREE
Q. What is happening to the characters with a low
number of occurrences?
40
HUFFMAN CODING SCHEME
BUILDING A TREE
41
HUFFMAN CODING SCHEME
BUILDING A TREE
42
HUFFMAN CODING SCHEME
BUILDING A TREE
43
HUFFMAN CODING SCHEME
BUILDING A TREE
44
HUFFMAN CODING SCHEME
BUILDING A TREE
45
HUFFMAN CODING SCHEME
BUILDING A TREE
46
HUFFMAN CODING SCHEME
BUILDING A TREE
47
HUFFMAN CODING SCHEME
BUILDING A TREE
After
enqueueing
this node there
is only one
node left in
priority queue.
48
HUFFMAN CODING SCHEME
BUILDING A TREE
Dequeue the single node left in the
queue.
This tree contains the new code
words for each character.
Frequency of root node should equal
number of characters in text.
Eerie eyes seen near lake. 26 characters
49
HUFFMAN CODING SCHEME
ENCODING THE FILE
TRAVERSE TREE FOR CODES
Perform a traversal of the tree to
26
obtain new code words
16
Going left is a 0 going right is a 1 10
4
code word is only completed when a
e 8
6 8
leaf node is reached 2 2 2 sp 4 4
4
E i y l k .
1 1 1 1 1 1 r s n a
2 2 2 2
50
HUFFMAN CODING SCHEME
ENCODING THE FILE
TRAVERSE TREE FOR CODES
Char Code
E 0000
i 0001 26
y 0010
l 0011 16
10
k 0100
. 0101
4
space 011 6
e
8
8
e 10
r 1100 2 2
2 sp 4 4
s 1101 4
n 1110 E i y l k .
1 1 1 1 1 1 n a
a 1111 r s
2 2 2 2
51
HUFFMAN CODING SCHEME
ENCODING THE FILE
Rescan text and encode Char Code
file using new code
E 0000
words i
y
0001
0010
l 0011
Eerie eyes seen near lake. k 0100
000010110000011001110001 . 0101
0101101101001111101011111 space 011
100011001111110100100101
e 10
r 1100
Q. Why is there no need for a
s 1101
n 1110
separator character? a 1111
.
52
HUFFMAN CODING SCHEME
ENCODING THE FILE
RESULTS
Have we made things any better?
73 bits to encode the text 0000101100000110011100010
10110110100111110101111110
ASCII would take 8 * 26 = 208 bits
0011001111110100100101
If modified code used 4 bits per
character are needed. Total bits
4 * 26 = 104. Savings not as great.
53
HUFFMAN CODING SCHEME
APPLICATIONS OF HUFFMAN CODING:
Supports various file type as:
ZIP (multichannel compression including text and other data
types) JPEG
MPEG (only upto 2 layers)
Also used in steganography for JPEG carrier compression.
54
HUFFMAN CODING SCHEME
CONCLUSION:
Like many other useful algorithms we do require Huffman
Algorithm for compression of data so it could be transmitted
over internet and other transmission channels properly.
Huffman algorithm works on Binary trees.
55
HUFFMAN CODING SCHEME
59
