Analog & Digital Communication

Presentation
on
Data Compression
1

Name:Muhamad Fahad
ID:3524-2016
Course Instructor Engr.Syed Abdul Latif Shah 6/9/2019
 Table of Contents:
2

 Data compression
 Source coding
 Reasons to compress
 Types of data compression
• Lossless data compression
• Lossy data compression
 Method for data compression
• Shannon’s fane coding algorithm
• Run time length coding algorithm
• huffman’s coding algorithm
• Lempel ziv coding algorithm
 Applications of data compression
 Conclusion

6/9/2019
3 Data Compression
 Data compression is a reduction in the number of bits needed to represent data. Compressing
data can save storage capacity, speed up file transfer, and decrease costs for storage hardware
and network bandwidth.
 Data compression is the process of encoding data so that it takes less storage space or less
transmission time than it would if it were not compressed
 Compression is possible because most real-world data is very redundant
 Source Coding – eliminate redundancy in the data, send same information in fewer bits

6/9/2019
4
Source coding :

 Source coding involves two main components:

 1)Encoding Algorithm
 2)Decoding Algorithm

Decoding Algorithm
Encoding Algorithm
This algorithm reconstructs the
This algorithm takes a message
original message or some
and generates a compressed
approximation of it from the
representation of that message
compressed representation

6/9/2019
 Reasons to Compress
5

 Reduce File Size

 Save disk space
 Increase transfer speed at a given data rate
 Allow real-time transfer at a given data rate
 To save on transmission capacity

6/9/2019
 Entropy
6

 Entropy is the measure of information content in a message.

 Messages with higher entropy carry more information than messages with lower
entropy.
 How to determine the entropy
 Find the probability p(x) of symbol x in the message
 The entropy H(x) of the symbol x is: H(x) = - p(x) • log21/p(x)
 The average entropy over the entire message is the sum of the entropy of all n
symbols in the message

6/9/2019
7 Data Compression Methods

6/9/2019
8
Lossless compression
• Lossless compression is a class of data compression that allows the original
data to be perfectly reconstructed from the compressed data.

• Redundant data is removed in compression and added during decompression.

• Most lossless compression programs do two things in sequence: the first step
generates a statistical model for the input data, and the second step uses this
model to map input data to bit sequences in such a way that "probable"

(e.g. frequently encountered) data will produce shorter output than "improbable" data.

6/9/2019
 Shannon-Fano Encoding:
9

 Shannon Fano Algorithm is an entropy encoding technique for lossless data

compression of multimedia.
 Named after Claude Shannon and Robert Fano, it assigns a code to each symbol
based on their probabilities of occurrence.
 It is a variable length encoding scheme, that is, the codes assigned to the symbols
will be of varying length.
 Sources without memory are such sources of information, where the probability of
the next transmitted symbol (message) does not depend on the probability of the
previous transmitted symbol (message).
 Shannon-Fano encoding constructs reasonably efficient separable binary codes
for sources without memory.

6/9/2019
10 Shannon-Fano Encoding(Example)

x1,x2,x3,x4,x5,x6,x7,x8
1
0
x1,x2 x3,x4,x5,x6,x7,x8
00 01 10 11
x2 x3,x4 x5,x6,x7,x8
x1
100 110 111
101 x5,x6
x3 1100 x7,x8
x4

1101 1110
x5
x6 x7 x8

6/9/2019
11 Shannon-Fano Encoding(Example)

Message X1 X2 X3 X4 X5 X6 X7 X8
Probability 0.25 0.25 0.125 0.125 0.0625 0.0625 0.062 0.062
5 5
Encoding 00 01 100 101 1100 1101 1110 1111
vector

1 1 1 1 1 1
Entropy H 2 log 2 log 4 log 2.75
4 4 8 8 16 16

Average length of the encoding vector

1
L P xi n i 2 2 2 3 4 4 2.75
16
The Shannon-Fano code gives 100% efficiency 6/9/2019
12 Method 1:Run-length encoding
• Simplest method of compression.
• How: replace consecutive repeating occurrences of a symbol by 1 occurrence of
the symbol itself, then followed by the number of occurrences

• The method can be more efficient if the data uses only 2 symbols (0s and 1s) in bit
patterns and 1 symbol is more frequent than another.

6/9/2019
13 Method 2:Huffman Coding
 Assign fewer bits to symbols that occur more frequently and more bits to symbols
appear less often.
 There’s no unique Huffman code and every Huffman code has the same average code
length.
 Algorithm:
 Make a leaf node for each code symbol
 Add the generation probability of each symbol to the leaf node
 Take the two leaf nodes with the smallest probability and connect them into
a new node
 Add 1 or 0 to each of the two branches
 The probability of the new node is the sum of the probabilities of the two
connecting nodes
 If there is only one node left, the code construction is completed. If not, go back to
(2)

6/9/2019
14 Huffman Algorithm:

6/9/2019
 How the encoding and decoding process takes
15 place ?
 Encoding:

 Decoding:

6/9/2019
16 Huffman Coding(Example):

6/9/2019
 Method 3:Lempel Ziv Encoding
17
• It is dictionary-based encoding

• Basic idea:
 Create a dictionary(a table) of strings used during communication.

 If both sender and receiver have a copy of the dictionary, then previously-
encountered strings can be substituted by their index in the dictionary.
 This compression has 2 phases:
 Building an indexed dictionary
 Compressing a string of symbols
• Algorithm for Lempel Ziv encoding:
 Extract the smallest substring that cannot be found in the remaining uncompressed string.
 Store that substring in the dictionary as a new entry and assign it an index value
 Substring is replaced with the index found in the dictionary
 Insert the index and the last character of the substring
 into the compressed string
6/9/2019
18 Lempel Ziv Compression
 Compression example,

6/9/2019
 Lempel Ziv Decompression
19

 It’s just the inverse

of compression process

6/9/2019
20 Lossy Compression
 Lossy compression is the converse of lossless data compression.
• It is Used for compressing images and video files.

• Methods of lossy compression:

 JPEG: compress pictures and graphics
 MPEG:compress video
 MP3: compress audio

6/9/2019
21 Method 1:JPEG Encoding
• Used to compress pictures and graphics.
• In JPEG, a grayscale picture is divided into 8x8 pixel blocks to decrease the
number of calculations.
• Basic idea:
 Change the picture into a linear (vector) sets of numbers that reveals the
redundancies.
 The redundancies is then removed by one of lossless.

6/9/2019
22 JPEG Encoding- DCT
• DCT: Discrete Concise Transform
• DCT transforms the 64 values in 8x8 pixel block in a way that the relative
relationships between pixels are kept but the redundancies are revealed.
• Example:
 A gradient grayscale

6/9/2019
23 Quantization & Compression
 Quantization:
 After T table is created, the values are quantized to reduce the number of bits
needed for encoding.
 Quantization divides the number of bits by a constant, then drops the fraction. This is
done to optimize the number of bits and the number of 0s for each particular
application.

• Compression:
 Quantized values are read from the table and redundant 0s are removed.
 To cluster the 0s together, the table is read diagonally in an zigzag fashion. The
reason is if the table doesn’t have fine changes, the bottom right corner of the table
is all 0s.
 JPEG usually uses lossless run-length encoding at the compression phase.

6/9/2019
 JPEG Encoding
24

6/9/2019
25 Method 2:MPEG Encoding
• Used to compress video.
• Basic idea:
 Each video is a rapid sequence of a set of frames. Each frame is a spatial
combination of pixels, or a picture.
 Compressing video =spatially compressing each frame + temporally
compressing a set of frames.

6/9/2019
26 Types of MPEG compression
• Spatial Compression
 Each frame is spatially compressed by JPEG.
• Temporal Compression
 Redundant frames are removed.
 For example, in a static scene in which someone is talking, most frames
are the same except for the segment around the speaker’s lips, which
changes from one frame to the next.

6/9/2019
27 Method 3:Audio(or)Mp3 Encoding

• Used for speech or music

 Speech: compress a 64 kHz digitized signal
 Music: compress a 1.411 MHz signal

• Two categories of techniques:

 Predictive encoding
 Perceptual encoding

6/9/2019
28 Application of data Compression :

 lossless data compression is often used to better use disk space on office
computers, or better use the connection bandwidth in a computer network.
 In other kinds of data such as sounds and pictures, a small loss of quality can be
tolerated without losing the essential nature of the data, so lossy data compression
methods can be used

6/9/2019
 Conclusion:
29
 The source coding and compress data gives and effective and response speed
and fast processes at transmission or communication and transfer files in order to
keep the time save and also save the required bandwidth of the system
 Compression is the process of encoding data more efficiently to achieve a
reduction in the file size

6/9/2019
30 References
 https://ocw.mit.edu/courses/aeronautics-and-astronautics/16-36-
communication-systems-engineering-spring-2009/lecture-
notes/MIT16_36s09_lec05.pdf
 https://www.britannica.com/technology/data-compression
 https://searchstorage.techtarget.com/definition/compression

6/9/2019
31

6/9/2019