UNIT III: Arithmetic Coding

Arithmetic Coding

Basic Principal of arithmetic coding

Like Huffman coding, this too is a Variable Length Coding (VLC) scheme
requiring a priori knowledge of the symbol probabilities. The basic principles of
arithmetic coding are as follows:

a) Unlike Huffman coding, which assigns variable length codes to a fixed group
symbols (usually of length one), arithmetic coding assigns variable length codes
to a variable group of symbols.

b) All the symbols in a message are considered together to assign a single

arithmetic code word.

c) There is no one-to-one correspondence between the symbol and its

corresponding code word.

d) The code word itself defines a real number within the half-open interval [0,1)
and as more symbols are added, the interval is divided into smaller and smaller
subintervals, based on the probabilities of the added symbols.

Algorithm for Arithmetic Coding

The steps of the encoding algorithm are as follows:

Step-1: Consider a range of real numbers in [0,1). Subdivide this range into a
number of sub-ranges that is equal to the total number of symbols in the source
alphabet. Each sub-range spans a real value equal to the probability of the
source symbol.

Step-2: Consider a source message and take its first symbol. Find to which
subrange does this source symbol belongs.

Step-3: Subdivide this sub-range into a number of next level sub-ranges,

according to the probability of the source symbols.

Step-4: Now parse the next symbol in the given source message and determine
the next-level sub-range to which it belongs.

Step-5: Repeat step-3 and step-4 until all the symbols in the source message are
parsed. The message may be encoded using any real value in the last sub- range
so formed. The final message symbol is reserved as a special end-of- symbol
message indicator.

1
Encoding a Sequence of Symbols using Arithmetic Coding

On the basis of a well-known alphabet the probability of all symbols has to be

determined and converted into intervals. The size of the interval depends
linearly on the symbol's probability. If this is 50% for example, then the
associated sub-interval covers the half of the current interval. Usually the initial
interval is [0: 1) for the encoding of the first symbol:

Example sub-division of the interval [0.0; 1.0)

Frequency
Symbol Probability Sub-Interval
a 4 0.4 [0.0; 0.4)
b 2 0.2 [0.4; 0.6)
c 2 0.2 [0.6; 0.8)
d 1 0.1 [0.8; 0.9)
e 1 0.1 [0.9; 1.0)
Sum 10 1.0 [0.0; 1.0)

Example sub-division of the interval [0.9; 1.0)

Frequency
Symbol Probability Sub-Interval
a 4 0.4 [0.90;
0.94)
b 2 0.2 [0.94;
0.96)
c 2 0.2 [0.96;
0.98)
d 1 0.1 [0.98;
0.99)
e 1 0.1 [0.99;
1.00)

Example sub-division of the interval [0.99; 1.00)

Frequency
Symbol Probability Sub-Interval
a 4 0.4 [0.990;
0.994)
b 2 0.2 [0.994;
0.996)
c 2 0.2 [0.996;
0.998)
d 1 0.1 [0.998;
0.999)
e 1 0.1 [0.999;
2
 1.000)

Accordingly a rational number in the range of 0.999 to < 1.000 represents the
string "eee".

3
Assignment of Intervals to Codes
The symbols are identified by any number, which falls into the interval
concerned. All values, which are greater or equal than 0.0 and less than 0.4, will
be interpreted as 'a' for example.

Examples: assigning codes and intervals:

Symbol Sub-Interval
a [0.0; 0.4)
b [0.4; 0.6)
c [0.6; 0.8)
d [0.8; 0.9)
e [0.9; 1.0)

Code Word Sub-Interval Symbol

0.1 [0.0; -> a
0.4)
0.2 [0.0; -> a
0.4)
0.4 [0.4; -> b
0.6)
0.55 [0.4; -> c
0.6)
0.99 [0.9; -> e
1.0)

All intervals have a closed lower endpoint (indicated by '[') and an open upper
endpoint (indicated by ')'). The lower endpoint always belongs to the interval,
while the upper endpoint is placed outside. I.e. the value 1 might be never
encoded in this context.

Sub-Intervals

After the initial encoding, any further symbol will not relate to the basic
intervall [0; 1). The sub-interval identified by the first symbol has to be used for
the next step. In the case of an 'a' the new interval would be [0; 0.4), for 'b'
[0.4; 0.6).

Providing a constant probability distribution and the first symbol 'a', the new
sub-intervals would be:

4
Divide interval after encoding of 'a'

total: [0.0; 1.0) total: [0.0; 0.4)

Sym. P(x) Intervals Sub-Intervals

a 0.4 [0.0; 0.4) ----- 0.16 [0.00; 0.16)
b 0.2 [0.4; 0.6) | 0.08 [0.16; 0.24)
c 0.2 [0.6; 0.8) | 0.08 [0.24; 0.32)
d 0.1 [0.8; 0.9) | 0.04 [0.32; 0.36)
e 0.1 [0.9; 1.0) -- 0.04 [0.36; 0.40)

Sum 1.0 0.40

Codes for 2 Symbols

If it is known that the current code identifies two symbols, any arbitrary number
between 0.00 and 0.16 represents the character string "aa". The same rules
apply to following examples:
Codes representing two symbols:
Symbol Interval String
0 [0.00; 0.16) aa
0.1 [0.00; 0.16) aa
0.14 [0.00; 0.16) aa

0.16 [0.16; 0.24) ab

0.2 [0.16; 0.24) ab

0.32 [0.32; 0.36) ad

0.34 [0.32; 0.36) ad

0.36 [0.36; 0.40) ae

0.399 [0.36; 0.40) ae

On the basis of the decimal number system the character strings "aa" and "ab"
could be encoded with one decimal position, "ac" and "ad" with two positions.

Scheme Intervals "aaaa"

With the parameters from the previous example the following scheme is the
result for the string "aaaa":

5
The code word for the string could be an arbitrary number greater or equal
than 0.0000 and less than 0.0256. The best choice would be a number
requiring a small amount of significant digits, in this case the 0.
Scheme Intervals "abcd"
Using the same parameter set, "abcd" will be encoded according to the
following scheme:

6
The string "abcd" is represented by an abritary number within the interval
[0,2208; 0,2224). In a binary system 10 digits or bit are sufficient to encode
such a number:
0.0011,1000,11 (binary)
0.221,679,687,5 (decimal)

Decoding an Arithmetic-Coded Bit Stream

The decoding process of arithmetic coded bit stream is very straightforward. We

should follow these steps:

Step-1: Identify the message bit stream. Convert this to the real decimal number
and determine its position within the subintervals identified at the beginning of
encoding process. The corresponding symbol is the first one in the message

Step-2: Consider the expanded subinterval of the previous decoded symbol and
map the real number within this to determine the next subinterval and obtain
the next decoded symbol. Repeat this step until the end-of-message indicator is
parsed.

7
Advantage of Arithmetic Coding

Arithmetic coding is generally suitable for adaptation strategies. It is easy to

implementation for system with multiple arithmetic codes. It is close to optimal
compression performance for sources with very low entropies.

Problems of arithmetic coding

In the arithmetic coding method infinite precision is required. Transmission of

coded data cannot begin until the whole sequence is encoded. Unlike Huffman
coding, the decoder requires knowledge of the number of symbols to be
decoded.

Comparison Arithmetic coding versus Huffman

Equivalently to the Huffman coding, the arithmetical coding tries to evaluate

the probability with which certain symbols appear and to optimize the length of
the required code. The AC achieves an optimum which exactly corresponds to
the theoretical specifications of the information theory. A slight degradation
results from inaccuracies, which are caused by correction mechanisms for the
interval division.

In contrast to this the Huffman coding always produces rounding errors,

because its code length is restricted to multiples of a bit. This deviation from
the theoretical optimum is much higher in comparison to the arithmetic
coding's inaccuracies.

The efficiency of an arithmetic code is always better or at least identical to a

Huffman code.

Several publications available in the internet quantify the advantage of

arithmetic coding somehow between 5 and 10%. Mostly they refer to the usage
according to the JPEG specification. Concrete comparisons or theoretical models
are not mentioned.

The high requirements for calculation were the most important disadvantage,
especially if taking older computer systems into consideration. Additionally the
patent situation had a crucial influence to decisions about the implementation of
an arithmetic coding.

8
 JBIG

JBIG is short for the 'Joint Bi-level Image experts Group'. This is a group of
experts nominated by national standards bodies and major companies to work to
produce standards for bi-level image coding. The 'joint' refers to its status as a
committee working on both ISO/IEC and ITU-T standards. The official title of the
committee is ISO/IEC JTC1 SC29 Working Group 1, and is responsible for both
JPEG and JBIG standards.
JBIG has developed ISO/IEC 11544 (ITU-T T.82) for the lossless compression of a
bi-level image. It can also be used for coding greyscale and colour images with
limited numbers of bits per pixel. It can be regarded as a form of facsimile
encoding, similar to Group 3 or Group 4 fax, offering between 20 and 80%
improvement in compression over these methods (about 20 to one over the
original uncompressed digital bit map).
JBIG2 is the successor (ISO/IEC 14492 | ITU-T T.88) format for bi-level
(black/white) image compression that offers significant advantages over other
compression formats:
 large increases in compression performance (typically 3-5 times smaller than
Group 4/MMR, 2-4 times smaller than JBIG1)
 special compression methods for text, halftones, and other binary image content
 Lossy and lossless compression
 multi-page document compression
 flexible format, designed for easy embedding in other image file formats, such
as TIFF
 high-performance decompression: using some coding modes, images can be
decompressed at over 250 million pixels/second in software

JBIG Basics

Bi-level images contain only two colors and are stored using a single bit per
pixel. Black-and-white images, such as the pages of a book, are the most
common type of bi-level images. However, any two colors may be represented
by the 1 (foreground color) or 0 (background color) state of a single bit.
Typical bi-level compression algorithms encode only a single scan line at a time
using a run-length encoding technique. Such algorithms are referred to as 1D
encoding methods. 2D encoding methods encode runs of pixels by describing the
differences between the pixel values in the current and the previous scan lines.
JBIG encodes redundant image data by comparing a pixel in a scan line with a
set of pixels already scanned by the encoder. These additional pixels are called
a template, and they form a simple map of the pattern of pixels that surround
the pixel that is being encoded. The values of these pixels are used to identify
redundant patterns in the image data. These patterns are then compressed
using an adaptive arithmetic compression coder.
The adaptive nature of templates allows the color of the pixel values being
encoded to be predicted with a high degree of success. For gray-scale images
with halftoning, compression ratios are increased by as much as 80 percent over
non-adaptive methods.

9
Although designed primarily as a method for compressing bi-level image data,
JBIG is capable of compressing color or gray-scale images with a depth of up to
255 bits per pixel. Such multi-bit pixel images are compressed by bitplane rather
than by pixel. For example, an 8-bit image compressed using JBIG would be
encoded into eight separate bitplanes.
This type of encoding may be used as an alternative to lossless JPEG. JBIG has
been found to produce better compression results than lossless JPEG (using the
Q-coder) on images with two to five bits per pixel and to produce identical
results on image data with pixels six to eight bits in depth.
It is recommended that each bitplane be preprocessed with a gray-coding
algorithm to normalize the changes between adjacent byte values in the image
data. This process increases the efficiency of the JBIG encoder.
JBIG images may be encoded sequentially or progressively. Sequentially
encoded images are stored in a single layer at full resolution and without other
lower resolution images being stored in the same data stream. This sequential
JBIG image is equivalent in function and application to a G4 image. Such an
image is decoded in a single pass and has at least as good a compression ratio
as G4.
Progressively encoded images start with the highest resolution image and end
with the lowest. The high-resolution image is stored in a separate layer and is
then used to produce a lower resolution image, also stored in its own layer. Each
layer after the first layer is called a resolution doubling. An image with three
layers is said to have two doublings.
There is no imposed limit to the number of doublings that may be encoded. For
example, a 1200-dpi image may be encoded as one layer (1200 dpi), three
layers (1200, 600, and 300 dpi), or five layers (1200, 600, 300, 150, and 75 dpi).
The lowest resolution is determined by whatever is considered useful. Even a 10-
dpi image, though not legible, is still useful as an icon.
Progressive decoding is the opposite process, with the lowest resolution image
being decoded first, followed by increased resolutions of the image until the full
resolution is achieved. This technique has the advantage of allowing data to
appear immediately on the output device. Only data up to the appropriate
resolution of the output device need be decoded and sent.
Both sequential and progressive JBIG encoding are completely compatible.
Images compressed using sequential encoding are readable by progressive JBIG
decoders. Sequential JBIG decoders are only capable of reading the first, lowest-
resolution layer within a progressively-encoded JBIG image.
Many applications that utilize JBIG may only have use for sequential encoding
and decoding, especially those used for facsimile transmission. It is therefore
possible to implement a simplified JBIG algorithm that encodes only the first
layer in a JBIG data stream. Such encoders produce a valid JBIG-encoded data
stream that is readable by all JBIG decoders.
Progressive encoding does not add much more data to a JBIG data stream than
does sequential encoding, but it does have greater memory requirements.
Because a lower resolution image is encoded from data of the next higher

10
resolution image (and vice versa when decoding), a frame buffer must be used
to store image data that is being used as a reference.
JBIG2

 JBIG2 is an image compression standard for bi-level images, developed by the

Joint Bi-level Image Experts Group. It is suitable for both lossless and lossy
compression.
 According to a press release from the Group, in its lossless mode JBIG2
typically generates files one third to one fifth the size of Fax Group 4and one
half to one quarter the size of JBIG, the previous bi-level compression
standard released by the Group. JBIG2 has been published in 2000 as the
international standard ITU

Ideally, a JBIG2 encoder will segment the input page into regions of text, regions
of halftone images, and regions of other data. Regions that are neither text nor
halftones are typically compressed using a context- dependent
arithmetic coding algorithm called the MQ coder. Textual regions are compressed
as follows: the foreground pixels in the regions are grouped into symbols. A
dictionary of symbols is then created and encoded, typically also using context-
dependent arithmetic coding, and the regions are encoded by describing which
symbols appear where. Typically, a symbol will correspond to a character of text,
but this is not required by the compression method. For lossy compression the
difference between similar symbols (e.g., slightly different impressions of the
same letter) can be neglected; for lossless compression, this difference is taken
into account by compressing one similar symbol using another as a template.
Halftone images may be compressed by reconstructing the gray scale image
used to generate the halftone and then sending this image together with a
dictionary of halftone patterns.

Difference between JBIG & JBIG2

Introduced as an ITU standard in 1993, JBIG (also called JBIG1) never achieved
the acceptance that TIFF G4 enjoyed even though it provided a 20-30% reduction
in file size over TIFF G4. Such a reduction rate never generated sufficient
enthusiasm among the digital imaging community to justify broad- based
industry support. Consequently, JBIG was mostly used for bitonal image
compression on a very limited range of (mostly Japanese) MFP devices and
digital copiers.

In contrast, the digital media industry has readily received the JBIG2 standard.
Almost from the time of its introduction, JBIG2 was supported for bitonal
compression in the JPEG 2000 Part 6 specifications, and as a compression filter in
Adobe PDF. It quickly became the format of choice for a number of document-

11
 heavy organizations including legal, media, financial, scanning and banking
firms.

One advantage held by both JBIG and JBIG2 over TIFF G3 and G4 is the JBIG
formats' ability to use arithmetic coding instead of Huffman coding. Again, the
key difference is the higher compression ratio arithmetic coding can bring to the
JBIG standard. Arithmetic coding allows for data to be represented by a fraction
of a bit. In comparison, Huffman coding requires whole bits to represent runs in
the image, resulting in a lower compression ratio for the TIFF formats.

JBIG2 introduced a number of features not available in JBIG, including:

 File size reduction of 90% or more.
 Support for several lossy and lossless compression modes.
 Machine learning of font classes.
 Readily available viewer support, Adobe Reader, when the JBIG2 file is PDF-
wrapped.

Progressive image transmission - transmitting all image data may not be

necessary under some circumstances - e.g., searching an image database
looking for a particular image.

 This approach is also commonly used to decrease the waiting time needed
for the image to start appearing after transmission and is used by World
Wide Web image transmissions.

 In progressive transmission, the images are represented in a pyramid

structure, the higher pyramid levels (lower resolution) being transmitted
first.
 The concept of smart compression is based on the sensing properties of
human visual sensors.
 The spatial resolution of the human eye decreases significantly with
increasing distance from the optical axis.
 Therefore, the human eye can only see in high resolution in a very small
area close to the point where the eye is focused.
 Similarly as with image displays, where it does not make sense to display
or even transmit an image in higher resolution than that of the display
device, it is not necessary to display an image in full resolution in image
areas where the user's eyes are not focused.
 The main difficulty remains in determining the areas of interest in the
image on which the user will focus.

12
 Predication by partial matching (PPM)

PPM was invented by John Cleary and Ian Witten back in 1984.

The basic algorithm initially attempted to use the largest context. The size of largest
context is predetermined.

PPM is an adaptive statistical data compression technique based on context

modelling and prediction. The name stands for Prediction by Partial Matching.
PPM models use a set of previous symbols in the uncompressed symbol stream
to predict the next symbol in the stream.

Predictions are usually reduced to symbol rankings. The number of previous

symbols, n, determines the order of the PPM model which is denoted as PPM (n).
Unbounded variants where the context has no length limitations also exist and
are denoted as PPM*. If no prediction can be made based on all n context
symbols a prediction is attempted with just n-1 symbols. This process is
repeated until a match is found or no more symbols remain in context. At that
point a fixed prediction is made. This process is the inverse of that followed by
DMC compression algorithms which build up from a zero-order model.

Suppose we are encoding a sequence X1, X2, . . . , Xn and we are trying to estimate
the probabilities of the next symbol, say X j+k given the previous k symbols. If
some symbols have never occurred before in this context of k symbols, then we
have the zero frequency problem mentioned last lecture. We don’t know how to
estimate the probabilities of these symbols. The idea of the PPM method 1 is to
avoid the zero frequency problem by switching to a lower order context. The
encoder tells the decoder that the symbol X j+k that comes next has never
occured in the present context, and that they should (both) switch to the context
of the previous k − 1 symbols, instead of the k symbol context. Here are some
details:

Rather than using one Markov model of order k, use several Markov models,
namely, models of order 0, 1, . . . , k max, where kmax is the largest order model
considered. (In fact, an order -1 model is used as well. See below.) As the
encoder encodes the sequence, it computes counts for each of these models.
This allows it to compute conditional probability functions: p(next symbol |
previous kmax symbols ) p(next symbol | previous k max − 1 symbols ) . . . p(next
symbol | previous symbol ) p(next symbol ) • Add one more symbol, AN+1 = ǫ
(for “escape”) to the alphabet. This escape symbol indicates that the encoder is
switching from the kth order model to the k − 1 th order model. • To choose the
probabilities of the next symbol, use the highest order model k such that the
previous k symbols followed by the next symbol has occurred at least once in
the past. That is, the encoder keeps escaping until it finds such an order. • What
if a symbol has never occurred before in any context? That is, what if it is the
first time that a symbol occurs? In this case, the encoder uses a “k = −1” order
model to encode the symbol. (Call the order “-1” is meaningless, of course. This
entire means is that we are decrementing the order from k = 0, i.e. we escaped
from k = 0.)

Example 1: “abracadabra” with kmax = 2

13
The symbols that are encoded are:

ǫ, ǫ, ǫ, a, ǫ, ǫ, ǫ, b, ǫ, ǫ, ǫ, r, ǫ, ǫ, a, ǫ, ǫ, ǫ, c, ǫ, ǫ, a, ǫ, ǫ, ǫ, d, ǫ, ǫ, a, ǫ, b, r, a

And corresponding orders are

2, 1, 0, −1, 2, 1, 0, −1, 2, 1, 0, −1, 2, 1, 0, 2, 1, 0, −1, 2, 1, 0, 2, 1, 0, −1, 2,

1, 0, 2, 1, 2, 2

Context Counts Context Counts Context Counts

Empty A=4 A C=3 AC B=1

B=2 B $=1 BA C=2
C=5 C A=2 CA $=2
$=3 $=1 CB C=1
A=1 CC $=1
B=2 C=1
C=2 $=1
$=3 A=2
$=1
A=1
B=1
$=2

You may wonder how this could achieve compression. The number of symbols
(including the ǫ) is far more than the original number of symbols. On the one
hand, one would think that this would require more bits! On the other hand,
notice that many of these “escapes” occur with probability 1 and so no bits need
to be sent.

PPM: Example Contexts

k=2 String = ACCBACCACBA $ means escape, values from PPMC

Exclusion: if we are using escape codes, when we are in a given context we

exclude lower orders from probability computing, because we just take in
account the probabilities of current one, that's PPMC using lazy exclusions. If we
exclude symbols which have appeared in higher orders from the probability
computation that's called full exclusion.

1: begin
2: while (not last character) do
3: begin

14
 4: readSymbol()
5: shorten context
6: while (context not found and context length not -1) do
7: begin
8: output(escape sequence)
9: shorten context
10: end
11: output(character)
12: while (context length not -1) do
13: begin
14: increase count of character (create node if
nonexistant) 15: shorten context
16: end
17: end
18: end

15
Dictionary Method

The compression techniques we have seen so far replace individual symbols with
a variable length codewords. In dictionary compression, variable length
substrings are replaced by short, possibly even fixed length codewords.
Compression is achieved by replacing long strings with shorter codewords. The
general scheme is as follows:

 The dictionary D is a collection of strings, often called phrases. For

The text T is parsed into a sequence of phrases: T = T1T2 . . . Tz, Ti ∈ D

completeness, the dictionary includes all single symbols.

 The sequence is called a parsing or a factorization of T with respect to D.
 The text is encoded by replacing each phrase Ti with a code that acts as a
pointer to the dictionary.

Here is a simple static dictionary encoding for English text:

 The dictionary consists of some set of English words plus individual

symbols.
 Compute the frequencies of the words in some corpus of English texts.
Compute the frequencies of symbols in the corpus from which the
dictionary words have been removed.
 Number the words and symbols in descending order of their frequencies.
 To encode a text, replace each dictionary word and each symbol that does
not belong to a word with its corresponding number. Encode the sequence
of number using γ coding.

Basic Principles of Lempel-Ziv Coding

 We now consider yet another popular lossless compression scheme, which

is originally called Lempel-Ziv coding, and also referred to as Lempel-Ziv-
Welch (LZW) coding, following the modifications of Welch for image
compression. This coding scheme has been adopted in a variety of
imaging file formats, such as the graphic interchange format (GIF), tagged
image file format (TIFF) and the portable document format (PDF). The
basic principles of this encoding scheme are:
 a) It assigns a fixed length codeword to a variable length of symbols.
 b) Unlike Huffman coding and arithmetic coding, this coding scheme does
not require a priori knowledge of the probabilities of the source symbols.
 c) The coding is based on a “dictionary” or “codebook” containing the
source symbols to be encoded. The coding starts with an initial dictionary,
which is enlarged with the arrival of new symbol sequences.
 d) There is no need to transmit the dictionary from the encoder to the
decoder. A Lempel-Ziv decoder builds an identical dictionary during the
decoding process.

16
In 1977 and 1978, Abraham Lempel and Jacob Ziv published two adaptive dictionary
compression algorithms that soon became to dominate practical text
compression. Numerous variants have been published and implemented, and
they are still the most commonly used algorithms in general purpose
compression tools.

The common feature of the two algorithms and all their variants is that the
dictionary consists of substrings of the already processed part of the text. This
means that the dictionary adapts to the text.

The two algorithms are known as LZ77 and LZ78, and most related methods can be
categorized as a variant of one or the other. The primary difference is the
encoding of the phrases:

 LZ77 uses direct pointers to the preceding text.

 LZ78 uses pointers to a separate dictionary.

LZ77

Jacob Ziv and Abraham Lempel have presented their dictionary-based scheme in
1977 for lossless data compression. The LZ77 compression algorithm is the most
used compression algorithm, on which program like PkZip has their foundation
along with a few other algorithms. LZ77 exploits the fact that words and phrases
within a text file are likely to be repeated. When there is repetition, they can be
encoded as a pointer to an earlier occurrence, with the pointer followed by the
number of characters to be matched. It is a very simple technique that requires
no prior knowledge of the source and seems to require no assumptions about
the characteristics of the source. In the LZ77 approach, the dictionary work as a
portion of the previously encoded sequence. The encoder examines the input
sequence by pressing into service of sliding window which consists of two parts:
Search buffer and Look-ahead buffer. A search buffer contains a portion of the
recently encoded sequence and a look-ahead buffer contains the next portion of
the sequence to be encoded. The algorithm searches the sliding window for the
longest match with the beginning of the look-ahead buffer and outputs a pointer
to that match. It is possible that there is no match at all, so the output cannot
contain just pointers. In LZ77 the sequence is encoded in the form of a triple ,
where ‘o’ stands for an offset to the match, ‘l’ represents length of the match,
and ‘c’ denotes the next symbol to be encoded. A null pointer is generated as
the pointer in case of absence of the match (both the offset and the match
length equal to 0) and the first symbol in the look-ahead buffer i.e.
(0,0,”character”). The values of an offset to a match and length must be limited
to some maximum constants. Moreover the compression performance of LZ77
mainly depends on these values.

Algorithm for LZ77

While (look-Ahead Buffer is not empty)

17
 {Get a pointer (position, length) to longest match;
If (length > 0)
{
Output (position, Longest match length, next symbol );
Shift the window by (length+1) positions along;
}
Else
{
Output (0, 0, first symbol in the look-ahead buffer); Shift
the window by 1 character along;
}
}

Shortcoming of LZ77 Algorithm

In the original LZ77 algorithm; Lempel and Ziv proposed that all string be
encoded as a length and offset, even string founds no match. In LZ77, search
buffer is thousands of bytes long, while the look-ahead buffer is tens of byte long.

 The encryption process is time consuming due to the large number of

comparison done to find matched Pattern.

 LZ77 doesn’t have its external dictionary which cause problem while
decompressing on another machine.

 In this algorithm whenever there is no match of any strings it encoded that

string as a length and offset which will take more space and this unnecessary
step also increases time taken by the algorithm.

LZ78

The LZ78 is a dictionary-based compression algorithm that maintains an explicit

dictionary. The encoded output consists of two elements: an index referring to
the longest matching dictionary entry and the first non-matching symbol. The
algorithm also adds the index and symbol pair to the dictionary. When the
symbol is not yet found in the dictionary, the codeword has the index value 0
and it is added to the dictionary as well. With this method, the algorithm
constructs the dictionary. LZ78 algorithm has the ability to capture patterns and
hold them indefinitely but it also has a serious drawback. The dictionary keeps
growing forever without bound. There are various methods to limit dictionary
size. the easiest one is to stop adding entries and continue like a static
dictionary coder or to throw the dictionary away and start from scratch after a
certain number of entries has been reached.

Algorithm for LZ78

W: = NIL;

18
While (there is input)
{
K: = next symbol from input; If (wK exists in the dictionary)
{
W := wK;
}
Else
{
Output (index(w), K); Add
wK to the dictionary;
W := NIL;
}
}

LZW

LZW is simple optimization of LZ78 used, e.g., in the unix tool compress.

Initially, the dictionary D contains all individual symbols:

D = fZ1; : : : ; Zσg.

Suppose the next phrase Tj starts at position i. Let Zk 2 D be the

longest phrase in the dictionary that is a prefix of T [i::n). Now the next text
phrase is Tj = Zk and the phrase added to the dictionary is Zσ+j = Tjti+jTjj.

The phrase Tj is encoded with the index k requiring dlog(σ + j - 1)e bits.

Exercise:

Compare MH, MR, MMR and JBIG

Data Compression Protocols

Besides error control protocols, all current high-speed modems also support data
compression protocols. That means the sending modem will compress the data
on-the-fly and the receiving modem will decompress the data to its original form.

MNP-5 and V.42bis

There are two standards for data compression protocols, MNP-5 and CCITT
V.42bis. Some modems also use proprietary data compression protocols.

A modem cannot support data compression without utilizing an error control

protocol, although it is possible to have a modem that only supports an error
control protocol but not any data compression protocol. A MNP-5 modem
requires MNP 4 error control protocol and a V.42bis modem requires V.42 error
control protocol.

19
Also note that although V.42 include MNP-4, V.42bis does not include MNP-5.
However, virtually all high-speed modems that support CCITT V.42bis also
incorporate MNP-5.

The maximum compression ratio that a MNP-5 modem can achieve is 2:1. That is
to say, a 9600 bps MNP-5 modem can transfer data up to 19200 bps. The
maximum compression ratio for a V.42bis modem is 4:1. That is why all those
V.32 modem manufacturers claim that their modems provide throughput up to
38400 bps.

Are MNP-5 and V.42bis useful?

Don't be fooled by the claim. It is extremely rare, if ever, that you will be able to
transfer files at 38400 bps. In fact, V.42bis and MNP-5 are not very useful when
you are downloading files from online services. Why?

How well the modem compression works depends on what kind of files are being
transferred. In general, you will be able to achieve twice the speed for
transferring a standard text file (like the one you are reading right now).
Decreasing by 50% means that you can double the throughput on the line so
that a 9600 bps modem can effectively transmit 19200 bps.

V.42bis and MNP-5 modem cannot compress a file which is already compressed
by software. In the case of MNP-5, it will even try to compress a precompressed
file and actually expand it, thus slow down the file transfer! Here are the test
results obtained by downloading the three compressed files using (1) MNP-4
without data compression, (2) MNP-5, (3) V.42 without data compression, and
(4) V.42bis.

Filename MNP-4 MNP-5 V.42 V.42bis

dayrpt.arc 1023 cps 946 1002 1010

sunset.arc 971 935 953 950
text109k.arc 1085 988 1064 1053
If you have ever downloaded files from a BBS or online service, you know that
almost all files are in a compressed format. Therefore, you should only expect to
see an actual throughput between 950 to 1100 cps even if your V.32/V.42bis
modem is supposed to offer throughput "up to" 38400 bps.

Most PC files are in the ZIP format. Macintosh files are typically in the .SIT
(Stuffit) or .CPT (Compact Pro) format. Amiga files are usually in the ZOO, ARC or
LZH format. Note that GIF files are also in a compressed format.

Compression by Software vs. MNP-5/V.42bis

There are several reasons why compression software programs (such as PKZIP or
Stuffit) are superior to MNP-5 or V.42bis.

1. Compressed files save disk storage space.

20
 2. Compression software programs are more versatile. Most of them allow
you to group several files in a compressed file archive to ensure that all
the related files get transferred at the same time.
3. Software compression is more efficient than on-the-fly modem
compression. In the case of a small file, this may not make much
difference. But the difference can be significant when you are transferring
large files.

Filename Size Time Throughput

the-wave.txt 143579 bytes 43 seconds 3296 cps

dayrpt.arc 8423 bytes 8 seconds 1010 cps
dayrpt.wks 19712 bytes 8 seconds 2337 cps
sunset.arc 5084 bytes 5 seconds 950 cps
sunset.pic 16391 bytes 6 seconds 2643 cps
text109k.arc 29775 bytes 28 seconds 1053 cps
text109k.txt 111386 bytes 39 seconds 2822 cps
As we can see from the test results, it is about 30% faster to transfer the
compressed file text109k.arc than to download the text file with V.42bis.

Hayes BBS does not provide a compressed version for the file the-wave.txt.
Using PKZIP (for PC) and Stuffit (for Macintosh), we obtain the following results:

the-wave.zip: 6812 bytes (PKZIP)

the-wave.sit: 6081 bytes (Stuffit)
Assuming a transfer speed of 1000 cps, the compressed file can be downloaded in
7 seconds. That's six times faster than downloading the text file with V.42bis!

Here is another example. Spider Island Software BBS (714-730-5785) has a test file
called One-Minute Max. It is a Macintosh TIFF file (file size 206,432 bytes).
According to Spider Island Software, the file can be downloaded in 56 seconds
(with an effective throughput of 3745cps) with a V.32/V.42bis modem.

The result may seem impressive at first. However, the file can be compressed to
6065 bytes (with Compact Pro) or 7385 bytes (with Stuffit). Assuming a transfer
speed of 1000 cps, it would only take 6-8 seconds to transfer. Again, it is seven
to nine times faster than downloading the file with V.42bis.

On-the-fly modem compression does have one advantage. It is more convenient.

You can send a file without compressing it first and the recipient does not need
to decompress the file.

Local Flow Control and Data Buffering

To get the most from a modem with data compression, you'll want to send data
from your PC to the modem as quickly as possible. If the modem is idle and
waiting for the computer to send data, you are not getting the maximum
performance from the modem.

21
 For example, you have a V.32/V.42bis modem and you want to send a text file to
a remote system which also has a V.32/V.42bis modem. Let's assume the
modem is able to send the file at 20000 bps using V.42bis. If your computer is
sending data to your modem at 9600 bps, your modem will have to stop and wait
to receive data from your computer.

To get the maximum performance, you want to set the computer to send data to
the modem at 38400 bps (the maximum a V.32/V.42bis modem can achieve).
Since the modem can only send the file to the other modem at 20000 bps, it will
never have to wait.

CALIC

CALIC stands for Context Adaptive Lossless Image Compression, a lossless image
compression technique. CALIC obtains higher lossless compression of
continuous-tone images than other techniques reported in the literature. This
high coding efficiency is accomplished with relatively low time and space
complexities. It's designed to achieve higher compression ratios than other
lossless methods, particularly for continuous-tone images. CALIC works by using
context modeling and a non-linear predictor that adapts to varying image
statistics.

Here's a more detailed breakdown:

Key Features and Techniques:

 Lossless Compression:

CALIC ensures that the original image data is preserved, meaning no information
is lost during compression or decompression.

 Context Modeling:

CALIC uses a large number of modeling contexts to condition a non-linear

predictor. This means the algorithm considers the surrounding pixels (context)
when predicting the value of the current pixel.

 Adaptive Predictor:

The predictor in CALIC is designed to adapt to the statistical variations within the
image. This allows it to handle different types of image content (e.g., smooth
areas, edges) more effectively.

 Error Feedback Mechanism:

CALIC uses an error feedback mechanism to refine the predictor's performance

over time.

 Side Information:

CALIC can also utilize side information (unaltered portions of the original image)
to further improve compression efficiency.

22
 Benefits of CALIC:

 High Compression Ratios:

CALIC is known for achieving higher lossless compression ratios compared to

some other techniques.

 Adaptive to Image Content:

The adaptive nature of CALIC allows it to handle various types of image content
effectively.

 Low Complexity:

Despite its effectiveness, CALIC has relatively low time and space complexities.

In essence, CALIC is a sophisticated lossless image compression algorithm that

leverages context modeling and a dynamic predictor to achieve high
compression performance while ensuring data integrity.

23