CHAPTER 2: INFORMATION

SOURCE BINARY CODING

By
Dr. Ebenezer Anohah
Department of Computer Science and Engineering
University of Mines and Technology
 A source coding in binary digital
• Considering the abstract model of information processing systems in
Chapter 1 lecture note, an information source produces a succession
of symbols as shown below.
.
 A source coding in binary digital
• These symbols or messeages produced are converted by the decoder
or transmitter into digital signals which are suitable for information.
• These digital signals that take discrete values corresponding to “0”
and “1” are widely used or preferred in modern communication
systems
• These digital signals have high noise immunity and can be transmitted
more reliably than analog signals
• When the inevitable corruption of the modern transmission systems
begin to degrade signals, the digital pulse can be detected, reshaped,
and amplified to stand form before relaying them to their final
destination
 A source coding in binary digital
• The figure below illustrates an ideal binary digital pulse propagating
along a digital communication or transmission line, where the pulse
shape is degraded as function of line length or distance; a) original
signal , b) slightly corrupted or distorted signal, c) corrupted or
distorted signal, d) intense corrupted or distorted signal, e)
regenerated or recovered signal by using filters to remove noise and
interference (low-pass filters or band-pass filters)
 source coding analog in binary digital
• On the other hand, analog signals (e.g. television and telephone networks)
is that cannot be so reshaped since they take an infinite variety of shapes
• The classical problem with analog signals is attenuation: that is, the
amplitude of the analog signal decays over time.
• Assuming we want to regularly boost the analog signals, we would also
amplify any noise that has been added to the analog signals.
• Analog encodings are possible, but whatever is used, no “boosting” process
can ever return a corrupted analog signal exactly to its original form.
• Analog encodings consider the amplitudes of analog signals including their
noises, that is , we interpret a positive signal-plus-noise in the received
symbol as a 1 and a negative signal-plus-noise as a 0.
A source coding analog in binary digital?
 What is coding?
• Coding is the process of finding one-to-one correspondence or a
mapping between sent symbols and codewords (string of bits or
bitstring) set obtained using the source alphabet.
• A code is a binary digit that stands for another symbol in sent symbol
from a source.
• Similarly, coding is representing one signal by another
 Types of codes?
• Codes can have a fixed or variable length, and can be static or adaptive
(dynamic)
• Fixed variable length codes are known as block codes. They are easy to
implement in software.
• A static code is constructed once and never changes, an example for such
codes are ASCII (American Standard Code for International Interchange)
which is pronounced “ascii” and Unicode
• A static code can also have variable length, where short codewords are
assigned to the commonly-occurring symbols or probabilities of the
symbols. These codes need code table included in transmission or
compressed file to help decoder know the symbols probabilities and
reconstruct the codewords independently.
 What are the advantages of variable-length
type of code?
• Compress data: It is possible to compress symbols by assigning short
codes to common symbols and long codes to the rare symbols
• Processor design: A processors fetch instructions from memory and
execute them. Modern computers processors reduce the overall size
of programs by fetching instructions set such that commonly-used
instructions are short.
What are examples of variable-length type of
codes?
What are the advantages of Fixed-length type
of codes?
• Uncompressed data: if symbols being compressed are ASCII codes,
then they may simply be assigned their ASCII codes as uncompressed
codes.
• We have different types of ASCII codes, for example English ASCII,
German ASCII, French ASCII, etc.
• Processor design: A processor easily handles fixed-length instructions.
 What are the examples of Fixed-length type
of codes?
• ASCII and Unicode: These are character codes that make it possible to
store characters of text as codewords in computers.
• ASCII codes assigns 7-bit or 8-bit to 128 characters including 26 letters
(uppercase and lowercase), 10 digits, certain punctuation marks, and
several control characters.
• Unicode assigns 32-bit codes to many characters, and has a provision
for longer codes. The long codes make it possible to store and
manipulate many thousands of characters, taken from many
languages.
 Octal and hexadecimal type of codes
• Although the symbols of an information source are usually encoded as
binary sequences, the binary code is sometimes inconvenient for human
use.
• People usually prefer to make a single discrimination among many things.
• Thus, for human use, it is often convenient to group the bits into three
groups of three at a time and call them the octal code (base 8) or there
groups of four and call them the hexadecimal code (base 16).
• Since computers usually work in bytes, which are 8 bits each, the
hexadecimal code fits into the machine architecture better than the octal
code.
• However, octal code seems to fit better into human’s psychology.
• In practice, neither hexadecimal or octal code has clear victory over the
other.
 What are Octal type of codes?
• When using the octal system, numbers are often enclosed in parentheses
with subscript 8.
• For example, the decimal number 81 is written in octal system as (121)8
since 81 = 1 × 82 + 2 × 81 + 1 × 80
• To translate the binary sequence 101011000111 to the octal system, we
first partition these bits into groups of three:

• Each group of bits is then mapped to an octal number by the next octal
table, resulting in the octal representation (5307)8
• Thus, octal system are made up of eight digits 0,1,2,3,4,5,6,7
Octal table
 Hexadecimal type of codes
• When using the Hexadecimal representation, numbers are often
enclosed in parentheses with subscript 16.
• To translate the binary sequence 101011000111 to the hexadecimal
system, we first partition these bits into groups of four:

• We can get the hexadecimal representation

AC7 16 by next hexadecimal table
What are hexadecimal type of codes?
 Prefix-free and suffix type of codes
• Prefix-free codes are like code(a), code(b) and code(c) whose codes
do not form part of another code word in a source alphabet
• Suffix codes are codes that are ending of code words
 Singular and nonsingular type of codes
• Singular codes are like code (i) that allows multiple codes, that is, 0
and 0 or 1 and1 to map onto the same received code word such as a
and b or c and d.
• However, nonsingular codes are like the ones in code (ii) that allow
unique codes, that is, 0, 010, 01 and 10 to match onto different
received code words such as a, b, c and d..
 Instantaneous type of codes
• These instantaneous codes are nonsingular and unique enough to
allow effective decodable.
• Instantaneous codes have least average word-length
• Instantaneous codes include prefix codes
Set of all codes
Set of all codes