CHAPTER 3: DATA REPRESENTATIONS

• 3.1. Graphic/Image Data Representation

• An image could be described as two-dimensional array of points where
every point is allocated its own color. Every such single point is called pixel,
short form of picture element.
• Pixel (picture element) contains the color or hue and relative brightness of
that point in the image.
• The number of pixels in the image determines the resolution of the image.
A digital image consists of many picture elements, called pixels.
The number of pixels determines the quality of the image  image
resolution.
Higher resolution always yields better quality.
 Cont..

• Bitmap resolution most graphics applications let you create bitmaps up to

300 dots per inch (dpi).

• Such high resolution is useful for print media, but on the screen most of the
information is lost, since monitors usually display around 72 to 96 dpi.

• A bit-map representation stores the graphic/image data in the same manner

that the computer monitor contents are stored in video memory.

• Most graphic/image formats incorporate compression because of the large

size of the data.
 3.2. Types of images
• There are two basic forms of computer graphics: bit-maps and vector graphics.
• The kind you use determines the tools you choose.
• Bitmap formats are the ones used for digital photographs. Vector formats are used only
for line drawings.
• 1. Bit-map images (also called Raster Graphics):
• They are formed from pixelsóa matrix of dots with different colors. Bitmap images are
defined by their dimension in pixels as well as by the number of colors they represent.
• For example, a 640X480 image contains 640 pixels and 480 pixels in horizontal and
vertical direction respectively.
• Using 24-bit color, each pixel can be set to any one of 16 million colors.
• All digital photographs and paintings are bitmapped, and any other kind of image can be
saved or exported into a bitmap format
• Cont.…

• To edit or modify bitmapped images you use a paint program. Bitmap

images are widely used but they suffer from a few unavoidable problems.

• They must be printed or displayed at a size determined by the number of

pixels in the image. Bitmap images also have large file sizes that are
determined by the image dimensions in pixels and its color depth. To reduce
this problem, some graphic formats such as GIF and JPEG are used to store
images in compressed format.
 Vector graphics:

• are really just a list of graphical objects such as lines, rectangles, ellipses, arcs, or curves called
primitives. Draw programs, also called vector graphics programs, are used to create and edit these
vector graphics.

• Draw programs have a number of advantages over paint-type programs. These include:

• Precise control over lines and colors.

• Ability to skew and rotate objects to see them from different angles or add perspective.

• Ability to scale objects to any size to fit the available space. Vector graphics always print at the best
resolution of the printer you use, no matter what size you make them.

• Color blends and shadings can be easily changed.

• Text can be wrapped around objects.

3. Monochrome/Bit-Map Images
• Each pixel is stored as a single bit (0 or 1)
• The value of the bit indicates whether it is light or dark
• A 640 x 480 monochrome image requires 37.5 KB of storage.
• Dithering is often used for displaying monochrome images
4. Gray-scale Images
• Each pixel is usually stored as a byte (value between 0 to 255)
• This value indicates the degree of brightness of that point. This
brightness goes from black to white
• A 640 x 480 grayscale image requires over 300 KB of storage
6. 8-bit Color Images
• One byte for each pixel
• Supports 256 out of the millions possible, acceptable color quality
• Requires Color Look-Up Tables (LUTs)
• A 640 x 480 8-bit color image requires 307.2 KB of storage (the same
as 8-bit greyscale)
• Examples: GIF
6. 24-bit Color Images
• Each pixel is represented by three bytes (e.g., RGB)
• Supports 256 x 256 x 256 possible combined colors (16,777,216)
• A 640 x 480 24-bit color image would require 921.6 KB of storage
• Most 24-bit images are 32-bit images,
o the extra byte of data for each pixel is used to store an alpha value
representing special effect information
• Image resolution: refers to the spacing of pixels in an image and is
measured in pixels per inch, ppi, sometimes called dots per inch, dpi.
The higher the resolution, the more pixels in the image
3.4. Popular File Formats
• Choosing the right file type for your image to save in is of vital
importance. If you are, for example, creating image for web pages, then
it should load fast. So such images should be small size. The other
criteria to choose file type is taking into consideration the quality of the
image that is possible using the chosen file type. You should also be
concerned about the portability of the image.
To choose file type:
• resulting size of the image large file size or small
• quality of image possible by the file type
• portability of file across different platforms
The most common formats used on internet are the GIF, JPG, and PNG
3.4.1. Standard System Independent Formats
• 1. GIF
-Graphics Interchange Format (GIF) devised CompuServe, initially for
transmitting graphical images over phone lines via modems.
- Uses the Lempel-Ziv Welch algorithm (a form of Huffman Coding),
modified slightly for image scan line packets (line grouping of pixels).
- LZW compression was patented technology by the UNISYS Corp.
- Limited to only 8-bit (256) color images, suitable for images with few
distinctive colors (e.g., graphics drawing)
- GIF format has long been the most popular on the Internet, mainly
because of its small size
2. PNG
-stands for Portable Network Graphics
-It is intended as a replacement for GIF in the WWW and image editing
tools.
-GIF uses LZW compression which is patented by Unisys. All use of GIF
may have to pay royalties to Unisys due to the patent.
-PNG uses unpatented zip technology for compression
3. JPEG/JPG
• A standard for photographic image compression
• created by the Joint Photographic Experts Group
• Intended for encoding and compression of photographs and similar
images
• Takes advantage of limitations in the human vision system to achieve
high rates of compression
• Uses complex lossy compression which allows user to set the desired
level of quality (compression). A compression setting of about 60%
will result in the optimum balance of quality and filesize.
• Though JPGs can be interlaced, they do not support animation and
transparency unlike GIF
 4. TIFF
• Tagged Image File Format (TIFF), stores many different types of images
(e.g., monochrome, grayscale, 8-bit & 24-bit RGB, etc.)
• Uses tags, keywords defining the characteristics of the image that is
included in the file. For example, a picture 320 by 240 pixels would include
a 'width' tag followed by the number '320' and a 'depth' tag followed by
the number '240'.
• TIFF is a lossless format (when not utilizing the new JPEG tag which allows
for JPEG compression)
• It does not provide any major advantages over JPEG and is not as user-
controllable.
• Do not use TIFF for web images. They produce big files, and more
importantly, most web browsers will not display TIFFs
 3.4.2. System Dependent Formats
Microsoft Windows: BMP
• A system standard graphics file format for Microsoft Windows
• Used in Many PC Graphics programs
• It is capable of storing 24-bit bitmap images
Macintosh: PAINT and PICT
• PAINT was originally used in MacPaint program, initially only for 1-bit
monochrome images.
• PICT is a file format that was developed by Apple Computer in 1984 as the
native format for Macintosh graphics.
• The PICT format is a meta-format that can be used for both bitmap images
and vector images though it was originally used in MacDraw (a vector
based drawing program) for storing structured graphics
• Still an underlying Mac format (although PDF on OS X)
X-windows: XBM
• Primary graphics format for the X Window system
• Supports 24-bit colour bitmap
• Many public domain graphic editors, e.g., xv
• Used in X Windows for storing icons, pixmaps, backdrops, etc.
Digital Audio and MIDI
• what is Sound? Sound is produced by a rapid variation in the average
density or pressure of air molecules above and below the current
atmospheric pressure. We perceive sound as these pressure fluctuations
cause our eardrums to vibrate. These usually minute changes in
atmospheric pressure are referred to as sound pressure and the fluctuations
in pressure as sound waves.
• Sound waves are produced by a vibrating body, be it a guitar string,
loudspeaker cone or jet engine.
• The back and forth oscillation of pressure produces a sound waves.
• Source  Generates Sound
• Air Pressure changes
• Electrical  Microphone produces electric signal
• Acoustic Direct Pressure Variations
• Destination  Receives Sound
• Electrical  Loud Speaker
• Ears  Responds to pressure hear sound
3.5.1 Common Audio Formats
• There are two basic types of audio files: the traditional discrete audio file, that you
can save to a hard drive or other digital storage medium, and the streaming audio
file that you listen to as it downloads in real time from a network/internet server to
your computer.
1. Discrete Audio File Formats : Common discrete audio file formats include WAV,
AIF, AU and MP3
2. WAV: The WAV format is the standard audio file format for Microsoft Windows
applications, and is the default file type produced when conducting digital
recording within Windows.
3. AIF/AIFF: The Audio Interchange File Format (AIFF) is the standard audio format
employed by computers using the Apple Macintosh operating system
4. AU: The AU file format is a compressed audio file format developed by Sun
Microsystems and popular in the Unix world.
5. MP3: MP3 stands for Motion Picture Experts Group, Audio Layer 3 Compression
6. MIDI
7. Streaming Audio File Formats: Streaming is a network technique for transferring
data from a server to client in a format that can be continuously read and
Popular audio file formats are:
au (Unix)
aiff (MAC)
wav (PC)
mp3
• MIDI MIDI stands for Musical Instrument Digital Interface.
Definition of MIDI:
• MIDI is a protocol that enables computer, synthesizers, keyboards, and
other musical device to communicate with each other. This protocol is a
language that allows interworking between instruments from different
manufacturers by providing a link that is capable of transmitting and
receiving digital data.
Figure 8 MIDI and Computer connection
Reading assignment
• Basic MIDI Concepts
 Chapter 4: COLOR IN IMAGE AND VIDEO
• Color is produced by the absorption of selected wavelengths of light by an
object.
• Objects can be thought of as
• absorbing all colors except the colors of their appearance which are
reflected back.
• A blue object illuminated by white light absorbs most of the wavelengths
except those corresponding to blue light.
• These blue wavelengths are reflected by the object.
Fig White light composed of all wavelengths of visible light incident on a pure blue
object. Only blue light is reflected from the surface.
4.2. Color Spaces
• Color space specifies how color information is represented. It is also
called color model.
• Any color could be described in a three dimensional graph, called a color
space. There are different ways of representing color. Some of these are:
• RGB color space
• YUV color space
• YIQ color space
• CMY/CMYK color space
• CIE color space
• HSV color space
• HSL color space
• YCbCr color spac
RGB Color Space:
• RGB stands for Red, Green, Blue. RGB color space expresses/defines
color as a mixture of three primary colors: Red, Green, Blue.
• All other colors are produced by varying the intensity of these three
primaries and mixing the colors. It is used self-luminous devices such
as TV, monitor, camera, and scanner
Cont.…
• Absence of all colors (zero values for all the components) create black.
The presence of the three colors form white.

• These colors are called additive colors since they add together the
way light adds to make colors, and is a natural color space to use with
video displays.
CYM and CYMK:
• A color model used with printers and other peripherals. Three
primary colors, cyan (C), magenta (M), and yellow (Y), are used to
reproduce all colors.

where K = min(C, M, Y),

C = C - K, M = M - K,
Y = Y - K.
CIE:
• In 1931, the CIE (Commite Internationale de Eíclairage) developed a
color model based on human perception.
• They are based on the human eyes’ response to red green and blue
colors, and are designed to accurately represent human color
perception.
• The CIE is a device-independent color model and because of this it is
used as a standard for other colors to compare with.
• Device-independent means color can be reproduced faithfully on any
type of device, such as scanners, monitors, and printers (color quality
does not vary depending on the device).
Cont.…
 YIQ Color Model
• YIQ is used in color TV broadcasting, it is downward compatible with Black and White TV.
The YIQ color space is commonly used in North American television systems. Note that if
the chrominance is ignored, the result is a "black and white" picture.

• Y (luminance) is the CIE Y primary Y = 0.299R + 0.587G + 0.114B

• the other two vectors I = 0.596R - 0.275G - 0.321B & Q = 0.212R - 0.528G + 0.311B

• I is red-orange axis, Q is roughly orthogonal to I.

• Eye is most sensitive to Y (luminance), next to I, next to Q. YIQ is intended to take

advantage of human color response characteristics. Eye is more sensitive to Orange-Blue
range (I) than in Purple-Green range (Q). Therefore less bandwidth is required for Q than
for I. NTSC limits I to 1.5 MHZ and Q to 0.6 MHZ. Y is assigned higher bandwidth, 4MHZ.
 YUV Color Model
• Established in 1982 to build digital video standard

• Works in PAL (50 fields/sec) or NTSC (60 fields/sec)

• The luminance (brightness), Y, is retained separately from the chrominance

(color) The Y component determines the brightness of the color (referred to as
luminance or luma), while the U and V components determine the color itself
(it is called chroma). U is the axis from blue to yellow and V is the axis from
magenta to cyan. Y ranges from 0 to 1 (or 0 to 255 in digital formats), while U
and V range from -0.5 to 0.5 (or -128 to 127 in signed digital form, or 0 to 255
in unsigned form).
YUV color space
Chapter 5:Basics of Digital Audio and Fundamental Concepts in Video
• 5.1. Digitizing Sound
• Microphone produces analog signal
• Computer deals with digital signal
Sampling Audio
• Analog Audio Most natural phenomena around us are continuous; they
are continuous transitions between two different states.
• Sound is not exception to this rule i.e. sound also constantly varies.
Continuously varying signals are represented by analog signal.
• Signal is a continuous function f in the time domain. For value y=f(t), the
argument t of the function f represents time.
• If we graph f, it is called wave. (see the following diagram)
Cont.…

Fig 1 analog signal

 Cont.…
• A wave has three characteristics:

• Amplitude: is the intensity of signal. This is can be determined by looking at the

height of signal. If amplitude increases, the sound becomes louder. Amplitude
measures the how high or low the voltage of the signal is at a given point of time.

• Frequency: is the number of times the wave cycle is repeated. This can be
determined by counting the number of cycles in given time interval. Frequency is
related with pitchness of the sound. Increased frequency high pitch.

• Phase: related to the wave’s appearance

Digitizing
 Analog to Digital Conversion
• Converting an analog audio to digital audio requires that the analog signal is
sampled. Sampling is the process of taking periodic measurements of the
continuous signal. Samples are taken at regular time interval, i.e. every T
seconds. This is called sampling frequency/sampling rate.

• Digitized audio is sampled audio. Many times each second, the analog signal is
sampled.

• How often these samples are taken is referred to as sampling rate.

• The amount of information stored ab out each sample is referred to as sample

size.
 Cont...
• Analog signal is represented by amplitude and frequency. Converting these
waves to digital information is referred to as digitizing. The challenge is to
convert the analog waves to numbers (digital information).

• In digital form, the measure of amplitude (the 7 point scale - vertically) is

represented with binary numbers (bottom of graph). The more numbers on
the scale the better the quality of the sample, but more bits will be needed
to represent that sample. The graph below only shows 3- bits being used
for each sample, but in reality either 8 or 16-bits will be used to create all
the levels of amplitude on a scale. (Music CDs use 16-bits for each sample
 CHAPTER 6: DATA COMPRESSION
• Data compression is often referred to as coding, where coding is a very
general term encompassing any special representation of data which
satisfies a given need.
• Definition: Data compression is the process of encoding information using
fewer number of bits so that it takes less memory area (storage) or
bandwidth during transmission. Two types of compression:
 Lossy data compression
Lossless data compression
• Lossless Data Compression: in lossless data compression, the original
content of the data is not lost/changed when it is compressed (encoded).
Examples:
• RLE (Run Length Encoding)
• Dictionary Based Coding
• Arithmetic Coding
Cont…
• Lossy data compression: the original content of the data is lost to
certain degree when compressed. Part of the data that is not much
important is discarded/lost. The loss factor determines whether there
is a loss of quality between the original image and the image after it
has been compressed and played back (decompressed). The more
compression, the more likely that quality will be affected. Even if the
quality difference is not noticeable, these are considered lossy
compression methods.
• Examples : JPEG (Joint Photographic Experts Group) ,MPEG (Moving
Pictures Expert Group),
6.2Information Theory
• Information theory is defined to be the study of efficient coding
and its consequences. It is the field of study concerned about the
storage and transmission of data. It is concerned with source
coding and channel coding.
Source coding: involves compression
Channel coding: how to transmit data, how to overcame noise, etc
Data compression may be viewed as a branch of information
theory in which the primary objective is to minimize the amount of
data to be transmitted.
 7.3 Need for Compression
• With more colors, higher resolution, and faster frame rates, you produce
better quality video, but you need more computer power and more
storage space for your video.
• Doing some simple calculations (see below) it can be shown that with 24-
bit color video, with 640 by 480 resolutions, at 30 fps, requires an
astonishing 26 megabytes of data per second! Not only does this surpass
the capabilities of the many home computer systems, but also
overburdens existing storage systems.
 6.4. Compression Algorithms
• Compression methods use mathematical algorithms to reduce (or compress) data by
eliminating, grouping and/or averaging similar data found in the signal. Although there
are various compression methods, including Motion JPEG, only MPEG-1 and MPEG-2 are
internationally recognized standards for the compression of moving pictures (video).

• Data compression has important application in the areas of data transmission and data
storage.

• Compressing data to be stored or transmitted reduces storage and/or communication

costs.

• When the amount of data to be transmitted is reduced, the effect is that of increasing
the capacity of the communication channel.
• Lossless compression: is a method of reducing the size of computer files
without losing any information. That means when you compress a file, it will
take up less space, but when you decompress it, it will still have the exact
same information. The idea is to get rid of any redundancy in the information,
this is exactly what happens is used in ZIP and GIF files.

• This differs from lossy compression, such as in JPEG files, which loses some
information that isn't very noticeable. Why use lossless compression.

• Modem uses LZW compression automatically to speed up transfers.

 6.4.1. Variable Length Encoding
• Claude Shannon and R.M. Fano created the first compression algorithm in
the 1950's. This algorithm assigns variable number of bits to letters/symbols.
• Shannon-Fano Coding Let us assume the source alphabet S={X1,X2,X3,Ö,Xn}
and Associated probability P={P1,P2,P3,Ö,Pn} The steps to encode data using
Shannon-Fano coding algorithm is as follows: Order the source letter into a
sequence according to the probability of occurrence in non-increasing order
i.e. decreasing order.
Shannon Fanon algorithm
• ShannonFano(sequence s)
If s has two letters
Attach 0 to the codeword of one letter and 1 to the codeword of
another;
Else if s has more than two letter
Divide s into two subsequences S1, and S2 with the minimal difference
between
probabilities of each subsequence;
extend the codeword for each letter in S1 by attaching 0, and by
attaching 1 to each
codeword for letters in S2;
ShannonFano(S1);
ShannonFano(S2);
• Example: Suppose the following source and with related probabilities
• S={A,B,C,D,E}
• P={0.35,0.17,0.17,0.16,0.15} Message to be encoded=ABCDE
• The probability is already arranged in non-increasing order.
• First we divide the message into AB and CDE. Why? This gives the smallest
difference between the total probabilities of the two groups.
• S1= {A, B} P= {0.35,0.17}=0.52
• S2={C, D, E} P= {0.17, 0.17, 0.16} =0.46
• The difference is only 0.52-0.46=0.06.
This is the smallest possible difference when we divide the message. Attach 0
to S1 and 1 to S2. Subdivide S1 into sub groups. S11= {A} attach 0 to this
S12={B} attach 1 to this.
• Again subdivide S2 into subgroups considering the probability again.
• S21={C} P={0.17}=0.17
• S22={D,E} P={0.16,0.15}=0.31
• Attach 0 to S21 and 1 to S22. Since S22 has more than one letter in it, we
have to subdivide it.
• S221={D} attach 0
• S222={E} attach 1
• The message is transmitted using the following code (by traversing
the tree)
• A=00 B=01
• C=10 D=110
• E=111
• Instead of transmitting ABCDE, we transmit 000110110111.
6.4.2Dictionary Encoding
• Dictionary coding uses groups of symbols, words, and phrases with
corresponding abbreviation. It transmits the index of the
symbol/word instead of the word itself. There are different variations
of dictionary based coding.
 LZW Compression
LZW compression has its roots in the work of Jacob Ziv and Abraham Lempel
• The Concept
• Many files, especially text files, have certain strings that repeat very often,
for example " the ".
• With the spaces, the string takes 5 bytes, or 40 bits to encode.
• But what if we were to add the whole string to the list of characters? Then
every time we came across " the ", we could send the code instead of
32,116,104,101,32. This would take less no of bits.
• The Algorithm:
LZWEncoding()
Enter all letters to the dictionary;
Initialize string s to the first letter from the input;
While any input is left
read symbol c;
if s+c exists in the dictionary
s = s+c;
else
output codeword(s); //codeword for s
enter s+c to dictionary;
s =c;
end loop
output codeword(s);
• Example: encode the ff string “aababacbaacbaadaa”
• The program reads one character at a time.
• Example: Encode the message aababacbaacbaadaaa using the above
algorithm
• Encoding Create dictionary of letters found in the message.
• Now instead of the original message, you transmit their indexes in the
dictionary. The code for the message is 112613791145.
Huffman Compression
• Huffman coding has the following properties:
• Codes for more probable characters are shorter than ones for less probable
characters.
• Each code can be uniquely decoded
• To accomplish this, Huffman coding creates what is called a Huffman tree,
which is a binary tree.
• First count the amount of times each character appears, and assign this as a
weight/probability to each character, or node. Add all the nodes to a list.
• Algorithm to create the tree
• Assume the source alphabet S={X1, X2, X3, …,Xn} and
• Associated Probabilities P={P1, P2, P3,…, Pn}
• Huffman()
• For each letter create a tree with single root node and order all trees
according to the probability of letter of occurrence;
• while more than one tree is left
• take two trees t1, and t2 with the lowest probabilities p1, p2 and create a
tree with probability in its root equal to p1+p2 and with t1 and t2 as its
subtrees;
• associate 0 with each left branch and 1 with each right branch;
• create unique codeword for each letter by traversing the tree the root to the
leaf containing the probability corresponding to this letter and putting all
encountered 0s and 1s together;
• Example: Suppose the following source and related probability
• S={A,B,C,D,E}
• P={0.15,0.16,0.17,0.17,0.35}
• Message=”abcde”
Hence, this unique prefix property assures
that each code can be uniquely decoded.
The code for each letter is:
The original message will be encoded to:
abcde=0000010100111

a=000 b=001
c=010 d=011

e=1