Chapter 3

MULTIMEDIA DATA
REPRESENTATIONS
Content
3.1. Graphic/Image Data Representation
3.2. Popular File Formats
3.3. Digital Audio and MIDI

1/5/2023 CH-2 2
 3.1. Graphic/Image Data Representation
• The number of file formats used in multimedia continues to proliferate.
GlF and JPG image file formats are the two formats that most web
browsers can decompress and display them.

1/5/2023 CH-2 3
 3.1. Graphic/Image Data Representation
• An image could be described as two-dimensional array of points where every point is
allocated its own color.
• Every such single point is called pixel, short form of picture element.
• Image is a collection of these points that are colored in such a way that they produce
meaningful information /data.
• Pixel (picture element) contains the color or hue and relative brightness of that point in the
image.
• The number of pixels in the image determines the resolution of the image.
❑ A digital image consists of many picture elements, called pixels.
❑ The number of pixels determines the quality of the image resolution.
❑ Higher resolution always yields better quality.
❑ Bitmap resolution most graphics applications let you create bitmaps up to 300 dots per
inch (dpi). Such high resolution is useful for print media, but on the screen most of the
information is lost, since monitors usually display around 72 to 96 dpi.
1/5/2023 CH-2 4
 3.1. Graphic/Image Data Representation
❑ A bit-map representation stores the graphic/image data in the same manner that the computer
monitor contents are stored in video memory.
❑ Most graphic/image formats incorporate compression because of the large size of the data.

1/5/2023 CH-2 5
 3.1. Graphic/Image Data Representation
Types of Images
There are two basic forms of computer graphics: bit-maps and vector graphics.
The kind you use determines the tools you choose. Bitmap formats are the ones
used for digital photographs. Vector formats are used only for line drawings.
Bit-map images (also called Raster Graphics)
They are formed from pixels – a matrix of dots with different colors. Bitmap
images are defined by their dimension in pixels as well as by the number of
colors they represent. For example, a 640X480 image contains 640 pixels and
480 pixels in horizontal and vertical direction respectively. If you enlarge a small
area of a bit-mapped image, you can clearly see the pixels that are used to create
it.

1/5/2023 CH-2 6
 3.1. Graphic/Image Data Representation
Bit-map images (also called Raster Graphics) …
Each of the small pixels can be a shade of gray or a color. Using 24-bit color, each pixel
can be set to any one of 16 million colors. All digital photographs and paintings are
bitmapped, and any other kind of image can be saved or exported into a bitmap format.
In fact, when you print any kind of image on a laser or ink-jet printer, it is first
converted by either the computer or printer into a bitmap form so it can be printed with
the dots the printer uses.
To edit or modify bitmapped images you use a paint program. Bitmap images are
widely used but they suffer from a few unavoidable problems. They must be printed or
displayed at a size determined by the number of pixels in the image. Bitmap images also
have large file sizes that are determined by the images dimensions in pixels and its color
depth. To reduce this problem, some graphic formats such as GIF and JPEG are used to
store images in compressed format.
1/5/2023 CH-2 7
 3.1. Graphic/Image Data Representation
Vector graphics
They are really just a list of graphical objects such as lines, rectangles, ellipses, arcs, or curves – called
primitives. Draw programs, also called vector graphics programs, are used to create and edit these vector
graphics. These programs store the primitives as a set of numerical coordinates and mathematical
formulas that specify their shape and position in the image. This format is widely used by computer-aided
design programs to create detailed engineering and design drawings. It is also used in multimedia when
3D animation is desired. Draw programs have a number of advantages over paint-type program.
These include:
▪ Precise control over lines and colors.
▪ Ability to skew and rotate objects to see them from different angles or add perspective.
▪ Ability to scale objects to any size to fit the available space. Vector graphics always print at the best
resolution of the printer you use, no matter what size you make them.
▪ Color blends and shadings can be easily changed.
▪ Text can be wrapped around objects.

1/5/2023 CH-2 8
 3.1. Graphic/Image Data Representation
Types of Bitmap Images
1. Monochrome/Bit-Map Images
▪ Images consist of pixels, or pels – picture elements in digital images. A 1-bit image consists of on and
off bits only and thus is the simplest type of image. Each pixel is stored as a single bit (0 or 1). Hence,
such an image is also referred to as a binary image. It is also called a I-bit monochrome image, since
it contains no color.
▪ The value of the bit indicates whether it is light or dark
Question: A 640 x 480 monochrome image requires ______KB of storage.
A640 × 480 monochrome image requires 37.5 kB of storage (= 640 × 480/8*1024). Monochrome
1-bit images can be satisfactory for pictures containing only simple graphics and text. Moreover, fax
machines use1-bit data, so in fact 1-bit images are still important even though storage capacities have
increased enough to permit the use of imaging that carries more information.
▪ Dithering is used to calculate patterns of dots such that values from 0 to 255 correspond to patterns
that are more and more filled at darker pixel values, for printing on a 1-bit printer.
▪ Dithering is often used for displaying monochrome images
1/5/2023 CH-2 9
 3.1. Graphic/Image Data Representation
Types of Bitmap Images
1. Monochrome/Bit-Map Images….
Figure 3. 2 Monochrome image

1/5/2023 CH-2 10
 3.1. Graphic/Image Data Representation
Types of Bitmap Images
2. Gray-scale Images
▪ Each pixel is usually stored as a byte (value between 0 to 255). The entire image can
be thought of as a two-dimensional array of pixel values. We refer to such an array as
a bitmap, a representation of the graphics/image data that parallels the manner in
which it is stored in video memory.
▪ This value indicates the degree of brightness of that point. This brightness goes from
black to white
▪ A 640 x 480 grayscale image requires over 300 KB of storage.

1/5/2023 CH-2 11
 3.1. Graphic/Image Data Representation
Types of Bitmap Images
2. Gray-scale Images …
Figure 3. 3 Gray-scale Images

1/5/2023 CH-2 12
 3.1. Graphic/Image Data Representation
Types of Bitmap Images
3. 8-bit Color Images
▪ One byte for each pixel
▪ Supports 256 out of the millions possible, acceptable color quality
▪ Requires Color Look-Up Tables (LUTs)
▪ A 640 x 480 8-bit color image requires 307.2 KB of storage (the same as 8-bit
greyscale).
Examples: GIF

1/5/2023 CH-2 13
 3.1. Graphic/Image Data Representation
Types of Bitmap Images
3. 8-bit Color Images

1/5/2023 CH-2 14
 3.1. Graphic/Image Data Representation
Types of Bitmap Images
3. 8-bit Color Images …
Such image files use the concept of a lookup table to store color
information. Basically, the image stores not color but instead just a set of
bytes, each of which is an index into a table with 3-byte values that specify
the color for a pixel with that lookup table index. In a way, it is a bit, as a
paint-by-number children’s art set, with number 1 perhaps standing for
orange, number 2 for green, and so on – there is no inherent pattern to the
set of actual colors.

1/5/2023 CH-2 15
 3.1. Graphic/Image Data Representation
Types of Bitmap Images
3. 8-bit Color Images …
Color lookup Tables (LUTs)
It used in 8-bit color images is to store only the index, or code value, for each pixel. Then, if a pixel
stores, say, the value 25, the meaning is to go to row 25 in a color lookup table (LUT). While images are
displayed as two-dimensional arrays of values, they are usually stored in row-column order as simply a
long series of values. For an 8-bit image, the image file can store in the file header information just what
8-bit values for R, G, and B correspond to each index. Figure 3.8 displays this idea. The LUT is often
called a palette.

Figure 3. 5 Color LUT for 8-bit color images.

1/5/2023 CH-2 16
 3.1. Graphic/Image Data Representation
Types of Bitmap Images
3. 8-bit Color Images …
4. 24-bit Color Images
▪ Each pixel is represented by three bytes (e.g., RGB)
▪ Supports 256 x 256 x 256 possible combined colors (16,777,216)
▪ A 640 x 480 24-bit color image would require 921.6 KB of storage
▪ Most 24-bit images are 32-bit images,
▪ The extra byte of data for each pixel is used to store an alpha value representing special effect
information
Image Resolution
Image resolution refers to the spacing of pixels in an image and is measured in pixels per inch, ppi,
sometimes called dots per inch, dpi. The higher the resolution, the more pixels in the image. A printed
image that has a low resolution may look pixelated or made up of small squares, with jagged edges and
without smoothness. Image size refers to the physical dimensions of an image.

1/5/2023 CH-2 17
 3.2. Popular File Formats
• Choosing the right file type for your image to save in is of vital
importance. If you are, for example, creating image for web pages, then it
should load fast. So such images should be small size. The other criteria to
choose file type is taking into consideration the quality of the image that is
possible using the chosen file type. You should also be concerned about the
portability of the image. To choose file type:
– resulting size of the image (large file size or small)
– quality of image possible by the file type
– portability of file across different platforms

• The most common formats used on internet are the GIF, JPG, and PNG.
1/5/2023 CH-2 18
 3.2. Popular File Formats…
Standard System Independent Formats
Graphics Interchange Format (GIF)
• Graphics Interchange Format (GIF) devised CompuServe, initially for transmitting
graphical images over phone lines via modems.
– Uses the Lempel-Ziv Welch algorithm (a form of Huffman Coding), modified slightly for image scan
line packets (line grouping of pixels).
– Limited to only 8-bit (256) color images, suitable for images with few distinctive colors (e.g., graphics
drawing)
– Supports one-dimensional interlacing (downloading gradually in web browsers. Interlaced images
appear gradually while they are downloading. They display at a low blurry resolution first and then
transition to full resolution by the time the download is complete.)
– Supports animation multiple pictures per file (animated GIF)
– GIF format has long been the most popular on the Internet, mainly because of its small size
– GIFs allow single-bit transparency, which means when you are creating your image, you can specify
one color to be transparent. This allows background colors to show through the image
1/5/2023 CH-2 19
 3.2. Popular File Formats…
Standard System Independent Formats
PNG
— stands for Portable Network Graphics
— It is intended as a replacement for GIF in the WWW and image editing tools.
— GIF uses LZW compression, which is patented by Unisys. All use of GIF may have to pay
royalties to Unisys due to the patent.
— PNG uses unpatented zip technology for compression
— One version of PNG, PNG-8, is similar to the GIF format. It can be saved with a maximum of
256 colors and supports 1-bit transparency. File sizes when saved in a capable image editor
like Fire Works will be noticeably smaller than the GIF counterpart will, as PNGs save their
color data more efficiently.
— PNG-24 is another version of PNG, with 24-bit color support, allowing ranges of color to a
high color JPG. However, PNG-24 is in no way a replacement format for JPG, because it is a
loss-less compression format, which results in large file size.
— Provides transparency using alpha value
— Supports interlacing
— PNG can be animated through the MNG extension of the format, but browser support is less
for this format.
1/5/2023 CH-2 20
 3.2. Popular File Formats…
Standard System Independent Formats
TIFF
— Tagged Image File Format (TIFF), stores many different types of images (e.g.,
monochrome, grayscale, 8-bit & 24-bit RGB, etc.)
— Uses tags, keywords defining the characteristics of the image that is included in the
file. For example, a picture 320 by 240 pixels would include a ‘width’ tag followed
by the number ‘320’ and a ‘depth’ tag followed by the number ‘240’.
— Developed by the Aldus Corp. in the 1980s and later supported by the Microsoft
— TIFF is a lossless format (when not utilizing the new JPEG tag which allows for
JPEG compression)
— It does not provide any major advantages over JPEG and is not as user-controllable.
— Do not use TIFF for web images. They produce big files, and more importantly, most
web browsers will not display TIFFs.
1/5/2023 CH-2 21
 3.2. Popular File Formats…
System Dependent Formats
Microsoft Windows: BMP
̶ Bit Map (BMP) is the major system standard graphics file format for
Microsoft Windows, used in Microsoft Paint and other programs.
̶ It makes use of run-length encoding compression and can efficiently
store 24-bit bitmap images.
̶ Note, however, that BMP has many different modes, including
uncompressed 24-bit images.
̶ A system standard graphics file format for Microsoft Windows
̶ Used in Many PC Graphics programs
̶ It is capable of storing 24-bit bitmap images
1/5/2023 CH-2 22
 3.2. Popular File Formats…
System Dependent Formats
Macintosh: PAINT and PICT
̶ PAINT was originally used in Mac Paint program, initially only for 1-bit
monochrome images.
̶ PICT is a file format that was developed by Apple Computer in 1984 as
the native format for Macintosh graphics.
̶ The PICT format is a meta-format that can be used for both bitmap
images and vector images though it was originally used in MacDraw (a
vector based drawing program) for storing structured graphics.
̶ Still an underlying Mac format (although PDF on OS X).

1/5/2023 CH-2 23
 3.3. Digital Audio and MIDI
What is Sound?
• Sound is produced by a rapid variation in the average density or pressure of air molecules above and
below the current atmospheric pressure
• We perceive sound as these pressure fluctuations cause our eardrums to vibrate. These usually minute
changes in atmospheric pressure are referred to as sound pressure and the fluctuations in pressure as
sound waves.
• Sound waves are produced by a vibrating body, be it a guitar string, loudspeaker cone or jet engine.
• The vibrating sound source causes a disturbance to the surrounding air molecules, causing them
bounce off each other with a force proportional to the disturbance.
• The back and forth oscillation of pressure produces a sound waves.
Source — Generates Sound
 Air Pressure changes
 Electrical —Microphone produces electric signal
 Acoustic (sound) — Direct Pressure Variations Destination
— Receives Sound
 Electrical — Loud Speaker
 Ears — Responds to pressure hear sound
1/5/2023 CH-2 24
 3.3. Digital Audio and MIDI
How to Record and Play Digital Audio
• In order to play digital audio (i.e WAVE file), you need a card with a Digital to Analog Converter
(DAC) circuitry on it.
• Most sound cards have both an ADC (Analog to Digital Converter) and a DAC so that the card can
both record and play digital audio. This DAC is attached to the Line Out jack of your audio card, and
converts the digital audio values back into the original analog audio. This analog audio can then be routed
to a mixer, or speakers, or headphones so that you can hear the recreation of what was originally recorded.
• Playback process is almost an exact reverse of the recording process.
– First, to record digital audio, you need a card, which has an Analog to Digital Converter (ADC) circuitry. The
ADC is attached to the Line In (and Mic In) jack of your audio card, and converts the incoming analog audio to a
digital signal. Your computer software can store the digitized audio on your hard drive, visually display on the
computer’s monitor, mathematically manipulate in order to add effects, or process the sound, etc. While the
incoming analog audio is being recorded, the ADC is creates many digital values in its conversion to a digital
audio representation of what is being recorded. These values must be stored for later playback.

1/5/2023 CH-2 25
 3.3. Digital Audio and MIDI
• Digitizing Sound
 Microphone produces analog signal
 Computers understands only discrete(digital) entities
This creates a need to convert Analog audio to Digital audio — specialized hardware. This is also known
as Sampling.
Common Audio Formats
There are two basic types of audio files:
• The Traditional Discrete Audio File:
In traditional audio file, you can save to a hard drive or other digital storage medium.
WAV: The WAV format is the standard audio file format for Microsoft Windows applications, and is
the default file type produced when conducting digital recording within Windows. It supports a variety
of bit resolutions, sample rates, and channels of audio. This format is very popular upon IBM PC
(clone) platforms, and is widely used as a basic format for saving and modifying digital audio data.

1/5/2023 CH-2 26
 3.3. Digital Audio and MIDI
• Digitizing Sound
Common Audio Formats
The Traditional Discrete Audio File:
AIF: The Audio Interchange File Format (AIFF) is the standard audio format employed by computers
using the Apple Macintosh operating system. Like the WAV format, it supports a variety of bit
resolutions, sample rates, and channels of audio and is widely used in software programs used to create
and modify digital audio.
• AU: The AU file format is a compressed audio file format developed by Sun Microsystems and
popular in the unix world. It is also the standard audio file format for the Java programming language.
Only supports 8-bit depth thus cannot provide CD-quality sound.
• MP3: MP3 stands for Motion Picture Experts Group, Audio Layer 3 Compression. MP3 files provide
near-CD-quality sound but are only about 1/10th as large as a standard audio CD file. Because MP3
files are small, they can easily be transferred across the Internet and played on any multimedia
computer with MP3 player software.

1/5/2023 CH-2 27
 3.3. Digital Audio and MIDI
• Digitizing Sound
Common Audio Formats
The Traditional Discrete Audio File:
• MIDI/MID: MIDI (Musical Instrument Digital Interface), is not a file format for storing or transmitting recorded
sounds, but rather a set of instructions used to play electronic music on devices such as synthesizers. MIDI files are
very small compared to recorded audio file formats. However, the quality and range of MIDI tones is limited.
2. Streaming Audio File Formats
• Streaming is a network technique for transferring data from a server to client in a format that can be continuously
read and processed by the client computer. Using this method, the client computer can start playing the initial
elements of large time-based audio or video files before the entire file is downloaded. As the Internet grows,
streaming technologies are becoming an increasingly important way to deliver time-based audio and video data. For
streaming to work, the client side has to receive the data and continuously feed it to the player application. If the
client receives the data more quickly than required, it has to temporarily store or buffer the excess for later play. On
the other hand, if the data does not arrive quickly enough, the audio or video presentation will be interrupted. There
are three primary streaming formats that support audio files: RealNetwork’s RealAudio (RA, RM), Microsoft’s
Advanced Streaming Format (ASF) and its audio subset called Windows Media Audio 7 (WMA) and Apple s
QuickTime 4.0+ (MOV).

1/5/2023 CH-2 28
 3.3. Digital Audio and MIDI
• Digitizing Sound
Common Audio Formats
The Traditional Discrete Audio File:
2. Streaming Audio File Formats
RA/RM
For audio data on the Internet, the de facto standard is RealNetwork’s RealAudio (.RA) compressed streaming audio
format. These files require a RealPlayer program or browser plug-in. The latest versions of RealNetworks server and
player software can handle multiple encodings of a single file, allowing the quality of transmission to vary with the
available bandwidth. Webcast radio broadcast of both talk and music frequently uses RealAudio. Streaming audio can
also be provided in conjunction with video as a combined RealMedia (RM) file.
ASF
Microsofts Advanced Streaming Format (ASF) is similar to designed to RealNetwork’s RealMedia format, in that it
provides a common definition for internet streaming media and can accommodate not only synchronized audio, but
also video and other multimedia elements, all while supporting multiple bandwidths within a single media file. Also
like RealNetwork’s RealMedia format, Microsofts ASF requires a program or browser plugin.
The pure audio file format used in Windows Media Technologies is Windows Media Audio 7 (WMA files). Like MP3
files, WMA audio files use sophisticated audio compression to reduce file size. Unlike MP3 files, however, WMA
files can function as either discrete or streaming data and can provide a security mechanism to prevent unauthorized
use.
1/5/2023 CH-2 29
 3.3. Digital Audio and MIDI
• Digitizing Sound
Common Audio Formats
The Traditional Discrete Audio File:
2. Streaming Audio File Formats
MOV
• Apple QuickTime movies (MOV files) can be created without a video channel and used as a sound-only format.
Since version 4.0, QuickTime provides true streaming capability. QuickTime also accepts different audio sample
rates, bit depths, and offers full functionality in both Windows as well as the Mac OS. Popular audio file formats are:
 au (Unix)
 aiff (MAC)
 wav (PC)
 mp3
MIDI
• MIDI stands for Musical Instrument Digital Interface.
• Definition of MIDI: It is a protocol that enables computer, synthesizers, keyboards, and other musical device to
communicate with each other. This protocol is a language that allows interworking between instruments from
different manufacturers by providing a link that is capable of transmitting and receiving digital data.
• MIDI transmits only commands; it does not transmit an audio signal. It was created in 1982.
1/5/2023 CH-2 30
 3.3. Digital Audio and MIDI
• Digitizing Sound
Common Audio Formats
The Traditional Discrete Audio File:
2. Streaming Audio File Formats
• Components of a MIDI System
1. Synthesizer: It is a sound generator (various pitch, loudness, tone color). A good (musician’s) synthesizer often has a
microprocessor, keyboard, control panels, memory, etc.
2. Sequencer: It can be a stand-alone unit or a software program for a personal computer. (It used to be a storage server for MIDI
data. Nowadays it is more a software music editor on the computer. It has one or more MIDI INs and MIDI OUTs.
• Basic MIDI Concepts
Track: Track in sequencer is used to organize the recordings. Tracks can be turned on or off on recording or playing back.
Channel: MIDI channels are used to separate information in a MIDI system. There are 16 MIDI channels in one cable. Channel
numbers are coded into each MIDI message.
Timbre: The quality of the sound, e.g., flute sound, cello sound, etc.
• Multi-timbral: capable of playing many different sounds at the same time (e.g., piano, brass, drums,..)
Pitch: The Musical note that the instrument plays
Voice: Voice is the portion of the synthesizer that produces sound. Synthesizers can have many (1Two, Two0, Two4, 36, etc.)
voices. Each voice works independently and simultaneously to produce sounds of Different timbre and pitch.
Patch: The control settings that define a particular timbre.

1/5/2023 CH-2 31
 Questions
1. What is the difference bit map and vectors graphics? Which one is better?
Why.
2. What is grayscale Image?
3. What are the different popular file formats of images(like, Exchangeable
Image File (EXIF), PTM (Polynomial Texture Mapping))?
4. What are the different popular file formats of video (Like, High Efficiency
Image File(HEIF))?
5. Explain PS and PDF file formats.
6. Explain briefly the common audio formats. Where we used?

1/5/2023 CH-2 32
 Exercises
1. Briefly explain why we need to be able to have less than 24-bit color and why this makes for a
problem. Generally, what do we need to do to adaptively transform 24-bit color values to 8-bit ones?
2. Suppose we decide to quantize an 8-bit grayscale image down to just 2 bits of accuracy. What is the
simplest way to do so? What ranges of byte values in the original image are mapped to what quantized
values?
3. Suppose we have a 5-bit grayscale image. What size of Ordered-dither matrix do we need to display
the image on a 1-bit printer?
4. Suppose we have available 24 bits per pixel for a color image. However, we notice that humans are
more sensitive to R and G than to B—in fact, 1.5 times more sensitive to R or G than to B. How could
we best make use of the bits available?
5. At your job, you have decided to impress the boss by using up more disk space for the company’s
grayscale images. Instead of using 8 bits per pixel, you’d like to use 48 bits per pixel in RGB. How
could you store the original grayscale images so that in the new format they would appear the same as
they used to, visually?
6. Write down an algorithm (pseudocode) for calculating a color histogram for RGB data.

1/5/2023 CH-2 33