Chapter 2:

Multimedia Basics and Representation

By: Dawit Teklu(MSc.)

Multimedia Information Systems 1

This chapter introduces:
2.1.Digital multimedia characteristics
2.2.audio formats and MIDI
2.3.Image formats and color models
2.4.video formats and color models

2
 What is “media” ?
Information represented in different formats/media
– text
Discrete media: time independent
– graphics
– images
– animation
Continuous media: time dependent
– audio
– video

• Analog vs Digital
– analog format: the time-varying feature (variable) of the signal is a continuous representation of
the input, i.e., analogous to the input audio, image, or video signal
– Physical world is analog !
 Multimedia System
• Multimedia: information represented through audio, graphics,
images, video, and animation in an integrated and interactive
manner (as contrast to traditional single-modality media, i.e.,
text and graphics drawing).

• Multimedia system: the generation, manipulation, storage,

presentation, and communication of multimedia information
 Digital Media
• Multimedia digitized
– Captured, stored, transmitted, processing in digital (discrete) domain
– By general purpose computers or dedicated embedded computers
• Today’s digital cameras’ have a number of CPUs inside, many of which are more powerful than a PC of 1990’s or even
2000’s.

• What do you mean by digitized ?

• Why digitized ?
 Digital Media
• What do you mean by digitized ?
– Audio/visual signals from the natural world is Analog
• Continuous in time and space
• Conventional storage/playback: LP (audio record), tape, CRT TV (old TV), film
• Can’t be handled by computer

– A/D conversion
• to 1/0 discrete signals

• Why digitized ?
– Bulky storage (space, cost, lifetime)
– Poor quality
– Poor/no compression
– Poor portability/mobility/editibility
MP3 player, iPod, YouTube ? No way
Film -> Polaroid -> Digital camera
 Image/Video Digitization
• Digital image is a 2-D array of pixels
• Each pixel represented by bits
– R:G:B Original
– Y:U:V Image
• Y = 0.299R + 0.587G + 0.114B (Luminance or Brightness)
U = B - Y (Chrominance 1, color difference)
V = R - Y (Chrominance 2, color difference)
• Video is sequence of images (frames) displayed at constant frame rate
– e.g. 24 images/sec
 Digital Representation of Media
There are established ways of representing images, video, animation, sound and text in
bits.
Media data may be represented as a textual description in a suitable language, or as
binary data with specific structural form.
Images come in many different format :
- Photographs - Paintings - Drawings
- Symbols - Corporate Logos - Flags
- Maps - Diagrams - Graphs
Images can be used to :
- Provide facts. - Explain a process.
- Pinpoint locations. - Tell stories.
- Compare. - Identify.
8
 File Format - Multimedia
• For Multimedia Databases –to allow searching for pictures using characteristics
such as
– colour
– texture
– shapes of objects
• to allow searching for audio (or say a song) characteristics of personal features
usingcertain specification of a program
• There are many applications that includes digital libraries (image catalog, musical
dictionary), multimedia directory services (yellow pages), broadcast media selection
(radio and TV channel)
• and multimedia edition (personalized electronic news services).

9
 Audio File Format -MIDI
• Musical Instrument Digital Interface (MIDI) provides a relatively simple
way for a single keyboard to control a variety of instruments
• MIDI is a simple digital protocol and was rapidly incorporated into
computer music applications.
• The MIDI is used as the foundation for Internet music playback and
other new kinds of music publishing.
• A MIDI file is a sequence of chunks. These chunk have the same
general format as the chunks used by AIFF, IFF and WAVE; each chunk
has a 4-character type, a 4-byte length code (in MSB format), and
some data
10
 Three types of MIDI files:

• Type zero files contain only one track:

– There are clearly the easiest to play, and preferred when you want to
make sure your MIDI files are easy for other people to play.
• Type one files contain multiple tracks that are played
simultaneously:
– A program to play type one files must somehow “flatten” the data into
a single event stream before playing.
• Type two file contain multiple tracks but without assuming any
timing relation among the tracks:
– Type two files are relatively uncommon
11
 MIDI Advantages
• Store a list of notes to be played is more compact than store
the entire song
– The music can be modified conveniently.
– Isolate a particular instrument in a symphony
– Great for games – as background music
– Music can be changed as the game progresses
– General MIDI specified 175 instruments

12
 Images Also known as pel is the smallest unit that can be
drawn

• Images are displayed as arrays of pixels and Pixel

represented using an internal model.

• The process of generating a pattern of pixels from the
model is called rendering.
• Images may be modeled as bitmaps graphics or vector
graphics.

13
 Image Representation
• Bitmap techniques or Bitmap Graphics
– Pixel-by-pixel representation of the color
– Wide range of colors and shades in complex images.
• Vector techniques or Vector Graphics
- Comprise mathematical representations : Stored as a mathematical description of a
collection of individual lines, curves and shapes making up the image
– e.g. line = two end points
- Displaying a vector image
– requires some computation to be performed in order to interpret the model and generate an
array of pixel to be displayed
– The process of interpreting the vector description known as rasterizing
– Scalable
– Small file size
14
 …Continued
• 8-bit GIF (Graphics Interchange Format) : one of the most
important formats because of its historical connection to the WWW and
HTML markup language as the first image type recognized by net
browsers.

• JPEG (Joint Photographic Experts Group): currently the most

important common file format.

15
 Media Data Types
Table 1 : Macromedia Director file formats
File import File export Native

Image Palette Sound Video Animation Image Video

.BMP , .DIB .PAL, .ACT .AIFF, .AVI, .DIR, .FLA .BMP .AVI , .DIR,
.GIF , .JPG, .PICT .AU, .MOV .FLC, .FLI .MOV .DXR,
, .PNG .MP3, .GIF, .PPT .EXE
.PNT , .PSD .WAV
.TGA , .TIFF
.WMF

16
 Resolution

• Resolution
– Image resolution is the number of pixels in a digital image.
– Device resolution is also a measure of how finely a device
approximates continuous images using finite pixels
– Higher resolution always yields better quality.
– Different monitors can have different resolutions
• The concept of resolution is simple, but it differ according to
where it is used.

17
 Image Resolution

• Bitmapped images is an array of pixel values, so it has pixel

dimensions (width & height).
• Unlike a printed photograph or original artwork on paper, a
bitmapped image has no physical dimensions
• The physical size of an image when it is displayed will depend on the
pixel density (dpi or ppi) of the device is to be displayed on.
• The equation to find the physical dimension of an image is:

18
 Example
For an image that has a logical pixel dimensions of 198 pixel wide and
149 high and to be displayed on a screen with resolution of 72 dpi,
What will be the image size on the screen?

19
 Which type of image to use?
• You have to take in your consideration the following factors to decide which type to
use:
1. Memory requirements
2. Visual characteristics
3. Possibilities for transformation & effects
4. The most important factor is the source of the image
• Photographs from digital camera, screen shots, scanned images and captured video
frames are all inherently bitmaps- because of the way the hardware from which they
originate works.
• Charts, diagrams and other data visualizations generated by a program from data
usually, but not invariably, use vector graphics
20
 Multimedia Presentation

• Bitmap: The two-dimensional array of pixel values that represents the

graphics/image data.
• Image resolution refers to the number of pixels in a digital image
(higher resolution always yields better quality).
– Fairly high resolution for such an image might be 1,600 x 1,200, whereas lower
resolution might be 640 x 480.
• Frame buffer: Hardware used to store bitmap.
– Video card (actually a graphics card) is used for this purpose.
– The resolution of the video card does not have to match the desired resolution of
the image, but if not enough video card memory is available then the data has to
be shifted around in RAM for display.
21
• Image file size = M x N x B
where: M = width
N = height
B = number of bits per pixel (bit depth)

• Note: Image file size is normally expressed in bytes or kbytes not

bits.

22
 Bit depth
• Bit depth quantifies how many unique colors are available in an
image's color palette in terms of the number of 0's and 1's, or "bits,"
which are used to specify each color.
• This does not mean that the image necessarily uses all of these colors,
but that it can instead specify colors with that level of precision.

23
 Bit Images

• Each pixel is stored as a single bit (0 or 1), so also referred to as binary image.
• Such an image is also called a 1-bit monochrome image since it contains no color.
• Fig. 1 shows a 1-bit monochrome image (called “Lena” by multimedia scientists —
this is a standard image used to illustrate many algorithms).

Fig. 1: Monochrome 1-bit Lena image.

24
 1-Bit Images Calculation

• What is the image file storage size for a 640 x 480 monochrome
image?
• Solution:
image size = width(M) × height(N) × bit depth(B)
image size = 640 × 480 × 1 Divide by 8 because
there are 8 bits in a byte
8
= 38 400 bytes Divide by 1024 because
there are 1024 bytes in
1024 1KB

= 37.5 kB
25
 8-Bit Gray-Level Images

• Each pixel takes 1 byte (8 bits) of storage resulting in 256

different states.
• Each pixel has a gray-value between 0 and 255.
• If these states are mapped onto a ramp of greys from black to
white the bitmap is referred to as a greyscale image.
• By convention 0 is normally black and 255 white.
• The grey levels are the numbers in between, for example, in a
linear scale 127 would be a 50% grey level.
26
• Fig. 2 shows the Lena image again, but this time in grayscale.

Fig.2 : Grayscale image of Lena

• What is the image file storage size for a 640 x 480 8-bit grayscale image?
• Solution:
image size = width(M) × height(N) × bit depth(B)
image size = 640 × 480 × 1
= 307,200 bytes
1024
= 300 kB
27
 Comparison

Monochrome 1-bit Lena image. Grayscale image of Lena.

38 400 Bytes 307 200 Bytes

28
 Color Image Data Types
• The most common data types for graphics and image file formats - 24-bit
color and 8-bit color.
• Some formats are restricted to particular hardware/operating system
platforms, while others are “cross-platform” formats.
• Even if some formats are not cross-platform, there are conversion
applications that will recognize and translate formats from one system to
another.
• Most image formats incorporate some variation of a compression
technique due to the large storage size of image files. Compression
techniques can be classified into either lossless or lossy.
29
 24-Bit Color Images
• In a color 24-bit image, each pixel is represented by three bytes, usually representing RGB.
– This format supports 256 x 256 x 256 possible combined colors, or a total of 16,777,216 possible
colors.
– However such flexibility does result in a storage penalty: A 640 x 480 24-bit color image would
require 900 kB of storage without any compression.

• An important point: many 24-bit color images are actually stored as 32-bit images, with the extra byte of
data for each pixel used to store an alpha value representing special effect information (e.g.,
transparency).

• The figure on the next slide shows the image forestfire.bmp, a 24-bit image in Microsoft Windows BMP
format. Also shown are the grayscale images for just the Red, Green, and Blue channels, for this image.

30
24 bit image “forestfire.bmp” From R channel

From G channel From B channel

31
 24-bit Color Images Calculation
• What is the image file storage size for a 640 x 480 24-bit color
image?
• Solution:
image size = width(M) × height(N) × bit depth(B)
image size = 640 × 480 × 3
= 921 600bytes there are 3 bytes
in 24-bit
1024
= 900 kB
32
 Color Model
• A system for representing colors is called a colors model.
• Color models are usually designed to take advantage of a particular type of display device
• The range of colors that can be represented by a color model is known as a colorspace.
• The RGB color model is used in computer system because of the CRT color monitor.
• 8-bit resolution per pixel is used which corresponding to 256 different shades of each
primary color.
• In telecommunication engineering, YUV(YCBCR) is used.
• Y is luminance which represent the brightness of the image and CBCR are chrominance
components.
•

33
 Assignment (5 marks)
Explain and discuss the following Colours and
Colour Systems? Give examples for each.
1. The CMY Colour model
2. RGB Colour model
3. HSV Colour model
4. HSB Colour model
5. YUV Colour model
 Text

Text is a Vital Element of Multimedia Presentations

Multimedia Information Systems 36

 …

• The study of how to display text is known as

typography. It concerns the precise shape of characters,
their spacing, the layout of the lines and paragraphs, etc
 Typefaces and fonts
• To display text, we need to have a visual representation of the characters
stored as codes in the computer.
• A typeface is a family of graphic characters with a coherent design and
usually includes many sizes and styles.
• A font is a set of graphic characters with a specific design in a specific
size and style.
• For example, the typeface used in this paragraph is ‘Arial’. The font is
‘Arial 28pt’. Arial may contain many fonts such as Arial Black, Arial
narrow etc
 Example fonts and sizes
• Arial 32: Multimedia systems
• Times New Roman 22: Multimedia systems
• Albertus extra bold 24: Multimedia systems
• Algerian 28: Multimedia systems
• Abyssinica 30: Multimedia systems
• Apple chancery 30: Multimedia systems
• Bauhaus 93 22: Multimedia systems

• Vladmir script 40: Multimedia systems

 Font measurement
• When putting characters on to a page, we need to
know some basic measurement of the types we
use.
• Each character has a bounding box. This is the
rectangle enclosing the entire character.
• Each character has an origin. It is usually place
on the baseline.
descender
• The width of the character determine where the
origin of the next character will be.
• The distance between the origin and the left side
of the bounding box is called left side bearing.
 Bitmap and outline fonts
Font formats can be divided into two main categories: bitmap fonts and
outline fonts.
• Bitmap fonts come in specific sizes and resolutions. Because the font
contain the bitmaps of the character shapes. The result will be very poor
if they are scaled to different sizes.

• Outline fonts contain the outline/drawing of the characters. They can be

scaled to a large range of different sizes and still have reasonable look.
They need a rasterizing process to display on screen.
 Bitmap font

W w
Outline font
 Font attributes
Five attributes are often used for specifying a font:
• Family — fonts in the same family have similar design, look and feel. Here are some of the common
families:
– Times, Helvetica, Courier, Garamond, Univers
• Shape — refers to the different appearance within a family.
– normal (upright), italic, SMALL CAP
• Weight — measures the darkness of the characters, or the thickness of the strokes. The commonly
used names are:
– ultra light, extra light, light, semi light, medium, semi bold, bold, extra bold, etc.
• Width — the amount of expansion or contraction with respect to the normal or medium in the family.
• Size — unit of measure is point.
– 1 inch = 72.27 point in printing industry.
– 1 inch = 72 point in PostScript systems.
–

tiny small normal large larger even larger huge

 Text compression
• What is compression?
• Lossless vs Lossy?
• How does text compression work?
 Video
• Video is a sequence of images viewed at a certain
fixed rate (attached to audio) to give us a sense of
motion.
• Both video and animation give a sense of motion
• It is viewed as continuous motion due to our
ability of interpreting images
 Frame rate
System Frame rate Remark

PAL(Phase Alternating Line) 25 fps A picture consists of 625 lines and

frame rate is 25Hz

SECAM(SEquential Couleur Avec 25 fps A picture consists of 625 lines and

Memoire) frame rate is 25Hz

NTSC(National Television Systems 30 fps A picture consists of 525 lines and

Committee) frame rate is
approximately 30Hz
HDTV 1080x720
up to
1920x1080 pixels.
 Interlacing vs Progressive scan
• Interlaced scan refers to a methods for "painting" a video image on an
electronic display screen by scanning or displaying each line of pixels.

• This technique uses two fields to create a frame. One field contains all the
odd lines in the image, the other contains all the even lines of the image.

• A PAL based television display, for example, scans 50 fields every second
(25 odd and 25 even). The two sets of 25 fields work together to create a
full frame every 1/25th of a second, resulting in a display of 25 frames per
second.
• Such scan of every second line is called interlacing.

• With progressive scan, an image is captured, transmitted and

displayed in a path similar to text on a page: line by line, from
top to bottom
 Video formats
• AVI (Audio Video Interleaved) format was defined by Microsoft for its Video for
Windows systems
– It supports video playback at up to 30 frames per second on a small window (typical size
300X200 with 8 or 16 bit color)
– It is a software-only system
– It supports a number of compression algorithms

• QuickTime was originally developed by Apple for storing audio and video in
Macintosh systems
– It supports video playback at up to 30 frames per second on a small window (typical size
300X200 with 8 or 16 bit color)
– It is a software-only system
– It supports a number of compression algorithms
Multimedia Information Systems 51
 …
• MPEG (Motion Picture Expect Group) is a working group under ISO
– There are several versions of mpeg standard. The most commonly used now
is mpeg-1
– It requires hardware support for encoding and decoding (on slow systems)
– The maximum data rate is 1.5Megabit/sec
– The next generation mpeg-2 is now getting popular
– Mpeg-2 improves mpeg-1 by increasing the maximum data rate to
15Mbit/sec
– It can interleave audio and video

Multimedia Information Systems 52

 Animation
• To animate something is, literally, to bring it to life
• An animation covers all changes that have a visual effect
• Visual effect can be of two major kinds:
– motion dynamic — time varying positions
– update dynamic — time varying shape, colour, texture, or even lighting, camera
position, etc.
• The visual effects is the result of exploiting the properties of human
vision system as described above (in the section about video)
• A computer animation is an animation performed by a computer using
graphical tools to provide visual effects
Multimedia Information Systems 53
 Input process
• The first step in producing computer animation is input process
• Key frames have to be created and input into the computer
• Key frames are the frames in which the objects being animated are at
extreme or characteristic positions
• They can be drawn using traditional artistic tools, such as pen and brush,
and then digitized
– The digital images may need to be cleaned up
• They can also be created using drawing or painting tools directly
• In composition stage, the foreground and background figures are combined
to generate the individual frames
Multimedia Information Systems 54
 Inbetween process
• The animation of movement from one position to another
needs a composition of frames with intermediate positions in
between the key frames

• The process of inbetweening is performed in computer

animation through interpolation
– The system is given the starting and ending positions
– It calculates the positions in between

Multimedia Information Systems 55

 …

Multimedia Information Systems 56

 …
• The easiest interpolation is linear interpolation
– It has many limitations: the object does no move smoothly, look unreal
• Spline interpolation can make object move more smoothly
• Inbetweening also involves interpolating the shapes of objects
• Some animation involves changing the colour of objects
– This is usually done using colour look-up table(CLUT)
– By cycling through the colours in the CLUT, the objects’ colours will change
• Morphing is a popular effect in which one image transforms into
another
Multimedia Information Systems 57
 Controlling animation
• Full explicit control — the animator provides a description of everything that occurs
in the animation
– either by specifying simple changes, such as scaling, transformation
– or by providing key frames
• Procedural control — using a program to calculate the position, angle, etc. of the
objects
– In physical systems, the position of one object may influence the motion of another
• Constraint-based systems — movement of objects that are in contact with each other is
constraint by physical laws An animation can be specified by these constraints
• Tracking live action —
– People or animals act out the parts of the characters in the animation
– The animator trace out the characters
Multimedia Information Systems 58
 Displaying animation
• The rules governing the showing of video apply to animation as well
• The frame rate should be at least 10, preferably 15 to 20, to give a reasonably
smooth effect
• There are basically three common ways to display animation
– Generate a digital video clip
Many Animation tools will export an animation in common digital video format,
e.g., QuickTime
– Create a package including runtime system of the animation tool
For example, Director can create a projector including all casts.
The projector can then be distributed and play the animation.
• Show the animation in the animation tool
Multimedia Information Systems 59
 Animation tools
• Macromedia Director and Flash
It is one of the most popular interactive animation tool for generating interactive
multimedia applications
• MetaCreations Poser
It understands human motion and inverse kinematics, e.g., move an arm the
shoulders will follow.
• Discreet 3D Studio Max
Very popular for creating 3D animations
• Animation language - VRML (Virtual Reality Modelling Language)

Multimedia Information Systems 60