k

Multimedia information

r
representation

a
2.1 Introduction
• All types of multimedia information are stored and
processed within a computer in a digital form.

h
• They can be integrated together and transmitted over a
single all-digital communications network.

S
2.2 Digitization principles
• A signal whose amplitude varies continuously with time
is known as an analog signal.

• Techniques involved in analog-to-digital conversion

include sampling and quantization.

• The range of frequencies of the sinusoidal components

that make up a signal is called the signal bandwidth.

• Any signal transmitted over a channel must have a signal

bandwidth less than the channel bandwidth.

CYH/MMT/Rep/p.1 CYH/MMT/Rep/p.2
 k
• Nyquist sampling theorem: The amplitude of a signal

r
must be sampled at a minimum rate that is equal to or
greater than twice the bandwidth of the signal.

h a
S • Quantization is the process that confines the amplitude
of a signal into a finite number of values.

• The difference between the actual signal amplitude and

the corresponding nominal amplitude is called the
quantization error.

• The ratio of the peak amplitude of a signal to its

minimum amplitude is known as the dynamic range.
D = 20 log10 (Vmax / Vmin ) in dB
CYH/MMT/Rep/p.3 CYH/MMT/Rep/p.4
 k
• Quantization interval : q = (Vmax / 2 n −1 )
where n is the number of bits used and Vmax is the

r
maximum positive and negative signal amplitude.

h a
S
CYH/MMT/Rep/p.5 CYH/MMT/Rep/p.6
 k
2.3 Text

r
• There are 3 types of text that are used to produce pages
of documents:

a
• Unformatted text:(=plaintext)
• comprise strings of fixed-size characters from a
limited character set such as ASCII code set.

h
• e.g. .txt file

• Hypertext:

S
• integrated set of documents (each comprising
formatted text) to be created which have defined
linkages (hyperlinks) between them.
• e.g. .htm file

• Formatted text: (=richtext)

• comprise of strings of characters of different styles,
size, and shape with tables, graphics, and images.
• e.g. word .doc file

CYH/MMT/Rep/p.7 CYH/MMT/Rep/p.8
 k
2.4 Images

r
• Images include computer-generated images (referred to
as computer graphics) and digitized images of
documents and pictures.

a
• All images are displayed in the form of a two-
dimensional matrix of individual picture elements called

h
pixels.

2.4.1 Graphics

S
• Graphics are composed of all kinds of visual objects
such as lines, arcs, squares, circles and so on, as well as
any form of hand-drawn objects.

• Each object has a number of associated attributes such as

color, shape, size, shadow and so on.
• There are 2 forms of representation of a computer
graphic:
• The representation of a graphic is analogous to the
structure of a program written in a high-level
programming language, which consists of a set of
commands that are necessary to draw the different
objects that make up the graphic.
• Another form of representation is the actual pixel-
image of the graphic (bit-map format).

CYH/MMT/Rep/p.9 CYH/MMT/Rep/p.10
 k
2.4.2 Digitized pictures • Raster-scan principles:

r
• A whole spectrum of colors can be produced by using
different proportions of the 3 primary colors red, green
and blue.

a
• There are 2 mixing techniques: additive color mixing and
subtractive color mixing.

S h • The frame refresh rate must be high enough to ensure

the eye is not aware the display is continuously being
refreshed.

• A low refresh rate leads to what is called flicker.

CYH/MMT/Rep/p.11 CYH/MMT/Rep/p.12
 k
• The number of bits per pixel is known as the pixel depth
and determines the range of different colors that can be

r
produced.

• The aspect ratio is the ratio of the number of pixels per

a
scanned line and the number of lines per frame.
(common ratios: 4:3, 16:9)

h
• Example display resolutions and memory requirements
Standard Resolution Number Memory required
of colors per frame (bytes)

S
VGA 640X480X8 256 307.2kB
XGA 640X480X16 64k 614.4kB
1024X768X8 256 786.432kB
SVGA 800X600X16 64k 960kB
1024X768X8 256 786.432kB
1024X768X24 16M 2359.296kB

CYH/MMT/Rep/p.13 CYH/MMT/Rep/p.14
 k
2.5 Audio

r
• There are 2 types of audio signal: speech and music-
quality audio.

a
• Audio can be produced either naturally by means of a
microphone or electronically using some form of

h
synthesizer.

• Output of all digitized audio signals the stream of

S
digitized values must be converted back into its analog
form as loudspeakers operate using an analog signal.

• Summary of characteristics
Speech Music
Typical bandwidth 50Hz - 10kHz 15Hz - 20kHz
Sampling rate 20kHz 40kHz
Bits per sample 12 16
No. of channels usually mono usually stereo

CYH/MMT/Rep/p.15 CYH/MMT/Rep/p.16
 k
2.5.1 PCM

r
• The bandwidth of a speech circuit in a PSTN was limited
to 200Hz through to 3.4kHz. ⇒ Sampling rate 8 kHz

a
• The digitization procedure is known as pulse code
modulation (PCM) and the international standard
relating to this is defined in ITU-T Recommendation

h
G.711.

• North America & Japan: 7 bits per sample, use µ-law

S
compression-expansion characteristics

• Europe: 8 bits per sample, use A-law compression-

expansion characteristics

CYH/MMT/Rep/p.17 CYH/MMT/Rep/p.18
 k
2.5.3 Synthesized audio
2.5.2 CD-quality audio

r
• Synthesized audio is often used in multimedia
• Sampling rates 44.1kHz applications since

a
(i) the amount of memory required can be 2 to 3 orders
• 16 bits per sample
of magnitude less than that required to store the
• 2 channels equivalent digitized waveform version,

h
(ii) it is much easier to edit synthesized audio and to mix
several passages together.

S
• The computer takes input commands from the keyboard
and outputs these to the sound generators, which, in turn,
produce the corresponding sound wave forms to drive
the speakers.

• A standardized set of messages have been defined for

both input and for output to the corresponding set of
sound generators.

• These are defined in a standard called Music Instrument

Digital Interface (MIDI).

CYH/MMT/Rep/p.19 CYH/MMT/Rep/p.20
 k
• The refresh rate can be halved with interlaced scanning.

r
• Progressive scanning: Frames are not split into fields
and each of them is transmitted as a whole.

a
• Two types of system (based on line resolution):
• NTSC: 525 lines per frame
• PAL/CCIR/SECAM: 625 lines per frame

h
• For historical reason, a different set of color signals from
R, G and B were selected for color television broadcasts.

S
• 1 luminance and 2 chrominance components are used to
describe the color of each pixel.

• Luminance components describe the brightness of pixels.

• Chronminance components carry the color information

of pixels.

2.6 Video • PAL system:

• The quality of the video required varies considerably Y = 0.299R + 0.587 G + 0.114 B
from one type of application to another. U = 0.493 (B-Y)
V = 0.877 (R-Y)
2.6.1 Broadcast television:
• NTSC system:
• Interlaced scanning: Each frame is divided into 2 fields.
Y = 0.299R + 0.587 G + 0.114 B
One comprises the odd scan lines and one comprises the
I = 0.74 (R-Y) – 0.27 (B-Y)
even scan lines. They are transmitted one by one and
Q = 0.48 (R-Y) + 0.41 (B-Y)
integrated together to form a complete frame in the
television receiver.
CYH/MMT/Rep/p.21 CYH/MMT/Rep/p.22
 k
2.6.2 Digital video • 4:2:2 format
• original digitization format used in Recommendation

r
• These are intended for use with standard television
receivers. CCIR-601 for use in television studios.

• The 3 component signals are usually digitized separately • Line sampling rate: 13.5MHz for luminance and

a
prior to their transmission. 6.75MHz for chrominance signals.

• The resolution of the eyes is less sensitive for color than • The number of bits per sample is 8 for all 3 signals.

h
it is for luminance.
• The International Telecommunications Union -

S
Radiocommunications Branch (ITU-R) - formerly known
as Consultative Committee for International
Radiocommunications (CCIR) - defined a standard for
the digitization of video pictures known as
Recommunication CCIR-601.
• Variants of this standard have been defined for use in
other application domains such as digital television
broadcasting, digital telephony and videoconferencing.
• Collectively these are known as digitization formats.

CYH/MMT/Rep/p.23 CYH/MMT/Rep/p.24
 k
• 4:2:0 format • SIF (Source intermediate format)
• used in digital video broadcast applications. • Give a picture quality comparable with that obtained

r
with VCRs
• Interlaced scanning is used

a
• Absence of chrominance samples in alternative lines is
the origin of the term 4:2:0

S h • CIF: common intermediate format

• Used in videoconferencing applications
• 4:1:1 digitization format
• spatial resolution: Y=360x288, Cb=Cr=180x144
• temporal resolution 30Hz using progressive scanning
• Derivatives of CIF: 4CIF & 16CIF
CYH/MMT/Rep/p.25 CYH/MMT/Rep/p.26
 k
• QCIF: 2.6.3 PC video
• for video telephony applications.

r
• PC video digitization formats:
• Quarter of CIF Digitization System Spatial resolution Temporal
format resolution

a
4:2:0 525-line Y= 640x480 60Hz
Cb=Cr= 320x240
625-line Y= 768x576 50Hz

h
Cb=Cr= 384x288
SIF 525-line Y= 320x240 30Hz
Cb=Cr= 160x240

S
625-line Y= 384x288 25Hz
Cb=Cr= 192x144
CIF Y= 384x288 30Hz
Cb=Cr= 192x144
QCIF Y= 192x144 15/7.5Hz
Cb=Cr= 96x72

• All PC monitors use progressive (non-interlaced)

scanning rather than interlaced scanning.

CYH/MMT/Rep/p.27 CYH/MMT/Rep/p.28