Chapter 2:Multimedia

information representation
Chapter 2:Multimedia
information representation
 2.1 Introduction
 2.2 Digitization principles
 2.3 Text
 2.4 Images
 2.5 Audio
 2.6 Video
Introduction
 The conversion of an analog signal into
a digital form
 Signal encoder, sampling, signal
decoder
 2.2 Digitization principles
2.2.1 Analog signals
 Fourier analysis can be used to show that any
time-varying analog signal is made up of a
possibly infinite number of single-frequency
sinusoidal signals whose amplitude and phase
vary continuously with time relative to each other
 Signal bandwidth
 Fig2.1
 The bandwidth of the transmission channel
should be equal to or greater than the bandwidth
of the signal─bandlimiting channel
 2.2.2 Encoder design
 A bandlimiting filter and an analog-to-digital
converter(ADC), the latter comprising a sample-
and-hold and a quantizer
 Fig2.2
 Remove selected higher-frequency components
from the source signal (A)
 (B) is then fed to the sample-and-hold circuit
 Sample the amplitude of the filtered signal at
regular time intervals (C) and hold the sample
amplitude constant between samples (D)
 2.2.2 Encoder design
 Quantizer circuit which converts each
sample amplitude into a binary value
known as a codeword (E)
 The signal to be sampled at a rate which
is higher than the maximum rate of
change of the signal amplitude
 The number of different quantization
levels used to be as large as possible
2.2.2 Encoder design
 Nyquist sampling theorem states that:
in order to obtain an accurate
representation of a time-varing analog
signal, its amplitude must be sampled
at a minimum rate that is equal to or
greater than twice the highest
sinusoidal frequency component that is
present in the signal
 2.2.2 Encoder design
 Nyquist rate: samples per second (sps)
 The distortion caused by sampling a signal
at a rate lower than the Nyquist rate
 Fig2.3
 Alias signals: they replace the
corresponding original signals
Figure 2.3 Alias signal generation
due to undersampling.
 2.2.2 Encoder design
 Quantization intervals
 A finite number of digits is used, each sample can
only be represented by a corresponding number
of discrete levels
 Fig2.4
 If Vmax is the maximum positive and negative
signal amplitude and n is the number of binary
bits used, then the magnitude of each
quantization interval, q
2Vmax
q  n
2
 2.2.2 Encoder design
 Each codeword corresponds to a nominal
amplitude level which is at the center of the
corresponding quantization interval
 The difference between the actual signal
amplitude and the corresponding nominal
amplitude is called the quantization error
(Quantization noise)
 The ratio of the peak amplitude of a signal to its
minimum amplitude is known as the dynamic
range of the signal, D (decibels or dB)
 2.2.2 Encoder design
D  20 log   dB
Vmax

10  Vmin 

 It is necessary to ensure that the level of
quantization noise relative to the smallest
signal amplitude is acceptable
 Example 2.2
 2.2.3 Decoder design
 Fig2.5
 Reproduce the original signal, the output
of the DAC is passed through a low-pass
filter which only passes those frequency
components that made up the original
filtered signal (C)
 Audio/video encoder-decoder or
audio/video codec
2.3 Text
 Three types of text
 Unformatted text
 Formatted text
 hypertext
 2.3.1 Unformatted text
 American Standard Code for Information
Interchange (ASCII character set)
 Fig2.6
 Mosaic characters create relatively simple
graphical images
 2.3.2 Formatted text
 Produced by most word processing
packages
 Each with different headings and with
tables, graphics, and pictures inserted at
appropriate points
 Fig2.8
 WYSIWYG: an acronym for what-you-see-
is-what-you-get
Figure 2.8 Formatted text: (a) an example formatted
text string;(b) printed version of the string.
 2.3.3 Hypertext
 Formatted text that enables a related set
of documents─normally referred to as
pages─to be created which have defined
linkage points─referred to as hyperlinks
─between each other
 Fig2.9
 2.4 Images
 Image are displayed in the form of a two-
dimensional matrix of individual picture
elements─known as pixels or pels
 2.4.1 Graphics
 Fig 2.10
 Two forms of representation of a computer
graphic: a high-level version (similar to the source
code of a high-level program) and the actual
pixel-image of the graphic (similar to the byte-
string corresponding to the low-level machine
code─bit-map format)
 Standardized forms of representation such as GIF
(graphical interchange format) and TIFF (tagged
image file format)
2.4.2 Digitized documents
 Fig 2.11
 A single binary digit to represent each
pel, a 0 for a white pel and a 1 for a
black pel
 2.4.3 Digitized pictures
 Color principles
 A whole spectrum of colors─known as a color
gamut ─can be produced by using different
proportions of red(R), green(G), and blue (B)
 Fig 2.12
 Additive color mixing producing a color image on
a black surface
 Subtractive color mixing for producing a color
image on a white surface
 Fig 2.13
 2.4.3 Digitized pictures
 Raster-scan principles
 Progressive scanning
 Each complete set of horizontal scan is
called a frame
 The number of bits per pixel is known as
the pixel depth and determines the range
of different colors
 2.4.3 Digitized pictures
 Aspect ratio
 Both the number of pixels per scanned line and
the number of lines per frame
 The ratio of the screen width to the screen height
 National Television Standards Committee (NTSC),
PAL(UK), CCIR(Germany), SECAM (France)
 Table 2.1
2.4.3 Digitized pictures
 2.4.3 Digitized pictures
 Digital cameras and scanners
 An image is captured within the
camera/scanner using an image sensor
 A two-dimensional grid of light-sensitive
cells called photosites
 A widely-used image sensor is a charge-
coupled device (CCD)
 Fig 2.16
 2.5 Audio
 The bandwidth of a typical speech signal is
from 50Hz through to 10kHz; music signal
from 15 Hz through to 20kHz
 The sampling rate: 20ksps (2*10kHz) for
speech and 40ksps (2*20kHz) for music
 Music stereophonic (stereo) results in a bit
rate double that of a monaural(mono)
signal
 Example 2.4
 2.5.2 CD-quality audio
 Bit rate per channel
=sampling rate*bits per sample
3
 44.110  16  705.6kbps

 Total bit rate = 2*705.6=1.411Mbps

 Example 2.5
 2.6 Video
2.6.1 Broadcast television
 Scanning sequence
 It is necessary to use a minimum refresh
rate of 50 times per second to avoid flicker
 A refresh rate of 25 times per second is
sufficient
 Field:the first comprising only the odd scan
lines and the second the even scan lines
 2.6.1 Broadcast television
 The two field are then integrated together
in the television receiver using a technique
known as interlaced scanning
 Fig 2.19
 The three main properties of a color source
 Brightness
 Hue:this represents the actual color of the
source
 Saturation:this represents the strength or
vividness of the color
 2.6.1 Broadcast television
 The term luminance is used to refer to the
brightness of a source
 The hue and saturation are referred to as
its chrominance
Ys  0.299 Rs  0.587Gs  0.144 Bs
 Where Ys is the amplitude of the luminance
signal and Rs,Gs and Bs are the magnitudes
of the three color component signals
 2.6.1 Broadcast television
 The blue chrominance (Cb), and the red
chrominance (Cr) are then used to
represent hue and saturation
 The two color difference signals:
Cb  Bs  Ys Cr  Rs  Ys
2.6.1 Broadcast television
 In the PAL system, Cb and Cr are referred to
as U and V respectively
PAL : Y  0.299 R  0.587G  0.114 B
U  0.493( B  Y )
V  0.877( R  Y )
 The NTSC system form two different signals
referred to as I and Q
NTSC : Y  0.299 R  0.587G  0.114 B
I  0.74( R  Y )  0.27( B  Y )
Q  0.48( R  Y )  0.41( B  Y )
 2.6.2 Digital video
 Eye have shown that the resolution of the eye is
less sensitive for color than it is for luminance
 4 ： 2 ： 2 format
 The original digitization format used in
Recommendation CCIR-601
 A line sampling rate of 13.5MHz for luminance
and 6.75MHz for the two chrominance signals
 The number of samples per line is increased to
720
 2.6.2 Digital video
 The corresponding number of samples for each
of the two chrominance signals is 360 samples
per active line
 This results in 4Y samples for every 2Cb, and 2Cr
samples
 The numbers 480 and 576 being the number of
active (visible) lines in the respective system
 Fig. 2.21
 Example 2.7
Figure 2.21 Sample positions with
4:2:2 digitization format.
 2.6.2 Digital video
 4 ： 2 ： 0 format is used in digital video
broadcast applications
 Interlaced scanning is used and the
absence of chrominance samples in
alternative lines
 The same luminance resolution but half the
chrominance resolution
 Fig2.22
Figure 2.22 Sample positions in
4:2:0 digitization format.
 2.6.2 Digital video
525-line system Y  720  480
Cb  Cr  360  240

625-line system Y  720  480

Cb  Cr  360  288

13.5 10  8  2 3.375 10  8  162Mbps

6 6
 2.6.2 Digital video
 HDTV formats: the resolution to the newer
16/9 wide-screen tubes can be up to
1920*1152 pixels
 The source intermediate format (SIF) give a
picture quality comparable with video
recorders(VCRs)
2.6.2 Digital video
 The common intermediate format (CIF) for
use in videoconferencing applications
 Fig 2.23
 The quarter CIF (QCIF) for use in video
telephony applications
 Fig 2.24
 Table 2.2
Figure 2.23 Sample positions for
SIF and CIF.
Figure 2.24 Sample positions for QCIF.
2.6.3 PC video