6/21/2020

CS-473: MULTIMEDIA APPLICATION

DESIGN
UNDERSTANDING MULTIMEDIA CONTENTS
(TEXT, AUDIO, IMAGE, GRAPHIC, VIDEO)

REPRESENTATION, ACQUISITION, MANIPULATION, AND

AUTHORING

UMER RASHID

Department of Computer Science

Quaid-i-Azam University, Paksitan

Introduction
2

 widely
 flexible means of presenting information on screen
 conveying ideas
 designer
 should not necessarily try to replace textual elements with others
 should consider how to present text in an acceptable way
 supplementing text with other media types
 Example
 Public System
 Tourist information
 Educational Websites
 E-Governance

1
 6/21/2020

Introduction
3

 Text
 a vital element of multimedia presentations
 visual representation of language
 Words and symbols  spoken or written
 most common system of communication
 deliver the most widely understood meanings
 very important to choose the suitable words and symbols
take the time to choose the right words
 Typography
 The study of how to display text
 Typography concerns
 precise shape of characters
 spacing between characters
 layout of the lines and paragraphs

Character Sets
4

 Visual appearance
 piece of text  many forms
 basic meaning of the text  not change
 Fundamentally  piece of text consists
 Letters
 Digits
 punctuations
 other symbols, abstract characters
 Abstract characters particular language
 Grouped  alphabets
 Upper and Lower case letters
 The digits and a number of punctuations

2
 6/21/2020

Character Sets
5

 Digital Text
 mapping between abstract and stored characters  mapping a
character set
 Domain of mapping
 character list  collection of abstract characters
 code values or code points  the values to be stored

Character Sets : Standardization

6

 Text is the most widely used means of communication among

computer systems

 If systems want to communicated  common language

common character

 ASCII 7-bit 127 characters 95 printable characters

 ANSI 8-bit 255 characters

 Single Language support

 Multilanguage  ?

3
 6/21/2020

Character Sets : Standardization

7

 Universal Multi-Octet Coded Character Set

 four bytes to encode a character
 organization  hierarchical
 four bytes (g; p; r; c) groups, planes, rows, columns

Character Sets : Standardization

8

 first plane (0,0,*,*)  Basic Multilingual Plane  BMP

 BMP  specify a character set
 Embrace/holds all languages of the world

4
 6/21/2020

Typefaces and fonts

9

 Display text  characters stored  visual representation  as

codes character  different glyphs
 Typeface a family of graphic characters
 coherent design and
 usually includes many sizes and styles
 E.g. Times, Arial
 Font  a set of graphic characters
 specific design in a specific size and style
 12, 14, 16
 collection of characters of a single size and style
belonging to a particular typeface family

 A ‘typeface’ is a
 family of graphic characters that usually includes many
type sizes and styles

10

10

5
 6/21/2020

Typefaces and fonts

11

 Measurements of the typefaces

 bounding box  rectangle enclosing the entire character
 Origin  usually place on the baseline
 Width determine origin of the next character
 left side bearing distance between origin and the left side of the
bounding box
 Ascender upper portion of character
 Descender lower portion of character
 X height  with out ascender and descender height

11

Typefaces and fonts

12

12

6
 6/21/2020

Typefaces and fonts

13

 Fonts  Five attributes

 Family fonts in the same family have
 coherent design, a similar look and feel
 common families Times, Helvetica, Courier
 Shape refers to the different appearance within a family
 normal (upright) , Italic, SMALL CAP
 Weight  measures the darkness of the characters, or the
thickness of the strokes
 ultra light, extra light, light, semi light, medium, semi bold, bold, extra
bold
 Width the amount of expansion or contraction with respect
to the normal or medium in the family
 Size unit is point

13

Typefaces and fonts

14

14

7
 6/21/2020

Typefaces and fonts

15
 Classification of Typeface
 Typefaces can be classified in many ways. One classification is
understood universally:
 serif and sans serif.

15

Bitmap Fonts Versus Outline Fonts

16

 Bitmap fonts
 come in specific sizes and resolutions
 font contain the bitmaps of the character shapes
 result will be very poor if they are scaled to different sizes
 Less common
 Outline fonts
 contain the outline of the characters
 can be scaled to a large range of different sizes and still
have reasonable look
 need a rasterizing process to display on screen
 More common

16

8
 6/21/2020

Measurements for Text Layout

17

 Leading
 the distance between the baselines of two adjacent lines
 Common used leadings
 14 points for 12 points text
 12 points for 10 points text
 Tracking
 spacing between characters in text lines
 Loose tracking means the space between characters are wider
 Less words can be put in a line of text
 Kerning
 extra adjustment between two specific characters

17

Using Text in multimedia Applications

18

 Fonts  suggestions text applications

 For small type most legible font
 decorative  useless
 Use as few different faces
 vary the weight and the size and using italic or bold styles
 Vary the size of a font
 in proportion to the importance of the message
 text block
 adjust the leading for the most pleasing line spacing
 too tightly packed  difficult to read
 In large size headline, do proper kerning so that the spacing
feels right
 Explore the effects of different colors and of placing the text
on various backgrounds

18

9
 6/21/2020

Cross platform issues

19

 multimedia project
 Platform
 Windows platform ?
 Macintosh platform  ?
 Medium
 Projector
 Screen
 Resolution difference
Fonts have perhaps the greatest cross-platform
concerns

19

Text in Multimedia Based Application

20

 Guidelines
1. Conventional upper and lower case text
 used for the presentation since reading is faster compared to all
upper case text
2. All upper case can be used
 if a text item has to attract attention as in warnings and alarm
messages
3. The length of text lines
 should be no longer than around 60 characters to achieve optimal
reading speed
4. Display
 Only one third of a display should be filled with text
5. Proportional spacing and ragged lines
 minimizes unpleasant visual effects

20

10
 6/21/2020

Guidelines
21

6. Font Size
 Best resolution
 12 point text is the practical minimum to adopt for PC based screens
 Poor Resolution
 14 point or higher for screens of poorer resolution than a normal desktop PC
 Visual impaired
 16 point
7. Sentences
 should be short and concise and not be split over pages
8. Technical expressions
 should
 be used only where the user is familiar with them from their daily routine
 made as understandable as possible
9. abbreviations
 As less as possible
 consistent way throughout an entire multimedia application

21

22

22

11
 6/21/2020

23

23

24

24

12
 6/21/2020

25

25

26

26

13
 6/21/2020

27

27

Using Text Elements in a Multimedia Presentation

28

1. Menus for Navigation:

 A user navigates through content using a menu
 A simple menu consists of a text list of topics
2. Interactive Buttons:
 A button is a clickable object that executes a command when activated
 Users can create their own buttons from bitmaps and graphics
 The design and labeling of the buttons should be treated as an industrial art
project
3. Fields for Reading:
 Reading a hard copy is easier & faster than reading from computer screen
 A document can be printed in one of two orientations - portrait or landscape
 The taller-than-wide orientation used for printing documents is called portrait
 The wider-than-tall orientation that is normal to monitors is called landscape

28

14
 6/21/2020

Using Text Elements in a Multimedia Presentation

29

4. HTML Documents:
 HTML stands for Hypertext Markup Language
 It is the standard document format used for Web pages
 HTML documents are marked using tags
 An advanced form of HTML is DHTML
 DHTML stands for Dynamic Hypertext Markup Language.
 DHTML uses CSS
5. Symbols and Icons
 Symbols are concentrated text in the form of stand-alone graphic
constructs
 They are used to convey meaningful messages
 Icons are symbolic representations of objects and processes
 Symbols used to convey human emotions are called emoticons
pictorial representation of a facial expression

29

Using Text Elements in a Multimedia Presentation

30

30

15
 6/21/2020

Using Text Elements in a Multimedia Presentation

31

6. Visualization
 Knowledge Maps
 Concept Graphs
 Self Organizing Maps

31

Computers and Text: The font wars

32

 PostScript:
 PostScript is a method of describing an image in terms of mathematical constructs
 PostScript characters are scalable and can be drawn much faster
 The two types of PostScript fonts
 Adobe developed Adobe Type Manager for displaying PostScript fonts on both Macintosh
and Windows

32

16
 6/21/2020

Computers and Text: The font wars

33

 TrueType
 Apple and Microsoft developed the TrueType methodology.
 TrueType is a system of scalable outline fonts, and can draw characters at low resolution.

33

Font Editing and Design Tools

34

1. Macromedia Fontographer:
 a specialized graphics editor
 It is compatible with both Macintosh and Windows platform
 It can be used to develop PostScript, TrueType, and bitmapped fonts
 It can also modify existing typefaces and incorporate PostScript artwork

34

17
 6/21/2020

Font Editing and Design Tools

35

2. Creating Attractive Texts

 Applications that are used to enhance texts and images include:
 Adobe Photoshop; TypeStyler; COOL 3D; HotTEXT; TypeCaster

35

36

36

18
 6/21/2020

Hypermedia Structures
37

 Hypermedia Structures
 Links Links are connections between conceptual elements. Links are the
navigation pathways and menus
 Nodes Nodes are accessible topics, documents, messages, and content
elements. Nodes and links form the backbone of a knowledge access system
 Anchors Anchor is defined as the reference from one document to another
document, image, sound, or file on the Web. The source node linked to the
anchor is referred to as a link anchor. The destination node linked to the anchor
is referred to as a link end
 Navigating Hypermedia Structures The simplest way to navigate hypermedia
structures is via buttons. Location markers must be provided to make navigation
user-friendly

37

38

CS-473: MULTIMEDIA APPLICATION

DESIGN
UNDERSTANDING MULTIMEDIA CONTENTS
(TEXT, AUDIO, IMAGE, GRAPHIC, VIDEO)

REPRESENTATION, ACQUISITION, MANIPULATION, AND

AUTHORING

UMER RASHID

Department of Computer Science

Quaid-i-Azam University, Paksitan

38

19
 6/21/2020

The Nature of Sound

39

 Voice
 predominant method of human communication

 We are so familiarized
 speaking and listening

 Sound exists in many different forms

 purpose and characteristics

 Here are some things we can do with sound

 listen to a song on a radio
 watch a music video
 listen to a song
 speak to a friend on the phone.

39

The Nature of Sound

40

40

20
 6/21/2020

What Is Sound?
41

 key component in communication

 Imagine
 What you would experience if the television program you are
watching becomes very noisy Or,
 If the sound system stops working in the middle of a gripping film
 There are some sounds that human ear cannot perceive

 Why important
 The presence of sound greatly enhances the effect of a mostly graphic
presentation
 In a multimedia project, you can use sound in many different ways
 Sound is the brain's interpretation of electrical impulses being sent by
the inner ear through the nervous system

41

How Do We Hear?
42

 If a tree falls in the forest, and there is no one to hear it, will
there be a sound?
 Philosophical dilemma which relies on using the word sound for
two different purposes
 Cause
 One use is as a description of a particular type of physical
disturbance, so sound is an organized movement of
molecules caused by a vibrating body in some medium
 Effect
 The other use is as a description of a sensation—sound is
the auditory sensation produced through the ear by the
alteration in pressure, particle displacement, or particle
velocity which is propagated in an elastic medium

42

21
 6/21/2020

How Do We Hear?
43
Steps
1. Object moves back and forth
2. Pushes the air immediately next to it a bit to one side and,
3. Coming back, creates a slight vacuum
4. Creates a wave
5. Air particles that move in waves make the eardrum
oscillate
6. Movement is registered by a series of small bones
7. Small bones transmit these vibrations to the inner ear nerve
endings
8. This in turn, send impulses to the brain
9. Perceived as sounds

43

Use of Audio in Multimedia

44

 Can be used in a multimedia project in two ways

 Content sound
 Provides information to audiences, for example, dialogs in movies or
theater
 Ambient sound
 consists of an array of background and sound effects

44

22
 6/21/2020

Use of Audio in Multimedia

45
Content Sound
1. Narration
 Provides information about an animation that is playing on the
screen

2. Testimonials
 These could be auditory or video sound tracks used in presentations
or movies

3. Voice-overs
 These are used for short instructions, for example, to navigate the
multimedia application

4. Music
 may be used to communicate (as in a song)

45

Use of Audio in Multimedia

46
Ambient sound
1. Message reinforcement
 The background sounds you hear in real life, such as the
crowds at a football match, can be used to reinforce the
message that you wish to communicate

2. Background music
 Set the mood for the audience to receive and process
information by starting and ending a presentation with music

3. Sound effects
 used in presentations to liven-up the mood and add effects
to your presentations, such as sound attached to lists

46

23
 6/21/2020

SOME PHYSICS BACKGROUND

47

1. Sound waves are disturbances in transmission mediums

2. Wave consists of compressions and
rarefactions/reductions
3. All waves can be represented by a standard
waveform
4. depicts the compressions and rarefactions
5. Compressions valley | the rarefactions peak

47

SOME PHYSICS BACKGROUND

48

6. Waveform
 gives a measurement of the speed of the air particles and
the distance that they travel for a given sound in a given
medium

7. Amplitude
 measures the relative loudness of the sound, which is the
distance between a valley and a peak
 determines the volume of the sound

8. Decibel
 unit of measurement of volume is a decibel

48

24
 6/21/2020

1. Properties of Sound
49
Frequency
 The difference in time between the formation of two
peaks/valleys is termed as the period
 Measured in seconds
 A number of peaks may occur within a second
 The number of peaks that occur in one second is the
frequency
 Measured by the number of cycles in second , hertz
 The human ear can perceive a range of frequencies
from 20-20,000 Hz
 However, it is most sensitive to sounds in the range of 2-
4 kHz.

49

2. Properties of Sound
50
Pitch
 If an object oscillates rapidly, it creates a "high-
pitched" sound
 A low-frequency sound on the other hand is produced
by an object that vibrates slowly

50

25
 6/21/2020

3. Properties of Sound
51
Bandwidth
 defined as the difference between the highest and
the lowest frequency contained in a signal

51

4. Properties of Sound
Harmonics
52

 Few instruments produce sound of a single

frequency
 Most of the instruments produce Multiple
frequencies
 The combinations of frequencies generated by an
instrument are known as the timbre
 A timbre consists of a fundamental or main
frequency and other minor frequencies known as
overtones /harmonics

52

26
 6/21/2020

5. Properties of Sound
53
Velocity
 Measure the time required for the waves to travel
a measured distance
 The velocity varies greatly with the medium through
which it travels
 The velocity of sound increases as the temperature
rises and decreases as the temperature falls

53

Introduction to Digital Audio

54
Digital Audio
 The sound heard by the ear (also called audio) is analog in
nature and is a continuous waveform
 Acoustic instruments produce analog sounds
 A computer needs to transfer the analog sound wave into its
digital representation, consisting of discrete numbers

54

27
 6/21/2020

Introduction to Digital Audio

55
Digital Audio
 Analog to digital
 microphone converts the sound waves into electrical signals
 signal is then amplified, filtered, and sent to an analog-to-
digital converter
 information can then be retrieved and edited using a computer

55

Introduction to Digital Audio

56
Digital Audio Representation
 SAMPLING
 audio input from a source is sampled several thousand times
per second
 Each sample is a snapshot of the original signal at a
particular time

 SAMPLING RATE
 the computer processes snapshots of the waveform

 The frequency of these snapshots is called the sampling rate

 The rate can vary typically from 5000-90,000 samples per

second

56

28
 6/21/2020

Introduction to Digital Audio

57
DIGITIZATION Process
 The process of assigning a discrete value to each of the sampled
values
 It is performed by an Integrated Chip (IC) called an A to D
Converter
 In the case of 8-bit digitization, this value is between 0
and 255 (or -128 and 127)
 In 16-bit digitization, this value is between 0 and
65,535

57

Introduction to Digital Audio

58
DIGITIZATION Process
 An essential thing to remember is that a digitized signal can
take only certain (discrete) values
 The process of digitization introduces noise in a signal
 This is related to the number of bits per sample
 A higher number of bits used to store the sampled value leads
to a more accurate sample, with less noise

58

29
 6/21/2020

Introduction to Digital
59
AUDIO FIDELITY
 Defined as the closeness of the recorded version to the original
sound
 In the case of digital speech, it depends upon the number of
bits per sample and the sampling rate
 A really high-fidelity (hi-fi) recording takes up a lot of memory
space
 176.4 Kb for every second of audio of stereo quality sampled
at 16 bits, 44.1 kHz per channel

59

Introduction to Digital Audio

60
Sound Sampling
 The sampling frequency determines the limit of audio
frequencies that can be reproduced digitally
 According to Nyquist theorem, a minimum of two samples (per
cycle) is necessary to represent a given sound wave
 To represent a sound with a frequency of 440 Hz, it is
necessary to sample that sound at a minimum rate of 880
samples per second.
 Therefore, Sampling rate = 2 x Highest frequency
 If it is not sampled above this rate, a distortion known as
"aliasing" occurs and it cannot be removed by post processing
the digitized audio signal

60

30
 6/21/2020

Introduction to Digital Audio

61
SOUND FORMATS AND SETTINGS
 Recording at high sampling rates produces a more accurate
capture of the high-frequency content of the sound
 Another aspect to consider is the "bit-resolution." Along with
sampling rate
 The resolution determines the accuracy with which a sound is
digitized
 Increase in the number of bits in a recording makes the sound
playback increasingly realistic

61

Introduction to Digital Audio

62
SOUND FORMATS AND SETTINGS
 Two Methods
1. Stereo Recordings
 made by recording on two channels, and are lifelike and realistic
 require twice the space as compared to mono recordings
 File size = Sampling rate x duration of recording in seconds x (bits
per sample/8) x 2
 Sampling rates of 8,11, 22, and 44 kHz are used more often
2. Mono Recordings
 less realistic, flat, and not as dramatic,
 have a smaller file size
 File size = Sampling rate x duration of recording in seconds x (bits
per sample/8) x 1

62

31
 6/21/2020

AUDIO COMPRESSION
63

 An important aspect of communication is transfer of data from

the creator to the recipient
 Transfer of data in the Internet age is very time-dependent
 The increase in the transfer time deteriorates the quality of the
sound at the receiver's end
 The time taken to transfer a file can be decreased using
compression
 Compression in computer terms means reducing the physical
size of data such that it occupies less storage space and
memory
 Compressed files are, therefore, easier to transfer

63

COMPRESSION
64

 Reduction in the bandwidth utilization thus providing good

sound quality over slow network
 an uncompressed audio signal of telephone quality (8-bit
sampled at 8 kHz) leads to a bandwidth requirement of 64
Kbps and storage requirement of 8 KB to store one second
of playback.
 an uncompressed stereo audio signal of CD quality (16-bit
sampled at 44.1 kHz) leads to a bandwidth requirement of
44.1 kHz x 16 bits = 705.6 Kbps and storage requirement
of 88.2 KB for one second of playback

64

32
 6/21/2020

Compression Requirements
65
 In the case of audio, processing data in a multimedia system
leads to storage requirements in the range of several megabytes
 Compressions in multimedia systems are subjected to certain
constraints
 The quality of the reproduced data should be adequate for
applications
 The complexity of the technique used should be minimal, to make
a cost-effective compression technique
 The processing of the algorithm should not take too long

 Various audio data rates should be supported. Thus, depending

on specific system conditions the data rates can be adjusted
 It should be possible to generate data on one multimedia system
and reproduce data on another system
 The compression technique should be compatible with various
reproduction systems

65

Compression Requirements
66

 Standardization
 CCITT (International Consultative Committee for

Telephone and Telegraph)

 ISO (International Standard Organization), and

 MPEG (Moving Picture Experts Group) are used

to achieve this compatibility

66

33
 6/21/2020

Common Compression Methods

67

 An array of compression techniques have been set

by the CCITT Group—an international organization
that develops communication standards known as
"Recommendations" for all digitally controlled forms
of communication
 There are two types of compression:
 Loss-less
Compression
 Lossy Compression

67

Lossless Compression
68

 In lossless compression, data are not altered or lost in the process of

compression or decompression
 Decompression produces a replica of the compressed object
 This compression technique is used for text documents, databases, and text-
related objects

68

34
 6/21/2020

Lossless Compression
69

 The following are some of the commonly used lossless

standards:
 Packbits encoding (run-length encoding)

 CCITT Group 3 1-D (compression standard based on run-

length encoding scheme)
 CCITT Group 3 2-D (compression standard based on run-
length encoding scheme modified by two-dimensional
encoding)
 CCITT Group 4 (compression standards based on two-
dimensional compression)

69

Lossy Compression
70
 There is loss of some information when Lossy compression is
used
 The loss of this data is such that the object looks more or less
like the original
 This method is used where absolute data accuracy is not
essential
 Most commonly used compression type
 This compression technique is used for image documents, audio,
and video objects

70

35
 6/21/2020

Lossy Compression
71
 The following are some of the commonly used Lossy standards:
 Motion Picture Experts Group (MPEG)
 Adaptive Differential Pulse Code Modulation (ADPCM)
 CCITT H.261 (Px64) Video Coding Algorithm
 Intel DVI (Digital Video Interactive)

71

Schemes of Audio Compression

72

 The most commonly used compression schemes for audio are:

 ADPCM (Adaptive Differential Pulse Code Modulation)
 The primary source of encoding algorithms is CCITT
 It is a family of speech compression and decompression
algorithms
 40-80% compression
 Digital encoding of the signal is achieved by sampling the
audio signal at different frequencies
 Performs compression by encoding and storing only the
differences between the values of successive samples in the
data stream
 For example, normal speech consists of 75% silence, which can
be compressed

72

36
 6/21/2020

Schemes of Audio Compression

73

 The most commonly used compression schemes for audio

are:
 MPEG (Motion Picture Experts Group)

 compress the original audio on a CD by a factor of 12

without losing the sound quality
 The family of audio coding schemes that achieve
compression without losing the CD sound quality

73

MPEG (Motion Picture Experts Group)

74

 compression ratio
 MPEG Layer 1 (corresponds with 384 Kbps of data
transferred to the player in case of a stereo signal)—
compression ratio is 1:4
 MPEG Layer 2 (corresponds with 256-192 Kbps of data
transferred to the player in case of a stereo signal)—
compression ratio is 1:6-1:8.
 MPEG Layer 3 (corresponds with 128-112 Kbps of data
transferred to the player in case of a stereo signal)—
compression ratio is 1:10-1:12

74

37
 6/21/2020

Audio Editing
75

 One can record or manipulate audio files using various audio

editors
 Recording sound for multimedia applications is only the first
step in the process of sound processing
 After the audio has been recorded and stored, it has to be
modified to improve the level of quality
 Unwanted sound or silences have to be removed
 Mistakes in recording can be erased or modified
 Sounds can be mixed to get a better effect
 Cool Edit
 Sound Forge XP
 Wave Flow
 MatLab

75

Sound Effects
76
Doppler Effect
 Sound waves are compressions and rarefactions of air
 When the object making the sound is moving toward you, the
frequency goes up due to the waves getting pushed more
tightly together and pitch goes up
 The opposite happens when the object moves away from you
and the pitch goes down This is called the Doppler effect.

76

38
 6/21/2020

Special Effects
77
 Reverb Special Effect-
 gradually fades away of sound
 A person could receive sound directly from the sound source as
well as reflected from other objects around him
 Different from echo
 In the case of an echo, there is a time gap between the finish of the
original sound and the arrival of the echo
 Echo sometimes causes irritation and undesirable effects
 Reverberation is pleasing to hear and should be incorporated in
the design of any room

77

Special Effects
78

 The Fade-in Effect

 progressive increase of the volume of the waveform
making the initial volume lower than the final volume
 The Fade-Out Effect
 Exactly opposite of Fade-out Effect
 Common Editing Packages
 COOL EDIT
 SOUND FORGE XP

 WAVE STUDIO

78

39
 6/21/2020

Music and Speech

79

 Digital audio has developed in two directions

 One direction is where we take analog audio,
transform it into digital format, and do various things
with the digital file such as add effects, edit it,
compress it, play it, or use it in applications
 Other direction of digital audio—creating or
synthesizing sounds using the computer
 audio synthesis
 Namely generation of music and of speech

79

SYNTHESIZED MUSIC
80

 Synthesized sounds are a set of instructions to the

hardware audio device on how and when to produce
sound
 The MIDI (Musical Instrument Digital Interface) format
 most widely used digital format for generating synthesized
sound
 Unlike digital audio, the actual data are not recorded
 Nearly all software that supports audio can play MIDI files
 Half an hour of stereo music takes about 100 KB of disk
space in MIDI format

80

40
 6/21/2020

SYNTHESIZED MUSIC
81

 MIDI
 Keys depressed
 Time when the key was depressed
 Duration for which the key was depressed
 How hard the key was struck
 Drawbacks
 A MIDI file may sound different on different MIDI
synthesizers
 FM synthesizer simulates the different instrument
effects, may be different

81

82

82

41
 6/21/2020

SYNTHESIZED MUSIC
83

 When to Use Sampled Sound?

 narration, testimonials, voice-overs, music, and/or sounds
are required
 They can be used to provide information about animation
on the screen, such as a manufacturing process or the
function of a device
 Narrations are invaluable for self learning trainings
because these take the place of the instructor
 These can be used as voice-overs where a short instruction
is provided in a multimedia application
 to explain what lies ahead in the presentation
 Sampled sound is more realistic

83

SYNTHESIZED MUSIC
84

 When to use Synthesized Sound

 can be used to create a soothing atmosphere during a
presentation or in a multimedia application
 Since these files occupy little space

 they can be used to play music for a long duration

 unrealistic or machine-like because the quality of the

sound depends on the rendering capability of the
hardware device on the computer

84

42
 6/21/2020

85

85

86

CS-473: MULTIMEDIA APPLICATION

DESIGN
(TEXT, AUDIO, IMAGE, GRAPHIC, VIDEO)

REPRESENTATION, ACQUISITION, MANIPULATION, AND

AUTHORING

UMER RASHID

Department of Computer Science

Quaid-i-Azam University, Paksitan

86

43
 6/21/2020

Understanding Multimedia Artifacts: Images

& Graphics
87

 Object  Spatial Representation

1. Two-dimensional or three-dimensional scene
2. Reflect the intensity of lights
 Images
1. stored and processed by computers
2. displayed on computer screens called digital
3. although  look like  continuous-tone
4. represented matrix of numeric values
5. Matrix entries
 Intensities
 Color
 Texture, etc.

87

Images: Basic Concept

88

 Pixel
 smallest element on a digital image/picture element
 Digital image
 matrix of pixels

88

44
 6/21/2020

Images: Basic Concept

89

 Depth
1. number of bits used to represent each pixel
2. 1-bit black-and-white or bit map image
3. 4-bit  16 colors used in low resolution screens (EGA/VGA)
4. 8-bit  256 colors 256 indexed colors  each indexed color mapped in
RGB intensity ranges

89

Depth
90

 8-bit grey
1. 256 grey-levels 
brightness/intensity
2. Without color information
 16-bit
1. 65536 colors
2. hi-color in Windows systems
3. Red  5 bits32 colors
4. Green  6 bits 64 colors
5. Blue5 bits32 colors

90

45
 6/21/2020

Depth
91

 24-bit
 16,777,216 colors
 Each byte  intensity of a primary color
 RED, GREEN and BLUE 0 ..255  256 levels
 32-bit
 4,294,967,296 (4G)
 3 bytes  three primary color
 fourth byte  Alpha channel  a portion of each pixel's data that is reserved
for transparency information
 Object level and pixel level transparency

91

Images: Basic Concepts

92

 Resolution  number of pixels in an image

 320X240 = 76800 pixels, 700X 400 = 280000 pixels
 Display (Monitor) resolution
 number of dots per inch (dpi) on a monitor
 Output resolution
 number of dots per inch (dpi) on a monitor
 Acquiring Digital Images
 paint program depth, resolution and size
 Grabbing depth, resolution and size
 Capturing  digital camera or a camcorder
 Scanning  a photograph or a print using a scanner
 Converting from existing digital media
 Synthesizing an image from numerical data

92

46
 6/21/2020

Images: Basic Concepts

93

 Vector Graphics
 objects represented by  attributes
 size, color, location,
 Vector graphics, or vector drawing
 abstract representation  2-D or 3-D scene
 Using graphics primitives
 rectangles, circles, lines
 languages
 PostScript  Adobe  page description language
 VRML  Virtual Reality Markup Language  description of a scene in a virtual
world
 SVG  Scalable Vector Graphic  describing two-dimensional graphics in XML

93

Bitmap versus Vector

94

 description
1. exact pixel-by-pixel value of an image
2. Mathematical model of objects
 resolution
1. fixed
2. independent
 file size
1. completely determined  image resolution, and depth
2. depends on the number of graphic elements in a file
 rendering
1. Easy
2. Complex

94

47
 6/21/2020

Color System
95

 Vital component of multimedia

 physical property of light
 perception is a physiological activity
 right colors  gives aesthetic judgments
 frequency/wave-length human eye responds
 light wave within the narrow band of the electromagnetic spectrum (380 – 760
mill microns)
 Light Properties
 Hue Wavelength
 Intensity  Brightness
 Spectral Purity  Saturation

95

Color System
96  How we describe color ?
 Color can only be described in terms of a person's report of his or her
perceptions (Wilbert O. Galitz, A.D(1997).the Essential guide to user interface
Design)
 Eye visual spectrum wavelength range(400-700) milli-microns
 The dominant wavelength being “seen” is the one that we come to associate with
a specific color name (Wilbert O. Galitz, A.D(1997).the Essential guide to user
interface Design

96

48
 6/21/2020

Color System
97

Approximate wavelength
colors In mill microns
Red 700
Orange 600
Yellow 570
Yellow-Green 535
Green 500
Blue-Green 493
Blue 470
Violet 400

97

Color System
98

 COLOR MODELS
 RGB Color Model
1. Use three primary colors (Red ,Green ,Blue)
2. Secondary Colors (cyan, magenta, yellow) by mixing RGB with Black
3. creates an accurate representation of the secondary colors by addition
4. produced on a computer or television screen
 CMY Color Model
1. Three primary colors (Cyan , Yellow, Magenta)
2. Secondary Colors (Red ,Blue ,Green ) by subtracting primary colors from
white
3. Printers produce color by CYM

98

49
 6/21/2020

99

RED
BLUE

GREEN

99

GREEN+RED=
100 GREEN(0,1,0) Black (0,0,0) YELLOW(1,1,0)

GREEN+BLUE=
CYAN(0,1,1) RED(1,0,0)

BLUE+RED=
GREEN+BLUE+RED= MAGENTA(1,0,1)
BLUE(0,0,1)
WHITE(1,1,1)

100

50
 6/21/2020

101

White (1,1,1)
White-yellow
magenta(0,1,0) blue(1,1,0)

White-cyan=
red(0,1,1) cyan(1,0,0)

White-magenta=
WHITE(0, 0 ,0) green(1,0,1)
yellow(0,0,1)

101

Color System
102

 HSB Color Model

 human perception of color
 Hue  is the wavelength of the light
1. often identified by the name of the color
2. measured as a location on the standard color
 Saturation
1. strength or purity of the color
2. represents the amount of gray in proportion to the hue and is measured
as a percentage from 0%(gray) to 100%(fully saturated)
 Brightness
1. relative lightness or darkness of the color
2. measured as a percentage from 0%(black) to 100%(white)

102

51
 6/21/2020

103
It is the purity of the color, measured
in percent from the center of the
cone (0) to the surface (100)

is measured in
It is the actual percent from
color. It is black (0) to
measured in white (100)
angular degrees
around the cone
starting and
ending at red = 0
or 360 both hue and
saturation are
meaningless.

hue is
meaningless

103

Color System
104

 YUV Color Model

 widely used in encoding color for use in television and video
 suitable coding distinguishes between luminance and chrominance
1. Luminance amount of light that passes through / emitted from a
particular area
2. Chrominance  convey the color information of the picture
 Y-signal  encodes the brightness information
 Black-and-white television system will use this channel only
 U and V channels encode the chromatic information

104

52
 6/21/2020

Color System
105

 Gamut
 range of colors that can be
A. Viewed
B. Displayed
C. printed

105

Color System
106
 Color Palette
 an index table to available colors in an indexed color system
 8-bit mode a system can display only 256 colors out of a total of 16 million
colors
 system keeps a default palette of available colors
 Palette flashing serious problem in multimedia applications cause an
annoying flash of strange colors
 Each program may have its own palette
 may replace the system palette with its own for the period it is active
 serious problem in multimedia applications

106

53
 6/21/2020

Some Image Techniques

107

 Human eye’s is capable to perform spatial integration

 Dithering
1. a technique to increase the number of colors to be perceived in an image
2. look at a number of closely placed small objects  distance  merged
together

107

Some Image Techniques

108

 Aliasing
 caused by the limited resolution of an output device
 makes edges seen as staircases
 Anti-aliasing
 reduce the staircase effect
 worked as
1. filling in pixels which should be half filled with different levels of gray or matching
colors
2. sharper edges  blurring or smoothing them

108

54
 6/21/2020

Image And Graphics File Formats

109

 Image File
 format

 Size

 Depth

 Color and palette

 Compression

109

110

110

55
 6/21/2020

Digital Image Processing

111

 sub-areas
1. Image analysis
 concerned with techniques for extracting descriptions from images that
are necessary for higher-level scene analysis methods
2. Image recognition
 concerned with the techniques for recovering information about objects
in the image
 E.g. character recognition or face recognition
3. Image enhancement
 concerned with the technique to improve the image and to correct some
defects
 color and tonal adjustment
 Transformations, e.g., scale, rotate
 Special effects, e.g., texture, blur, sharpen

111

Image Types, properties and

Representation
112

1. binary image
 binary image
 logical array of 0's and 1's black and white respectively
2. indexed image
 pixel values are direct indices into an RGB color map
 map to represent the color map
3. intensity image
 image consisting of intensity (grayscale) values
 array of class uint8, uint16, or double
 Image m-by-n array
 colormap is always an m-by-3 array of class double
4. Multi-frame image
 Image array that contains more than one image
 related by time or view
 4 D  1 D represents frame number
5. RGB image
 Image in which each pixel is specified by three values
 m-by-n-by-3 array
 class uint8, uint16, or double
 true-color image

112

56
 6/21/2020

Converting Image Types

113

113

114

114

57
 6/21/2020

Spatial Transformations
115

 Image Resizing
 I = imread('circuit.tif');
 J = imresize(I,1.25);
 imshow(I);
 imshow(J);
 Y = imresize(X,[100 150])

115

Spatial Transformations
116

 Image Rotation
 I = imread('circuit.tif');
 J = imrotate(I,35,'bilinear');
 imshow(I)
 figure, imshow(J)

116

58
 6/21/2020

Spatial Transformations
117

 Image Cropping
 I = imcrop;
 imshow(I);
 a = imcrop(I,[50 50 15 15]);

117

JPEG
118

 JPEG: Joint Photographic Expert Group

 An international standard in 1992
 Works for both color and grayscale images
 Targets  natural images
 Image types 
1. Applications include satellite
2. medical imaging
3. general photography

118

59
 6/21/2020

The System of JPEG

119

8x8 block
DCT Quantization

Quantization
tables
Header
DC
Tables
Difference coding
Entropy
coding
Data Run length coding
AC

119

JPEG Extensions
120

 The basic mode of JPEG supports sequential coding

 order is from top to bottom and left to right
 JPEG extensions support progressive modes
 Progressive
 Hierarchical
 Lossless
 Compression—JPEG and GIF
 JPEG works well with 24-bit images
 JPEG for images that contain many colors (such as photographs),
 GIF is a format that only supports 8 bits of color information
 JPEG is desirable, and for cartoons containing at most 256 colors, GIF is useful
 Both JPEG and GIF support an interesting feature that is especially useful when
they are
 used in Web pages

120

60
 6/21/2020

Compression—JPEG and GIF

121

 GIFs can be saved so that they appear "interlaced“

1. Essentially the image is shown in stages in interlaced GIFs
2. First every 8th row is shown, then every 4th row, and so on
3. In effect, the viewer sees a low-resolution image quickly, which slowly comes
into focus
 We can use the "consecutive" GIF format that
1. shows the image from top to bottom as it downloads
2. Thus the user does not have to wait for the entire image to load, before
seeing something on a Web page
 Progressive JPEGs
 appear from top to bottom

121

Strengths and Weaknesses of JPEG

Compression
122

 Strengths
1. It provides support for full 24-bit color images. In contrast, GIF only supports 8-bit images
2. The compressed image size and image quality trade-off can be user determined
3. It is ideally suited to images of real-world scenes or complex computer-generated images
4. It is platform independent for displaying 24-bit images.
5. It is quite good for compressing photographs
 Weaknesses
1. JPEG compression is a trade-off between degree of compression, resultant image quality, and
time required for compression/decompression
2. Blockiness results in high image compression ratios
3. Bad quality is obtained when compressing an image with sharp edges or lines (as in text)
4. It is not suitable for 2-bit black and white images
5. The degree of compression is greater for full color images than it is for gray scale
6. Images
7. It is not intended for moving images/video.

122

61
 6/21/2020

Strengths and Weaknesses of GIF Compression

123

 Strengths
 It is lossless for 8-bit images
 It is ideally suited to images such as line drawings, those containing a limited number of
colors, and for images containing text
 It is preferred for Vector Graphics over the Internet
 Animated GIFs are easy to make by using GIF constructing packages such as Animation
Shop (this comes packaged with Paint Shop Pro)
 Weaknesses
 GIF is not suitable for 24-bit images. When compressing such images, much of the color
information is lost due to the reduction of color depth
 Good algorithms can still optimize this process so that the resultant image still has
reasonable quality
 The compression ratios are low

123

124

CS-473: MULTIMEDIA APPLICATION

DESIGN
UNDERSTANDING MULTIMEDIA CONTENTS
(TEXT, AUDIO, IMAGE, GRAPHIC, VIDEO)

REPRESENTATION, ACQUISITION, MANIPULATION, AND

AUTHORING

UMER RASHID

Department of Computer Science

Quaid-i-Azam University, Paksitan

124

62
 6/21/2020

Understanding Video
125

 Types of Video Signals

 Composite Video Male F-Connector, Connecting co-
axial cable with the device
 S-Video
 Component Video
 Composite Video
 Used in broadcast TV’s
 Compatible with B/W TV Dot Crawl, due to interference in
composite video
 Chrominance ( I & Q or U & V) & Luminance signals are
mixed into a single carrier wave, which can be
separated at the receiver end
 Mixing of signals causes interference & create dot
crawl

125

Types of Video Signals

126

 S-Video
 S stands Super / Separated
 Uses 2 wires, one for luminance & the other for
chrominance signals
 Less cross talk
 Humans are able to differentiate spatial resolution
in gray-scale images with a much higher acuity than
for the color part of color images.
 As a result, we can send less accurate color
information than must be sent for intensity
information
 Component Video
 Each primary is sent as a separate video signal.
 The primaries can either be RGB or a luminance-
chrominance transformation of them (e.g., YIQ, YUV).
 Best color reproduction
 Requires more bandwidth and good synchronization of
the three components

126

63
 6/21/2020

Analog Video
127

 Represented as a continuous (time varying) signal

With interlaced scan, the odd

and even lines are displaced in
time from each other.

127

Analog Video
128

128

64
 6/21/2020

NTSC (National Television System

Committee)
129

 It uses the familiar 4:3 aspect ratio (i.e., the ratio of picture width
to its height)
 Uses 525 scan lines per frame at 30 frames per second (fps).
 NTSC follows the interlaced scanning system, and each frame is
divided into two fields, with 262.5 lines/field.
 Thus the horizontal sweep frequency is 525x 29.97 =15,734
lines/sec, so that each line is swept out in 63.6 µ sec

 63.6 µ sec = 10.9 µ sec for Horizontal retrace + 52.7 µ sec active line
signal
 For the active line signal during which image data is displayed

129

NTSC (National Television System

Committee)
130

 20 lines at the beginning of

every field is for Vertical
retrace control information
leaving 485 lines per frame
 1/6 of the raster at the left side
is blanked for horizontal retrace
and sync. The non-blanking pixels
are called active pixels.
 Pixels often fall in-between the
scan lines. NTSC TV is only
capable of showing about 340
(visually distinct) lines

130

65
 6/21/2020

NTSC (National Television System

Committee)
131

 NTSC video is an analog signal with no fixed horizontal

resolution
 Pixel clock is used to divide each horizontal line of
video into samples. Different video formats provide
different numbers of samples per line
 Uses YIQ Color Model
 Quadrature Modulation is used to combine I & Q to
produce a single chroma signal

131

NTSC (National Television System Committee)

132

 Fsc is 3.58MHz
 Composite signal is formed by

 The available bandwidth is 6MHz, in which the audio

is signal centered at 5.75MHz and the lower
spectrum carries picture information

132

66
 6/21/2020

PAL (Phase Alternating Line)

133

 Widely used in Western Europe, China, India, and

many other parts of the world.
 Uses 625 scan lines per frame, at 25 frames/second,
with a 4:3 aspect ratio and interlaced fields
 Uses the YUV color model
 Uses an 8 MHz channel and allocates a bandwidth of
5.5 MHz to Y, and 1.8 MHz each to U and V.

133

Digital Video
134

 Advantages over analog:

 Direct random access  good for nonlinear video editing
 No problem for repeated recording

 No need for blanking and sync pulse

 Almost all digital video uses component video

134

67
 6/21/2020

Chroma Subsampling
135

 The human eye responds more

precisely to brightness information
than it does to color, chroma
subsampling (decimating) takes
advantage of this.
 In a 4:4:4 scheme, each 8×8 matrix of
RGB pixels converts to three YCrCb 8×8
matrices: one for luminance (Y) and one
for each of the two chrominance bands
(Cr and Cb)
8x8 : 8x8 : 8x8

4:4:4

135

Chroma Subsampling
136

 A 4:2:2 scheme also creates one

8×8 luminance matrix but
decimates every two horizontal
pixels to create each
chrominance-matrix entry. Thus
reducing the amount of data to
2/3rds of a 4:4:4 scheme.

4:2:2

136

68
 6/21/2020

Chroma Subsampling
137

 Ratios of 4:2:0 decimate

chrominance both horizontally
and vertically, resulting in four 4:1:1
Y, one Cr, and one Cb 8×8
matrix for every four 8×8
pixel-matrix sources. This
conversion creates half the
data required in a 4:4:4
chroma ratio
4:2:0

137

Chroma Subsampling
138

Luma sample
4:2:0
Chroma sample

138

69
 6/21/2020

Chroma Subsampling
139

 h 4:1:1 and 4:2:0 are used in JPEG and MPEG

 256-level gray-scale JPEG images aren't usually

much smaller than their 24-bit color counterparts,
because most JPEG implementations aggressively
subsample the color information. Color data therefore
represents a small percentage of the total file size

139

High Definition TV (HDTV)

140

 The main thrust of HDTV (High Definition TV) is not to

increase the definition in each unit area, but rather to
increase the visual field especially in its width.
 The first generation of HDTV was based on an analog
technology developed by Sony and NHK in Japan in the
late 1970s.
 Uncompressed HDTV will demand more than 20 MHz
bandwidth, which will not fit in the current 6 MHz or 8 MHz
channels
 More than one channels even after compression.

140

70
 6/21/2020

High Definition TV (HDTV)

141

 The salient difference between conventional TV and

HDTV:
 HDTV has a much wider aspect ratio of 16:9 instead of 4:3.
 HDTV moves toward progressive (non-interlaced) scan. The
rationale is that interlacing introduces serrated edges to
moving objects and flickers along horizontal edges.

141

71