Multimedia - IT451

Lecture 1

BY…
DR. NOURA SEMARY
Lecture 1: What is
Multimedia
Multimedia Definition
A computer system perspective definition:
Multimedia means that computer information can be represented through audio, video, and animation in
addition to traditional media (i.e., text, graphics/drawings, images).
General Definition
A good general working definition for this module is:
Multimedia is the field concerned with the computer-controlled integration of text, graphics, drawings,
still and moving images (Video), animation, audio, and any other media where every type of
information can be represented, stored, transmitted and processed digitally.
Multimedia Application Definition
A Multimedia Application is an application which uses a collection of multiple media sources e.g. text,
graphics, images, sound/audio, animation and/or video
Definition of Multimedia Image

• Multimedia is a combination of
text, graphic, sound, animation,
and video that is delivered Video Audio
interactively to the user by
electronic or digitally Multimedia
manipulated means.

Animation Text
 Historical Perspective
Newspaper was perhaps the first mass communication medium to employ Multimedia -- they used mostly text,
graphics, and images.
◦ In 1895, Gugliemo Marconi sent his first wireless radio transmission at Pontecchio, Italy. A few years later (in 1901) he
detected radio waves beamed across the Atlantic. Initially invented for telegraph, radio is now a major medium for audio
broadcasting.
Television was the new media for the 20th century. It brings the video and has since changed the world of mass
communications.
Some of the important events in relation to Multimedia in Computing include: 
1945 - Bush wrote about Memex

1971 - Email
1989 - Tim Berners-Lee proposed the World Wide Web to CERN (European Council for Nuclear Research)
1994 - Jim Clark and Marc Andreesen: Netscape
1996 - Microsoft, Internet Explorer.
Multimedia Application
A Multimedia Application is an application which uses a collection of
multiple media sources e.g. text, graphics, images, sound/audio,
animation and/or video.
 Examples of Multimedia Applications
Examples of Multimedia Applications include:
1. World Wide Web
2. Multimedia Authoring, e.g. Adobe/Macromedia Director
3. Hypermedia courseware
4. Video-on-demand
5. Interactive TV
6. Computer Games
7. Virtual reality
8. Digital video editing and production systems
9. Multimedia Database systems
Hyper Multimedia
Hypertext is a text which contains links to
other texts. The term was invented by
Ted Nelson around 1965.
Traversal through pages of hypertext is
therefore usually non-linear.
This has implications in layout and organization of
material — and depends a lot on the application at hand.
Hyper Multimedia
Hyper Media
◦ A combination of hypertext, graphics,
audio, video, (linked elements) and
interactivity culminating in a complete,
non-linear computer-based experience.
Multimedia Systems
Definition:
A Multimedia System is a system capable of processing multimedia
data and applications.

A Multimedia System is characterised by the processing, storage,

generation, manipulation and rendition of Multimedia information.
Multimedia Systems
A Multimedia system has four basic characteristics:
1. Multimedia systems must be computer controlled.
2. Multimedia systems are integrated.
3. The information they handle must be represented
digitally.
4. The interface to the final presentation of media is
usually interactive.
Challenges for Multimedia Systems
1. Distributed Networks
2. Temporal relationship between data
◦ Render different data at same time — continuously.
◦ Sequencing within the media:
◦ playing frames in correct order/time frame in video
◦ Synchronisation — inter-media scheduling
◦ E.g. Video and Audio — Lip synchronisation is clearly important for humans to watch playback of video
and audio and even animation and audio. Ever tried watching an out of (lip) sync film for a long time?
Key Issues for Multimedia Systems
The key issues multimedia systems need to deal with here are:
◦ How to represent and store temporal information.
◦ How to strictly maintain the temporal relationships on play back/retrieval
◦ What process are involved in the above.
◦ Data has to represented digitally — Analog–Digital Conversion, Sampling etc.
◦ Large Data Requirements — bandwidth, storage,

Data compression is usually mandatory

Desirable Features for a Multimedia
System
Given the above challenges the following feature a desirable (if not a
prerequisite) for a Multimedia System:
◦ Very High Processing Power — needed to deal with large data processing and
real time delivery of media. Special hardware commonplace.
◦ Multimedia Capable File System — needed to deliver real-time media — e.g.
Video/Audio Streaming.
◦ Special Hardware/Software needed – e.g. RAID technology.
◦ Data Representations — File Formats that support multimedia should be easy to
handle yet allow for compression/decompression in real-time.
Desirable Features for a Multimedia
System (Cont.)
Efficient and High I/O — input and output to the file subsystem needs to be
efficient and fast. Needs to allow for real-time recording as well as
playback of data. e.g. Direct to Disk recording systems.

Special Operating System — to allow access to file system and process data
efficiently and quickly. Needs to support direct transfers to disk, real- time
scheduling, fast interrupt processing, I/O streaming etc.
Desirable Features for a Multimedia
System
Storage and Memory — large storage units (of the order of hundreds of Tb if not
more) and large memory (several Gb or more). Large Caches also
required and high speed buses for efficient management.

Network Support — Client-server systems common as distributed systems

common.

Software Tools — user friendly tools needed to handle media, design and develop
applications, deliver media.
Components of a Multimedia System
Now let us consider the Components (Hardware and Software) required for a multimedia
system:
Capture Devices
• Video Camera, Video Recorder, Audio Microphone, Keyboards, mice, graphics tablets, 3D input devices,
tactile sensors, VR devices. Digitising Hardware

Storage Devices
• Hard disks, CD-ROMs, DVD-ROM, etc

Communication Networks
• Local Networks, Intranets, Internet, Multimedia or other special high speed networks.

Computer Systems
• Multimedia Desktop machines, Workstations, MPEG/VIDEO/DSP Hardware

Display Devices
• CD-quality speakers, HDTV,SVGA, Hi-Res monitors, Colour printers etc.
Multimedia Data:
Input and Format Text and Static Data
Source: keyboard, speech input, optical character recognition, data stored on disk.
Stored and input character by character:
• Storage of text is 1 byte per char / more bytes for Unicode.
• For other forms of data (e.g. Spreadsheet files). May store format as text (with formatting) others may use binary encoding.

Format: Raw text or formatted text e.g HTML, Rich Text Format (RTF), Word or a program
language source (Java, Python, MATLAB etc.)
Not temporal — BUT may have natural implied sequence e.g. HTML format sequence,
Sequence of C program statements.
Size Not significant w.r.t. other Multimedia data.
Graphics
Format: constructed by the composition of primitive objects such as lines, polygons, circles,
curves and arcs.
Input: Graphics are usually generated by a graphics editor program (e.g. Illustrator) or
automatically by a program (e.g. Postscript).
Graphics are usually editable or revisable (unlike Images).
Graphics input devices: keyboard (for text and cursor control), mouse, trackball or graphics
tablet.
Graphics standards : OpenGL, PHIGS, GKS Graphics files usually store the primitive assembly
Do not take up a very high storage overhead.
Images
Still pictures which (uncompressed) are represented as a bitmap (a grid of pixels).
Input: digitally scanned photographs/pictures or direct from a digital camera.
Input: May also be generated by programs “similar” to graphics or animation programs.
Stored at 1 bit per pixel (Black and White), 8 Bits per pixel (Grey Scale, Colour Map) or 24 Bits
per pixel (True Colour)
Size: a 512x512 Grey scale image takes up 1/4 MB, a 512x512 24 bit image takes 3/4 MB with
no compression.
This overhead soon increases with image size — modern high digital camera 10+ Megapixels ≈
29MB uncompressed!
Compression is commonly applied.
Audio
Audio signals are continuous analog signals.
Input: microphones and then digitised and stored
CD Quality Audio requires 16-bit sampling at 44.1 KHz Even higher audiophile rates (e.g. 24-bit,
96 KHz) 1 Minute of Mono CD quality (uncompressed) audio requires 5 MB.
1 Minute of Stereo CD quality (uncompressed) audio requires 10 MB.
Usually compressed (E.g. MP3, AAC, Flac, Ogg Vorbis).
Video
Input: Analog Video is usually captured by a video camera and then digitised.
There are a variety of video (analog and digital) formats
Raw video can be regarded as being a series of single images. There are typically 25, 30 or 50
frames per second.
E.g. A 512 × 512 size monochrome video images take 25 × 0.25 = 6.25MB for a second to store
uncompressed.
Typical PAL digital video (720 × 576 pixels per colour frame) ≈ 1.24 × 25 = 31MB for a second to store
uncompressed.

High Definition video on Blu-ray (up to 1920 × 1080 = 2 Megapixels per frame) ≈ 6.2 × 25 =
155MB for a second to store uncompressed. (There are higher possible frame rates!)
Digital video clearly needs to be compressed for most times.
 Summary:
This Course is Essentially
about Multimedia Data
Compression
How can we compress data?
Lossy vs Lossless :
◦ Lossless : Ideal (e.g. zip, unix compress) not good enough for MM
data!
◦ Lossy :Throw away nonessential (perceptually less relevant) parts
of the data stream FILTER the data somehow. Examples:
MP3, JPEG, MPEG Video/Audio
Compression: Is there another way?
Compression via Synthesis :
◦ Encode how to make (synthesise) the data can be
done in many less bits in certain cases.
◦ Examples: Vector Graphics (Flash), MPEG Video, MP4
(Audio), MIDI: