Multimedia Systems

Ch-01: Introduction

Tsedeniya K.
 What is Multimedia?
• Multimedia is used exclusively to describe multiple forms of media
and content.
–Multimedia concerns the representation of mixed modes of information as
digital signals.
–It includes a combination of text, audio and speech, video, images and
graphics content forms
–The content may be in the form of animation based on which multimedia is
divided into conventional media (text, image, …) and new media (audio, video,
…).

• Multimedia represents the convergence of text, image, video and

sound into a single form.
– The power of multimedia and the Internet lies in the way in which information
is linked.
 Media Types: Two broad classes
• Discrete (or Static) media: it refers to text, graphics, and images as
they are composed of time independent information item.
– Information in these media consist exclusively of a sequence of individual
elements without a time component. Indeed they may be displayed according
to a wide variety of timing or even sequencing and still remain meaningful

• Continuous (or Dynamic) media: refers to time-dependent media like

sound, and video, where the presentation requires a continuous play-out as
time passes.
– Information is expressed as not only of its individual value, but also by the time
of its occurrence. In other word, time dependency between information items
is part of the information itself.
– Thus if the timing of the items change, or the sequence is modified, the
meaning of the items is altered.
– Note: These notions of time-dependent, discrete and continuous media do not
have any connection to the internal representation.
 Multimedia System
• Multimedia system represents a technology/tool used to combine two
or more categories of information having different transport signal
characteristics.
– Examples of media: Image, audio, video, text and graphics

• Discuss:
–Can we say a word processor that handles embedded graphics/image in
addition to text a multimedia system?
–What about macromedia flash that helps to create animated objects?
–Do you think PDF, PS and DAT files are multimedia?
 Properties of Multimedia System
• Multimedia system handles discrete & continuous media
– It is a system capable of handling at least one discrete and one continuous
media in digital form
– E.g. Multiple forms of information content are often not considered
multimedia if they don't contain, say, audio or video.
• It integrates and synchronizes the different medias simultaneously
into a coherent framework.
– integrate and use minimal number of different devices for presentation of
multimedia
– Synchronize well the presentation of the result of the different media such
that information flows in the correct order
• Multimedia system must be computer controlled.
– A digital device, like computer must be involved in the presentation of the
multimedia information to users
 Application
• Multimedia finds its application in various areas with the help of
Web technologies
–Education: hypermedia courseware, e-learning, audio/video conferencing
–Business: e.g. multimedia authoring, video-on-demand
–Digital and virtual libraries
–Entertainment: e.g. games
–Advertisements: e.g. interactive television
–Scientific research: e.g. virtual worlds
–Engineering: e.g. AutoCAD
–Art, Medicine, Mathematics, etc.
 Challenges of Multimedia Computing
Developing a successful multimedia system is non-trivial.
• Memory space requirement: multimedia data need a lot of space to
store
– At least GB’s main memory; Up to TB’s secondary storage; PB’s of tertiary storage
• Data transmission bandwidth: Multimedia needs very high bandwidth
to transmit.
– Faster network (up to 25Mbs per video stream) with very high bandwidth to
transmit
• Complexity of multimedia data management: Multimedia needs more
complex and more efficient algorithms, say
– Automatically analyzing, indexing and organizing information in audio, image and
video is much harder than from text. They involve many different research issues.
• Hardware platforms: Multimedia data need efficient hardware
platforms
– Better CPU, graphics card, video card, sound card
 Multimedia is a Multidisciplinary subject

Operating
system,
Computer
networks
Image, audio, Computer vision,
Multimedia
speech processing pattern recognition
computing

Computer
Human-computer graphics
interaction
 Multimedia Computing
• It involves extensive practices from multimedia capturing, processing,
transmitting, content representing to multimedia database
management
• Multimedia systems involve multimedia information handling and
some basic enabling techniques:
– Multimedia Data Generation: production/authoring tools
• Software Authoring Tools: e.g., Adobe flash, MS silver light, Macromedia, etc.
• For Web publishing: HTML, Java, Macromedia Director, etc.

– Multimedia Data Representation: compression and formats

• Storage and file systems design

– Multimedia Database Management: Multimedia Data Processing and

Analysis, indexing, searching and retrieval.

– Multimedia Data Transmission through communication and computer

networks.
 Evolution of Multimedia
• 1945 - Vannevar Bush (1890-1974) wrote about Memex
• Generally, the concept of multimedia is based on the concept of
hypertext which was invented in the early 1940ies.
• In 1945 Vannevar Bush wrote an article about a virtual machine
(Memex: Memory Extender) which was able to browse and annotate
large collections of documents including a mechanism to create links
between various documents. (BUSH 1945)
• From Bush's concept evolved the concept of hypertext systems.
Hypertext is text which is extended by links. These links act as pointers
to other pieces of text that are located elsewhere, either in the same
document or in another document or both. Using these links enables
users to "browse around" in one or more documents.
• 1960s - Ted Nelson started Xanadu project and coined the term
“HyperText”. Xanadu was the first attempt at a hypertext system – Nelson
called this a magic place of literary memory”.
 Evolution of Multimedia
• 1967 - Nicholas Negroponte formed the Architecture Machine
Group at MIT
• 1968 - Douglas Engelbart demonstrated the “on –Line System”
(NLS), another hypertext program. NLS consisted of critical ideas
as an outline editor for idea development, hypertext links,
teleconferencing, word processing, and email and made use of
mouse pointing device, windows software and help system.
• 1969 - Nelson & Van Dam created hypertext editor called PRESS.
 Evolution of Multimedia
• 1976 - Architecture Machine Group proposal a project called “Multiple Media” this results in the
Aspen Movies Map, the first hypermedia videodisc
• 1985 - Negroponte, Wiesner: opened MIT Media Lab, a leading research institution investigating
digital video and multimedia.
• 1989 - Tim Berners-Lee proposed the World Wide Web to CERN
– WWW is a hypertext-based system that operates over the internet (be aware that the World Wide Web is not
the same as the Internet; the difference will be discussed later on in this module). Distributed documents,
stored on remote computers, can be accessed electronically via web browser software. The WWW is made
up of three standards:
– The Uniform Resource Locator (URL) which specifies how each page of information is given a unique address
at which it can be found (e.g. www.karto.ethz.ch)
– The Hyper Text Transfer Protocol (HTTP) which determines how a web browser requests information from
the server and how the server makes the data available.
– The Hypertext Markup Language (HTML) is a mark up language designed for the creation of web pages, i.e.,
information presented on the WWW. HTML encodes the information of a web-page so it can be displayed on
a variety of devices. HTML is generally written in text files stored on computers connected to the internet.
 Evolution of Multimedia
• 1990 - K. Hooper Woolsey, Apple Multimedia Lab
• 1991 – MPEG-1 was approved as an international standard for digital
video.
• 1992 - The first M-Bone audio multicast on the Net was made. And
on same year JPEG was accepted a the international standard for
digital image processing.
• 1993 - U. Illinois National Center for Supercomputing Applications:
NCSA Mosaic
• 1994 - Jim Clark and Marc Andreesen created Netscape program.
• 1995 - JAVA for platform-independent application development
Hypermedia/HyperText/Multimedia

• HyperText
• Ted Nelson invented the term “HyperText” in 1965.
• HyperText systems read nonlinearly, by following the links that
point to other part of the documents.

• Hypermedia
• is not constrained to be text based. It can include graphics,
images and especially the continues media –sound and video.
World Wide Web is the best example for this.
 Hypermedia/HyperText/Multimedia
• “Multimedia” is fundamentally means that computer
information represented through audio, graphics, image
and animation in addition to the traditional media (text
and graphics). HyperMedia can be considered as one
part of multimedia application.
• Example MM: Digital video editing and production
system, electronic newspaper and magazine, the WWW,
online reference works such as encyclopedia, games
groupware home shopping interactive TV …
 Overview
Digital Audio, Image and Video
 Overview
• Audio: used to record sound.
– In the past 20 years, audio has moved from analog recording on tape cassettes to totally digital
recording using computers.
– Today, the Musical Instrument Digital Interface (MIDI) allow anyone to create music right on their
desktop. MIDI is a digital standard that defines how to code musical scores, such as sequences of
notes, timing conditions, and the instrument to play each note.
• Image:
–It is a 2-D object, which is stored as a specific arrangement of dots, or pixels.
–It differs from graphics in that images can be captured from the real world, where as graphics are
created by software & internally represented as an assemblage of primitive objects such as lines,
curves, circles, polygons, and arcs.
–Unlike Images, graphics are usually editable or revisable.
• Video:
– A series of framed images put together, one after another, to simulate motion and interactivity. A
video can be transmitted by number of frames per second and/or the amount of time between
switching frames.
– The difference between video and animation is that video is broken down into individual frames.
 Digital Media
• In computers, audio, image and video are stored as files just like
other text files (e.g. DOC, TXT, TEX, etc.).

– For images, these files can have an extension like

• BMP, JPG/JPEG, GIF, TIF, PNG, PPM, …

– For audios, the file extensions include

• WAV, MP3, m4a, AMR, WMA…

– The videos files usually have extensions:

• MOV, AVI, MPEG, MP4, 3gp, …

• What about PDF file? PS file? DAT file?

 Analog and Digital Signals
• The world we sense is full of analog signal; electrical sensors such as transducers,
thermocouples, microphones convert the medium they sense into electrical
signals.
• These are usually continuous and still analog. These analog signals must be
converted or digitized into discrete digital signals that computer can readily deal
with.
• Special hardware devices called Analog-to-Digital converters perform this task.
• For playback Digital-to-Analog must perform a converse operation.
• Note that Text, Graphics and some images are generated directly by computer
and do not require digitizing: they are generated directly in binary format.
• Handwritten text would have to digitize either by electronic pen sensing of
scanning of paper based form.
 Advantage of digital media over analog ?
• Is digital cameras do things that are not done by still cameras?
• The following are some of the advantages of digital media
– Displaying images/audios/videos on a screen immediately after they are
recorded
– Storing thousands of images/audios/videos on a single small memory device
– Deleting images/audios/videos to free storage space
– Digital camera enables recording video with sound; and camcorder enables
capturing image.
– What else???
 Digital Media Capturing
• To get a digital image, an audio or a video clip, we need some media
capturing devices

• Image:
– is captured using devices such as a digital camera or a digital scanner

• Audio:
– is recorded using a digital audio recorder (or Microphone), such as Olympus
Voice Recorder, MP3 digital recorder, SONY Voice Recorder, etc.

• Video:
– is recorded using a digital camcorder.
– Camcorder is a video camera that records video and audio using a built-in
recorder unit. The camcorder contains both a video camera and a video
recorder in one unit and hence its compound name
 Image Storing format & compression
• The most popular image storing formats include: BMP, JPG, GIF, TIF,
PNG, PPM, …
–To store an image, the image is represented in a 2-D matrix, in which each value
corresponds to the data associated with one image pixel.
–The image formats also influence storage requirements of image. If storage space
is scarce, images should be compressed in a suitable way. If sufficient memory
available, the image can be stored uncompressed.

• For instance;
–BMP (Bit Map) format does not compress the original image.
–GIF (Graphics Interchange Format ) type images are compressed to more than
99% of their original size with no loss in image quality. It supports up to 256
colors.
– JPEG (Joint Photographic Experts Group) became an ISO international standard
for compression of images. It applies to color and gray-scaled images.
 Video Storing format & compression

• Each video formats support various resolutions and color presentation. The
following are the well-known video formats
• The Color Graphics Adaptor (CGA):
– Has a resolution of 320 x 200 pixels with simultaneous display of four colors
– What the necessary storage capacity per frame ?
• The Enhanced Graphics Adaptor (EGA):
– Supports display resolution of 640 x 350 pixels with 16 simultaneous display colors
– What the necessary storage capacity per frame ?
• The Video Graphics Array (VGA):
– Works mostly with a resolution of 640 x 480 pixels with 256 simultaneous display colors
– What the necessary storage capacity per frame ?
• The Supper Video Graphics Array (VGA):
– Can present 256 colors at a resolution of 1024 x 768 pixels.
– What the necessary storage capacity per frame ?
– Other SVGA modes include 1280 x 1024 pixels and 1600 x 1280 pixels.
 Output Devices
• The output devices for a basic multimedia system include
• A High Resolution Color Monitor
• CD Quality - - Audio Output
• Color Printer
• Video Output to save Multimedia presentations to (Analog) -
Video Tape, CD-ROM DVD.
• Audio Recorder (DAT, DVD, CD-ROM, (Analog) Cassette)
• Storage Medium (Hard Disk, Removable Drives, CD-ROM)
• one unit and hence its compound name
 Overview of Multimedia Software Tools
• Digital Audio
– Macromedia Sound edit -- Edits a variety of different format audio files, apply a variety of
effects.
– CoolEdit -- Edits a variety of different format audio files. Many Public domain tools on the Web.
• Music Sequencing and Notation
– Cakewalk
• Supports General MIDI. l Provides several editing views (staff, piano roll, event list) and
Virtual Piano
• Can insert WAV files and Windows MCI commands (animation and video) into tracks
– Cubase
• A better software than Cakewalk Express
• Intuitive Interface to arrange and play Music
• Wide Variety of editing tools including Audio.
– Logic Audio: l Cubase Competitor, similar functionality
– Marc of the Unicorn Performer : Cubase/Logic Audio Competitor, similar functionality
 Overview of Multimedia Software Tools
• Image/Graphics Editing
– Adobe Photoshop
• Allows layers of images, graphics and text
• Includes many graphics drawing and painting tools
• Sophisticate lighting effects filter
• A good graphics, image processing and manipulation tool
– Adobe Premiere
• l Provides large number (up to 99) of video and audio tracks, superimpositions and
• virtual clips
• l Supports various transitions, filters and motions for clips
• l A reasonable desktop video editing tool
– Macromedia Freehand
• Graphics drawing editing package
• Many other editors in public domain and commercially
 Overview of Multimedia Software Tools
• Animation
– Many packages available including:
• Avid SoftImage
• Animated Gif building packages e.g. GifBuilder
Multimedia Authoring
• Tools for making a complete multimedia presentation where users usually have a lot of
interactive controls.
• Macromedia Director: Movie metaphor (the cast includes bitmapped sprites, scripts,
music, sounds, and palettes, etc.)
• Can accept almost any bitmapped file formats, l Lingo script language with own debugger
allows more control including external, devices, e.g., VCRs and video disk players
• Authorware: Professional multimedia authoring tool, Supports interactive applications
with hyperlinks, drag-and-drop controls, and integrated animation. Compatibility between
files produced from PC version and MAC version
End of Chapter- 01