Multimedia

Multimedia Concept
Concept &
& Topics
Topics

•• Multimedia
Multimedia Concept
Concept
•• Multimedia
Multimedia Computing
Computing
•• Multimedia
Multimedia Classification
Classification
•• Multimedia
Multimedia Topics
Topics
•• Multimedia
Multimedia Driving
Driving Forces
Forces
•• Multimedia
Multimedia Applications
Applications

•• Course
Course Outline
Outline
 What
What is
is Multimedia
Multimedia
• Multi: more than one
• Medium (singular): middle, intermediary, mean
• Media (plural): means for conveying information

- Media in the press, newspaper, radio and TV context - mass media

- Media in communications: cables, satellite, network – transmission media
- Media in computer storage: floppy, CD, DVD, HD, USB – storage media
- Media in HCI context: text, image, audio, video, CG – interaction media

• Multimedia: refers to various information forms text, image, audio,

video,
graphics, and animation in a variety of application environments

Multimedia … :
product, application, technology, platform, board, device, network
computer, system, classroom, school, …
Word “multimedia” is widely used to mean many different things
 What
What is
is Multimedia
Multimedia in
in terms
terms of
of Computing
Computing
Computing: Computer-based technologies and applications
 What computers?  Various forms of computers/devices!
In terms of computing, four fundamental multimedia attributes:
- Digitized: All media including audio/video are represented in digital format
- Distributed: The information conveyed is remote, either pre-produced and
stored or produced in realtime, distributed over networks
- Interactive: It is possible to affect the information received, and send own
information, in a non-trivial way beyond start, stop, fast forward
- Integrated: The media are treated in a uniform way, presented in an
orchestrated way, but are possible to manipulate independently

Definition of Multimedia:
Computer-based techniques of text, images, audio, video, graphics, animation,
and any other medium where every type of information can be represented,
processed, stored, transmitted, produced and presented digitally.

This course focus  Audio and Video

 Benefits
Benefits of
of Multimedia
Multimedia
Some authors claim that humans get their information in the following way:

• more than 80 % by sight - of which 20 % is remembered

• 11 % by hearing - of which 30 % is remembered
• 3.5 % by smell
• 1.5 % by touch and taste.

... where 50 % of what is both seen and heard is remembered

... further 80 % of what is seen, heard and done, is remembered

That is, multiple, media, and interactive should be a good thing

 A
A Classification
Classification of
of Multimedia
Multimedia
• Text - ASCII/Unicode, HTML, Postscript, PDF
• Audio – Sound, music, speech, structured audio (e.g. MIDI)
• Still Image - Facsimile, photo, scanned image
• Video (Moving Images) – Movie, a sequence of pictures
• Graphics – Computer produced image
• Animation – A sequence of graphics images

• Discrete Media (DM, Static): text, image, graphics

• Continuous Media (CM, Dynamic): audio, video, animation

• Captured vs Synthesized media

• Standalone vs Networked media
 System
System Implications
Implications of
of Multimedia
Multimedia
Multimedia imposes new requirements on all parts of the system architecture :
• Representation
- digitization and coding (compressing)
• Storage
- database, larger volumes and new access patterns
• Processing
- OS, scheduling, indexing, searching
• Understanding
- speech/object recognition, content analysis
• Production
- more complex authoring and user interface software
• Presentation
- user perception, user friendly in HCI (Human Computer Interface)
• Protection
– media encryption, copyright, privacy
• Distribution
- media delivery and broadcast
• Communication
- media transmission over network/internet, session control
 Why
Why is
is Multimedia
Multimedia Important
Important ??

• Digital audio/video is revolutionizing music, film, game, and video &

audio industries
• Convergence of computers, telecommunication, radio, and TV
– Caused by technology and competition
– Dramatic changes in products and infrastructure
• New application potential
– Huge potential markets
– Improving our lives (learning, entertainment, and work)
• Interesting technical issues

Multimedia has become hot and been emerged in CS/IT since 1985
Forces
Forces Driving
Driving the
the Multimedia
Multimedia Revolution
Revolution

• Evolution of communication and data networks: Increasing

availability of bandwidth on demand in the office, home,
road.... Thanks to high-speed data modems, cable modems,
hybrid fiber-coax systems, xDSL, wireless.
• Ubiquitous access to network. Via local-area networks
(LAN), wireline and wireless networks, Internet, world wide
web,  “anywhere, anytime”.
• Fast processor and large capacity storage devices, including
3-D hardware. Moore’s law: computation and memory
capacity of chips doubles every 18 months or so.
Forces
Forces Driving
Driving the
the Multimedia
Multimedia Revolution
Revolution (Cont…)
(Cont…)
• New algorithms and data structures. Compression
techniques, graphics, computer vision, speech
understanding...
• Smart terminals such as digital phones, screen phones,
multimedia PC’s, web-TV, personal digital assistants, etc.,
accessing and interacting the network with wired and
wireless connections.
• And of foremost importance, the digitization of virtually any
device : cameras, video capture and playback devices,
handwriting terminals, sound capture, etc., together with
plug-and-play standards; and the digitization of
text/audio/video documents and libraries that allows better
communications, storage, and fast access and browsing.
 Technological
Technological Aspects
Aspects
• Techniques for compressing and coding the various media: models,
algorithms, forms, standards, etc.
• Communications aspects: downloading and streaming techniques,
synchronization, layering of signals, issues involved in the definition of
QoS (quality of service.)
• Techniques for accessing multimedia signals by providing tools that
match user to the machine: “natural” spoken language queries, media
conversion tools and multimodal user interface (speech recognition, lip
reading, face tracking, OCR,..), agents that monitor the multimedia
sessions and provide assistance in all phases of access and utilization.
• Techniques for organizing, storing and retrieving multimedia, for
searching and browsing individual multimedia documents and libraries.
 Are
Are Multimedia
Multimedia Applications
Applications Hard?
Hard?
• Large size of multimedia objects
– Speech: 8000 samples/s – 8 Kbytes/s
– CD audio: 44,100 samples/sec, 2 bytes/sample,
stereo audio – 176 Kbytes/s
– NTSC video: 30 frames/s, 640x480 pixels, 3
bytes/pixel – 30 Mbytes/s
(too big, 2-8 Mbits/s if compressed)
– More storage required
– More main memory
– 10-30 GB secondary storage
– TB’s of tertiary storage
• Real-time performance requirements
Are
Are Multimedia
Multimedia Applications
Applications Hard?
Hard? (Cont…)
(Cont…)
• Higher data rates
– Fast I/O subsystem (SCSI, fiber channel, HIPPI)
• E.g., Ultra SCSI2 – 80 Mbytes/s
– High speed backplane (PCI or faster)
– Faster network (1-25Mbs per video stream)
• 1-4 Gbits/s network
• Hardware CODEC, modified CPU (?), and modified
frame buffer/graphics subsystem

Essentially, new hardware and software

Further, audio/image/video “content” processing
 Examples
Examples of
of Multimedia
Multimedia Applications
Applications

• Residential services • Science and technology

– Video-On-Demand – Virtual environment
– Video phone, A/V – Scientific visualization,
conferencing prototyping
– Home shopping • Entertainment
• Business services – Games
– Corporate education – Interactive TV
– E-business – Post production of movie
• Education and music
– Digital libraries • Medicine, Web applications, etc.
– Distance learning
General
General Overview
Overview of
of aa Multimedia
Multimedia System
System
Timeline
Timeline Audio/Video
Audio/Video Editing
Editing Interface
Interface
Adobe Premier
Audio/Video
Audio/Video Broadcast
Broadcast over
over the
the Internet
Internet
Viewers
Streaming Listen
Media Server

A/V
Internet

Encoder
Encoder
Shared
Shared Applications
Applications and
and CSCW
CSCW
Desktop
Desktop Audiovisual
Audiovisual Conferencing
Conferencing
 Digital
Digital Library
Library
Library Creation Library Exploration
Offline Online

Video Audio Text

Digital Compression
Spoken
Natural Semantic-
Language Expansion
Speech Image Natural Language Query
Recognition Extraction Interpretation
Story Requested
Segmentation Choices Segment

Indexed Database Indexed Database

Indexed Segmented Indexed Segmented
Transcript Compressed DISTRIBUTION Transcript Compressed
Audio/Video
TO USERS Audio/Video
 Teaching
Teaching Plan
Plan

Part I: Multimedia Fundamentals and Coding

Techniques

Lesson 1.  Multimedia Concept and Topics

Lesson 2.  Audio Fundamentals
Lesson 3.  Audio Coding and Standard
Lesson 4.  Image/Video Fundamentals
Lesson 5.  Image/Video Coding: JPEG and H.26x
Lesson 6.  MPEG Coding Standards

Lesson 7. Review of Advanced MM Coding

Quiz Test I. Questions related to Part I

Report I.  Summary of Audio and Video Coding, or

A Study on a Specific Coding Technique
 Teaching
Teaching Plan
Plan

Part II: Multimedia Technologies and Applications

Lesson  8. Media Object Production

Lesson  9. Media Integration and Presentation
Lesson 10. Media Protection
Lesson 11. Media Retrieval
Lesson 12. Media Distribution Across Internet
Lesson 13. Media Communication - IP Telephony & Teleconference
Lesson 14. Mobile Multimedia Service over Wireless Networks

Report  II.  Summary of Multimedia Technologies, or

A Study on a Specific Multimedia Technology

Quiz Test II. Questions related to Part II