Scribd
Upload
2 views

Lecture -1 - Introduction

The document outlines a course on Multimedia Systems and Applications, covering topics such as multimedia signal processing, video compression, and real-time coding. It discusses the challenges faced in multimedia communication, including bandwidth limitations, synchronization issues, and the need for efficient indexing and retrieval of multimedia files. Prerequisites include basic knowledge of digital signal and image processing, and the document lists several key resources for further reading.

Uploaded by

arkaisalegend0006
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
2 views

Lecture -1 - Introduction

The document outlines a course on Multimedia Systems and Applications, covering topics such as multimedia signal processing, video compression, and real-time coding. It discusses the challenges faced in multimedia communication, including bandwidth limitations, synchronization issues, and the need for efficient indexing and retrieval of multimedia files. Prerequisites include basic knowledge of digital signal and image processing, and the document lists several key resources for further reading.

Uploaded by

arkaisalegend0006
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 23

Multimedia Systems and Applications

Resources:

1. Video Processing and Communications by Y. Wang, J.


Ostermann, Y.-Q. Zhang, Prentice-Hall, 2002.
2 Introduction to Multimedia Systems and Processing, nptel.
3. H.264 and MPEG-4 Video Compression: Video Coding for
Next-generation Multimedia by I. E. Richardson, Wiley, 2003.
4. Fundamentals of Multimedia by Li and Drew, PHI, 2012.
5. Introduction to Data Compression by K. Sayood, Morgan
Kaufmann, 2006.
Overview of the course

• Introduction to Multimedia Systems


• Basics of Image Compression and Coding
• Video Compression
• Real-time Video Coding
• Multimedia standards
• Multimedia applications
Prerequisites

Basic knowledge of the following subjects are useful.


• Digital Signal Processing

• Digital Image Processing


Introduction to Multimedia Systems
Outline

• Multimedia signal

• Motivation behind the growth of multimedia technology

• Elements of multimedia communication system

• Challenges involved with multimedia signal processing


Multimedia signal

• A multimedia signal is one that integrates signals from several


media sources such as video, audio, graphics, animation etc.,

• The integration of different media must be done to convey


some information.

• Television is an example, where there is unidirectional flow of


multimedia information.
Motivation

• There is no single factor that prompted the growth of


multimedia technology.

• Multiple factors contributed to its growth:


- Advancements of processing speed
- Effective data compression tools and algorithms
- Availability of VLSI circuits for real-time processing
- Advancement of computer networking
Elements of Multimedia Communication System

Fig. 1: Elements of Multimedia Transmitter


Elements of Multimedia Communication System

Fig. 2: Elements of Multimedia Receiver


Challenges

• Bandwidth limitations of communication channels

• Real-time processing requirements

• Inter-media synchronization

• Intra-media continuity

• End-to-end delays and delay jitters

• Multimedia indexing and retrieval


Bandwidth Limitation

• Limited bandwidth of communication channels poses the most


serious challenge.

Example 1

• Transmission a color image of 1024 x 768 pixels through a


telephone channel having a speed of 14.4 Kbps

• Assuming 24-bits for each pixel, i.e., 8-bits for each color
components (Red, Green and Blue)

• The total number of bits to be transmitted for the entire image is


given by
B = 1024 768 24 = 18.8
• Total time required to transmit this image is i.e.,
approximately 22 minutes

Example 2

• Transmission a video sequence of 352 x 288 pixels through a


leased line of 384 Kbps

• Assuming 24-bits for each pixel acquired at a frame rate of


30 frames per second

• The raw video bit rate to be transmitted is given by


B = 352 288 24 30 = 72.9Mbps
• Hence, we cannot transmit the video through the given
channel bandwidth, unless the data is significantly
compressed.

Fig. 3: Foreman Sequence (123rd frame)

• We can observe that most of the pixels have almost the same
intensity as those of the neighbors.
• There is considerable redundancy present in the image, since
the pixels are spatially correlated to a great extent.

• If this redundancy could be exploited, significant data


compression can be achieved.

122nd frame 123 rd frame 124th frame

Fig. 4: Foreman Sequence


• We can observe that the redundancies are not only present
within a frame (spatial redundancy), but also between
successive frames (temporal redundancy).

• We can see that the successive frames are very similar to each
other.

• Although the data to be handled is much more in video, there


is a scope to exploit both spatial and temporal redundancy.

• However, we should not think that we can perform data


compression to whatever extent possible.

• Data compression techniques that achieve significant


compression are irreversible.
• These therefore lead to loss of data and quality degradation.

• There is always a trade-off that is present between compression,


i.e., bandwidth reduction and quality.

Real-time processing requirements

• Irrespective of techniques adopted for data compression,


significant amount of processing will be involved.

• If the processing time is high, the advantage of data


compression may be lost.

• In our still image example, if we are able to achieve a


compression of 20:1, the entire image can be transmitted in one
minute.
• If the processing time to achieve the compression had been in
the order of minutes, the advantage of compression will be
lost.

• Challenges are much more in the case of video.

• Video frames are captured at a rate of 30 frames per second


and this leaves a time of 33 milliseconds between successive
frames.

• Hence, video compression and whatever additional processing


are required, needs to be completed within one frame time.

• Development of multimedia processing hardware with real-


time capability is a highly challenging.
Inter-media synchronization

• Media streams are available from different and independent


sources and are asynchronous with respect to each other.

• Lack of lip-synchronization is a commonly observed problem


in multimedia systems involving audio and video.

• Lack of synchronization between the different media may


even defeat the purpose of multimedia presentation.

• Multimedia standards have addressed this problem and


adopted time-stamping to ensure synchronization.
• Time-stamps, with respect to a system clock reference (SCR)
are appended with the different media packets before
integrating the individual streams into a multimedia one.

• At the receiver end, the individual media streams are decoded


and presented to the playback units in accordance with the
time-stamps obtained from the pack and packet headers.

Intra-media continuity
• The extent of data compression with acceptable reconstruction
quality is highly data-dependent.

• Wherever redundancy is more, high compression ratios are


achievable.
• For example each frame in a video sequence will undergo
different extent of compression.

• This in turn will vary the bit rate from one frame to the other.

• A channel that supports constant bit rate, accommodating a


variable bit rate source will be a challenging task.

• This is achieved by providing a buffer before the bit-stream


generation.

• The buffer may be filled up at variable rate, but emptied


during transmission at a constant rate.
• One must therefore ensure that at no instant should the buffer
be completely emptied and the channel starves for data.

• On the other hand, at no instant should the buffer be


completely filled up and further incoming data is lost.

• In both these extreme situations of buffer overflow and


underflow, the continuity is lost during presentation.

• Inter-media continuity may happen because of inadequate


processing speed, packet loss, channel errors and many other
conditions.
End-to-end delays and Delay jitters

• In a multimedia conferencing, if the users receive the


multimedia contents after considerable delays the interactivity
will be lost.

• The multimedia standards available till date have addressed


this problem and specified what is acceptable.

Multimedia Indexing and Retrieval

• There is a requirement to store large number of multimedia


files since the storage capacity increases in recent times.
• Unless these files are properly indexed, retrieval of desired
multimedia file becomes a tough task, in view of the search
complexities.

• If the multimedia files are based on their contents and then a


content-based query system is developed, efficient retrieval
can be obtained.

• For quick browsing of multimedia files, video summaries are


needed and automated generation of video summaries is a very
challenging task.

You might also like