Still Images

Types of Still Images

Still images are generated in two ways:
– Bitmaps (or raster-based) .
– Vector-drawn graphics.
 Bitmaps
• Bitmap is derived from the words ‘bit’, which
means the simplest element in which only two
digits are used, and ‘map’, which is a two-
dimensional matrix of these bits.

• A bitmap is a data matrix describing the

individual dots of an image that are the
smallest elements (pixels) of resolution on a
computer screen or printer.
• Example
 Bitmaps
• Bitmaps are an image format suited for
creation of:
– Photo-realistic images.
– Complex drawings.
– Images that require fine detail.

• Bitmapped images are known as paint

graphics.
• Bitmapped images can have varying bit and
color depths.
 Bitmaps
• More bits provide more color depth, hence more photo-realism;
• but require more memory and processing power

Available binary Combinations for

Describing a Color
• Monochrome just requires one bit per pixel,
representing black or white

BMP – 16 KB
• 8 bits per pixel allows 256 distinct colors

BMP – 119KB
• 16 bits per pixel represents 32K distinct colors (Most graphic
chipsets now supports the full 65536 colors and the color
green uses the extra one bit)

BMP – 234 KB
• 24 bits per pixel allows millions of colors

• 32 bits per pixel – trillion of colors

BMP – 350KB
• Bitmaps are best for photo-realistic images or
complex drawings requiring fine detail
• Bitmaps picture and their suitability of use:-
– Use the native Microsoft bmp format as a raw image that will later be processed. It
faster to process.
– Use JPEG, for photo sharing on the web because of its size and quality.

– GIF is normally used for diagrams, buttons, etc., that have a small number of colours
• It is also suitable for simple animation because it
supports interlaced images.

– PNG is almost equal to gif except that it didn’t support the animation format.
 Bitmaps
Bitmaps can be inserted by:
– Using clip art galleries. Clip Art
– Using bitmap software.
– Capturing and editing images.
– Scanning images. Drawn

Scan Capture
 Using Clip Art Galleries
• A clip art gallery is an assortment of graphics,
photographs, sound, and video.
• Clip arts are a popular alternative for users
who do not want to create their own images.
• Clip arts are available on CD-ROMs and on the
Internet.
 Using Bitmap Software
The industry standard for bitmap painting and
editing programs are:
• Adobe's Photoshop and Illustrator.
• Macromedia's Fireworks.
• Corel's Painter.
• CorelDraw.
• Quark Express.
 Capturing and Editing Images
• Capturing and storing images directly from the
screen is another way to assemble images for
multimedia.
• The PRINT SCREEN button in Windows and
COMMAND-CONTROL-SHIFT-4 keystroke on
the Macintosh copies the screen image to the
clipboard.
 Capturing and Editing Images
Image editing programs enable the user to:
– Enhance and make composite images.
– Alter and distort images.
– Add and delete elements.
– Morph (manipulate still images to create animated transformations).
 Scanning Images
• Users can scan images from conventional
sources and make necessary alterations and
manipulations.
 Applications of Vector-Drawn Images

• Vector-drawn images - created from geometric objects such as lines, rectangles,

ovals, polygons using mathematical formulas

• Vector-drawn images are used in the following areas:

– Computer-aided design (CAD) programs.

– Graphic artists designing for the print media.

– 3-D animation programs.

– Applications requiring drawing of graphic shapes.

 How Vector-Drawn Images Work
• A vector is a line that is described by the
location of its two endpoints.
• Vector drawing makes use of Cartesian co-
ordinates.
• Cartesian coordinates are numbers that
describe a point Yin two or three-dimensional
space as the intersection of X, Y, and Z axis.
Z

X
• Example
RECT 0,0,200,300,RED,BLUE says
– “Draw a rectangle starting at 0,0 (upper left corner of screen) going 200 pixels
horizontally right and 300 pixels downward, with a RED boundary and filled
with BLUE.”
200 pixel

300
pixel
 Vector-Drawn Images v/s Bitmaps
• Vector images use less memory space and
have a smaller file size as compared to
bitmaps.
• For the Web, pages that use vector graphics in
plug-ins download faster, and when used for
animation, draw faster than bitmaps.
 Vector-Drawn Images v/s Bitmaps
• Vector images cannot be used for
photorealistic images.
• Vector images require a plug-in for Web-based
display.
• Bitmaps are not easily scalable and resizable.
• Bitmaps can be converted to vector images
using autotracing.
 3-D Drawing and Rendering
• 3D graphics tools, such as Macromedia
Extreme3D, or Form-Z, typically extend vector-
drawn graphics in 3 dimensions (x, y and z)

X
Z
y
 3-D Drawing and Rendering
• A 3D scene consist of object that in turn contain many small
elements, such as blocks, cylinders, spheres or cones
(described in terms of vector graphics)
• The more elements, the finer the object’s resolution and
smoothness.
 3-D Drawing and Rendering
• Objects as a whole have properties such as
shape, color, texture, shading & location.
• A 3D application lets you model an object’s
shape, then render it completely.
 Features of a 3-D Application
1. Modeling involves drawing a shape, such as a 2D letter, then extruding it or
lathing it into a third dimension.
– extruding : extending its shape along a defined path
– lathing : rotating a profile of the shape around a defined axis
 Features of a 3-D Application
• Modeling also deals with lighting, setting a
camera view to project shadows
 Features of a 3-D Application
2. Rendering : produces a final output of a
scene and is more compute-intensive.
 3-D Animation Tools
3-D animation, drawing, and rendering tools
include:
– Ray Dream Designer.
– Caligari True Space 2.
– Specular Infini-D.
– Form*Z.
– NewTek's Lightwave.
 Natural Light and Color
• Light comes from an atom where an electron
passes from a higher to a lower energy level.
• Each atom produces uniquely specific colors.
• Color is the frequency of a light wave within
the narrow band of the electromagnetic
spectrum, to which the human eye responds.
 Understanding Natural Light and Color
• The tools we use to describe color are
different when the color is printed than from
when it is projected
– Additive color (projected color).
– Subtractive color (printed color).
– Monitor-specific color.
– Color models.
 Additive Color
• In the additive color method, a color is created
by combining colored light sources in three
primary colors - red, green, and blue (RGB).
• TV and computer monitors use this method.
 Subtractive Color
• In the subtractive color method, color is
created by combining colored media such as
paints or ink.
• The colored media absorb (or subtract) some
parts of the color spectrum of light and reflect
the others back to the eye.
 Subtractive Color
• Subtractive color is the process used to create
color in printing.
• The printed page consists of tiny halftone dots
of three primary colors- cyan, magenta, and
yellow (CMY).
 Monitor-Specific Colors
• Colors should be used according to the target
audience's monitor specifications.
• The preferred monitor resolution is 800x600
pixels.
• The preferred color depth is 32 bits.
 Color Models
• Different ways of representing information
about color.
• Models used to specify color in computer
terms are:
– RGB model - A 24-bit methodology where color is specified in terms of red,
green, and blue values ranging from 0 to 255.
– HSB and HSL models – Color is specified as an angle from 0 to 360 degrees on
a color wheel.
– Other models include CMYK, CIE, YIQ, YUV, and YCC.
 RGB Model
 Add red, green and blue to create colors, so it is an
additive model.
 Assigns an intensity value to each pixel ranging from
0 (black) to 255 (white)
 A bright red color might have R 246, G 20, B 50
 HSB Model
• Based on human perception of color,
describe three fundamental properties of
color:
– Hue
– Saturation (or chroma)
– Brightness - relative lightness or darkness of color, also measured
as %

• There is no HSB mode for creating or editing

images
 HSB Model
• Hue - color reflected from or transmitted through
an object, measured on color wheel
 HSB Model
• Saturation (or chroma) - strength or purity of color
(% of grey in proportion to hue)
 HSB Model
• Brightness - relative lightness or darkness of color, also measured as %

0% 50% 100%

Black white
 CMYK Model
• Based on light-absorbing quality of ink printed on paper
• As light is absorbed, part of the spectrum is absorbed and part
is reflected back to eyes
• Associated with printing; called a subtractive model
• Four channels: Cyan (C ), magenta (M), yellow (Y) and black (K)

 In theory, pure colors should produce

black, but printing inks contain impurities,
so this combination produces muddy
brown
 K is needed to produce pure black, hence
CMYK is four-color process printing
 Color Palettes
• Palettes are mathematical tables that define
the color of pixels displayed on the screen.
• Palettes are called ‘color lookup tables’ or
CLUTs on Macintosh.
• The most common palettes are 1, 4, 8, 16, and
24-bit deep.
 Color Palettes
Dithering:
– Dithering is a process whereby the color value of each pixel is changed to the
closest matching color value in the target palette.
– This is done using a mathematical algorithm.
 Image File Types used in Multimedia
• Macintosh formats.
• Windows formats.
• Cross-platform formats.
 Macintosh Formats

• On the Macintosh, the most commonly used format is PICT.

• PICT is a complicated and versatile format developed by Apple.

• Almost every image application on the Macintosh can import

or export PICT files.

• In a PICT file, both vector-drawn objects and bitmaps can

reside side-by-side.
 Windows Formats
• The most commonly used image file format on
Windows is DIB.
• DIB stands for Device-independent bitmaps.
• The preferred file type for multimedia
developers in Windows is Resource
Interchange File Format (RIFF).
 Windows Formats

Bitmap formats used most often by Windows

developers are:
– BMP - A Windows bitmap file.
• Native bitmap file format of the Microsoft Windows environment

– TIFF - Extensively used in DTP packages.

• Used to exchange documents between different applications and
platforms

– PCX - Used by MS-DOS paint software.

• One of the oldest bitmapped formats popularized by MS-DOS paint
programs that first appeared in the early 1980's
 Cross-Platform Formats
The image file formats that are compatible
across platforms are:
• DXF - Used by CAD applications.
• Initial Graphics Exchange Standard (IGS or IGES) - Standard for transferring
CAD drawings.
• JPEG and GIF - Most commonly used formats on the Web.
 Most Popular Image File Formats

• JPEG (Joint-Photographic Experts Group)

• GIF (Graphical Interchange Format)
• PNG (Portable Network Graphic)
• Other formats:
– BMP, PSD, TIFF/TIF, TGA, EPS, PCX, ICO
 Most Popular Image File Formats

• JPEG
– For continuous tone images, such as full-color
photographs
– Supports more than 16 millions of color (24-bit)
– Uses lossy compression (averaging may lose
information)
 Most Popular Image File Formats

• GIF
– For large areas of the same color and a moderate
level of detail.
– Supports up to 256 colors
– Allows transparency and interlacing
– Uses lossless compression
 Most Popular Image File Formats

• PNG
 lossless, portable, well-compressed storage of
raster images
 patent-free replacement for GIF
 also replace many common uses of TIFF
 Support indexed-color, grayscale, and true color
images + an optional alpha channel for
transparency
Speech Recognition
 Definition
• Speech recognition is the process of converting an
acoustic signal, captured by a microphone or a
telephone, to a set of words.
• The recognised words can be an end in themselves,
as for applications such as commands & control, data
entry, and document preparation.
• They can also serve as the input to further linguistic
processing in order to achieve speech understanding
 Speech Processing
• Signal processing:
– Convert the audio wave into a sequence of feature vectors
• Speech recognition:
– Decode the sequence of feature vectors into a sequence of
words
• Semantic interpretation:
– Determine the meaning of the recognized words
• Dialog Management:
– Correct errors and help get the task done
• Response Generation
– What words to use to maximize user understanding
• Speech synthesis (Text to Speech):
– Generate synthetic speech from a ‘marked-up’ word string
 Dialog Management
• Goal: determine what to accomplish in response to
user utterances, e.g.:
– Answer user question
– Solicit further information
– Confirm/Clarify user utterance
– Notify invalid query
– Notify invalid query and suggest alternative
• Interface between user/language processing
components and system knowledge base
 What you can do with Speech
Recognition
• Transcription
– dictation, information retrieval
• Command and control
– data entry, device control, navigation, call routing
• Information access
– airline schedules, stock quotes, directory
assistance
• Problem solving
– travel planning, logistics
 Transcription and Dictation
• Transcription is transforming a stream of
human speech into computer-readable form
– Medical reports, court proceedings, notes
– Indexing (e.g., broadcasts)
• Dictation is the interactive composition of text
– Report, correspondence, etc.
 Speech recognition and understanding

• Sphinx system
– speaker-independent
– continuous speech
– large vocabulary
• ATIS system
– air travel information retrieval
– context management
 Speech Recognition and Call Centres

• Automate services, lower payroll

• Shorten time on hold
• Shorten agent and client call time
• Reduce fraud
• Improve customer service
 Applications related to Speech
Recognition
• Speech Recognition
• Figure out what a person is saying.
• Speaker Verification
• Authenticate that a person is who she/he claims to
be.
• Limited speech patterns
• Speaker Identification
• Assigns an identity to the voice of an unknown
person.
• Arbitrary speech patterns
Many kinds of Speech Recognition Systems

• Speech recognition systems can be

characterised by many parameters.
• An isolated-word (Discrete) speech
recognition system requires that the speaker
pauses briefly between words, whereas a
continuous speech recognition system does
not.
 Spontaneous V Scripted
• Spontaneous, speech contains disfluencies,
periods of pause and restart, and is much
more difficult to recognise than speech read
from script.
MIDI
 What is MIDI?
• MIDI stands for Musical Instrument Digital Interface

Some Clarification:
• MIDI doesn’t directly describe musical sound
• MIDI is not a language
• It is a data communications protocol
 MIDI Ports
• It use a five-pin DIN connector

– Inexpensive and readily available

– Only 3 pins among 5 are used until now
– Both ends of MIDI line are the same.
 MIDI Ports
• Serial transfer, data are sent bit by bit
Hence:
- transmission rate is slow at only 31,250 bits/sec.
- Too slow to transmit samples in real-time
- have to do off-line sample dump
 MIDI Interface
MIDI In
• MIDI data enters each item of MIDI equipment through the MIDI In port.
MIDI Out
• All the MIDI data generated by individual pieces of equipment are sent out
through the MIDI Out port.
A common error for MIDI setup is: inverted connection of MIDI IN/OUT
MIDI Thru
• These are used to re-transmit all information received at the MIDI In port
using the MIDI Thru port connections.
• Often these ports are used to create a chain of connected devices in a
single MIDI data path, called a 'daisy chain'.
 Limitations of MIDI
1. Slow -- Serial transfer
• When there have too much continuous data
transfer (e.g. a lot of control data) MIDI
choke
– Solution: can be solved by EVENT FILTERING
– e.g., discard less important messages (esp., system
exclusive messages)
 Limitations of MIDI
2. Slow -- MIDI is only control information (like
Csound score), and time is needed to
synthesize the sound
• computation time  MIDI lag
– Solution: users have to avoid using patch
(instrument) which uses a lot of memory
– e.g. Cymbal in channel 10 of Nokia Cellular phone
 Limitations of MIDI
3. Sound quality varies
• It depends on which synthesizer you use
Solution:
– users have to judge by ear, to see which sound is
good
– Standardized with General MIDI (GM) (discussed
later)
 Limitations of MIDI
3. Sound quality varies
• the size of MIDI file is very small!
– e.g. :
– a three minutes wav file, 48kHz, stereo:
• size of 40MB
– a three minutes MIDI file, with 10 channels:
• size of 40kb
• It is because MIDI file doesn’t actually contain audio
data, but only control information (like Csound score)
MIDI Transmission Protocol
MST 1 0 LST

• Each message begin with ONE start

bit (logical 0)
• Then followed by EIGHT message
bits
• End with ONE stop bit (logical 1)
• Each 8-bit MIDI message byte,
specifies either a status value, or
data value
MIDI message types
 General MIDI
• Optional to manufacturer
• But it is a good addendum
to the MIDI 1.0 Detailed Specification

• MIDI itself doesn’t specify message or data

• Program number 1  What does it mean?
– Piano? Flute? It is up to Manufacturer’s decision!
• Program number 3 can be “flute” on
synthesizer A, but can be “horn” on
synthesizer B!
 What is General MIDI
• So, we have GM
• Define a set of available sound patches, with their
program numbers fixed
– Sequence recorded on one GM synthesizer is then
recognizable on other synthesizers.
 General MIDI specification
1. Instrument Patch Map
– a list of 128 sounds, with assigned program numbers
– Loosely grouped into 16 families, each with 8 variations
2. Percussion Key Map
3. Other specification generally follow MIDI 1.0
– 32 simultaneous notes
– MIDI Channels: 16
– 60 = Middle C
 Limitations of GM
1. Dynamics
• How should a note of “pressure 120” on program
number 1 be performed?
• Different samplers use different voice samples
• what if manufacturer A uses a Steinway piano,
manufacturer B uses a Yamaha piano?
• The dynamics can be very different!
 Limitations of GM
2. Instrument definition
• We know what is a “flute”

• But, what is “FX2 (sound track)” ?

?
Applications of MIDI
1. Studio Production
– recording, playback, cut-and-splice editing
– creative control/effect can be added
2. Making score
– with score editing software, MIDI is
excellent in making score
– some MIDI software provide function of
auto accompaniment/intelligent chord
arrangement
3. Learning
– You can write a MIDI orchestra, who are
always eager to practice with you!
Applications of MIDI
4. Commercial products
– mobile phone ring tones, music box
music…..
5. Musical Analysis
– MIDI has detailed parameters for every
input note
– It is useful for doing research
– For example, a pianist can input his
performance with a MIDI keyboard, then
we can analyze his performance style by the
parameters
 Introduction to Multimedia
• Chapter 8
 Objectives

• How multimedia products are created and

authored.
• What kinds of multimedia authoring tools you
have to choose from to suit your project needs.
• How various multimedia authoring tools work.
 What is an Authoring Tools
• The tools by which various media components
are brought together (integrated) into a
structure and flow.
 Features of Authoring Tools
• Most Authoring tool have the following features
in their development interface:
1. A list of media events
2. Iconic flowchart / timeline
3. Card stack
4. A series of figures
 Features of Authoring Tools
1. A list of media events
– Occurrences of scene, images, sound,action in the
end product.
 Features of Authoring Tools
2. Iconic flowchart / timeline
– Controls event sequence.
 Features of Authoring Tools
3. Card stack (tabs)
– Easy viewing if there is too many item on screen.

Card stack
 Features of Authoring Tools
4. A series of figures
– List of images / objects in the project
– Information about objects

figures
 Categories of Authoring Tools
• Can be categorized into
1. Presentation software
2. Tools for creating production
3. Interactive training and education

 Some authoring tools

can fit into more than
one category.
 1. Presentation software
• Enable users to create and deliver business presentations
in the form of the slide show.
• Each major heading is the beginning of a new slide.

heading
 1. Presentation software
• Templates are used to determine how the
heads and subheads are formatted and
displayed over backgrounds, including:
(etc PowerPoint)
– position
– size
– font
– style
– color
 Example: PowerPoint Interface

Style and format

templates
 2. Tools for creating production
• Typically oriented toward producing content that is more ambitious than the
slide-show level
– (more interactive & dynamic)

• Usually integrate all types of' multimedia data into a multitrack timeline that
determines the evolution of events

 Interactivity takes the form of conditional

branching (multiple linking) that can make
navigation decisions based on user input and
other conditions

 Production software creates dynamic content

 EXAMPLE: mTropolis / AppleMedia Tool /
MediaForge
 3. Interactive training and education tools
• To create education or training content.
• Designed primarily to present information in an Interactive
book.
• Multimedia-oriented programming languages such as Visual
Basic represent another authoring alternative.
• Offers the higher flexibility, performance speed and power
• Example: Macromedia Flash, Authorware & Director
Flash Interface
Director Interface
Authorware Interface
 Types of Authoring Tools
1. Card based
2. Icon based
3. Time based
4. Object based
 1. Card Based
• Cards are developed that have different
elements associated with them.
• Cards are put in stack.
• Cards or pages combine to make up a book.
– example of authoring tools
• HyperCard (Mac)
• ToolBook (Mac / Windows)
 2. Icon Based
• Icons are gathered along the line.
• Provide visual development.
• Flow chart is created to show the organisation of
icons or elements:
– including activity list, results and done with dragging the
icon/elements along the lines
– each Icon represents a particular event – button, graphics,
text, video
– examples of authoring tools
• Authorware(Mac/Windows)
• IconAuthor (Windows)
2. Icon Based
 3. Time Based
• The most popular used.
• Using “timeline” for organizing activities
• Also using “framing” – timely adjusted
depending on the frame size
• Example:- Macromedia Director / Flash
(Mac/Windows)
 3. Time Based

Macromedia Director

Macromedia Flash
 4. Object Based
• Support environment based on object.
• Every object is modified using properties & modifiers
• The environment is based on ‘Hierarchy’ (section
and sub-section).
• Examples of the tools :
– mTropolis (Mac/Windows)
– AppleMedia Tool (Mac/Windows)
– MediaForge (Windows)
4. Object Based