UNIT-V

Introductory Concepts:
Multimedia Definition

Multimedia is an interactive media and provides multiple ways to represent information

to the user in a powerful manner. It provides an interaction between users and digital
information. It is a medium of communication.Some of the sectors where multimedias
is used extensively are education, training, reference material, business presentations,
advertising and documentaries.

Definition of Multimedia

By definition Multimedia is a representation of information in an attractive and

interactive manner with the use of a combination of text, audio, video, graphics and
animation. In other words we can say that Multimedia is a computerized method of
presenting information combining textual data, audio, visuals (video), graphics and
animations. For examples: E-Mail, Yahoo Messenger, Video Conferencing, and
Multimedia Message Service (MMS).
Multimedia as name suggests is the combination of Multi and Media that is many types
of media (hardware/software) used for communication of information.

150
Components of Multimedia

Following are the common components of multimedia:

Text- All multimedia productions contain some amount of text. The text can have
various types of fonts and sizes to suit the profession presentation of the
multimedia software.
Graphics- Graphics make the multimedia application attractive. In many cases
people do not like reading large amount of textual matter on the screen.
Therefore, graphics are used more often than text to explain a concept, present
background information etc. There are two types of Graphics:
o Bitmap images- Bitmap images are real images that can be captured
from devices such as digital cameras or scanners. Generally bitmap
images are not editable. Bitmap images require a large amount of
memory.
o Vector Graphics- Vector graphics are drawn on the computer and only
require a small amount of memory. These graphics are editable.
Audio- A multimedia application may require the use of speech, music and
sound effects. These are called audio or sound element of multimedia.Speech is
also a perfect way for teaching. Audio are of analog and digital types. Analog
audio or sound refers to the original sound signal. Computer stores the sound in
digital form. Therefore, the sound used in multimedia application is digital audio.

151
 Video- The term video refers to the moving picture, accompanied by sound such
as a picture in television. Video element of multimedia application gives a lot of
information in small duration of time. Digital video is useful in multimedia
application for showing real life objects. Video have highest performance
demand on the computer memory and on the bandwidth if placed on the
internet. Digital video files can be stored like any other files in the computer and
the quality of the video can still be maintained. The digital video files can be
transferred within a computer network. The digital video clips can be edited
easily.
Animation- Animation is a process of making a static image look like it is
moving. An animation is just a continuous series of still images that are
displayed in a sequence. The animation can be used effectively for attracting
attention. Animation also makes a presentation light and attractive. Animation is
very popular in multimedia application

Applications of Multimedia

Following are the common areas of applications of multimedia.

Multimedia in Business- Multimedia can be used in many applications in a
business. The multimedia technology along with communication technology has
opened the door for information of global wok groups. Today the team members
may be working anywhere and can work for various companies. Thus the work
place will become global. The multimedia network should support the following
facilities:
o Voice Mail
o Electronic Mail
o Multimedia based FAX
o Office Needs
o Employee Training
o Sales and Other types of Group Presentation
o Records Management
Multimedia in Marketing and Advertising- By using multimedia marketing of
new products can be greatly enhanced. Multimedia boost communication on an
affordable cost opened the way for the marketing and advertising personnel.
Presentation that have flying banners, video transitions, animations, and sound
effects are some of the elements used in composing a multimedia based

152
advertisement to appeal to the consumer in a way never used before and
promote the sale of the products.
Multimedia in Entertainment- By using multimedia marketing of new products
can be greatly enhanced. Multimedia boost communication on an affordable
cost opened the way for the marketing and advertising personnel. Presentation
that have flying banners, video transitions, animations, and sound effects are
some of the elements used in composing a multimedia based advertisement to
appeal to the consumer in a way never used before and promote the sale of the
products.
Multimedia in Education- Many computer games with focus on education are
now available. Consider an example of an educational game which plays
various rhymes for kids. The child can paint the pictures, increase reduce size of
various objects etc apart from just playing the rhymes.Several other multimedia
packages are available in the market which provide a lot of detailed information
and playing capabilities to kids.
Multimedia in Bank- Bank is another public place where multimedia is finding
more and more application in recent times. People go to bank to open
saving/current accounts, deposit funds, withdraw money, know various financial
schemes of the bank, obtain loans etc. Every bank has a lot of information which
it wants to impart to in customers. For this purpose, it can use multimedia in
many ways. Bank also displays information about its various schemes on a PC
monitor placed in the rest area for customers. Today on-line and internet
banking have become very popular. These use multimedia extensively.
Multimedia is thus helping banks give service to their customers and also in
educating them about banks attractive finance schemes.
Multimedia in Hospital- Multimedia best use in hospitals is for real time
monitoring of conditions of patients in critical illness or accident. The conditions
are displayed continuously on a computer screen and can alert the doctor/nurse
on duty if any changes are observed on the screen. Multimedia makes it
possible to consult a surgeon or an expert who can watch an ongoing surgery
line on his PC monitor and give online advice at any crucial juncture.
In hospitals multimedia can also be used to diagnose an illness with CD-ROMs/
Cassettes/ DVDs full of multimedia based information about various diseases
and their treatment.Some hospitals extensively use multimedia presentations in
training their junior staff of doctors and nurses. Multimedia displays are now
extensively used during critical surgeries.
Multimedia Pedagogues- Pedagogues are useful teaching aids only if they
stimulate and motivate the students. The audio-visual support to a pedagogue

153
 can actually help in doing so. A multimedia tutor can provide multiple numbers
of challenges to the student to stimulate his interest in a topic. The instruction
provided by pedagogue have moved beyond providing only button level control
to intelligent simulations, dynamic creation of links, composition and
collaboration and system testing of the user interactions.
Communication Technology and Multimedia Services- The advancement of
high computing abilities, communication ways and relevant standards has
started the beginning of an era where you will be provided with multimedia
facilities at home. These services may include:
o Basic Television Services
o Interactive entertainment
o Digital Audio
o Video on demand
o Home shopping
o Financial Transactions
o Interactive multiplayer or single player games
o Digital multimedia libraries
o E-Newspapers, e-magazines

CD-ROM and the multimedia highway

Multimedia require large amount t of digital memory when stores in an end users library,
or large amounts of bandwidth when distributed over wires, glass Fiber, or airwaves on
network. The greater the bandwidth, the bigger the "pipeline", so more content can be
delivered to end users quickly.

154
CD-ROM, DVD and Multimedia:

CD-ROM (compact disc read-only memory, has become the most cost-effective
distribution medium for multimedia projects: a CD-ROM disc can be mass-produced for
pennies and can contain up to 80 minutes of full-screen video or sound. Or it can come
rain unique mixes of images, sound, text, video and animations controlled by an
authoring system to provide ultimates user interaction.

Discs can be stamped out of poly-carbonate plastic as fast as cookies on a baker's

production line and just as cheaply. Virtually all personal computers sold today include a
least a CD-ROM player, and the software that drives these computers is commonly
available on a CD-ROM disc applications that required inserting as many as 16 or more
floppy disk one after another are now installed from a CD-ROM without muss or fuss.

155
Many systems now come with a DVD-ROM player, Multilayered Digital Versatile Disk
(DVD) technology increases the capacity and multimedia capability or current optical
technology to 18 GB. CD and DVD burners are used for reading discs and for making
them, too, in audio, video, and data formats. DVD authoring and integration software
allows the creation of interactive front-end menus for films and games.

In the very long term, however, CD-ROM and DVD discs are but interim memory
technologies that will be replaced by new devices that do not require moving parts. As
the data highway described below becomes more and more pervasive and users
become better "connected", copper wire, glass Fiber, and radio/cellular technologies
may prevail as the most common delivery means for interactive multimedia files, served
across the broadband internet or from dedicated computer farms and storage facilities.

The Multimedia Highway:

Now, that telecommunications networks are global, and when information provides and
content owners determines the worth of theirs products and how to charge money for
them, information elements will ultimately link up online's as distributed resources on a
data highway (actually more like a toll road). Where you will pay to acquire and use
multimedia-based information.

Curiously, the actual glass Fiber cables that makes up much of the physical backbone
of the data highway are, in many cases. Owned by railroad and pipelines companies
who simply buried the cables on existing rights of way where no special permits and
156
environmental reports are necessary. One railroad in the United States invested more
than a million dollars in a special cable laying trenching car; in the United Kingdom,
there is talk of placing a fiber-optic cables backbone along the decaying 19th century
canal and barge system. Bandwidth on these lines is leased to other, so competing
retailers such as AT&T, MCI, and Sprint may even share the same cable. Full-text
content from books and magazines is accessible by modem and electronic link; features
movies are played at home; real-time new reports from anywhere on earth are
available; lectures from participating universities are monitored for education credits;
street maps of any city are view-able with recommendations for restaurants, in any
language-and online travelogues include testimonials and video tracks. This is not
science fiction; it is happing now. For each of these interfaces or gateways to
information is a Multimedia projects just waiting to be developed.

Computer Animation (Design, types of animation, using different functions) Uses

of Multimedia

Animation

Animation refers to the movement on the screen of the display device created by
displaying a sequence of still images. Animation is the technique of designing, drawing,
making layouts and preparation of photographic series which are integrated into the
multimedia and gaming products. Animation connects the exploitation and management
of still images to generate the illusion of movement. A person who creates animations is
called animator. He/she use various computer technologies to capture the pictures and
then to animate these in the desired sequence.

Animation includes all the visual changes on the screen of display devices. These are:

1. Change of shape as shown in fig:

157
2. Change in size as shown in fig:

3. Change in color as shown in fig:

4. Change in structure as shown in fig:

158
5. Change in angle as shown in fig:

Animation means giving life to any object in computer graphics. It has the power of
injecting energy and emotions into the most seemingly inanimate objects. Computer-
assisted animation and computer-generated animation are two categories of computer
animation. It can be presented via film or video.
The basic idea behind animation is to play back the recorded images at the rates fast
enough to fool the human eye into interpreting them as continuous motion. Animation
can make a series of dead images come alive. Animation can be used in many areas
like entertainment, computer aided-design, scientific visualization, training, education,
e-commerce, and computer art.

Animation Techniques

Animators have invented and used a variety of different animation techniques.

Basically there are six animation technique which we would discuss one by one in this
section.
Traditional Animation framebyframe
Traditionally most of the animation was done by hand. All the frames in an animation
had to be drawn by hand. Since each second of animation requires 24 frames filmfilm,
the amount of efforts required to create even the shortest of movies can be
tremendous.

Keyframing
In this technique, a storyboard is laid out and then the artists draw the major frames of
the animation. Major frames are the ones in which prominent changes take place. They
are the key points of animation. Keyframing requires that the animator specifies critical
or key positions for the objects. The computer then automatically fills in the missing
frames by smoothly interpolating between those positions.

Procedural

not by keyframing. The animator specifies rules and initial conditions and runs
simulation. Rules are often based on physical rules of the real world expressed by
mathematical equations.

Behavioral
In behavioral animation, an autonomous character determines its own actions, at least
to a certain extent. This gives the character some ability to improvise, and frees the
animator from the need to specify each detail of every character's motion.

159
Performance Based MotionCaptureMotionCapture
Another technique is Motion Capture, in which magnetic or vision-based sensors
record the actions of a human or animal object in three dimensions. A computer then
uses these data to animate the object.
This technology has enabled a number of famous athletes to supply the actions for
characters in sports video games. Motion capture is pretty popular with the animators
mainly because some of the commonplace human actions can be captured with
relative ease. However, there can be serious discrepancies between the shapes or
dimensions of the subject and the graphical character and this may lead to problems of
exact execution.
Physically Based DynamicsDynamics
Unlike key framing and motion picture, simulation uses the laws of physics to generate
motion of pictures and other objects. Simulations can be easily used to produce slightly
different sequences while maintaining physical realism. Secondly, real-time simulations
allow a higher degree of interactivity where the real person can maneuver the actions
of the simulated character.
In contrast the applications based on key-framing and motion select and modify
motions form a pre-computed library of motions. One drawback that simulation suffers
from is the expertise and time required to handcraft the appropriate controls systems.

Key Framing

A keyframe is a frame where we define changes in animation. Every frame is a

keyframe when we create frame by frame animation. When someone creates a 3D

object on every single frame. They create keyframes.

Keyframes are important frames during which an object changes its size, direction,
shape or other properties. The computer then figures out all the in-between frames and
saves an extreme amount of time for the animator. The following illustrations depict the
frames drawn by user and the frames generated by computer.

160
Morphing

The transformation of object shapes from one form to another form is called morphing.
It is one of the most complicated transformations.

161
 A morph looks as if two images melt into each other with a very fluid motion. In
technical terms, two images are distorted and a fade occurs between them.

Types Of Animaiton

In this post we listed 20 different types of animation techniques and styles. Animation is
the process of creating the illusion of motion and shape change by means of the rapid
display of a sequence of static images that minimally differ from each other. Animation
is all around us, be it your favourite tv commercials, music, movies or even videos you
can see the stop motion animation type. Movement creation techniques incorporate the
conventional traditional animation and stop motion animation techniques of two and
three-dimensional figures, for example, paper set patterns, puppets and clay figures.
Keeping Stop motion as the base of all animation, different styles of animation
techniques can be used to create the animated sequences. In this post we included 20
different types of animaiton and animtion styles,

1. Traditional animation
2. 2D animation
3. 3D animation

Traditional animation

Traditional animation involved animators drawing by hand for each and every frame. If
you love the feel of pencils on a paper, then the traditional approach is very fascinating.
Traditional animation is creating the drawings one by one on the frame. 2D animation
involves creating numerous drawings then feeding into a plastic cells, hand painting them
and create the animated sequence on a painted background image.
Traditional Aniamtion Movies : Snow White and the Seven Dwarfs, Peter Pan, and
Sleeping Beauty, Aladdin

162
Computer animation - 2D, 3D: The famous Mickey Mouse animation was created
using the 2d animation technique.The first 2D animation was called Fantasmagorie,
it's a short cartoon made by Emile Cohl. It's shot entirely in black and white, the
cartoon is all about a simple stick man in live action. The cartoon is 75 seconds long
and it took about 700 different drawings to create. This historic animation was
released in 1908. During the 1960s many popular cartoons like the Jetsons and the
Flintstones were created using 2d animation.

163
2D animation

Creating animations in the 2 dimensional space with the help of digital technologies is known as

Create 100s of drawing and animating them to show some kind of movement is technically
known as digital 2d animation. Using Adobe flash, animators can limit the number of drawings
used, which makes them easier to create digital 2d animation. Small variations like changing the
color or frame rate can be changed almost instantly, thus making it easier for the animators to
work on.

164
 Simple 2D walk cyle : Watch Video

3D animation

If you are interested in making the unreal characters into a realistic one, then
3d animation. Digital 3d animation characters are much faster to create and they are
quite popular in the movie making industry. Using a computer software 3d animated
images are used to create many short films, full length movies and even tv commercials
and a career in digital 3d animation is highly rewarding. Comparing to 2D animation and
the traditional approach, 3d animation models are highly realistic.

Introduction to making multimedia The stage of Project

Multimedia is an interactive media and provides multiple ways to represent information

to the user in a powerful manner. It provides an interaction between users and digital
information. It is a medium of communication.Some of the sectors where multimedias
is used extensively are education, training, reference material, business presentations,
advertising and documentaries.

Definition of Multimedia

By definition Multimedia is a representation of information in an attractive and

interactive manner with the use of a combination of text, audio, video, graphics and
animation. In other words we can say that Multimedia is a computerized method of
presenting information combining textual data, audio, visuals (video), graphics and
animations. For examples: E-Mail, Yahoo Messenger, Video Conferencing, and
Multimedia Message Service (MMS).
Multimedia as name suggests is the combination of Multi and Media that is many types
of media (hardware/software) used for communication of information.
165
Components of Multimedia

Following are the common components of multimedia:

Text- All multimedia productions contain some amount of text. The text can have
various types of fonts and sizes to suit the profession presentation of the
multimedia software.

166
 Graphics- Graphics make the multimedia application attractive. In many cases
people do not like reading large amount of textual matter on the screen.
Therefore, graphics are used more often than text to explain a concept, present
background information etc. There are two types of Graphics:
o Bitmap images- Bitmap images are real images that can be captured
from devices such as digital cameras or scanners. Generally bitmap
images are not editable. Bitmap images require a large amount of
memory.
o Vector Graphics- Vector graphics are drawn on the computer and only
require a small amount of memory. These graphics are editable.
Audio- A multimedia application may require the use of speech, music and
sound effects. These are called audio or sound element of multimedia.Speech is
also a perfect way for teaching. Audio are of analog and digital types. Analog
audio or sound refers to the original sound signal. Computer stores the sound in
digital form. Therefore, the sound used in multimedia application is digital audio.
Video- The term video refers to the moving picture, accompanied by sound such
as a picture in television. Video element of multimedia application gives a lot of
information in small duration of time. Digital video is useful in multimedia
application for showing real life objects. Video have highest performance
demand on the computer memory and on the bandwidth if placed on the
internet. Digital video files can be stored like any other files in the computer and
the quality of the video can still be maintained. The digital video files can be
transferred within a computer network. The digital video clips can be edited
easily.
Animation- Animation is a process of making a static image look like it is
moving. An animation is just a continuous series of still images that are
displayed in a sequence. The animation can be used effectively for attracting
attention. Animation also makes a presentation light and attractive. Animation is
very popular in multimedia application

Applications of Multimedia

Following are the common areas of applications of multimedia.

Multimedia in Business- Multimedia can be used in many applications in a
business. The multimedia technology along with communication technology has
opened the door for information of global wok groups. Today the team members
may be working anywhere and can work for various companies. Thus the work

167
place will become global. The multimedia network should support the following
facilities:
o Voice Mail
o Electronic Mail
o Multimedia based FAX
o Office Needs
o Employee Training
o Sales and Other types of Group Presentation
o Records Management
Multimedia in Marketing and Advertising- By using multimedia marketing of
new products can be greatly enhanced. Multimedia boost communication on an
affordable cost opened the way for the marketing and advertising personnel.
Presentation that have flying banners, video transitions, animations, and sound
effects are some of the elements used in composing a multimedia based
advertisement to appeal to the consumer in a way never used before and
promote the sale of the products.
Multimedia in Entertainment- By using multimedia marketing of new products
can be greatly enhanced. Multimedia boost communication on an affordable
cost opened the way for the marketing and advertising personnel. Presentation
that have flying banners, video transitions, animations, and sound effects are
some of the elements used in composing a multimedia based advertisement to
appeal to the consumer in a way never used before and promote the sale of the
products.
Multimedia in Education- Many computer games with focus on education are
now available. Consider an example of an educational game which plays
various rhymes for kids. The child can paint the pictures, increase reduce size of
various objects etc apart from just playing the rhymes.Several other multimedia
packages are available in the market which provide a lot of detailed information
and playing capabilities to kids.
Multimedia in Bank- Bank is another public place where multimedia is finding
more and more application in recent times. People go to bank to open
saving/current accounts, deposit funds, withdraw money, know various financial
schemes of the bank, obtain loans etc. Every bank has a lot of information which
it wants to impart to in customers. For this purpose, it can use multimedia in
many ways. Bank also displays information about its various schemes on a PC

168
monitor placed in the rest area for customers. Today on-line and internet
banking have become very popular. These use multimedia extensively.
Multimedia is thus helping banks give service to their customers and also in
educating them about banks attractive finance schemes.
Multimedia in Hospital- Multimedia best use in hospitals is for real time
monitoring of conditions of patients in critical illness or accident. The conditions
are displayed continuously on a computer screen and can alert the doctor/nurse
on duty if any changes are observed on the screen. Multimedia makes it
possible to consult a surgeon or an expert who can watch an ongoing surgery
line on his PC monitor and give online advice at any crucial juncture.
In hospitals multimedia can also be used to diagnose an illness with CD-ROMs/
Cassettes/ DVDs full of multimedia based information about various diseases
and their treatment.Some hospitals extensively use multimedia presentations in
training their junior staff of doctors and nurses. Multimedia displays are now
extensively used during critical surgeries.
Multimedia Pedagogues- Pedagogues are useful teaching aids only if they
stimulate and motivate the students. The audio-visual support to a pedagogue
can actually help in doing so. A multimedia tutor can provide multiple numbers
of challenges to the student to stimulate his interest in a topic. The instruction
provided by pedagogue have moved beyond providing only button level control
to intelligent simulations, dynamic creation of links, composition and
collaboration and system testing of the user interactions.
Communication Technology and Multimedia Services- The advancement of
high computing abilities, communication ways and relevant standards has
started the beginning of an era where you will be provided with multimedia
facilities at home. These services may include:
o Basic Television Services
o Interactive entertainment
o Digital Audio
o Video on demand
o Home shopping
o Financial Transactions
o Interactive multiplayer or single player games
o Digital multimedia libraries
o E-Newspapers, e-magazines
169
hardware & software requirements to make good multimedia skills and Training
opportunities in Multimedia Motivation for Multimedia usage

The computer technology that allows us to develop three-dimensional virtual

environments (VEs) consists of both hardware and software. The current popular,
technical, and scientific interest in VEs is inspired, in large part, by the advent and
availability of increasingly powerful and affordable visually oriented, interactive,
graphical display systems and techniques. Graphical image generation and display
capabilities that were not previously widely available are now found on the desktops of
many professionals and are finding their way into the home. The greater affordability
and availability of these systems, coupled with more capable, single-person-oriented
viewing and control devices (e.g., head-mounted displays and hand-controllers) and an
increased orientation toward real-time interaction, have made these systems both more
capable of being individualized and more appealing to individuals.

Limiting VE technology to primarily visual interactions, however, simply defines the

technology as a more personal and affordable variant of classical military and
commercial graphical simulation technology. A much more interesting, and potentially
useful, way to view VEs is as a significant subset of multimodal user interfaces.
Multimodal user interfaces are simply human-machine interfaces that actively or
purposefully use interaction and display techniques in multiple sensory modalities (e.g.,
visual, haptic, and auditory). In this sense, VEs can be viewed as multimodal user
interfaces that are interactive and spatially oriented. The human-machine interface
hardware that includes visual and auditory displays as well as tracking and haptic
interface devices is covered in Chapters

In this chapter, we focus on the computer technology for the generation of VEs. One
possible organization of the computer technology for VEs is to decompose it into
functional blocks. In three distinct classes of blocks are shown: (1) rendering hardware
and software for driving modality-specific display devices; (2) hardware and software for
modality-specific aspects of models and the generation of corresponding display
representations; (3) the core hardware and software in which modality-independent
aspects of models as well as consistency and registration among multimodal models
are taken into consideration. Beginning from left to right, human sensorimotor systems,
such as eyes, ears, touch, and speech, are connected to the computer through human-
machine interface devices. These devices generate output to, or receive input from, the
human as a function of sensory modal drivers or renderers. The auditory display driver,
for example, generates an appropriate waveform based on an acoustic simulation of the
VE. To generate the sensory output, a computer must simulate the VE for that particular
sensory mode. For example, a haptic display may require a physical simulation that
includes compliance and texture. An acoustic display may require sound models based
on impact, vibration, friction, fluid flow, etc. Each sensory modality requires a simulation
170
tailored to its particular output. Next, a unified representation is necessary to coordinate
individual sensory models and to synchronize output for each sensory driver. This
representation must account for all human participants in the VE, as well as all
autonomous internal entities. Finally, gathered and computed information must be
summarized and broadcast over the network in order to maintain a consistent
distributed simulated environment.

To date much of the design emphasis in VE systems has been dictated by the
constraints imposed by generating the visual scene. The nonvisual modalities have
been relegated to special-purpose peripheral devices. Similarly, this chapter is primarily
concerned with the visual domain, and material on other modalities can be found in
Chapters 3-7. However, many of the issues involved in the modeling and generation of
acoustic and haptic images are similar to the visual domain; the implementation
requirements for interacting, navigating, and communicating in a virtual world are
common to all modalities. Such multimodal issues will no doubt tend to be merged into
a more unitary computational system as the technology advances over time.

In this section, we focus on the computer technology for the generation of VEs. The
computer hardware used to develop three-dimensional VEs includes high-performance
workstations with special components for multisensory displays, parallel processors for
the rapid computation of world models, and high-speed computer networks for
transferring information among participants in the VE. The implementation of the virtual
world is accomplished with software for interaction, navigation, modeling (geometric,
physical, and behavioral), communication, and hypermedia integration. Control devices
and head-mounted displays are covered elsewhere in this report.

VE requires high frame rates and fast response because of its inherently interactive
nature. The concept of frame rate comes from motion picture technology. In a motion
picture presentation, each frame is really a still photograph. If a new photograph
replaces the older images in quick succession, the illusion of motion in engendered. The
update rate is defined to be the rate at which display changes are made and shown on
the screen. In keeping with the original motion picture technology, the ideal update rate
is 20 frames (new pictures) per second or higher. The minimum acceptable rate for VE
is lower, reflecting the trade-offs between cost and such tolerances. With regard to
computer hardware, there are several senses of frame rate: they are roughly classified
as graphical, computational, and data access. Graphical frame rates are critical in order
to sustain the illusion of presence or immersion in a VE. Note that these frame rates
may be independent: the graphical scene may change without a new computation and
data access due to the motion of the user's point of view. Experience has shown that,
whereas the graphical frame rate should be as high as possible, frame rates of lower
than 10 frames per second severely degrade the illusion of presence. If the graphics
being displayed relies on computation or data access, then computation and data
171
access frame rates of 8 to 10 frames per second are necessary to sustain the visual
illusion that the user is watching the time evolution of the VE.

Fast response times are required if the application allows interactive control. It is well
known (Sheridan and Ferrell, 1974) that long response times (also called lag or pure
delay) severely degrade user performance. These delays arise in the computer system
from such factors as data access time, computation time, and rendering time, as well as
from delays in processing data from the input devices. As in the case of frame rates, the
sources of delay are classified into data access, computation, and graphical categories.
Although delays are clearly related to frame rates, they are not the same: a system may
have a high frame rate, but the image being displayed or the computational result being
presented may be several frames old. Research has shown that delays of longer than a
few milliseconds can measurably impact user performance, whereas delays of longer
than a tenth of a second can have a severe impact. The frame rate and delay required
to create a measurable impact will in general depend on the nature of the environment.
Relatively static environments with slowly moving objects are usable with frame rates as
low as 8 to 10 per s and delays of up to 0.1 s. Environments with objects exhibiting high
frequencies of motion (such as a virtual handball game) will require very high frame
rates (> 60 Hz) and very short delays. In al
alll cases, however, if the frame rate falls below
8 frames per s, the sense of an animated three-dimensional environment begins to fail,
and if delays become greater than 0.1 s, manipulation of the environment becomes very
difficult. We summarize these results to the following constraints on the performance of
a VE system:

Frame rates must be greater than 8 to 10 frames/s.

Total delay must be less than 0.1 s.

Both the graphics animation and the reaction of the environment to user actions require
extensive data management, computation, graphics, and network resources. All
operations that take place to support the environment must operate within the above
time constraints. Although one can imagine a system that would have the graphics,
computation, and communications capability to handle all environments, such a system
is beyond current technology. For a long time to come, the technology necessary will
generally be dependent on the application domain for which the VE is being built. Real-
world simulation applications will be highly bound by the graphics and network protocols
and by consistency issues; information visualization and scientific visualization
applications will be bound by the computational performance and will involve issues of
massive data management (Bryson and Levit, 1992; Ellis et al., 1991). Some
applications, such as architectural visualization, will require photorealistic rendering;
others, such as information display, will not. Thus the particular hardware and software
required for VE implementation will depend on the application domain targeted. There

172
are some commonalities of hardware and software requirements, and it is those
commonalities on which we focus in our examination of the state of the art of computer
hardware and software for the construction of real-time, three-dimensional virtual
environments.

HARDWARE FOR COMPUTER GRAPHICS

The ubiquity of computer graphics workstations capable of real-time, three-dimensional

display at high frame rates is probably the key development behind the current push for
VEs today. We have had flight simulators with significant graphics capability for years,
but they have been expensive and not widely available. Even worse, they have not
been readily programmable. Flight simulators are generally constructed with a specific
purpose in mind, such as providing training for a particular military plane. Such
simulators are microcoded and programmed in assembly language to reduce the total
number of graphics and central processing unit cycles required. Systems programmed
in this manner are difficult to change and maintain. Hardware upgrades for such
systems are usually major undertakings with a small customer base. An even larger
problem is that the software and hardware developed for such systems are generally
proprietary, thus limiting the availability of the technology. The graphics workstation in
the last 5 years has begun to supplant the special-purpose hardware of the flight
simulator, and it has provided an entry pathway to the large numbers of people
interested in developing three-dimensional VEs. The following section is a survey of
computer graphics workstations and graphics hardware that are part of the VE
development effort.

Notable Graphics Workstations and Graphics Hardware

Graphics performance is difficult to measure because of the widely varying complexity

of visual scenes and the different hardware and software approaches to computing and
displaying visual imagery. The most straightforward measure is given in terms of
polygons/second, but this only gives a crude indication of the scene complexity that can
be displayed at useful interactive update rates. Polygons are the most common building
blocks for creating a graphic image. It has been said that visual reality is 80 million
polygons per picture (Catmull et al., 1984). If we wish photorealistic VEs at 10 frames/s,
this translates into 800 million polygons/s. There is no current graphics hardware that
provides this, so we must make approximations at the moment. This means living with
less detailed virtual worlds, perhaps via judicious use of hierarchical data structures
(see the software section below) or off-loading some of the graphics requirements by
utilizing available CPU resources instead.

For the foreseeable future, multiple processor workstations will be playing a role in off-
loading graphics processing. Moreover, the world modeling components, the

173
communications components, and the other software components for creating virtual
worlds also require significant CPU capacity. While we focus on graphics initially, it is
important to note that it is the way world modeling effects picture change that is of
ultimate importance.

Graphics Architectures for VE Rendering

This section describes the high-level computer architecture issues that determine the
applicability of a graphics system to VE rendering. Two assumptions are made about
the systems included in our discussion. First, they use a z-buffer (or depth buffer), for
hidden surface elimination. A z-buffer stores the depth or distance from the eye
point of the closest surface ''seen" at that pixel. When a new surface is scan
converted, the depth at each pixel is computed. If the new depth at a given pixel is
closer to the eye point than the depth currently stored in the z-buffer at that pixel, then
the new depth and intensity information are written into both the z-buffer and the frame
buffer. Otherwise, the new information is discarded and the next pixel is examined. In
this way, nearer objects always overwrite more distant objects, and when every object
has been scan converted, all surfaces have been correctly ordered in depth. The
second assumption for these graphic systems is that they use an application-
programmable, general-purpose processor to cull the database. The result is to provide
the rendering hardware with only the graphics primitives that are within the viewing
volume (a perspective pyramid or parallel piped for perspective and parallel projections
respectively). Both of these assumptions are valid for commercial graphics workstations
and for the systems that have been designed by researchers at the University of North
Carolina at Chapel Hill.

174