Multimedia Authoring and User Interfaces (IT/CS 2nd Sem) Unit 4

Multimedia Authoring Systems

Multimedia authoring systems are designed with two primary target users: They
are
(i) Professionals who prepare documents, audio or sound tracks, and full motion
video clips for wide distribution.
(il) Average business users preparing documents, audio recordings, or full
motion video clips for stored messages' or presentations.
The authoring system covers user interface. The authoring system spans issues
such as data access, storage structures for individual components embedded in a
document, the user's ability to browse through stored objects, and so on.
Most authoring systems are managed by a control application.
Design Issues for Multimedia Authoring
Enterprise wide standards should be set up to ensure that the user requirements
are fulfilled with good quality and made the objects transferable from one
system to another.
So standards must be set for a number of design issues
1. Display resolution
2. Data formula for capturing data
3. Compression algorithms
4. Network interfaces
5. Storage formats.
Display resolution
A number of design issues must be considered for handling different display
outputs. They are:
(a) Level of standardization on display resolutions.
(b) Display protocol standardization.
(c) Corporate norms for service degradations
(d) Corporate norms for network traffic degradations as they relate to resolution
issues Setting norms will be easy if the number of different work station types,

1
Multimedia Authoring and User Interfaces (IT/CS 2nd Sem) Unit 4

window managers, and monitor resolutions are limited in number. But if they
are more in number, setting norms will be difficult.
Another consideration is selecting protocols to use. Because a number of
protocols have emerged, including AVI, Indeo, Quick Time and so on. So, there
should be some level of convergence that allows these three display protocols to
exchange data and allow viewing files in other formats.
Design Approach to Authoring
Designing an authoring system spans a number of design issues. They include:
Hypermedia application design specifics, User Interface aspects,
Embedding/Linking streams of objects to a main document or presentation,
Storage of and access to multimedia objects. Playing back combined streams in
a synchronized manner.
A good user interface design is more important to the success of hypermedia
applications.
Types of Multimedia Authoring Systems
There are varying degrees of complexity among the authoring systems. For
example, dedicated authoring systems that handle only one kind of an object for
a single user is simple; where as programmable systems are most complex.
Dedicated Authority Systems
Dedicated authoring systems are designed for a single user and generally for
single streams.
Designing this type of authoring system is simple, but if it should be capable of
combining even two object streams, it becomes complex. The authoring is
performed on objects captured by the local video camera and image scanner or
an objects stored in some form of multimedia object library. In the case of
dedicated authoring system, users need not to be experts in multimedia or a
professional artist. But the dedicated systems should be designed in such a way
that. It has to provide user interfaces that are extremely intuitive and follow
real-world metaphors.
2
Multimedia Authoring and User Interfaces (IT/CS 2nd Sem) Unit 4

A structured design approach will be useful in isolating the visual and

procedural design components.
TimeLine –based authoring
In a timeline based authoring system, objects are placed along a timeline. The
timeline can be drawn on the screen in a window in a graphic manner, or it
created using a script in a manner similar to a project plan. But, the user must
specify a resource object and position it in the timeline.
On playback, the object starts playing at that point in the time Scale.
In most timeline based approaches, once the multimedia object has been
captured in a timeline, .it is fixed in location and cannot be manipulated easily,
So, a single timeline causes loss of information about the relative time lines for
each individual object.

Structured Multimedia Authoring

A structured multimedia authoring approach was presented by Hardman. It is an
evolutionary approach based on structured object-level construction of complex
presentations. This approach consists of two stages:
(i) The construction of the structure of a presentation.
(ii) Assignment of detailed timing constraints.
3
Multimedia Authoring and User Interfaces (IT/CS 2nd Sem) Unit 4

A successful structured authoring system must provide the following

capabilities for navigating through the structure of presentation.
1. Ability to view the complete structure.
2. Maintain a hierarchy of objects.
3. Capability to zoom down to any specific component.
4. View specific components in part or from start to finish.
5. Provide a running status of percentage full of the designated length of the
presentation.
6. Clearly show the timing relations between the various components.
7. Ability to address all multimedia types including text, image, audio, video
and frame based digital images.
The author must ensure that there is a good fit within each object hierarchy
level. The navigation design of authoring system should allow the author to
view the overall structure while examining a specific object segment more
closely.
Programmable Authoring Systems
Early structured authoring tools were not able to allow the authors to express
automatic function for handling certain routine tasks. But, programmable
authoring system bas improved in providing powerful functions based on image
processing and analysis and embedding program interpreters to use image-
processing functions.
The capability of this authoring system is enhanced by Building user
programmability in the authoring tool to perform the analysis and to manipulate
the stream based on the analysis results and also manipulate the stream based on
the analysis results. The programmability allows the following tasks through the
program interpreter rather than manually. Return the time stamp of the next
frame. Delete a specified movie segment. Copy or cut a specified movie
segment to the clip board. Replace the current segment with clip board contents.

4
Multimedia Authoring and User Interfaces (IT/CS 2nd Sem) Unit 4

Multisource Multi-user Authoring Systems

We can have an object hierarchy in a geographic plane; that is, some objects
may be linked to other objects by position, while others may be independent and
fixed in position".
We need object data, and information on composing it. Composing means
locating it in reference to other objects in time as Well as space. Once the object
is rendered (display of multimedia object on the screen) the author can
manipulate
it and change its rendering information must be available at the same time for
display. If there are no limits on network bandwidth and server performance, it
would be possible to assemble required components on cue at the right time to
be rendered. In addition to the multi-user compositing function A multi user
authoring system must provide resource allocation and scheduling of
multimedia objects.
Telephone Authoring systems
There is an application where the phone is linking into multimedia electronic
mail application
1. Telephone can be used as a reading device by providing fill text to-speech
synthesis capability so that a user on the road can have electronic mail messages
read out on the telephone.
2. The phone can be used for voice command input for setting up and managing
voice mail messages. Digitized voice clips are captured via the phone and
embedded in electronic mail messages.
3. As the capability to recognize continuous speech is deploy phones can be
used to create electronic mail messages where the voice is converted to ASCII
text on the fly by high performance voice recognition engines. Phones provide a
means of using voice where the alternative of text on a screen is not available. A
phone can be used to provide interactive access to electronic mail, calendar
information databases, public information database and news reports, electronic

5
Multimedia Authoring and User Interfaces (IT/CS 2nd Sem) Unit 4

news papers and a variety of other applications. Integrating of all these

applications in a common authoring tool requires great skill in planning.

Hypermedia Application Design Consideration

The user interface must be highly intuitive to allow the user to learn the tools
quickly and be able to use them effectively. In addition, the user interface
should be designed to cater to the needs of both experienced and inexperienced
user.
In addition to control of their desktop environments, user also needs control of
their system environment. This control should include some of the following:
 The ability to specify a primary server for each object class within a
domain specified by the system administrative. A domain can be viewed
as a list of servers to which they have unrestricted access.

6
Multimedia Authoring and User Interfaces (IT/CS 2nd Sem) Unit 4

 The ability to specify whether all multimedia -objects or only references

should be replicated. The ability to specify that the multimedia object
should be retrieved immediately for display versus waiting for a signal to
"play" the object. This is more significant if the object must be retrieved
from a remote server.
 Display resolution defaults for each type of graphics or video object.
Essential for good hypermedia design:
1. Determining the type of hypermedia application.
2. Structuring the information.
3. Determining the navigation throughout the application.
4. Methodologies for accessing the information.
5. Designing the user interface.
Integration of Applications
The computer may be called upon to run a diverse set of applications, including
some combination of the following:
1. Electronic mail.
2. Word processing or technical publishing.

3. Graphics and formal presentation preparation software.

4. Spreadsheet or some other decision support software.
5. Access to a relational on object-oriented database.
6. Customized applications directly related to job function:
* Billing
* Portfolio management
* Others.
Integration of these applications consists of two major themes: the appearance
of the applications and the ability of the applications to exchange of data.

7
Multimedia Authoring and User Interfaces (IT/CS 2nd Sem) Unit 4

Common UI and Application Integration

Microsoft Windows has standardized the user interface for a large number of
applications by providing standardization at the following levels:
Overall visual look and feel of the application windows
 Menus
 Dialog Boxes
 Buttons
 Help Features
 Scroll Bars
 Tool Bars
 File open and save etc
This standardization level makes it easier for the user to interact with
applications designed for the Microsoft Windows operational environment.
Standardization is being provided for Object Linking and Embedding (OLE),
Dynamic Data Exchange (DOE), and the Remote Procedure Call (RPC).
Data Exchange
The Microsoft Windows Clipboard allows exchanging data in any format. It can
be used to exchange multimedia objects also. We can cut and copy a multimedia
objects in one document and pasting in another. These documents can be
opened under different applications. The windows clipboard allows the
following formats to be stored:
 Text
 Bitrnap
 Image
 Sound
 Video (AVI format).
Distributed Data Access
If all applications required for a compound object can access the sub objects that
they manipulate, then only application integration succeeds.

8
Multimedia Authoring and User Interfaces (IT/CS 2nd Sem) Unit 4

Fully distributed data access implies that any application at any client
workstation in the enterprise wide WAN must be able to access any data object
as if it were local. The underlying data management software should provide
transport mechanisms to achieve transparence for the application.
Hypermedia Application Design
Hypermedia applications are applications consisting of compound objects that
include the multimedia objects. An authoring application may use existing
multimedia objects or call upon a media editor to CD create new object.
Structuring the Information
A good information structure should consist the following modeling primitives:
Object types and object hierarchies.
 Object representations.
 Object connections.
 Derived connections and representations.

The goal of information Structuring is to identify the information objects and to

develop an information model to define the relationships among these objects.
Object Types and Object Hierarchies
Object types are related with various attributes and representations of the
objects. The nature of the information structure determines the functions that
can be performed on that information set. The object hierarchy defines a
contained-in relationship between objects. The manner in which this hierarchy
is approached depends on whether the document is being created or played
back. Users need the ability to search for an object knowing very little about the
object. Hypermedia application design should allow for such searches.
The user interface with the application depends on the design of the application,
particularly the navigation options provided for the user.

9
Multimedia Authoring and User Interfaces (IT/CS 2nd Sem) Unit 4

Object representations
Multimedia objects have a variety of different object representations. A
hypermedia object is a compound object, consists of s~ information elements,
including data, text, image, and video Since each of these multimedia objects
may have its own sub objects, the design must consider the representation of
objects.
An object representation may require controls that allow the user to alter the
rendering of the object dynamically. The controls required for each object
representation must be specified with the object.
Object connection
In the relational model, the connections are achieved through joins, and in the
object oriented models, through pointers hidden inside objects. Some means of
describing explicit connections is required for hypermedia design to define the
relationships among objects more clearly and to help in establishing the
navigation.
Derived Connections and Representations
Modeling of a hypermedia system should attempt to take derived objects into
consideration for establishing connection guidelines.

10
Multimedia Authoring and User Interfaces (IT/CS 2nd Sem) Unit 4

User Interface Design

Multimedia applications contain user interface design. There are four kinds of
user interface development tools. They are
1. Media editors
2. An authoring application
3. Hypermedia object creation
4. Multimedia object locator and browser
A media editor is an application responsible of the creation and editing of a
specific multimedia object such as an image, voice, or Video object. Any
application that allows the user to edit a multimedia object contains a media
editor. Whether the object is text, voice, or full-motion video, the basic
functions provided by the editor are the same: create, delete, cut, copy, paste,
move, and merge.
Navigation through the application
Navigation refers to the sequence in which the application progresses and
objects are created, searched and used. Navigation can be of three modes:
(i) Direct: It is completely predefined. In this case, the user needs to know what
to expect with successive navigation actions.
Free-form mode: In this mode~ the user determines the next sequence of
actions.
Browse mode: In this mode, the user does not know the precise question and
wants to get general information about a particular topic. It is a very common
mode in application based on large volumes of non-symbolic data. This mode
allows a user to explore the databases to support the hypothesis.

11
Multimedia Authoring and User Interfaces (IT/CS 2nd Sem) Unit 4

Information Access
Access structure defines the way objects can be accessed and how navigation
takes place through the information objects.
The common forms of navigations for information access are:
Direct: Direct information access is completely predefined. User must have
knowledge about the object that needs to be accessed. That information includes
object representations in a compound object.
Indexed: Index access abstracts the real object from the access to the object. If
the object ID of the object is an index entry that resolves to a filename on a
specific server and disk partition, then the information access mechanism is an
indexed mechanism.
Random Selection: In this fonn, the user can pick one of several possible
items. The items need not arranged in any logical sequence; and they need not
to be displayed sequentially. The user need not have much knowledge about the
information. They must browse through the information.
Path selection or Guided tour: In guided tour, the application guides the user
through a predefined path across a number of objects and operations. The user
may pause to examine the objects at any stage, but the overall access is
controlled by the application. Guided tours can also be used for operations such
as controlling the timing for discrete media, such as slide show. It can be used
for control a sound track or a video clip.
Browsing: It is useful when the user does not have much knowledge about the
object to access it directly.
Object Display Playback Issues
User expects some common features apart from basic functions for authoring
systems. And to provide users with same special control on the display/
playback of these objects, designers have to address some of these issues for
image, audio and video objects.

12
Multimedia Authoring and User Interfaces (IT/CS 2nd Sem) Unit 4

Image Display Issues Scaling: Image scaling is performed on the fly after
decompression the image is scaled to fit in an application defined window at t:'
full pixel rate for the window. The image may be scaled by using factors. For
e.g.: for the window 3600 x 4400 pixels can be scaled by a factor of 6 x 10 ie.60
x 440 (60 times).
Zooming: Zooming allows the user to see more detail for a specific area of the
image. Users can zoom by defining a zoom factor (eg: 2: 1,5: 1 or 10: 1). These
are setup as preselected zoom values.
Rubber banding: This is another form of zooming. In this case, the user uses a
mouse to define two comers of the rectangle. The selected area can be copied to
the clipboard, cut, moved or zoomed.
Panning: If the image window is unable to display the full image at the
·selected resolution for display. The image can be panned left to right or right to
left as well as top to bottom or bottom to top. Panning is useful for finding
detail that is not visible in the full image.
Audio Quality: Audio files are stored in one of a number of formats, including
WAVE and A VI. Playing back audio requires that the audio file server be
capable of playing back data at the rate of 480 kbytes/min uncompressed or 48
kbytes/min for compressed 8 bit sound or 96 kbytes/min for 16 bit sound.
The calculation is based on an 8 MHz sampling rate and ADCPM compression
with an estimated compression ratio. 32 bit audio will need to be supported to
get concert hall quality in stored audio. Audio files can be very long. A 20
minute audio clip is over 1 MB long. When played back from the server, it must
be transferred completely in one burst or in a controlled manner.
Special features for video playback: Before seeing the features of video
playback let us learn what is isochronous playback. The playback at a constant
rate to ensure proper cadence (the rise and fall in pitch of a person's voice) is
known as isochronous playback. But isochronous playback is more complex
with video than It is for sound. . If video consists of multiple clips of video and

13
Multimedia Authoring and User Interfaces (IT/CS 2nd Sem) Unit 4

multiple soundtracks being retrieved from different servers and combined for
playback by accurately synchronizing them, the problem becomes more
complex. To achieve isochronous playback, most video storage systems use
frame interleaving concepts. Video Frame Interleaving: Frame interleaving
defines the structure of the video file in terms of the layout of sound and video
components.
Programmed Degradation: When the client workstation is unable to keep up
with the incoming data, programmed degradation occurs. Most video servers are
designed to transfer data from storage to the client at constant rates. The video
server reads the file from storage, separate the sound and video components,
and feeds them as a separate stream over the network to the client workstations.
Unless specified by the user, the video server defaults to favoring sound and
degrades video playback by dropping frames. So, sound can be heard on a
constant basis. But the video loses its smooth motion and starts looking shaky.
Because intermediate frames are not seen.
The user can force the ratio of sound to video degradation by changing the
interleaving factor for playback; ie the video server holds back sound until the
required video frames are transferred. This problem becomes more complex
when multiple streams of video and audio are being played back from multiple
source servers. .
Scene change Frame Detection: The scene we see changes every few seconds
or minutes and it replaced by a new image. Even within the same scene, there
may be a constant motion of some objects in a scene.
Reason for scene change detection: Automating scene change detection is
very useful for browsing through very large video clips to find the exact frame
sequence of interest. Spontaneous scene change detection provides an automatic
indexing mechanism that can be very useful in browsing. A user can scan a
complete video clip very rapidly if the key frame for each new scene is
displayed in an iconic (poster frame) form in a slide sorter type display. The

14
Multimedia Authoring and User Interfaces (IT/CS 2nd Sem) Unit 4

user can then click on a specific icon to see a particular scene. This saves the
user a significant amount of time and effort and reduces resource load by
decompressing and displaying only the specific scene of interest rather than the
entire video.
Scene change detection is of real advantage if it can be performed without
decompressing the video object. Let us take a closer-look at potential techniques
that can be employed for this purpose. Techniques:
(i) Histogram Generation: Within a scene, the histogram changes as the
subject of the scene mover. For example, if a person is running and the camera
pans the scene, a large part of the scene is duplicated with a little shift. But if
the scene changes from a field to a room, the histogram changes quite
substantially. That is, when a scene cuts over to a new scene, the histogram
changes rapidly. Normal histograms require decompressing the video for the
successive scenes to allow the optical flow of pixels to be plotted on a
histogram. The foot that the video has to be decompressed does help in that the
user can jump from one scene to the next. However, to show a slide sorter view
requires the entire video to be decompressed. So this solution does not really of
the job.
Since MPEG and JPEG encoded video uses DCT coefficients, DCT
quantization analysis on uncompressed video or Audio provides the best
alternatives for scene change detection without decompressing video
The efficiency can be managed by determining the frame interval for checks
and by deciding on the regions within the frame that are being checked. A new
cut in a scene or a scene change can be detected by concentrating on a very
small portion of the frame
The scene change detection technology as is the case with video compression
devices as well as devices that can process compressed video, the
implementations of scene change detection can be significantly enhanced.
Video scaling, Panning and Zooming:

15
Multimedia Authoring and User Interfaces (IT/CS 2nd Sem) Unit 4

Scaling:
Scaling is a feature since users are used in changing window sizes. When the
size of the video window is changed, scaling take place.
Panning:
Panning allows the user to move to other parts of the window. Panning is useful
in combination with zooming. Only if the video is being displayed at full
resolution and the video window is not capable of displaying the entire window
then panning is useful. Therefore panning is useful only for video captured
using very high resolution cameras.
Zooming:
Zooming implies that the stored number of pixels is greater than the number
that can be displayed in the video window. In that case, a video scaled to show
the complete image in the video window can be paused and an area selected to
be shown in a higher resolution within the same video window. The video can
be played again from that point either in the zoomed mode or in scaled to fit
window mode.
Three Dimensional Object Display and VR (Virtual Reality)
Number of 3D effects are used in home entertainment a advanced systems used
for specialized applications to achieve find in results.
Let us review the approaches in use to determine the impact 0 multimedia
display system design due to these advanced systems.
Planar Imaging Technique: The planar imaging technique used in computer-
aided tomography (CAT scan) systems, displays a two dimensional [2D] cut of
X-ray images through multi dimensional data specialized display techniques try
to project a 3D image constructed from the 2D data. An important design issue
is the volume of data being displayed (based on the image resolution and
sampling rate) and the rate at which 3D renderings need to be constructed to
ensure a proper time sequence for the changes in the data.

16
Multimedia Authoring and User Interfaces (IT/CS 2nd Sem) Unit 4

Computed tomography has a high range of pixel density and can be used for a
variety of applications. Magnetic resonance imaging, on the other hand, is not
as fast, nor does it provide as high a pixel density as CT. Ultrasound is the third
technique used for 3D imaging in the medical and other fields. .

17