Multimedia Databases

1
Multimedia Databases
 To provide such database functions as indexing
and consistency, it is sometimes desirable to store
multimedia data in a database
 rather than storing them outside the database,
in a file system
 The database must handle large object
representation.
 Similarity-based retrieval must be provided by
special index structures.
 Must provide guaranteed steady retrieval rates for
continuous-media data.
2
Multimedia Data Formats
 Store and transmit multimedia data in compressed form
 JPEG and GIF the most widely used formats for image
data.
 MPEG standard for video data use commonalties among
a sequence of frames to achieve a greater degree of
compression.
 MPEG-1 quality comparable to VHS video tape.
 stores a minute of 30-frame-per-second video and audio
in approximately 12.5 MB
 MPEG-2 designed for digital broadcast systems and digital
video disks; negligible loss of video quality.
 Compresses 1 minute of audio-video to approximately 17
MB.
 Several alternatives of audio encoding
 MPEG-1 Layer 3 (MP3), RealAudio, WindowsMedia
format, etc. 3
Continuous-Media Data
 Most important types are video and audio data.
 Characterized by high data volumes and real-time information-
delivery requirements.
 Data must be delivered sufficiently fast that there are no
gaps in the audio or video.
 Data must be delivered at a rate that does not cause
overflow of system buffers.
 Synchronization among distinct data streams must be
maintained
 video of a person speaking must show lips moving
synchronously with the audio

4
 Video Servers
 Video-on-demand systems deliver video from central video
servers, across a network, to terminals
 Must guarantee end-to-end delivery rates
 Current video-on-demand servers are based on file systems;
existing database systems do not meet real-time response
requirements.
 Multimedia data are stored on several disks (RAID
configuration), or on tertiary storage for less frequently accessed
data.
 Head-end terminals - used to view multimedia data
 PCs or TVs attached to a small, inexpensive computer called
a set-top box.

5
 Data Type
 Binary Large Object (BLOB)
 Character Large Object (CLOB)
 Advantages:
 Multimedia Data is integrated into the database system
 Not dependent on the physical location of multimedia
data (disk/directory/filename)
 Disadvateges:
 Database storage can be very large
 Sometimes this will decrease the performance

4/14/
15
 Modelling
Activities in information system analysis
and design that identifies and develops
a model of the information requirements
for a proposed computer-based
application
Identifying and describing the information
requirements for an information system,
Specifying the data to be maintained by the data
management system, and
Specifying the data structures to be used for data
storage that best support the information
requirements by providing efficient and effective
information retrieval
User Requirement

Data Model
 Data Model Type
Data structure
data types and inter-relationships,

Constraints
allowable values, relationships, and cardinalities

Basic operations
for data storage, retrieval, modification,
maintenance, and control.
 Data structure
entity types: real world phenomena about which data is
collected.
f.ex.: person, teacher, student, course, report, …

relationship types: between entity-types.

associative relationships: between different types of entities. f.ex:
student takes course, teacher teaches course
generic relationships: which identify sub-classes or roles within an entity
type. f.ex student IS_A person, teacher IS_A person, …

attribute types: for each entity and associative relationship

type.
f.ex.: name and address for the "person" entity type,
semester and grade for the "takes" relationship type.

data types: to be used for recording the values of each

attribute type,
f.ex. name ::= character string, picture ::= image
 Constraints
rules for determining valid values and
structures
entity identifier, synonym: primary key, PK
one or more attributes whose value uniquely identifies an
individual entity.
f.ex: a social security number (SSN) or the person.Id,
course.Id, and date for a grade.
reference link, synonym: foreign key, FK, uniquely identifies a
related entity.
f.ex: person.spouse::= SSN that references a given
person's spouse.
relationship cardinality: for each entity to relationship type.
f.ex.: Student (0:m) takes (0:m) courses, meaning that a
student may take more than 1 course and a course may
be taken by more than 1 student.
 Constraints
participation constraints: for generic hierarchies,
specify the relationship between the parent and sub-class
entity-types
f.ex. a partial, overlapping (p,o) participation constraint for
the person heirarchy indicates that each person (in the
DB) may be a student, a teacher, both or neither.
domain constraints: define the valid value sets for attributes,
f.ex: course.level must have a value between [1..6]
 Basic operations
define how data values are
calculated when some event
occurs.
F.ex: age and wage can be calculated upon request
using a function definition:
person.age = todays_date - birth_date
student.wage = hourly_wage *
number_of_hours_worked
Functions can also check that data constraints are
maintained.
F.ex: ON INSERT person CHECK UNIQUE person.Id
 Modelling Processes
Development of a semantic model of the
information requirements for an information
system.
Translation of the semantic model to a structural
model that describes the data types and
structures using the concepts of a particular
data model type (relational, semantic, object-
oriented, ...).
Adaptation and/or translation of the structural
model to an implementation model, as
required by the specific DMS chosen to
manage the user's data.
 Semantic Model
A conceptual data model where
semantic information is included
Semantic information is a fact
oriented as opposed to object
oriented
Fact oriented is usually expressed as
a binary relationship
Mount Merapi [is located in] Indonesia
Jokowi [is the president of] Indonesia
 Data Model
Graphical
Easier for a human reader to interpret and check for
completeness and correctness than list models

List
Meta Data, Data Definition Language

Tabular
DB Schema
 Model Type
Entity-Relationship
Extended Entity-Relationship
Object Oriented Model
Structural Semantic Model
Entity-Relationship
Model
Extended Entity-
Relationship Model
Object-Oriented
Model
 Structural Semantic
Model
Base and weak entity (SSM)

Hierarchic Relationship

Subclass Entiry Type

associative
relationships
with (min,max) cardinality
specification

Base entity
SSM Attribute and
Data Type
Meta Data
Data about Data

Uses
Specification and interpretation of user requests -
queries - for information
Determining storage, indexing, and retrieval of the data
to/from the database
 Meta Data
Specification
Database design
a semantic data model
metadata standard for description of user requirements, data
structures, and constraints

Data storage –
using a data definition language, DDL, to add metadata
values.
 Meta Data for
Multimedia
Semantic metadata
characterize the subject matter of the
document,
Context metadata
describe relationships to external (to the
meaning of the document) objects, such as
author and publisher
Structural metadata
describe the internal structure and
 Semantic Metadata
Speficy features that describe the
semantic content of the media
object
Index terms or keywords
Shapes
Colors
Textures
Categories
Context Metadata
specify the relationships that
the media object has to its
environment
author/creator, publisher, the date of
creation/purchase/publication, and the current
location of the object
typically represent 'facts' about the object that need
to be determined and recorded manually
can be modelled as objects, attributes and/or
relationships in a traditional data model
 Structural Metadata
describe the implementation, layout
spatial and temporal placement of objects within a document
presentation style of the media object
Language, length, presentation media
 Multimedia
Metadata
Dublin Core
originally developed for description of text-based documents,

Mpeg-7
developed for description of streamed multimedia such as film

CIDOC/CRM
developed for description of museum artifacts
 Dublin Core
• Metadata for Electronic
Documents
Metadata Type DC element
Semantic Title, Subject, Description, Type, Coverage
Context Creator, Contributor, Publisher, Date, Rights,
Source, Relation

Structural
1. TITLE Type,given
The name Format,
to theLanguage,
resource byIdentifier
the CREATOR or PUBLISHER. 2.
CREATOR The person(s) or organization(s) primarily responsible for the
intellectual content of the resource; the author. 3. SUBJECT The topic of the
resource; also keywords, phrases or classification descriptors that describe the
subject or content of the resource.
 Dublin Core
4. DESCRIPTION A textual description of the content of the
resource, including abstracts in the case of document-like objects;
also may be a content description in the case of visual resources.
5. PUBLISHER The entity responsible for making the resource
available in its present form, such as a publisher, university
department or corporate entity. 6. CONTRIBUTORS Person(s)
or organization(s) in addition to those specified in the CREATOR
element, who have made significant intellectual contributions to
the resource but on a secondary basis. 7. DATE The date the
resource was made available in its present form. 8. TYPE The
resource type, such as home page, novel, poem, working paper,
technical report, essay or dictionary. It is expected that TYPE will
be chosen from an enumerated list of types.

9. FORMAT The data representation of the resource, such as

text/html, ASCII, Postscript file, executable application or JPG
image. FORMAT will be assigned from enumerated lists such as
registered Internet Media Types (MIME types). MIME types are
defined according to the RFC2046 standard.
 Dublin Core
10. IDENTIFIER A string or number used to uniquely identify
the resource. Examples from networked resources include URLs
and URNs (when implemented).11. SOURCE The work, either
print or electronic, from which the resource is delivered (if
applicable). 12. LANGUAGE The language(s) of the
intellectual content of the resource. 13. RELATION The
relationship to other resources. Formal specification of
RELATION is currently under development. 14. COVERAGE
The spatial locations and temporal duration characteristics of
the resource. Formal specification of COVERAGE is also now
being developed. 15. RIGHTS MANAGEMENT A link (URL
or other suitable URI as appropriate) to a copyright notice, a
rights-management statement or perhaps a server that would
provide such information in a dynamic way.
 MPEG-7
Dublin Code
does not describe basic characteristics of moving
pictures (streamed data), such as the temporal
and sequencing requirements or the object
identification and inter-relationships depicted in
the film media.

The Moving Picture Experts

Groups
establishes standards for transmitting digital films
and videos on the Internet
focused on describing the semantic and structural
content of multimedia
formally named as "Multimedia Content
 Elements of MPEG-7
Standards
Description Tools: Descriptors (D), that define the syntax and
the semantics of each feature (metadata element); and
Description Schemes (DS), that specify the structure and
semantics of the relationships between their components,
that may be both Descriptors and Description Schemes;
A Description Definition Language (DDL) to define the syntax
of the MPEG-7 Description Tools and to allow the creation
of new Description Schemes and, possibly, Descriptors and
to allow the extension and modification of existing
Description Schemes;
System tools, to support binary coded representation for
efficient storage and transmission, transmission
mechanisms (both for textual and binary formats),
multiplexing of descriptions, synchronization of
descriptions with content, management and protection of
intellectual property in MPEG-7 descriptions
Possible Application
 CIDOC-CRM
Developed by The International
Committee for Documentation,
CIDOC, of international council of
museums, ICOM
for describing cultural heritage
objects, particularly those found in
museum collections
descriptions include the spatial,
temporal and event aspects of
museum objects, in addition to the
semantic, context and structural
aspects emphasized in the Dublin
CIDOC-CRM
Component
Relationships
CIDOC-CRM Class Hierarchy
 Multiple Media
Documents
A report, containing text, images, as well as
structured data for title, author, date, ...
A map, containing points, lines, areas or
regions, as well as title, place names,
facts (distances, heights), and icons, ...
A film, containing image stream, and
(multiple) audio streams as well as a title,
actor list, producer, ...
 Text Documents
can be described from 3 perspectives:
Semantic content of the document, i.e. representation of its meaning,
Context of the document, e.g. its author, publisher
Structure of the document, e.g. its language, style, length, .
 Images
do not have a standard 'vocabulary'
or grammar that can be used for
automatic interpretation of the
semantic content, or meaning of
the images
Through text annotations
Using content descriptors or
features of images
Level Feature examples

1 structural color, texture, shape

location in image

2 Objects in image building, trees, people, band

spatial relationships between objects

3
Content Descriptors
Identification of The White House, US army fife and
objects in image drum corps

4 Event/action marching, welcome ceremony

representation

5 Emotion formal, official, at 'attention'

represented
 Image color-
signature example
low level features, such as
color, texture and shape
can be used to compose
an image signature

Color Cell-1 Cell-2 Cell-3 Cell-4 Cell-5 Cell-6 Cell-7 Cell-8 Cell-9
red - 1 - 1 5 1 20 20 -
green 40 2 30 89 20 85 70 75 70
blue 60 48 20 - - 1 - - 25
white - 49 50 10 75 13 5 5 5
 Modelling
Multimedia Data in
media
SSM objects can be modelled as:
An attribute of an entity-type: ex. Person.picture or
Person.geographic_location
An entity-type: ex. Report with such attributes as {id, title,
keywords, summary, content}
A set of related entity-types: ex. Report consists of Media
Objects of text and image type.
Media Objects as
Attributes
 Media Objects as
Ternary Relationship

useful when a balanced set of related media objects

occur in the 'parent' multimedia object
 Media Objects as a
Classification
Hierarchy

useful when the sets of media objects are relatively

independent and used in multiple multimedia objects