MetaData and its Classification

• Metadata is data about the data or documentation about the

information which is required by the users. In data warehousing,
metadata is one of the essential aspects.
• Metadata includes the following:
• The location and descriptions of warehouse systems and components.
• Names, definitions, structures, and content of data-warehouse and
end-users views.
• Identification of authoritative data sources.
• Integration and transformation rules used to populate data.
• Integration and transformation rules used to deliver information to
end-user analytical tools.
• Subscription information for information delivery to analysis
subscribers.
• Metrics used to analyze warehouses usage and performance.
• Security authorizations, access control list, etc.

b) Use of Metadata
• Metadata is used for building, maintaining, managing, and using the
data warehouses. Metadata allow users access to help understand the
content and find data.
Examples of Metadata
• A library catalog may be considered metadata. The directory metadata consists
of several predefined components representing specific attributes of a resource,
and each item can have one or more values. These components could be the
name of the author, the name of the document, the publisher's name, the
publication date, and the methods to which it belongs.
• The table of content and the index in a book may be treated metadata for the
book.
• Suppose we say that a data item about a person is 80. This must be defined by
noting that it is the person's weight and the unit is kilograms. Therefore,
(weight, kilograms) is the metadata about the data is 80.
• Another examples of metadata are data about the tables and figures in a report
like this book. A table (which is a record) has a name (e.g., table titles), and
there are column names of the tables that may be treated metadata. The figures
also have titles or names.
Is metadata necessary in a data warehouses
• First, it acts as the glue that links all parts of the data warehouses.
• Next, it provides information about the contents and structures to the
developers.
• Finally, it opens the doors to the end-users and makes the contents
recognizable in their terms.
• Metadata is Like a Nerve Center. Various processes during the
building and administering of the data warehouse generate parts of
the data warehouse metadata. Another uses parts of metadata
generated by one process. In the data warehouse, metadata assumes
a key position and enables communication among various methods. It
acts as a nerve centre in the data warehouse.
Types of Metadata
Metadata in a data warehouse fall into three major parts:
• Operational Metadata
• Extraction and Transformation Metadata
• End-User Metadata
a) Operational Metadata
• As we know, data for the data warehouse comes from various operational systems of the
enterprise. These source systems include different data structures. The data elements
selected for the data warehouse have various fields lengths and data types.
• In selecting information from the source systems for the data warehouses, we divide
records, combine factor of documents from different source files, and deal with multiple
coding schemes and field lengths.
• When we deliver information to the end-users, we must be able to tie that back to the
source data sets. Operational metadata contains all of this information about the
operational data sources.
Extraction and Transformation Metadata

b) Extraction and transformation metadata include data about the

removal of data from the source systems, namely, the extraction
frequencies, extraction methods, and business rules for the data
extraction. Also, this category of metadata contains information about
all the data transformation that takes place in the data staging area.
c) End-User Metadata
• The end-user metadata is the navigational map of the data
warehouses. It enables the end-users to find data from the data
warehouses. The end-user metadata allows the end-users to use their
business terminology and look for the information in those ways in
which they usually think of the business.
ETL Environment
• Traditional extraction, transformation and loading (ETL) software is a
computer programming tool that enables a developer to create custom
ETL code.
• Custom coding is a management challenge with expensive overhead
and complex coordination between the business, application, and data
teams. In addition, the code can be unreliable because it is hard-coded
and tightly binds to database schema, so any environmental changes
causes the program to fail.
• Metadata-based extraction, transformation, and loading (ETL) can
support a new approach to any organization’s data integration and
development practices
• Data integration is a data issue that should be resolved with a data
methodology.
• Metadata ETL provides this flexibility by managing schema, the data
model, and ETL together in a holistic solution.
• Metadata based ETL implements the Declarative Programming
Paradigm which differs from Object Oriented and Procedural
Programming. Declarative separates “what should done” from the
“how to do it”.
• The “what should be done” is defined in metadata, similar to a data
dictionary which defines the data mappings, data model, datatypes,
data transformations, etc.
Security mechanism in DW environment
• Access Controls: Setting up access restrictions to make sure that only people
with permission can access the data warehouse. In accordance with their work
responsibilities within a company, this also entails allowing appropriate levels
of access and assigning user accounts and passwords to specific people.
• Authentication and Authorization: Implementing authorization mechanisms
to ensure that users only have access to the data they need to carry out their
job functions and implementing authentication mechanisms such as two-factor
authentication, which we have in our email, biometric authentication, and
single sign-on to ensure that users are who they claim to be.
• Encryption: Protecting data from illegal access by encrypting it both in
transport and at rest To safeguard data, this calls for the use of encryption
methods and secure key management procedures.
• Auditing: Establishing auditing procedures to keep an eye on user behavior
and spot any security breaches This entails recording user activity and
inspecting it to look for irregularities or suspect conduct.
• Data Masking: Adopting a data masking approach, which protects
sensitive data by replacing it with fake data that has the same format and
properties but hides the actual data in data warehousing. Data masking
helps firms retain the confidentiality of sensitive data and adhere to rules,
making it a crucial part of data security in data warehousing.
• Data Loss Prevention: Data backup and recovery procedures are
implemented along with security measures like firewalls and intrusion
detection and prevention systems to prevent unwanted access, alteration,
or destruction of data.
• Regulatory Compliance: Since there are numerous rules, regulations, and
standards relating to data security and privacy that apply to different
businesses, regulatory compliance is a crucial component of data
warehousing. This entails putting in place the proper security processes
and controls to safeguard data and guarantee legal compliance.
Advantages of Security in Data Warehousing
• By guaranteeing the confidentiality and integrity of customers’ data,
strong data security will improve consumer trust.

• In hopes of enhancing decision-making, access restrictions, and audit

trails can assist assure the quality and integrity of data.

• A business may reduce the enormous financial implications of security

breaches, including such attorney costs and reputation damage, by
using more efficient security solutions.
Disadvantages of Security in Data Warehousing
• Data warehousing procedures may become more complicated as a
result of the implementation of security measures, making it more
challenging to administer and maintain them.

• Access restrictions caused by security measures like access controls

can slow down corporate agility and decision-making processes.

• Access limitations brought on by security precautions like access

controls might impede business agility and decision-making.