Introduction of database management

system.

What is a database management system?

A database management system (DBMS) is a software tool that

enables users to manage a database easily. It allows users to access
and interact with the underlying data in the database. These actions
can range from simply querying data to defining database schemas
that fundamentally affect the database structure.

Furthermore, DBMS allow users to interact with a database securely

and concurrently without interfering with each user and while
maintaining data integrity.

Database tasks in a DBMS

The typical database administrative tasks that can be performed
using a DBMS include:

 Configuring authentication and authorization. Easily

configure user accounts, define access policies, modify
restrictions, and access scopes. These operations allow
administrators to limit access to underlying data, control
user actions, and manage users in databases.
 Providing data backups and snapshots. DBMS can
simplify the backup process of databases by providing a
 simpler and straightforward interface to manage backups
and snapshots. They can even move these backups to third-
party locations such as cloud storage for safekeeping.
 Performance tuning. DBMS can monitor the performance
of databases using integrated tools and enable users to
tune databases by creating optimized indexes. It
reduces I/O usage to optimize SQL queries, enabling the
best performance from the database.
 Data recovery. In a recovery operation, DBMS provides a
recovery platform with the necessary tools to fully or
partially restore databases to their previous state—
effortlessly.

All these administrative tasks are facilitated using a single

management interface. Most modern DBMS support handling
multiple database workloads from a centralized DBMS software,
even in a distributed database scenario. Furthermore, they allow
organizations to have a governable top-down view of all the data,
users, groups, locations, etc., in an organized manner.

(Explore the role of DBAs, or database administrators.)

DBMS system schematic

The following diagram illustrates the schematic of a DBMS system:
Components of a database management
system
All DBMS comes with various integrated components and tools
necessary to carry out almost all database management tasks.
Some DBMS software even provides the ability to extend beyond the
core functionality by integrating with third-party tools and services,
directly or via plugins.
In this section, we will look at the common components that are
universal across all DBMS software, including:

 Storage engine
 Query language
 Query processor
 Optimization engine
 Metadata catalog
 Log manager
 Reporting and monitoring tools
 Data utilities
Storage engine

The storage engine is the core component of the DBMS that

interacts with the file system at an OS level to store data. All SQL
queries which interact with the underlying data go through the
storage engine.

Query language

A database access language is required for interacting with a

database, from creating databases to simply inserting or retrieving
data. A proper DBMS must support one or multiple query languages
and language dialects. Structured query language (SQL) and
MongoDB Query Language (MQL) are two query languages that are
used to interact with the databases.

In many query languages, the query language functionality can be

further categorized according to specific tasks:

 Data Definition Language (DDL). This consists of

commands that can be used to define database schemas or
modify the structure of database objects.
 Data Manipulation Language (DML). Commands that
directly deal with the data in the database. All CRUD
operations come under DML.
 Data Control Language (DCL). This deals with the
permissions and other access controls of the database.
 Transaction Control Language (TCL). Command which
deals with internal database transactions.

Query processor

This is the intermediary between the user queries and the database.
The query processor interprets the queries of users and makes them
actionable commands that can be understood by the database to
perform the appropriate functionality.
Optimization engine

The optimization Engine allows the DBMS to provide insights into the
performance of the database in terms of optimizing the database
itself and queries. When coupled with database monitoring tools, it
can provide a powerful toolset to gain the best performance out of
the database.

Metadata catalog

This is the centralized catalog of all the objects within the database.
When an object is created, the DBMS keeps a record of that object
with some metadata about it using the metadata catalog. Then, this
record can be used to:

 Verify user requests to the appropriate database objects

 Provide an overview of the complete database structure

Log manager

This component will keep all the logs of the DBMS. These logs will
consist of user logins and activity, database functions, backups and
restore functions, etc. The log manager ensures all these logs are
properly recorded and easily accessible.

(Compare logs to monitoring.)

Reporting & monitoring tools

Reporting and monitoring tools are another standard component

that comes with a DBMS. Reporting tools will enable users to
generate reports while monitoring tools enable monitoring the
databases for resource consumption, user activity, etc.
Data utilities

In addition to all the above, most DBMS software comes with

additional inbuilt utilities to provide functionality such as:

 Data integrity checks

 Backup and restore
 Simple database repair
 Data validations
 Etc.

Scale operational effectiveness with an artificial intelligence for IT

operations. Learn more about AIOps with BMC! ›

Types of database management systems

There are many different types of DBMS, yet we can categorize the
most commonly used DBMS into three types.

Relational database management systems (RDBMS)

This is the most common type of DBMS. They are used to interact
with databases that contain structured data in a table format with
predefined relationships. Moreover, they use structured query
language (SQL) to interact with databases. Microsoft SQL, MySQL,
and Oracle Database are some popular DBMS that come under this
category.

Document database management systems (DoDBMS))

These DoDBMS are used to manage databases that contain data

stored in JSON-like structures with limited or no relationship
structure. They are powered by query languages such as MongoDB
query language (MQL) for database operations. MongoDB, Azure
Cosmos DB are some prominent examples of DoDBMS.

Columnar database management systems (CDBMS)

As the name suggests, this type of DBMS is used to manage

columnar databases that store data in columns instead of rows,
emphasizing high performance. Some databases that use columnar
format are Apache Cassandra, Apache HBase, etc.

Advantages of a DBMS
DBMS was introduced to solve the fundamental issues associated
with storing, managing, accessing, securing, and auditing data in
traditional file systems. Software users and organizations can gain
the following benefits by using DBMS:

Increased data security

DBMS provides the ability to control users and enforce policies

for security and compliance management. This controlled user
access increases the database security and makes the data less
vulnerable to security breaches.

Simple data sharing

DBMS enables users to access the database securely regardless of

their location. Thus, they can handle any database-related task
promptly without the need for complex access methods or worrying
about database security. On top of that, DBMS allows multiple users
to collaborate effectively when interacting with the database.
Data integration

DBMS allows users to gain a centralized view of databases spread

across multiple locations and manage them using a single interface
rather than operating them as separate entities.

Abstraction & independence

DBMS enables users to change the physical schema of a database

without changing the logical schema that governs database
relationships. As a result, organizations can scale the underlying
database infrastructure without affecting the database operations.

Furthermore, any change to the logical schema can also be carried

out without affecting applications that access the databases.

Streamlined backup & recovery mechanism

Most databases have built-in backup and recovery tools. Yet, DBMS
offers centralized tools to facilitate backup and recovery
functionality more conveniently and thereby provide a better user
experience. Securing data has become easier than ever with
functionality like:

 Automated snapshots
 Backup scheduling
 Backup verifications
 Multiple recovery methods

Uniform management & monitoring

DBMS provides a single interface to carry out all the management

and monitoring tasks, thus simplifying the workload of database
administrators. These tasks can range from database creation and
schema modifications to reporting and auditing.

DBMSs are essential

DBMS is an essential component for any organization when it comes
to managing databases. The scale, complexity, and feature set of a
DBMS will depend on the specific DBMS and requirements of the
organizations.

With different DBMS providing different feature sets, it is paramount

that organizations rigorously evaluate the DBMS software before
committing to a single system. However, a properly configured
DBMS will greatly simplify the management and maintenance of
databases at any scale.

Data base Administrator

What is a database administrator (DBA)

A database administrator, or DBA, is responsible for maintaining,
securing, and operating databases and also ensures that data is
correctly stored and retrieved.

In addition, DBAs often work with developers to design and

implement new features and troubleshoot any issues. A DBA must
have a strong understanding of both technical and business needs.

The role of DBA is becoming increasingly important in today’s

information-driven business environment. Thoroughout the world,
more and more organizations depend on data to discover analytical
insights on market conditions, new business models, and cost-
cutting measures. The global cloud computing market is also
expected to expand as companies move their business operations to
the cloud. Consequently, the need for qualified DBAs will only
continue to grow.

Different types of DBAs

There are several types of database administrators, each with
specific duties and responsibilities. The most common types of DBAs
include system administrators, database architects, database
analysts, data modelers, application DBAs, task-oriented DBAs,
performance analysts, data warehouse administrators, and cloud
DBAs.

 System administrators are responsible for the overall

management and upkeep of a computer system, including
installing and configuring software, applying security patches,
and monitoring system performance.
 Database architects design databases to meet the specific
needs of an organization.
 Database analysts collect and analyze data to help improve
database performance. They may also be responsible for
developing reports and providing recommendations to
database administrators.
 Data modelers create and maintain data models that depict
the relationship between data elements. Data modeling is a
critical component of effective database design.
 Application DBAs are responsible for administrating databases
that support applications. Specific tasks include installing and
configuring applications, ensuring that data is synchronized
correctly between databases, and troubleshooting application-
related issues.
 Task-oriented DBAs focus on a particular area of database
administration, such as backup and recovery, security, or
performance tuning. They typically have in-depth knowledge of
a specific database management system (DBMS).
  Performance analysts monitor database performance and
identify areas where improvement is needed. They may also
be responsible for creating performance reports and providing
recommendations to database administrators.

 Data warehouse administrators manage databases that store

data for business intelligence or decision-support applications.
They are responsible for extracting data correctly,
transforming the data, and loading it into the data warehouse.
 Cloud DBAs are responsible for administrating databases
hosted in a cloud computing environment, provisioning and
managing database instances, setting up replication and high
availability, and monitoring database performance.

How has the role of a DBA evolved with cloud

computing?
The role of a database administrator has evolved significantly with
the advent of cloud computing. Rather than being responsible for
managing on-premises hardware and software, DBAs now need to
be able to work with cloud-based platforms. This requires a different
set of skills and knowledge and a different approach to work.

DBAs need to be able to work with different types of databases,

such as MySQL, MongoDB, and Cassandra. They also need to be
familiar with cloud-based tools and platforms, such as Amazon Web
Services (AWS) and Microsoft Azure.

One of the most significant changes is that DBAs are no longer

responsible for managing the underlying infrastructure. With cloud
computing, this is all managed by the provider. As a result, DBAs
now perform more strategic tasks, such as data analytics, user
experience design, and cybersecurity. DBAs often work directly with
users and business leaders on developing new ways to use data and
software to automate processes, reduce costs, and stay
competitive.
This requires a new set of skills from DBAs. In the past, having
strong technical skills was the most important requirement. There is
less need for these skills with cloud computing. Instead, DBAs need
to communicate and collaborate with users to understand their
needs and business environment. They also need to work with other
teams, such as DevOps, to help deliver software that will solve
business problems.

Learning database administration

There are several education institutes that offer professional

courses, including late-night programs, to allow candidates to learn
database administration. DBMS vendors such as Oracle, Microsoft
and IBM offer certification programs to help companies to hire
qualified DBA practitioners. College degree in Computer Science or
related field is helpful but not necessarily a prerequisite.

Schema , Sub Schema , Instance

What is schema?
The overall design of the database is called schema or description of
database.

 Schema is a physical representation of data which is present in

the database management system.
 In simple words we can call a schema the structure of any
database.
 It defines how the data was stored in a database and also
shows the relationship among those data, but it does not show
the data present in those tables.
 The database schema includes the definition of the database,
entities and the components.
 Schema of a database can only modify the DDL statement but
does not change by performing certain operations like
insertion, updating, and deletion.
 Database schema explains the integrity constraints of the
database, domains of all attributes, foreign, and primary key of
all the relations.

Sub schema
It is the subset of the schema and inherits the same property that a
schema has. It gives the users a window through which he/she can
view only that part of the database which he wants.

For example − Student table in a database the programmer can

access all fields of table, but the user can access only two or three
fields of it. Subschema describes both views of the database.

Types of Schema
Schema is of three types, which are as follows −
View Schema − The design of a database at a view level is called
view schema. This schema generally shows the user interaction with
the database system.

Logical Schema − The design of a database at the logical level is

called a logical schema. A database administrator (DBA) and the
programmers used to work at this level. This level describes all the
entities, attributes and their relationship with the integrity
constraints.

Physical Schema − The design of a database at the physical level is

called a physical schema. This schema describes how the data is
stored in the secondary storage devices. There is only 1 logical and
1 physical schema per database and more than 1 view schema.

Schema is also called Intention and is shown as below –

What is DBMS Instance?

In DBMS, the data is stored for a particular amount of time and is

called an instance of the database. The database schema defines
the attributes of the database in the particular DBMS. The value of
the particular attribute at a particular moment in time is known as
an instance of the DBMS.

For example, in the above example, we have taken the example of

the attribute of the schema. In this example, each table contains
two rows or two records. In the above schema of the table, the
employee table has some instances because all the data stored by
the table have some instances.

Let's take another example: Let's say we have a single table student
in the database; today, the table has 100 records, so today, the
instance of the database has 100 records. We are going to add
another 100 records to this table by tomorrow, so the instance of
the database tomorrow will have 200 records in the table. In short,
at a particular moment, the data stored in the database is called the
instance; this change over time as and when we add, delete or
update data in the database.

Differences between Database Schema and

Instance
Database Schema Database Instance

i. It is the definition of the i. It is a snapshot of a

database, or it is database at a specific
 defined as the moment.
description of the
database.

ii. It rarely changes. ii. It changes frequently.

iii. This corresponds to the iii. The value of the variable in

variable declaration of a a program at a point in
programming language. time corresponds to an
instance of the database
schema.

iv. Defines the basic iv. It is the set of Information

structure of the stored at a particular time.
database, i.e., how the
data will be stored in
the database.

v. Schema is same for v. Data in instances can be

whole database. changed using addition,
deletion, updation.

vi. It does not change very vi. It changes very frequently

frequently.

Both of these help in describing the data available in a database,

but there is a fundamental difference between Schema and Instance
in DBMS. Schema refers to the overall description of any given
database. Instance basically refers to a collection of data and
Information that the database stores at any particular moment.

What Is a Data Warehouse?

 Data Warehouse Defined
A data warehouse is a type of data management system that is
designed to enable and support business intelligence (BI) activities,
especially analytics. Data warehouses are solely intended to
perform queries and analysis and often contain large amounts of
historical data. The data within a data warehouse is usually derived
from a wide range of sources such as application log files and
transaction applications.

A data warehouse centralizes and consolidates large amounts of

data from multiple sources. Its analytical capabilities allow
organizations to derive valuable business insights from their data to
improve decision-making. Over time, it builds a historical record that
can be invaluable to data scientists and business analysts. Because
of these capabilities, a data warehouse can be considered an
organization’s “single source of truth.”

A typical data warehouse often includes the following elements:

 A relational database to store and manage data

 An extraction, loading, and transformation (ELT) solution for
preparing the data for analysis
 Statistical analysis, reporting, and data mining capabilities
 Client analysis tools for visualizing and presenting data to
business users
 Other, more sophisticated analytical applications that generate
actionable information by applying data science and artificial
intelligence (AI) algorithms, or graph and spatial features that
enable more kinds of analysis of data at scale
Organizations can also select a solution combining transaction
processing, real-time analytics across data warehouses and data
lakes, and machine learning in one MySQL Database service—
without the complexity, latency, cost, and risk of extract, transform,
and load (ETL) duplication.

Benefits of a Data Warehouse

 Data warehouses offer the overarching and unique benefit of
allowing organizations to analyze large amounts of variant data and
extract significant value from it, as well as to keep a historical
record.

Four unique characteristics (described by computer scientist William

Inmon, who is considered the father of the data warehouse) allow
data warehouses to deliver this overarching benefit. According to
this definition, data warehouses are

 Subject-oriented. They can analyze data about a particular subject

or functional area (such as sales).
 Integrated. Data warehouses create consistency among different
data types from disparate sources.
 Nonvolatile. Once data is in a data warehouse, it’s stable and
doesn’t change.
 Time-variant. Data warehouse analysis looks at change over time.
 A well-designed data warehouse will perform queries very quickly,
deliver high data throughput, and provide enough flexibility for
end users to “slice and dice” or reduce the volume of data for
closer examination to meet a variety of demands—whether at a
high level or at a very fine, detailed level. The data warehouse
serves as the functional foundation for middleware BI
environments that provide end users with reports, dashboards,
and other interfaces.

Data Warehouse Architecture

The architecture of a data warehouse is determined by the
organization’s specific needs. Common architectures include

 Simple. All data warehouses share a basic design in which

metadata, summary data, and raw data are stored within the
central repository of the warehouse. The repository is fed by data
 sources on one end and accessed by end users for analysis,
reporting, and mining on the other end.
 Simple with a staging area. Operational data must be cleaned
and processed before being put in the warehouse. Although this
can be done programmatically, many data warehouses add a
staging area for data before it enters the warehouse, to simplify
data preparation.
 Hub and spoke. Adding data marts between the central
repository and end users allows an organization to customize its
data warehouse to serve various lines of business. When the data
is ready for use, it is moved to the appropriate data mart.
 Sandboxes. Sandboxes are private, secure, safe areas that allow
companies to quickly and informally explore new datasets or ways
of analyzing data without having to conform to or comply with the
formal rules and protocol of the data warehouse.

The Evolution of Data Warehouses—From

Data Analytics to AI and Machine Learning
When data warehouses first came onto the scene in the late 1980s,
their purpose was to help data flow from operational systems into
decision-support systems (DSSs). These early data warehouses
required an enormous amount of redundancy. Most organizations
had multiple DSS environments that served their various users.
Although the DSS environments used much of the same data, the
gathering, cleaning, and integration of the data was often replicated
for each environment.

As data warehouses became more efficient, they evolved from

information stores that supported traditional BI platforms into broad
analytics infrastructures that support a wide variety of applications,
such as operational analytics and performance management.

Data warehouse iterations have progressed over time to deliver

incremental additional value to the enterprise with enterprise data
warehouse (EDW).
 Ste
Capability Business Value
p

Provides relational information to

Transactional
1 create snapshots of business
reporting
performance

Slice and dice, ad hoc Expands capabilities for deeper

2
query, BI tools insights and more robust analysis

Predicting future
Develops visualizations and forward-
3 performance (data
looking business intelligence
mining)

Offers “what-if” scenarios to inform

Tactical analysis
4 practical decisions based on more
(spatial, statistics)
comprehensive analysis

Stores many months

5 Stores data for only weeks or months
or years of data

Supporting each of these five steps has required an increasing

variety of datasets. The last three steps in particular create the
imperative for an even broader range of data and analytics
capabilities.

Today, AI and machine learning are transforming almost every

industry, service, and enterprise asset—and data warehouses are no
exception. The expansion of big data and the application of new
digital technologies are driving change in data warehouse
requirements and capabilities.

The autonomous data warehouse is the latest step in this evolution,

offering enterprises the ability to extract even greater value from
their data while lowering costs and improving data warehouse
reliability and performance.
Find out more about autonomous data warehouses and get started
with your own autonomous data warehouse.

Data Warehouses, Data Marts, and Operation

Data Stores
Though they perform similar roles, data warehouses are different
from data marts and operation data stores (ODSs). A data mart
performs the same functions as a data warehouse but within a much
more limited scope—usually a single department or line of business.
This makes data marts easier to establish than data warehouses.
However, they tend to introduce inconsistency because it can be
difficult to uniformly manage and control data across numerous data
marts.

ODSs support only daily operations, so their view of historical data is

very limited. Although they work very well as sources of current
data and are often used as such by data warehouses, they do not
support historically rich queries.

What is a Cloud Data Warehouse?

A cloud data warehouse uses the cloud to ingest and store data
from disparate data sources.

The original data warehouses were built with on-premises servers.

These on-premises data warehouses continue to have many
advantages today. In many cases, they can offer improved
governance, security, data sovereignty, and better latency.
However, on-premises data warehouses are not as elastic and they
require complex forecasting to determine how to scale the data
warehouse for future needs. Managing these data warehouses can
also be very complex.

On the other hand, some of the advantages of cloud data

warehouses include:
 Elastic, scale-out support for large or variable compute or storage
requirements
 Ease of use
 Ease of management
 Cost savings

The best cloud data warehouses are fully managed and self-driving,
ensuring that even beginners can create and use a data warehouse
with only a few clicks. An easy way to start your migration to a cloud
data warehouse is to run your cloud data warehouse on-premises,
behind your data center firewall which complies with data
sovereignty and security requirements.

In addition, most cloud data warehouses follow a pay-as-you-go

model, which brings added cost savings to customers.

What is a Modern Data Warehouse?

Whether they’re part of IT, data engineering, business analytics, or
data science teams, different users across the organization have
different needs for a data warehouse.

A modern data architecture addresses those different needs by

providing a way to manage all data types, workloads, and analysis.
It consists of architecture patterns with necessary components
integrated to work together in alignment with industry best
practices. The modern data warehouse includes:

 A converged database that simplifies management of all data

types and provides different ways to use data
 Self-service data ingestion and transformation services
 Support for SQL, machine learning, graph, and spatial processing
 Multiple analytics options that make it easy to use data without
moving it
 Automated management for simple provisioning, scaling, and
administration
 A modern data warehouse can efficiently streamline data workflows
in a way that other warehouses can’t. This means that everyone,
from analysts and data engineers to data scientists and IT teams,
can perform their jobs more effectively and pursue the innovative
work that moves the organization forward, without countless delays
and complexity.

Designing a Data Warehouse

When an organization sets out to design a data warehouse, it must
begin by defining its specific business requirements, agreeing on the
scope, and drafting a conceptual design. The organization can then
create both the logical and physical design for the data warehouse.
The logical design involves the relationships between the objects,
and the physical design involves the best way to store and retrieve
the objects. The physical design also incorporates transportation,
backup, and recovery processes.

Any data warehouse design must address the following:

 Specific data content

 Relationships within and between groups of data
 The systems environment that will support the data warehouse
 The types of data transformations required
 Data refresh frequency

A primary factor in the design is the needs of the end users. Most
end users are interested in performing analysis and looking at data
in aggregate, instead of as individual transactions. However, often
end users don’t really know what they want until a specific need
arises. Thus, the planning process should include enough
exploration to anticipate needs. Finally, the data warehouse design
should allow room for expansion and evolution to keep pace with
the evolving needs of end users.
The Cloud and the Data Warehouse
Data warehouses in the cloud offer the same characteristics and
benefits of on-premises data warehouses but with the added
benefits of cloud computing―such as flexibility, scalability, agility,
security, and reduced costs. Cloud data warehouses allow
enterprises to focus solely on extracting value from their data rather
than having to build and manage the hardware and software
infrastructure to support the data warehouse.

Index

Topic 1…………………………………………
Introduction of database management
system?

Topic 2…………………………………………
Data base Administrator?

Topic 3…………………………………………
Schema , Sub Schema , Instance?
Topic 4…………………………………………
What Is a Data Warehouse?