Data management & Databases

Data management refers to the processes and techniques used to control and organize data
throughout its lifecycle. It encompasses various activities aimed at ensuring data quality,
accessibility, security, and compliance. Effective data management is a crucial piece of
components that constitute information systems that run business applications and provide
analytical information to help drive operational decision-making and strategic planning by
corporate executives, business managers and other end users.

Here are some key aspects of data management:

● Data Governance: Data governance involves establishing policies, procedures, and

guidelines for managing data within an organization. It defines roles, responsibilities, and
processes for data management, ensuring data are handled consistently and in accordance
with regulatory requirements.
● Data Quality: Data quality management focuses on maintaining accurate, consistent, and
reliable data. It involves processes to identify and rectify data errors, inconsistencies,
duplicates, and incomplete or outdated information. Data cleansing, validation, and
standardization techniques are used to improve data quality.
● Data Integration: Data integration involves combining data from multiple sources into a
unified view. It enables organizations to have a holistic understanding of their data by
consolidating information from different systems or departments.
● Data Storage and Retrieval: Data management includes determining the most appropriate
storage infrastructure for different types of data. This may involve using databases, data
repositories, data warehouses, data lakes, or cloud-based storage solutions. Efficient
retrieval mechanisms, such as indexing and search algorithms, filters and queries are
implemented to access data quickly and accurately.
● Data Security and Privacy: Data security is crucial to protect sensitive and confidential
information from unauthorized access, data breaches, or data leaks, SQL injection attacks,
and malware/Ransomware. Data security measures includes implementing access controls,
encryption, authentication mechanisms, regular security audits, input validation, have
Backup and recovery and security protocols to safeguard data. Regular audits and
monitoring help detect and respond to suspicious activities promptly. Backup and recovery

1
 procedures contribute to resilience, ensuring data integrity and availability even in the face
of unexpected events. A comprehensive data security strategy is essential for maintaining
confidentiality, integrity, and availability in database systems.
● Data Lifecycle Management: Data lifecycle management involves managing data from
its creation or acquisition to its archival or disposal. It includes defining data retention
policies, data archiving, and data purging strategies. This ensures data is retained for the
required period, compliant with legal and regulatory requirements, and reduces storage
costs.
● Data Analytics and Business Intelligence: Data management supports data analysis and
business intelligence initiatives by providing clean, organized, and accessible data. It
involves creating data models, data warehouses, or data marts for efficient data analysis,
reporting, and decision-making.
● Data Documentation: Effective data management involves documenting data structures,
definitions, and relationships. Metadata management involves capturing and managing
metadata, which provides information about data attributes, sources, transformations, and
usage. Proper documentation and metadata management facilitate data understanding and
enhance data lineage and traceability.
● Data Compliance and Regulatory Requirements: Data management ensures compliance
with industry-specific regulations, legal requirements, and data privacy laws. It includes
establishing data governance frameworks, data protection policies, and procedures to meet
compliance standards. Compliance with data privacy regulations, such as the Data
Protection & Regulatory laws is also essential.

These aspects of data management are crucial for organizations to derive value from their data
assets, ensure data reliability and security, and make informed decisions based on accurate and
high-quality data.

Data management best practices

Implementing robust data management best practices is crucial for organizations with distributed
data environments, for instance:
● Prioritize a well-designed data management program, placing a strong emphasis on
achieving high data quality.

2
 ● Active involvement of business executives and users is essential to ensure alignment with
data needs and adherence to quality standards.
● Deploying databases and data platforms in accordance with adopted policy standards to
maintain consistency.
● IT and data managers to ensure that implemented information systems align with intended
purposes, delivering the necessary data processing capabilities and analytics information
vital for organizational business operations.
This comprehensive approach ensures effective data management and contributes to overall
organizational success.

Data Hierarchy
Listed below are the components of the data hierarchy in the ascending order of complexity:

Simple Bit (1s and 0s)

Byte (8 bits)
Field or item
Record
File
Most complex Database Database

This is called a data hierarchy because databases are composed of files; files are composed of
records, and so on.
Bit - The term bit is short for binary digit. It can assume either of the two states, representing
numeric value 0s or 1s. e.g. 1001012
Byte - In a computer system, basic unit of information is called a byte. A byte of information is
generally stored by using 8 bits in a specified combination, e.g., 111011002
Field or Item - A field or item of data is one or more bytes that contain data about attributes of
an entity.
Record - A record is a collection of fields relating to a specific entity.
File - A file is a collection of related records. The concept of a computer file is very similar to
the manual file in a filing cabinet.
Database - A database consists of all the files of an organization, structured and integrated to
facilitate updating of files and retrieval of information from them.

3
Databases
Collection information which is in an organized form for easier access, management, and various
updating is known as a database. A database is a collection of related data stored in a well-defined
structure. Data can be defined as a collection of facts and records on which we can apply reasoning
or can-do discussion or some calculation. Data is a numeric or alphanumeric group of symbols,
such as 2231971. The data is always easily available and is in plenty. It can be used for processing
some useful information from it. Also, it can be in redundant, can be irrelevant. Data can exist in
form of graphics, reports, tables, text, etc. that represents every kind of information, that allows
easy retrieval, updating, analysis, and output of data by systematically organized or structured
repository of indexed information.

Record: Collection of related data items, e.g. in the above example the three data items had no
meaning. But if we organize them in the following way, then they collectively represent
meaningful information.

Roll Name Age

1 ABC 19

Table or Relation: Collection of related records.

4
 Roll Name Age
1 ABC 19
2 DEF 22
3 XYZ 28

NB: The columns of this relation (table) are called Fields, Attributes or Domains. The rows are
called Tuples or Records. Several tables or relations constitute a database.

Database management
Database management refers to the process of organizing, storing, and retrieving data efficiently
and securely in a database system. It involves various tasks and techniques aimed at ensuring data
integrity, accessibility, and reliability. It involves a range of activities aimed at ensuring efficient
and secure handling of data. The process begins with database design, where the structure and
relationships between data elements are defined.
Data modeling and schema creation follow, providing a blueprint for organizing information.
Implementation involves deploying the database using a database management system (DBMS),
with administrators overseeing user access and permissions. Regular monitoring and optimization
activities maintain database performance, involving tasks such as indexing, query optimization,
and space management.
Backups and recovery plans are crucial for data integrity and business continuity. Database
security measures, including authentication and encryption, protect against unauthorized access.
Ongoing maintenance includes software updates, addressing potential vulnerabilities, and ensuring
data remains consistent and accurate. Database management encompasses these activities to
facilitate reliable, secure, and efficient handling of data throughout its lifecycle within an
organization.
Here are some key aspects of database management:
a) Database Design: It involves designing the structure and layout of the database, including
tables, relationships, and data types. A well-designed database ensures efficient data
storage and retrieval.
b) Data Modeling: Data modeling is the process of creating a conceptual representation of
the database. It involves defining entities, attributes, and relationships between them.

5
 Commonly used data modeling techniques include Entity-Relationship (ER) diagrams and
Unified Modeling Language (UML).
c) Data Storage and Organization: Databases store data in structured formats using tables,
rows, and columns. The data is organized based on predefined schemas, such as Relational
Database Management Systems (RDBMS) or NoSQL databases. Efficient indexing and
partitioning strategies can be employed for faster data access and management.
d) Data Retrieval: Retrieving data from a database involves querying the database using
query languages such as SQL (Structured Query Language). SQL allows users to retrieve
specific data based on conditions and criteria defined in the query.
e) Data Security, Backup and Recovery: Database management includes implementing
security measures to protect data from unauthorized access, modification, or loss. Regular
backups of the database are crucial to prevent data loss in case of system failures, disasters,
or accidental deletion. Database administrators should establish backup schedules and
implement recovery procedures to restore data to a consistent state.
f) Performance Tuning: Optimizing the performance of a database involves analyzing and
improving factors like query execution time, indexing strategies, database configuration,
and hardware resources. Performance tuning ensures efficient data retrieval and
processing.
g) Database Scalability: As data grows, a database should be scalable to handle increased
storage requirements and user demand. Scaling techniques include vertical scaling (adding
more resources to a single server) or horizontal scaling (distributing data across multiple
servers).
h) Data Integration: In many cases, data needs to be integrated from multiple sources into a
centralized database. This may involve data extraction, transformation, and loading
processes to ensure consistency and accuracy across different data sources.

Thus, effective database management is essential for businesses and organizations to ensure data
availability, reliability, and security, enabling informed decision-making and efficient operations.

Database Management System

Databases are managed by software system known as Database Management Systems (DBMS).
It is software that enables the creation, organization, management, and retrieval of data in a

6
structured and efficient manner. It provides a set of tools and capabilities for interacting with
databases and allows users to store, manipulate, and retrieve data easily.
A DBMS is a set of rules and procedures which help us to create organize and manipulate the
database. It also helps to add, modify, delete, edit data items/records in the database. It is important
because without the existence of such programs it is not possible to maintain the database.
Examples of DBMS’s are Microsoft Access, msSQL, Oracle, and MySQL, PostQress among many
others. Just as a word processor (e.g., Microsoft Word) is used to create and edit documents, a
DBMS is used to create and manage databases. Inside a database, the data is recorded in a table
which is a collection of rows, columns, and it is indexed so that to find relevant information
becomes an easier task. As new information is added, data gets updated, expanded and deleted.
The various processes of databases create and update themselves, querying the data they contain
and running applications against it.

Advantages of DBMS
• Reduction of Redundancy: This is perhaps the most significant advantage of using DBMS.
Redundancy is the problem of storing the same data item in more one place. Redundancy
creates several problems like requiring extra storage space, entering same data more than
once during data insertion, and deleting data from more than one place during deletion.
Anomalies may occur in the database if insertion, deletion etc are not done properly.
• Sharing of Data: In a paper-based record keeping, data cannot be shared among many users.
But in computerized DBMS, many users can share the same database if they are connected
via a network.
• Data Integrity: We can maintain data integrity by specifying integrity constrains, which are
rules and restrictions about what kind of data may be entered or manipulated within the
database. This increases the reliability of the database as it can be guaranteed that no wrong
data can exist within the database at any point of time.
• Data security: We can restrict certain people from accessing the database or allow them to
see certain portion of the database while blocking sensitive information. This is not possible
very easily in a paper-based record keeping.

Disadvantages of DBMS

7
However, there could be a few disadvantages of using DBMS. They can be as following:
• As DBMS needs computers, we have to invest a good amount in acquiring the hardware,
software, installation facilities and training of users.
• We have to keep regular backups because a failure can occur any time. Taking backup is a
lengthy process and the computer system cannot perform any other job at this time.
• While data security system is a boon for using DBMS, it must be very robust. If someone
can bypass the security system then the database would become open to any kind of
mishandling.