Coursework Electronic Submission Cover Sheet

Person Number: PN1097197 / AU2200612

Student Name: Arkar Hmue Htet

Module Code: 5502CSMM

Module Title: Database System

Coursework Title: Developing for NoSQL

Name of Tutor: Dr. Yuzana Win

Due Date: 13th December 2023

Word Count: 3751

Marker (If different from tutor):

Group Submission? NO

Group Members: -
Please include the names of all group
members, and if you are submitting
anonymously include their six digit
Person Numbers.

Contents
Introduction................................................................................................................................................2
History of DBMS..........................................................................................................................................2
Rise of NoSQL..............................................................................................................................................3
Types of No SQL..........................................................................................................................................4
Key-Value Store.......................................................................................................................................4
Document Store......................................................................................................................................4
Column DB...............................................................................................................................................4
Graph DB.................................................................................................................................................4
Advantages of NoSQL:................................................................................................................................5
Disadvantages of NoSQL:............................................................................................................................5
Difference Between SQL and No SQL.........................................................................................................6
Data Model..............................................................................................................................................6
Scalability.................................................................................................................................................6
Querying Language..................................................................................................................................6
Schema Flexibility....................................................................................................................................7
ACID Properties.......................................................................................................................................7
Use Cases.................................................................................................................................................7
Data Models / Type of Schema..................................................................................................................7
Berkeley DB.............................................................................................................................................7
MongoDB.................................................................................................................................................7
Berkeley DB and MongoDB similarities......................................................................................................8
Data Persistence......................................................................................................................................8
Embeddable Architecture........................................................................................................................8
ACID Properties.......................................................................................................................................8
Indexing Capabilities................................................................................................................................8
Functionalities between Berkeley DB, MongoDB & RDBMS......................................................................8
Data Storage............................................................................................................................................9
Data Models and Relationships...............................................................................................................9
Query Language and Connectivity...........................................................................................................9
Indexing and Performance......................................................................................................................9
Concurrency and Transactions................................................................................................................9
Management and Backups....................................................................................................................10
Query Mechanisms...................................................................................................................................10
 Berkeley DB...........................................................................................................................................10
MongoDB...............................................................................................................................................13
Management of NoSQL.............................................................................................................................17
Security..................................................................................................................................................17
Indexing.................................................................................................................................................17
Concurrency..........................................................................................................................................17
Backup & Recovery................................................................................................................................17
Tools for Design & Dev.............................................................................................................................18
NetBeans...............................................................................................................................................18
Visual Studio..........................................................................................................................................18
MongoDB Shell (mongo)........................................................................................................................18
MongoDB Compass...............................................................................................................................18
Conclusion.................................................................................................................................................19
References................................................................................................................................................19

Introduction

The evolution of Database Management Systems (DBMS) marks a dynamic journey from early data
processing systems in the 1960s to modern innovations reshaping data handling today. Beginning with
pioneering systems like IBM's IMS and IDS, the emergence of Edgar Codd's relational model in the 1970s
revolutionized data storage, giving rise to the widely adopted Relational Database Management Systems
(RDBMS) such as Oracle and SQL Server. However, the evolving landscape of data needs led to the
advent of NoSQL databases in response to challenges faced by traditional RDBMS in managing diverse
data types and scalability. This evolution reflects a transformative shift from structured models to more
adaptable and scalable databases, catering to the dynamic requirements of contemporary applications.

History of DBMS

The evolution of Database Management Systems (DBMS) commenced in the early 1960s when
computers began their journey into data processing. It marked the genesis of systems aiming to
structure and manage data effectively. Two pioneering systems that emerged during this period were
IBM's IMS (Information Management System) and the Integrated Data Store (IDS). IMS, a hierarchical
database and information management system, specialized in transaction processing. Simultaneously,
IDS served as an early network database management system, widely utilized in industrial applications,
setting the initial foundation for systematic data handling in hierarchical or networked structures to
streamline business operations.

In the 1970s, Edgar Codd introduced a groundbreaking concept in his paper titled “A Relational Model of
Data for Large Shared Data Bank.” This revolutionary model proposed storing data in tables with
relationships defined by keys, introducing the relational model. This concept transformed data storage,
heralding the emergence of Relational Database Management Systems (RDBMS) like Oracle, IBM DB2,
and SQL Server. RDBMS gained immense popularity owing to their structured representation of data in
tables, offering simplicity, flexibility, and stringent adherence to the ACID (Atomicity, Consistency,
Isolation, Durability) properties. This marked a significant shift in data management practices, fostering
the widespread adoption of relational databases across various industries.

Rise of NoSQL

The rise of NoSQL databases emerged as a response to the limitations posed by traditional relational
databases in handling specific types of data and accommodating scalable, distributed systems. The term
"NoSQL" initially stood for "Not Only SQL," signifying a departure from the rigid structures and
constraints of SQL-based systems.

The need for alternatives became evident with the exponential growth of web applications and the
advent of big data, where traditional SQL databases faced challenges in efficiently managing
unstructured or semi-structured data. NoSQL databases surfaced to address these challenges by offering
flexible data models and enhanced scalability.

The roots of the NoSQL movement can be traced back to the early 2000s. Initially, various internet
companies, such as Google, Amazon, and Facebook, encountered difficulties managing vast volumes of
data generated by their web applications using conventional relational databases. They required
systems capable of handling massive amounts of unstructured or semi-structured data while also
providing better scalability and performance.

This led to the development of various NoSQL database types (Document Stores, Key-Value Stores,
Column DB, Graph DB) each catering to specific use cases and data storage requirements:

The rise of NoSQL databases brought a paradigm shift in data management, offering solutions that
better fit the needs of modern applications dealing with diverse data types and requiring high scalability.

Types of No SQL
Key-Value Store
Key-Value databases, like Berkeley DB, manage Associative Arrays, also known as Dictionaries,
employing a Search-Tree or a Hash-Table. These databases operate with collections of Records
accessible through specific Keys, which may or may not pre-define a Data-Type. They excel in simplicity
and performance, refraining from complex Parsing or Processing. While lacking significant support for
NULL and imposing no logical limits on Scaling or Partitioning, this flexibility may serve as both an
advantage or a disadvantage based on varying needs. They offer layers facilitating extensible data
models, such as Object-Oriented Programming, allowing dynamic changes in Data Models.

Document Store
Document databases, a subset of Key-Value databases, differ notably in their handling of data. In Key-
Value databases, data is opaque, while Document databases, like MongoDB using JSON (JavaScript
Object Notation), emphasize the structure of data. JSON, a lightweight data format, represents data with
Key-Value pairs and allows values that can be primitive (e.g., int, string), objects, or arrays. These
databases share many traits with Key-Value databases, often providing dedicated Integrated
Development Environments (IDEs) like MongoDB Compass. They support Native Querying due to the
structured nature of the data, which can be advantageous or limiting depending on the situation.

Column DB
Column databases organize data using Tables, employing Column-Wise storage as opposed to Row-Wise
storage in traditional Relational Database Systems (RDBS). The key distinction lies in Column-Wise
indexing, where the data becomes the primary focus rather than the key itself. This storage strategy
benefits Data Warehousing, especially in querying Fact Tables based primarily on their Columns rather
than Rows. It allows for compression and efficient storage due to storing data of the same domain in
each storage unit.

Graph DB
Graph databases consist of nodes representing entities and edges denoting relationships between
nodes. Nodes and edges hold identities and properties, often in a Key-Value or JSON format. Nodes are
directional by name but bi-directional by navigation. Graph databases excel in native support for many-
to-many relationships, simplifying data storage and processing in hierarchical structures. Their key
advantage lies in handling complex relationships, making them particularly suitable for scenarios such as
Social Networking.

Advantages of NoSQL:
Scalability: NoSQL databases are designed to handle large volumes of data and can easily scale
horizontally across multiple servers, allowing for seamless expansion compared to traditional relational
databases.

Flexible Schema: NoSQL databases often offer schema flexibility, allowing for the storage of various
types of unstructured, semi-structured, or polymorphic data without a fixed schema, which can be
advantageous in handling diverse data types.

Performance: In certain use cases, NoSQL databases can offer better performance compared to
traditional SQL databases due to their optimized data retrieval and storage mechanisms, especially
when dealing with large-scale, distributed systems.

High Availability: Many NoSQL databases are designed with features for high availability, ensuring
continuous data access even in the event of hardware failures or network issues.

Horizontal Scaling: NoSQL databases, particularly some types like document-oriented or key-value
stores, are well-suited for horizontal scaling, distributing data across multiple nodes or servers to
improve performance and handle increased loads.

Disadvantages of NoSQL:

Consistency and Transactions: Some NoSQL databases sacrifice strong consistency and ACID
transactions in favor of scalability and performance. This trade-off might not be suitable for applications
requiring strict consistency.

Query Functionality: Not all NoSQL databases offer powerful querying capabilities similar to SQL
databases. Depending on the type of NoSQL database, querying functionalities might be limited, making
complex queries more challenging to execute.

Maturity and Support: Some NoSQL databases might lack maturity compared to established SQL
databases. Consequently, they might have limited toolsets, less documentation, or a smaller user
community, leading to potential challenges in adoption and support.
Learning Curve: Transitioning from SQL to NoSQL databases might require a learning curve for
developers accustomed to relational databases. Understanding the specific nuances of each NoSQL type
and its associated query language can be time-consuming.

Data Integrity and Joins: NoSQL databases might not handle complex relationships or data integrity
constraints as effectively as SQL databases, which can lead to challenges in maintaining data consistency
in certain scenarios.

Difference Between SQL and No SQL

Data Model
SQL: SQL databases follow a rigid schema-based structure. They organize data into tables with
predefined schemas, enforcing relationships between tables through foreign keys.

NoSQL: NoSQL databases offer flexible data models. They can accommodate unstructured, semi-
structured, or structured data without enforcing a fixed schema. Different types of NoSQL databases,
like document stores, key-value stores, column-family stores, and graph databases, handle data in
diverse ways, allowing more adaptable structures.

Scalability
SQL: Scaling SQL databases vertically (adding more resources to a single server) might have limitations.
Horizontal scaling across multiple servers can be complex.

NoSQL: NoSQL databases are generally designed for horizontal scalability. They excel in distributed
architectures and can scale out across various servers more efficiently, making them suitable for
handling vast amounts of data and high traffic applications.

Querying Language
SQL: SQL databases use the Structured Query Language (SQL) for data manipulation and querying. SQL
offers a standard language for accessing and managing relational databases.

NoSQL: NoSQL databases often use different query languages tailored to their specific data models. For
instance, document-oriented databases might use specialized query languages like MongoDB's query
language or JSON-based querying.

Schema Flexibility
SQL: SQL databases enforce a predefined schema. Any changes to the schema often involve complex
migration procedures and may require downtime.

NoSQL: NoSQL databases support schema flexibility, allowing changes to the data structure without
disrupting the application. This flexibility enables agile development and easier handling of evolving data
requirements.
ACID Properties
SQL: SQL databases usually adhere to ACID (Atomicity, Consistency, Isolation, Durability) properties,
ensuring transactional integrity and data consistency.

NoSQL: NoSQL databases might sacrifice some ACID properties in favor of scalability and performance.
They often offer eventual consistency instead of immediate consistency.

Use Cases
SQL: SQL databases are suitable for applications with structured data and established relationships, such
as accounting systems, financial applications, and systems requiring complex transactions.

NoSQL: NoSQL databases are preferred for applications dealing with unstructured, semi-structured data,
high scalability, and high-speed data processing, such as social media platforms, IoT (Internet of Things)
applications, real-time analytics, and content management systems.

Data Models / Type of Schema

Berkeley DB
Berkeley DB is an open-source embedded database library that operates as a Key-Value store, managing
Associative Arrays or Dictionaries using diverse data structures such as B-trees or Hash-tables. It stands
out for its simplicity, high performance, and flexibility. Opting for efficiency over complexity, Berkeley DB
refrains from performing intricate parsing or processing, making it well-suited for handling simple data
operations. Moreover, it permits the implementation of more complex data models, like Object-
Oriented Programming (OOP), by providing layered structures. However, Berkeley DB lacks support for
NULL values and doesn't impose logical scaling or partitioning limits. Commonly utilized in embedded
systems and applications requiring high-performance data storage with relatively straightforward data
access patterns, Berkeley DB remains a favored choice for its efficient and effective data management
capabilities.

MongoDB
MongoDB is a widely used document-oriented NoSQL database. It distinguishes itself by storing data in
flexible, schema-less documents utilizing the JSON-like BSON format. These documents, organized into
collections, offer dynamic schemas and accommodate diverse data types. MongoDB provides robust
querying and indexing capabilities, supporting complex data structures, and thereby proving highly
adaptable for modern application development needs. Offering a balance between flexibility and
structure, MongoDB seamlessly adjusts to evolving data requirements. Its support for native querying
and dynamic data models makes it a versatile and scalable solution. Predominantly applied in web
applications, content management systems, real-time analytics, and various scenarios dealing with
unstructured or semi-structured data, MongoDB remains a favored choice due to its scalability and
adaptability in managing modern data challenges.

Berkeley DB and MongoDB similarities

Berkeley DB and MongoDB, despite their categorization as different types of NoSQL databases (Berkeley
DB as a Key-Value Store and MongoDB as a Document Store), share some fundamental similarities in
their core technology.

Data Persistence
Both Berkeley DB and MongoDB prioritize data persistence. They are designed to durably store data,
ensuring that data remains available even in the event of system failures or crashes.

Embeddable Architecture
Both databases offer an embeddable architecture, allowing them to be integrated directly into
applications. This feature enables developers to work with the database without the need for separate
processes or servers.

ACID Properties
Both databases support ACID (Atomicity, Consistency, Isolation, Durability) properties to varying
degrees. This ensures that database transactions are reliable, maintain data integrity, and are robust
against concurrent access.

Indexing Capabilities
Both Berkeley DB and MongoDB offer indexing capabilities to optimize query performance. They allow
developers to create indexes on fields, improving search efficiency and speeding up data retrieval.

Functionalities between Berkeley DB, MongoDB &

RDBMS

Data Storage
Berkeley DB operates as a Key-Value Store, storing data in a simplistic key-value pair format, ideal for
quick retrieval of data through unique keys. MongoDB functions as a Document Store, where data is
stored in JSON-like flexible documents, allowing for nested structures and accommodating diverse data
types within a single document. On the other hand, RDBMS, such as MySQL or PostgreSQL, adopts a
table-based structure with predefined schemas, organizing data into rows and columns, facilitating the
management of structured data with defined relationships between tables.

Data Models and Relationships

In Berkeley DB, the primary focus revolves around key-value pair storage, making it suitable for
straightforward data storage but less adept at managing complex relationships. MongoDB excels in
handling more intricate data structures by supporting embedded documents and arrays, allowing for the
representation of complex relationships within documents. RDBMS, through its normalized tables and
JOIN operations, establishes relationships between entities using primary and foreign keys, enabling the
management of complex data relationships within a relational schema.

Query Language and Connectivity

Berkeley DB, lacking a dedicated query language, provides APIs for languages like Java or C++, requiring
developers to build custom querying mechanisms. In contrast, MongoDB employs its query language,
MongoDB Query Language (MQL), equipped with various operators and aggregation pipelines,
facilitating efficient data retrieval and manipulation. RDBMS uses SQL (Structured Query Language), a
standardized language offering robust querying capabilities for complex data retrieval, manipulation,
and management of relational databases.

Indexing and Performance

Berkeley DB offers indexing functionalities for optimizing key-value pair retrieval, ensuring swift access
to specific data elements. MongoDB provides indexing on fields, enhancing query performance by
enabling faster retrieval based on indexed criteria. RDBMS utilizes indexes extensively to expedite data
retrieval, particularly in large datasets, employing indexing on columns to enhance query performance
significantly.

Concurrency and Transactions

Berkeley DB supports concurrency control mechanisms and ACID transactions, ensuring data
consistency and reliability, especially in multi-user scenarios. MongoDB offers support for concurrency
and transactions in certain configurations (like replica sets), albeit with some limitations compared to
traditional RDBMS systems. RDBMS, renowned for its support of ACID properties and robust
concurrency control, assures data integrity and reliability in multi-user environments through
comprehensive transactional support.

Management and Backups

Berkeley DB provides backup and recovery features but might require additional tools or custom
solutions for comprehensive database management. MongoDB offers tools for backup, recovery, and
cluster management, providing robust management capabilities within its ecosystem. RDBMS, boasting
mature and comprehensive solutions, offers extensive tools and functionalities for backup, recovery,
and administration, providing robust management features for maintaining data integrity and security in
diverse scenarios.

Query Mechanisms

Berkeley DB
Language Support & Connectivity

Language Support: Berkeley DB offers language support primarily through native APIs, with
implementations primarily in C and C++. While it provides native support for these languages, it also
offers some language-specific APIs or bindings, such as a Java API for Java developers. However,
compared to MongoDB's extensive list of officially supported languages, Berkeley DB might have more
limited language bindings, often relying on community-contributed libraries or wrappers for languages
like Python, Perl, and PHP.

Connectivity: Berkeley DB, as a database library, integrates directly into applications rather than being
accessed via a separate server process. It primarily offers connectivity through its native APIs, allowing
applications to interact directly with the database within their codebase. Some versions, like SQLite,
provide SQL APIs for simpler interaction and querying. However, Berkeley DB typically doesn't have the
variety of connectivity options seen in full-fledged databases like MongoDB, relying more on direct API
usage within the application.

course
Module

Adding new courses and modules

Function to show course and module

Result
MongoDB

Language Support & Connectivity

Language Support: MongoDB boasts extensive language support, offering official drivers for a multitude
of programming languages such as JavaScript, Python, Java, Ruby, C#, C++, PHP, and more. This wide
array of language bindings enables developers to seamlessly integrate MongoDB into their applications,
facilitating efficient data interaction and manipulation using their language of choice. These drivers are
regularly updated and maintained by MongoDB, ensuring compatibility and ease of use across diverse
development environments.

Connectivity: MongoDB provides diverse connectivity options, including MongoDB Compass, a graphical
user interface tool for database interaction, and MongoDB Atlas, a cloud-based service offering
connectivity, hosting, and management solutions. Additionally, MongoDB offers robust official drivers,
enabling seamless integration with applications through direct API calls in languages like JavaScript,
Python, Java, and others. Furthermore, third-party ODBC/JDBC connectors facilitate connections with
applications that adhere to these standards, offering flexibility and compatibility in various
environments.

Json File of Table Course and Module Training Center Database to Use in MongoDB
Adding Json file to database db1 collection course

Establishing Connection with the MongoDatabase db1

Function to Display All the Data in JSON

Result

Function to Display Course that has Module Called ‘Oracle’

Result

Function to Insert a Module

Result (Module Inserted)

Function to Delete a Module

Result (Module with code ‘A1’ removed)

Function to show the Maximum Credit

Result
Management of NoSQL
Security
NoSQL databases have varying levels of security features. Most provide authentication and
authorization mechanisms to control access to data. They might support encryption both at rest and in
transit to secure data. Some NoSQL databases offer role-based access control (RBAC), auditing
capabilities, and integration with external security systems for enhanced security compliance.

Indexing
Indexing in NoSQL databases varies based on the type of database. Some NoSQL databases like
MongoDB offer flexible indexing options, allowing users to create indexes on specific fields to optimize
queries. Others, like key-value stores or document stores, might have limited indexing capabilities or
predefined indexing structures for efficient data retrieval.

Concurrency
NoSQL databases are designed to handle high concurrency and distributed architectures. They typically
use optimistic concurrency control, allowing multiple users to access and modify data concurrently
without locking the entire database. Some employ techniques like multi-version concurrency control
(MVCC) or distributed locking mechanisms to manage concurrency in distributed environments.

Backup & Recovery

NoSQL databases often provide backup and recovery mechanisms tailored to their architecture. Many
offer features for taking consistent snapshots of data, incremental backups, point-in-time recovery, and
replication across multiple nodes or data centers to ensure fault tolerance and data durability. Backup
and recovery strategies can vary widely among different NoSQL databases.

ACID vs CAP

The ACID (Atomicity, Consistency, Isolation, Durability) and CAP (Consistency, Availability, Partition
tolerance) are two fundamental sets of principles that define different aspects of database systems,
especially in the context of distributed systems. ACID properties ensure the reliability and robustness of
transactions within a database. Atomicity guarantees that transactions are either executed completely
or not at all, ensuring that partial transactions don't occur. Consistency maintains the integrity of data,
enforcing predefined rules and constraints. Isolation ensures that concurrent transactions don’t
interfere with each other, while Durability guarantees that committed transactions persist even in the
event of system failure. On the other hand, the CAP theorem in distributed systems describes the trade-
offs between Consistency, Availability, and Partition tolerance. It states that in a distributed system, it's
impossible to simultaneously achieve all three aspects. Distributed databases can prioritize either
Consistency and Partition tolerance (CP) or Availability and Partition tolerance (AP). CP databases
prioritize data consistency across distributed nodes, even in the face of network partitions, potentially
sacrificing some availability. In contrast, AP databases prioritize high availability and responsiveness,
potentially relaxing strict consistency guarantees under certain circumstances to maintain operations
despite network partitions. The choice between ACID and CAP principles often guides the design and
implementation of database systems, reflecting the trade-offs and priorities suited to specific use cases
and requirements, especially in distributed and fault-tolerant environments.

Tools for Design & Dev

NetBeans
A powerful open-source IDE primarily used for Java development but supports multiple languages
through plugins. For Java-based projects that involve Berkeley DB Java Edition, NetBeans provides a rich
development environment with features like code editing, debugging, and project management,
facilitating seamless integration with Berkeley DB applications.

Visual Studio
Code (VS Code): A popular and versatile code editor with extensive support for various programming
languages. It offers extensions for MongoDB and database-related development, providing syntax
highlighting, IntelliSense, and debugging features.

MongoDB Shell (mongo)

A command-line interface for interacting with MongoDB. Developers can perform CRUD operations, run
administrative commands, and execute JavaScript functions.

MongoDB Compass
As previously mentioned, MongoDB Compass is a GUI tool offering a visual way to interact with
MongoDB databases. It allows developers and administrators to view and manipulate data, manage
indexes, analyze performance, and more.
Conclusion
The evolution of Database Management Systems (DBMS) from structured, relational models to flexible
NoSQL solutions highlights the dynamic responses to evolving data needs. Each phase offers distinct
advantages—RDBMS with structured simplicity and NoSQL with scalability. However, challenges like
trade-offs between consistency and scalability persist. The differences between SQL and NoSQL
underscore their suitability for specific data types. Exploring NoSQL databases like Berkeley DB and
MongoDB reveals their unique functionalities and applications. Understanding management aspects—
security, indexing, concurrency, and backup & recovery—is crucial in selecting an optimal database
system. Overall, this evolutionary journey showcases the adaptability of databases to meet the dynamic
demands of modern applications.

References

https://www.techopedia.com/definition/24361/database-management-systems-dbms

https://www.mongodb.com/

https://en.wikipedia.org/wiki/Berkeley_DB

https://docs.oracle.com/cd/E17275_01/html/programmer_reference/intro_dbis.html

https://www.simplilearn.com/rise-of-nosql-and-why-it-should-matter-to-you-article

https://www.integrate.io/blog/the-sql-vs-nosql-difference/

https://www.xcubelabs.com/blog/nosql-databases-unlocking-the-power-of-non-relational-data-
management/