What is NoSQL?

Traditionally, the software industries use relational databases to store and manage data persistently. Not only SQL or
NoSQL is a new set of a database that has emerged in the recent past as an alternative solution to relational
databases.

Carl Strozzi introduced the term NoSQL to name his file-based database in 1998.

NoSQL refers to all databases and data stores that are not based on the Relational Database Management Systems
or RDBMS principles. It relates to large data sets accessed and manipulated on a Web scale. NoSQL does not
represent single product or technology. It represents a group of products and a various related data concepts for
storage and management. NoSQL was a hashtag that was chosen for a tech meetup to discuss the new databases.

Database Features of NoSQL

NoSQL Databases can have a common set of features such as:

 Non-relational data model.

 Runs well on clusters.
 Mostly open-source.
 Built for the new generation Web applications.
 Is schema-less.

Why NoSQL?

With the explosion of social media, user-driven content has grown rapidly and has increased the volume and type of
data that is produced, managed, analyzed, and archived. In addition, new sources of data, such as sensors, Global
Positioning Systems or GPS, automated trackers, and other monitoring systems generate huge volumes of data on a
regular basis.
These large volumes of data sets, also called big data, have introduced new challenges and opportunities for data
storage, management, analysis, and archival. In addition, data is becoming increasingly semi-structured and sparse.
This means that RDBMS databases which require upfront schema definition and relational references are examined.
To resolve the problems related to large-volume and semi-structured data, a class of new database products has
emerged. These new classes of database products consist of column-based data stores, key/value pair databases,
and document databases. Together, these are called NoSQL. The NoSQL database consists of diverse products with
each product having unique sets of features and value propositions.

Difference Between RDBMS and NoSQL Databases

Given below is a table that explains the difference between RDBMS (Relationship Database Management Systems)
and NoSQL.
RDBMS NoSQL
 Data is stored in a relational model, with
rows and columns.
 A row contains information about an item  Data is stored in a host of different databases, with
while columns contain specific information, different data storage models.
such as ‘Model’, ‘Date of Manufacture’,  Follows dynamic schemas. Meaning, you can add
‘Color’. columns anytime.
 Follows fixed schema. Meaning, the  Supports horizontal scaling. You can scale across
columns are defined and locked before data multiple servers. Multiple servers are cheap commodity
entry. In addition, each row contains data hardware or cloud instances, which make scaling cost-
for each column. effective compared to vertical scaling.
 Supports vertical scaling. Scaling an  Not ACID Compliant.
RDBMS across multiple servers is a
challenging and time-consuming process.
 Atomicity, Consistency, Isolation &
Durability(ACID) Compliant
Benefits of NoSQL

The desired technical characteristics of an enterprise-class NoSQL solution are as follows.

1. Primary and Analytic Data Source Capability

The first criterion of an enterprise-class NoSQL solution is it must serve as a primary or active data source that
receives data from different business applications. It also must act as a secondary data source or analytic database
that enhances business intelligence applications. From business perspective, the NoSQL database must be capable
of quickly integrating all types of data viz structured, semi-structured, or unstructured. In addition, it must be able to
execute high-performance queries.
Once all the required data is collected in the database, data administrators may want to perform an analysis in real
time and in map-reduce form. An enterprise-class NoSQL database can easily handle such requests using the same
database. It does not require loading the data into a separate analytic database for analysis.

2. Big Data Capability

NoSQL databases are not restricted to working with big data. However, an enterprise-class NoSQL solution can scale
to manage large volumes of data from terabytes to petabytes. In addition to storing large volumes of data, it delivers
high performance for data velocity, variety, and complexity.

3. Continuous Availability or No Single Point of Failure

To be considered enterprise-class, a NoSQL database must offer continuous availability, with no single point of failure.
Moreover, rather than providing the continuous availability feature outside the software, the NoSQL solution delivers
inherent continuous availability.
The NoSQL databases must include the following key features:

 All nodes in a cluster must be able to serve read request even if some machines are down.
 Must be capable of easily replicating and segregating data between different physical shelves in a data center.
This helps avoid hardware outages.
 Must be able to support data distribution designs that are multi-data centers, on-premises or in the cloud.

4. Multi-Data Center Capability

Typically, business enterprises own highly distributed databases that are spread across multiple data centers and
geographic locales. Data replication is a feature offered by all legacy RDBMS. However, none can offer a simple
model of data distribution between various data centers without any performance issue.
A simple method includes the ability to handle multiple data centers without concerning the occurrences of the read
and write operations. A good NoSQL enterprise solution must support multi data-center deployment and must provide
configurable option to maintain a balance between performance and consistency.

5. Easy Replication for Distributed Location-Independent Capabilities

To avoid data loss affecting an application, a good NoSQL solution provides strong replication abilities. These include
a read-anywhere and write-anywhere capability with full location-independence support.
This means you can write data to any node in a cluster, have it replicated on other nodes, and make it available to all
users irrespective of their location. In addition, the write capability on any node must ensure that the data is safe in the
event of a power failure or any other incident.

6. No Need for Separate Caching Layer

A good NoSQL solution is capable of using multiple nodes and distributing data among all the participating nodes.
Thus, it does not require a specific caching layer to store data. The memory caches of all participating nodes can store
data for quick input and output or I/O access. NoSQL database eliminates the problem of synchronizing cache data
with the persistent database. Thus, it supports simple scalability with fewer management issues.
Cloud-Ready
As an adaptation of cloud infrastructure is increasing day by day, an enterprise-class NoSQL solution must be cloud-
ready.
A NoSQL database cluster must be able to function in a cloud setting, such as Amazon EC2, and also must be able to
expand and contract a cluster when necessary. It also must support a hybrid solution where part of the database is
hosted within the enterprise premise and another part is hosted in a cloud setting.

7. High Performance with Linear Scalability

An enterprise-class NoSQL database can enhance performance by adding nodes to a cluster. Typically, the
performance of database systems may go down when additional nodes are added to a cluster. However, a good
NoSQL solution increase performance for both read and writes operations when additional nodes are added. These
performance gains are linear in nature.
 8. Flexible Schema Support
An enterprise-class NoSQL database offers a flexible or dynamic schema design to manage all types of data—
structured, semi-structured, and non-structured. Therefore, the need to have different vendors to support the different
data types does not arise.
NoSQL databases may support various schema formats, such as columnar/Bigtable and document.
Therefore, choosing an appropriate database based on application requirement is a key design decision.
The flexible or dynamic schema support ensures that you can make schema changes to a structure without making
the structure offline. This support is critical considering the near-zero downtime and round-the-clock availability for
business applications.

9. Support Key Developer Languages and Platforms

Ideally, an enterprise-class NoSQL solution must support all major operating systems. In addition, it must run on a
product hardware that does not require any tweaks or other proprietary add-ons.
The NoSQL database must provide client interfaces and drivers for all common developer languages. It must offer a
structured query language or SQL or a similar language that helps store and access data in a NoSQL database.

10. Easy to Implement, Maintain, and Grow

A NoSQL database must be simple but robust. In other words, it must be easy to implement and use and must offer
sturdy functionality to handle various enterprise applications.
In addition, the NoSQL vendor must supply good management tools that assist data professionals to perform various
administrative tasks, such as adding capacity to a cluster, running utility tasks, and so on.
The NoSQL database must allow easy growth without making any change to the front-end of the business application.

11. Thriving Open Source Community

For an open source NoSQL database, having a vibrant community is essential to make a regular contribution to
enhance the core software.
Moreover, open source communities generally provide excellent quality assurance or QA testing. This sometimes
eliminates the need for software companies to hire, train, and retain a QA team.
To encourage a thriving open source community, including activities on mailing lists and technical forums, initiate
technical discussions, and participate in conferences.

Types of NoSQL

There are four basic types of NoSQL databases.

1. Key-Value database
Key-Value database has a big hash table of keys and values. Riak (Pronounce as REE-awk), Tokyo Cabinet, Redis
server, Memcached ((Pronounce as mem-cached), and Scalaris are examples of a key-value store.

2. Document-based database
Document-based database stores documents made up of tagged elements. Examples: MongoDB, CouchDB,
OrientDB, and RavenDB .

3. Column-based database
Each storage block contains data from only one column, Examples: BigTable, Cassandra, Hbase, and Hypertable.
 4. Graph-based database
A graph-based database is a network database that uses nodes to represent and store data. Examples are Neo4J,
InfoGrid, Infinite Graph, and FlockDB. The availability of choices in NoSQL databases has its own advantages and
disadvantages.
The advantage is, it allows you to choose a design according to your system requirements. However, because you
have to make a choice based on requirements, there is always a chance that the same database product may not be
used properly.

Key-Value Database

From an Application Program Interface or API perspective, a key-value database is the simplest NoSQL database.

This database stores every single item as a key with a value. You can get the value for a key, add a value for a key, or
delete a key. The value is a blob that the database stores without knowing its content. The responsibility lies with the
application to understand what is stored. Typically, key-value databases use primary-key access.
Therefore, they generally offer enhanced performance and scalability. All key-value databases may not have the same
features.
For example, data is not persistent in Memcached while it is in Riak.
These features are important when implementing certain solutions.
For example, you need to implement caching of user preferences.
If you implement them in Memcached, you may lose all the data when the node goes down and may need to get them
from the source database. If you store the same data in Riak, you may not lose data but must consider how to update
the stale data. It is important to select a key-value database based on your requirements.

Advantages & Disadvantages of Key-Value Database

The key value store does not have a defined schema. It contains client defined semantics for understanding what the
values are. A key value store is simple to build and easy to scale. It also tends to have great performance because the
access pattern can be optimized to suit your requirement.
The advantages & disadvantages of the key-value store include the following.

Advantages Disadvantages
 Queries: You can perform a query  It does not provide any
by using the key. Even range traditional database capabilities,
queries on the key are usually not such as consistency when
possible. multiple transactions are
 Schema: Key value databases executed simultaneously.
 have the following schema - the  As the volume of data
key is a string, the value is a blob. increases, maintaining unique
The client determines how to values as keys become difficult.
parse data.
 Usages: Key value databases can
access data using a key. Key-
value type database suffers from
major weaknesses.

Document Database

This NoSQL database type is based on the concept of documents.

 It stores and retrieves various documents in formats, such as XML, JavaScript Object Notation or JSON
(Pronounce as JAY- Sahn), Binary JSON or BSON.
 These documents are self-descriptive, hierarchical tree data structures which consist of maps, collections, and
scalar values.
 The stored documents can be similar to each other, but not necessarily the same. It stores documents in the
value part of the key-value database. You can consider the document databases as key-value stores where
you can examine the values.

Document Database Example

Examples of Document databases are:

 MongoDB: Provides a rich query language and many useful features such as built-in support for MapReduce-
style aggregation and geospatial indexes.
 Apache CouchDB: Uses JSON for documents, JavaScript for MapReduce indexes, and regular HTTP for its
API.

Example: (MongoDB) document

{Name:“SimpliLearn",
Address:" 10685 Hazelhurst Dr, Houston, TX 77043, United States”,
Courses: [“Big Data”,”Python”,”Android”,”PMP”,”ITIL”],
Offices: [ “NYK”,”Dubai”,”BLR”],
}

Column-Based Database

Column-based databases store data in column families as rows. These rows contain multiple columns associated with
a row key.

Column families are groups of related data that is accessed together.

For example, you may access customer profile information at the same time, but not their order history.

 Each column family is like a container of rows in an RDBMS table where the key identifies the rows.
 Each row consists of multiple columns. However, the various rows need not have the same columns.
Moreover, you can add a column to any row at any time without adding it to other rows.

Goals of Column-Based Database

The goal of a Column-based database is to efficiently read and write data to and from hard disk storage to quickly
return a query. In this database, all column one values are physically together, followed by all the column two values.

The data is stored in record order so that the 100th entry for column one and the column two are from the same input
record. This allows you to access individual data elements, such as customer name, as a group in columns, rather
than individually row-by-row.

The compression permits columnar operations like MIN, MAX, SUM, COUNT, and AVG— to be performed very
rapidly. A column-based database management system or DBMS is self-indexing, therefore it uses less disk space
than an RDBMS containing the same data.

Diagramatic Representation of Column-Based Database

The diagram provided in the section depicts that data is getting stored in a column rather than row format. It shows
columns for the same column family are stored together in one file on the hard disk.
Therefore, these data can be retrieved fast in an efficient manner.

Column-Based Database Example

Cassandra is a column-based database.

 Cassandra is fast and easily scalable with write operations spread across the cluster.
 The cluster does not have a master node, hence, any node can handle the read and write operations.

Graph Database

A graph database lets you store data and its relationships with other data in the form of nodes and edges. Each
relation can have a set of properties. Edges have a direction which has its own significance and enables you to
explore the relationship in both the direction.
All the nodes in the graph are organized by relationships that help explore interesting and hidden patterns between the
nodes.
Examples of Graph Database

 Examples of graph database are Neo4J (pronounce as Neo- four-J), Infinite Graph, OrientDB, and FlockDB.
 Neo4J is one of the most popular graph databases, which is ACID compliant. It is the product of the company
Neo Technologies. It is Java based but has bindings for other languages, including Ruby and Python.
 FlockDB was created by Twitter for relationship related analytics.

In the graph database, the labeled property graph model is used for modeling the data. It is same as the entity
relationships or ER model used in RDBMS. The property graph contains connected entities, such as the nodes which
can hold any number of attributes or key-value-pairs.

CAP Theorem

In a distributed system, the following three properties are important.

  Consistency: Each client must have consistent or the same view of the data.
 Availability: The data must be available to all clients for read and write operations.
 Partition toleration: System must work well across distributed networks.

The CAP theorem was proposed by Eric Brewer.

According to this theorem, in any distributed system, you can use only two of the three properties—consistency,
availability, or partition tolerance simultaneously.

Many NoSQL databases provide options for a developer to choose to adjust the database as per requirement. For
this, understanding the following requirements is important:

 How the data is consumed by the system.

 Whether the data is read or write heavy.
 If there is a need to query data with random parameters.
 If the system is capable of handling inconsistent data.

Consistency

Consistency in CAP theorem and ACID are explained below.

Consistency in CAP theorem

Consistency in CAP theorem refers to atomicity and isolation. Consistency means consistent read and write
operations for the same sets of data so that concurrent operations see the same valid and consistent data state,
without any stale data.

Consistency in ACID

Consistency in ACID means if the data does not satisfy predefined constraints, it is not persisted. Consistency in CAP
theorem is different. In a single-machine database, consistency is achieved using the ACID semantics.
However, in the case of NoSQL databases which are scaled out and distributed providing consistency gets
complicated.

Availability

According to the CAP theorem, availability means:

  The database system must be available to operate when required. This means that a system that is busy,
uncommunicative, unresponsive, or inaccessible is not available.
 If a system is not available to serve a request at a time it is needed, it is unavailable.

Partition Tolerance

Partition tolerance or fault-tolerance is the third element of the CAP theorem. Partition tolerance measures the ability
of a system to continue its service when some of its clusters become unavailable.
In the next section, we will learn about MongoDB in terms of the CAP theorem.

MongoDB as Per CAP

By default, MongoDB offers strong consistency. This means after you perform a write operation, you cannot read the
same data until the write operation is successful. MongoDB is a single-master system and by default, all reads go to
the primary node.
Optionally, if you enable reading from the secondary node, MongoDB becomes eventually consistent and allows
reading of out-of-date results. In addition, MongoDB handles network partition very well by keeping same data on
multiple nodes or replica set.
Therefore, MongoDB is a consistent and partition tolerant database which comprises the availability aspect.

Summary

Here is a quick recap of what was covered in this lesson:

 NoSQL represents a class of products and a collection of diverse or related data concepts for storage and
manipulation.
 NoSQL databases are used to efficiently manage large-volume and semi-structured data.
 The four basic NoSQL database types are— Key-Value, Document-based, Column-based, and Graph-based.
 According to the CAP theorem, a distributed computer system cannot provide all the three properties together
—consistency, availability, and partition tolerance.