Module 7
NoSQL
Dr. S. RENUKA DEVI
Professor
SCOPE
VIT Chennai Campus
What is NoSQL?
Not Only SQL
Most NoSQL systems are distributed databases or
distributed storage systems, with a focus on semi-
structured data storage, high performance, availability,
data replication, and scalability
A structured relational SQL system may not be appropriate
for applications storing vast amount of data because
SQL systems offer too many services which the
application may not need
Traditional relational model may be too restrictive
Some of the organizations decided to develop their own
systems:
Google developed a proprietary NoSQL system known as
BigTable - used in many of Google’s applications such as
Gmail, Google Maps, and Web site indexing.
Apache Hbase is an open source NoSQL system based on
similar concepts.
Google innovation - Column-based or wide column
stores; also referred to as column family stores.
Amazon developed a NoSQL system called DynamoDB –
available through Amazon’s cloud services.
This innovation led to the category known as key-value
data stores or sometimes key-tuple or key-object data
stores.
Facebook developed a NoSQL system called Cassandra,
which is now open source and known as Apache Cassandra.
It uses concepts from both key-value stores and column-
based systems.
Other software companies started developing their own
solutions—for example, MongoDB and CouchDB, which
are classified as document-based NoSQL systems or
document stores.
Another category of NoSQL systems is the graph-based
NoSQL systems, or graph databases
Neo4j, graphbase
Characteristics of NOSQL Systems
NOSQL characteristics related to distributed databases and
distributed systems
Scalability -In NoSQL systems, horizontal scalability is generally
used, where the distributed system is expanded by adding more nodes
for data storage and processing as the volume of data grows.
Availability, Replication and Eventual Consistency
Replication Models
Master-slave replication requires one copy to be the master copy;
all write operations must be applied to the master copy and then
propagated to the slave copies
master-master replication allows reads and writes at any of the
replicas but may not guarantee that reads at nodes that store
different copies see the same values.
Sharding of Files (also known as horizontal partitioning) -
distribute the load of accessing the file records to multiple nodes
High-Performance Data Access: - hashing or range partitioning on
object keys
Characteristics of NOSQL Systems
NoSQL characteristics related to data models and query
languages
NoSQL systems emphasize performance and flexibility over
modeling power and complex querying
Not Requiring a Schema - allows semi-structured, self-
describing data
Less Powerful Query Languages - CRUD operations
Versioning
Categories of NOSQL Systems
Document-based NOSQL systems: These systems store data
in the form of documents using well-known formats, such as
JSON (JavaScript Object Notation)
Documents are accessible via their document id
NoSQL key-value stores: a simple data model based on fast
access by the key to the value associated with the key
Column-based or wide column NoSQL systems: These
systems partition a table by column into column families where
each column family is stored in its own files.
Graph-based NOSQL systems: Data is represented as graphs,
and related nodes can be found by traversing the edges using
path expressions.
Hybrid NoSQL systems
Object databases
XML databases
The CAP Theorem
CAP refer to three desirable properties of distributed systems with
replicated data: consistency , availability and partition
tolerance
The CAP theorem states that it is not possible to guarantee all
three of the desirable properties at the same time in a distributed
system with data replication.
If this is the case, then the distributed system designer would have
to choose two properties out of the three to guarantee.
In a N0SQL distributed data store, a weaker consistency level is
often acceptable, and guaranteeing the other two properties
(availability, partition tolerance) is important.
Hence, eventual consistency is often adopted in NoSQL systems.
Document-Based NOSQL Systems
It stores data as collections of similar documents
Sometimes known as document stores
Examples include MongoDB and CouchDB
MongoDB Data Model
MongoDB documents are stored in BSON (Binary JSON)
format
Individual documents are stored in a collection.
The operation createCollection is used to create each
collection.
For example, the following command can be used to create a
collection called project to hold PROJECT objects from the
COMPANY database
db.createCollection(“project”, { capped : true, size : 1310720, max :
500 } )
db.createCollection(“worker”, { capped : true, size : 5242880, max
: 2000 } ) )
MongoDB Data Model
Each document in a collection has a unique ObjectId
field, called _id, which is automatically indexed in the
collection
The value of ObjectId can be specified by the user, or it
can be system-generated
System-generated ObjectIds have a specific format,
which combines the timestamp when the object is
created (4 bytes, in an internal MongoDB format), the
node id (3 bytes), the process id (2 bytes), and a counter
(3 bytes) into a 16-byte Id value.
User-generated ObjectsIds can have any value specified
by the user as long as it uniquely identifies the document
MongoDB Data Model
A collection does not have a schema
The structure of the data fields in documents is
chosen based on how documents will be accessed and
used
Denormalized document design with embedded
subdocuments
Embedded array of document references
MongoDB CRUD Operations
MongoDb has several CRUD (create, read, update,
delete) operations
Documents can be created and inserted into their
collections using the insert operation, whose format is:
db.<collection_name>.insert(<document(s)>)
The parameters of the insert operation can include either a
single document or an array of documents
Example:
db.project.insert( { _id: “P1”, Pname: “ProductX”, Plocation:
“Bellaire” } )
db.worker.insert( [ { _id: “W1”, Ename: “John Smith”,
ProjectId: “P1”, Hours: 32.5 },{ _id: “W2”, Ename: “Joyce
English”, ProjectId: “P1”,Hours: 20.0 } ] )
MongoDB CRUD Operations
The delete operation is called remove, and the format
is:
db.<collection_name>.remove(<condition>)
There is also an update operation, which has a
condition to select certain documents, and a $set
clause to specify the update.
For read queries, the main command is called find,
and the format is:
db.<collection_name>.find(<condition>)
MongoDB Distributed Systems Characteristics
MongoDB is a distributed system, the two-phase
commit method is used to ensure atomicity and
consistency of multi-document transactions
Replication in MongoDB
The concept of replica set is used in MongoDB to
createmultiple copies of the same data set on different
nodes in the distributed system
It uses a variation of the master-slave approach - all
write operations must be applied to the primary copy
and then propagated to the secondaries
MongoDB Distributed Systems Characteristics
Sharding in MongoDB
Sharding of the documents in the collection—also known as
horizontal partitioning— divides the documents into disjoint
partitions known as shards.
This allows the system to add more nodes as needed by a
process known as horizontal scaling of the distributed
system
It store the shards of the collection on different nodes to
achieve load balancing
Each node will process only those operations pertaining to
the documents in the shard stored at that node
MongoDB Distributed Systems Characteristics
Two ways to partition a collection into shards – range
partitioning and hash partitioning
Both require that the user specify a particular document field to
be used as the basis for partitioning the documents into shards.
The partitioning field, known as the shard key must have two
characteristics:
it must exist in every document in the collection
it must have an index
Range partitioning creates the chunks by specifying a range of key
values
Hash partitioning applies a hash function h(K) to each shard key
K, and the partitioning of keys into chunks is based on the hash
values
MongoDB Distributed Systems Characteristics
When sharding is used, MongoDB queries are submitted to a
module called the query router, which keeps track of which
nodes contain which shards based on the particular
partitioning method used on the shard keys
The query (CRUD operation) will be routed to the nodes that
contain the shards that hold the documents that the query is
requesting
Sharding and replication are used together; sharding focuses
on improving performance via load balancing and horizontal
scalability, whereas replication focuses on ensuring system
availability when certain nodes fail in the distributed system
NoSQL Key-Value stores
The key is a unique identifier associated with a data item
and is used to locate this data item rapidly
The value is the data item itself, and it can have very
different formats for different key-value storage systems
The main characteristic of key-value stores is the fact that
every value (data item) must be associated with a unique
key, and that retrieving the value by supplying the key must
be very fast
NoSQL Key-Value stores
DynamoDB
an Amazon product and is available as part of Amazon’s
AWS/SDK platforms
The basic data model in DynamoDB uses the concepts of
tables, items, and attributes
A table in DynamoDB does not have a schema; it holds a
collection of self-describing items
Each item will consist of a number of (attribute, value) pairs,
and attribute values can be single-valued or multivalued
DynamoDB also allows the user to specify the items in JSON
format, and the system will convert them to the internal
storage format of DynamoDB
NoSQL Key-Value stores
When a table is created, it is required to specify a table
name and a primary key
The primary key will be used to rapidly locate the items in
the table
Thus, the primary key is the key and the item is the value
for the DynamoDB key-value store
The primary key attribute must exist in every item in the
table
Column-Based or Wide Column NOSQL
Systems
BigTable - Google distributed storage system for big data
An open source system known as Apache Hbase is similar
to Google BigTable, but it typically uses HDFS (Hadoop
Distributed File System) for data storage
HDFS is used in many cloud computing applications
sAnother well-known example of column-based NOSQL
systems is Cassandra
Column-Based or Wide Column NOSQL
Systems
BigTable (and Hbase) is sometimes described as a sparse
multidimensional distributed persistent sorted map, where
the word ‘map’ means a collection of (key, value) pairs
One of the main differences that distinguish column-based
systems from key-value stores is the nature of the key
In Hbase, the key is multidimensional and so has several
components: typically, a combination of table name, row
key, column, and timestamp
Hbase data model
. The data model in Hbase organizes data using the concepts of
namespaces, tables, column families, column qualifiers, columns,
rows, and data cells
A column is identified by a combination of (column family:column
qualifier)
Data is stored in a self-describing form by associating columns with
data values, where data values are strings
Hbase also stores multiple versions of a data item, with a timestamp
associated with each version, so versions and timestamps are also
part of the Hbase data model
Hbase data model
Column Families, Column Qualifiers, and Columns
A table is associated with one or more column families
Each column family will have a name
Column families must be specified when the table is created and cannot
be changed later
The table name is followed by the names of the column families
associated with the table.
When the data is loaded into a table, each column family can be
associated with many column qualifiers
A column is specified by a combination of
ColumnFamily:ColumnQualifier.
The concept of column family is somewhat similar to vertical partitioning
because columns (attributes) that are accessed together because they
belong to the same column family are stored in the same files..
Examples in Hbase
Examples in Hbase
A cell holds a basic data item in Hbase. The key (address)
of a cell is specified by a combination of (table, rowid,
columnfamily, columnqualifier, timestamp)
If timestamp is left out, the latest version of the item is
retrieved
A namespace is a collection of tables.
NOSQL Graph Databases
The data is represented as a graph, which is a collection of
vertices (nodes) and edges
Both nodes and edges can be labeled to indicate the types
of entities and relationships they represent
It is generally possible to store data associated with both
individual nodes and individual edges
Any Queries?
