NoSQL UNIT -3

NoSQL DATABASES

© Kalasalingam academy of research and education

 NOSQL DATABASES

 NoSQL Database is a non-relational Data Management System, that does not require a fixed schema. It avoids
joins, and is easy to scale.
 The major purpose of using a NoSQL database is for distributed data stores with humongous data storage needs.
 NoSQL is used for Big data and real-time web apps. For example, companies like Twitter, Facebook and Google
collect terabytes of user data every single day.
 Traditional RDBMS uses SQL syntax to store and retrieve data for further insights.
 Instead, a NoSQL database system encompasses a wide range of database technologies that can store structured,
semi-structured, unstructured and polymorphic data.
Brief History of NoSQL Databases
1998- Carlo Strozzi use the term NoSQL for his lightweight, open-source
relational database
2000- Graph database Neo4j is launched
2004- Google BigTable is launched
2005- CouchDB is launched
2007- The research paper on Amazon Dynamo is released
2008- Facebooks open sources the Cassandra project
2009- The term NoSQL was reintroduced

© Kalasalingam academy of research and education

Why Need NoSQL?

Non- Relational
Multi-Model • NoSQL databases never follow the relational model
Easily Scalable • Never provide tables with flat fixed-column records
Big Data Applications • Work with self-contained aggregates or BLOBs
Redundancy and Zero Downtime • Doesn’t require object-relational mapping and data normalization
Distributed • No complex features like query languages, query planners, referential
integrity joins, ACID

Schema-free
• NoSQL databases are either schema-free or have relaxed schemas
• Do not require any sort of definition of the schema of the data
• Offers heterogeneous structures of data in the same domain
Types of NoSQL

 Document databases – JSON,BSON or XML documents(not word documents or Google Docs)

 Key- Value store- Key-value pairs
 Column –Oriented databases- a column store is organized as a set of columns.
 Graph databases – graph databases focuses on the relationship between data elements
 Comparing SQL and UNIT -3
NoSQL DATABASES
NoSQL
© Kalasalingam academy of research and education
 Comparing SQL and NOSQL Databases

Relational Database NoSQL

It is used to handle data coming in low velocity.  It is used to handle data coming in high velocity.
It gives only read scalability. It gives both read and write scalability.
It manages structured data. It manages all type of data.
Data arrives from one or few locations. Data arrives from many locations.
It supports complex transactions. It supports simple transactions.
It has single point of failure. No single point of failure.
It handles data in less volume. It handles data in high volume.
Transactions written in one location. Transactions written in many locations.
support ACID properties compliance doesn’t support ACID properties
Its difficult to make changes in database once it is defined Enables easy and frequent changes to database schema
schema is mandatory to store the data design is not required
Deployed in vertical fashion. Deployed in Horizontal fashion.

© Kalasalingam academy of research and education

 COMPARING MongoDB, HBase, CASSANDRA AND Neo4j

Name MongoDB Hbase Cassandra Neo4j

Architecture Document Store Wide Column Store Wide Column Store Graph DBMS
Server OS Linux, OS X, Solaris, Linux, Unix, Windows FreeBSD, Linux, OS X, Windows, MacOS,
Windows Windows Debian, Ubuntu, SuSE

Distributed System Eventual and Immediate Immediate Consistency Eventual and Immediate Eventual and Immediate
Consistency Consistency Consistency Consistency

Owner and Developer MongoDB, Inc. Apache Software Apache Software neo4j.com
Foundation Foundation

Replication Master-Slave Replication Master-Slave Replication Masterless Ring Master-Slave Replication

Programming Language C++ Java Java Java, Scala

(Base Code)

Editions Community (Free) and Community Community with Option Free source
Enterprise of Third-Party Support

© Kalasalingam academy of research and education

 COMPARING MongoDB, HBase, CASSANDRA AND Neo4j

Name MongoDB Hbase Cassandra Neo4j

Supported C, C#, C++, Erlang, C, C#, C++, Groovy, Java, C#, C++, Clojure, Erlang, .Net, Clojure, Elixir
Programming Haskell, Java, JavaScript, PHP, Python, Scala Go, Haskell, Java, Go, Groovy,Haskell,Java
Languages Perl, PHP, Python, Ruby, Node.js, Perl, PHP, JavaScript, Perl, PHP
Scala Python, Ruby, Scala Python, Ruby,Scala
Popular Use Cases Operational Online Log Analytics, Sensor Data, Messaging Data visualization, fraud
Intelligence, Product Hadoop, Write Heavy Systems, E-commerce detection and analytics,
Data Management, Applications, Websites, Always-On data management
Content Management MapReduce Applications, Fraud
Systems, IoT, Real-Time Detection for Banks
Analytics
Key Customers Adobe, eBay, Google, 23andMe, Salesforce, eBay, McDonald’s, medium.com., Stack,
Cisco, SAP, Facebook, Netflix, Bloomberg, Walmart, Comcast, Fiverr, Fundamentei,
Royal Bank of Scotland, Sophos, Adobe, Xiaomi, Facebook, Instagram, Trendyol Group, double
Trend Micro Yahoo GitHub, CERN, Netflix, slash, ingsmen
Reddit Software, GAPO, etc.

© Kalasalingam academy of research and education

 Applications of NOSQL

 Data Mining
 Social Media Networking Sites
 Software Development

© Kalasalingam academy of research and education

 Thank you !

© Kalasalingam academy of research and education

 CAP THEORAM UNIT -3
NoSQL DATABASES

© Kalasalingam academy of research and education

 DISTRIBUTED DATABASE SYSTEMS

• Multiple computers, or nodes, work together to give an

impression of a single working database unit to the user
• They store the data in these multiple nodes. And each of these
nodes runs an instance of the database server and they
communicate with each other in some way.
• A user simply interacts with the system as if it is interacting with a
single database.
• Internally the nodes communicate with each other, retrieving
data that the user is looking for, from the relevant node, or
storing the data provided by the user.
Benefits:
1. With the increase in traffic from the users, we can easily scale
our database by adding more nodes to the system. Since these
nodes are commodity hardware, they are relatively cheaper than
adding more resources to each of the nodes individually, which is
called as horizontal scaling

© Kalasalingam academy of research and education

 CAP THEORAM

• CAP theorem, also known as Brewer’s theorem, stands for Consistency, Availability and Partition Tolerance

 Consistency - Consistency in CAP means

having the most up-to-date information

 Availability - All reads contain data, but it

might not be the most recent.

 Partition tolerance - The system continues to

operate despite network failures (ie; dropped
partitions, slow network connections, or
unavailable network connections between
nodes.)

© Kalasalingam academy of research and education

 Cont…

• In normal operations, your data store provides all three functions. But the CAP theorem maintains that when a
distributed database experiences a network failure, you can provide either consistency or availability.
• It’s a tradeoff. All other times, all three can be provided. But, in the event of a network failure, a choice must be
made.
• In the theorem, partition tolerance is a must. The assumption is that the system operates on a distributed data store
so the system, by nature, operates with network partitions.
• Network failures will happen, so to offer any kind of reliable service, partition tolerance is necessary—the P of CAP.
• That leaves a decision between the other two, C and A. When a network failure happens, one can choose to
guarantee consistency or availability:
 High consistency comes at the cost of lower availability.
 High availability comes at the cost of lower consistency.

© Kalasalingam academy of research and education

 AVAILABILITY or CONSISTENCY in NoSQL

NoSQL databases do not require a schema, and don’t enforce

relations between tables. All its documents are JSON
documents, which are complete entities one can readily read
and understand. They are widely
recognized for:
• Ease-of-use
• Scalable performance
• Strong resilience
• Wide availability

© Kalasalingam academy of research and education

 Thank you !

© Kalasalingam academy of research and education

 Installing MongoDB UNIT -3
NoSQL DATABASES

© Kalasalingam academy of research and education

 CONTENT

Installation
Creating database
Creating collection
Insert documents
Finding documents
Update documents

© Kalasalingam academy of research and education

 INSTALLATION

•Mongodb Community server

•Mongodb Atlas database

•Mongoshell

© Kalasalingam academy of research and education

 Step-1

https://www.mongodb.com/try/download/community
© Kalasalingam academy of research and education
 Step-2

© Kalasalingam academy of research and education

 BASIC IMPLEMENTATION-CREATING DATABASE

© Kalasalingam academy of research and education

 VISUALIZATION IN COMPASS

© Kalasalingam academy of research and education

CREATING MULTIPLE COLLECTIONS
 Contd…

© Kalasalingam academy of research and education

 GETTING ALL THE COLLECTIONS

© Kalasalingam academy of research and education

 GETTING A SPECIFIC DETAIL

© Kalasalingam academy of research and education

 UPDATING COLLECTION

© Kalasalingam academy of research and education

© Kalasalingam academy of research and education
 Thank you!

© Kalasalingam academy of research and education

 MONGODB CRUD UNIT -4
Working with NoSQL
operations Databases

© Kalasalingam academy of research and education

 MONGODB TERMINOLOGIES FOR RDBMS CONCEPTS

RDBMS MongoDB
Database Database
Table, View Collection
Row Document (JSON, BSON)
Column Field
Index Index
Join Embedded Document
Foreign Key Reference
Partition Shard

© Kalasalingam academy of research and education

JSON Vs BSON

“Binary JSON”
“JavaScript Object Notation”
• Lightweight

• name/value pairs • Traversable

• Efficient (decoding and encoding)

• Ordered list of values

Easy for humans to write/read, easy Binary-encoded serialization of JSON-like docs
for computers to parse/generate
Embedded structure reduces need for joins
Objects can be nested

Built on Goals

http://json.org/ http://bsonspec.org/
 MONGODB TERMINOLOGIES FOR RDBMS CONCEPTS

RDBMS MongoDB
Database Database
Table, View Collection
Row Document (JSON, BSON)
Column Field
Index Index
Join Embedded Document
Foreign Key Reference
Partition Shard

© Kalasalingam academy of research and education

 MONGODB Datatypes

Integer
Date Boolean

Binary data Double

Object ID String

Null Arrays

© Kalasalingam academy of research and education

 DATATYPES

• String : This is most commonly used datatype to store the data. String in mongodb must be UTF-8
valid.
• Integer : This type is used to store a numerical value. Integer can be 32 bit or 64 bit depending
upon your server.
• Boolean : This type is used to store a boolean (true/ false) value.
• Double : This type is used to store floating point values.
• Min/ Max keys : This type is used to compare a value against the lowest and highest BSON
elements.
• Arrays : This type is used to store arrays or list or multiple values into one key.
• Timestamp : ctimestamp. This can be handy for recording when a document has been modified or
added.
• Object : This datatype is used for embedded documents.

© Kalasalingam academy of research and education

 Contd…..

• Null : This type is used to store a Null value.

• Symbol : This datatype is used identically to a string however, it's generally reserved
for languages that use a specific symbol type.
• Date : This datatype is used to store the current date or time in UNIX time format.
You can specify your own date time by creating object of Date and passing day,
month, year into it.
• Object ID : This datatype is used to store the document’s ID.
• Binary data : This datatype is used to store binay data.
• Code : This datatype is used to store javascript code into document.
• Regular expression : This datatype is used to store regular expression

© Kalasalingam academy of research and education

 BASIC DATABASE OPERATIONS

use <database name> • switched to database provided with command

• To check currently selected database use the command

db
db

show dbs • Displays the list of databases

db.dropDatabase() • To Drop the database

© Kalasalingam academy of research and education

 DATABASE COLLECTION OPERATIONS

db.createCollection
(name) • To create collection
Ex:- db.createCollection(Stud)

>show collections • List out all names of collection in current database

db.databasename.insert • In mongodb you don't need to create collection. MongoDB

({Key : Value})
creates collection automatically, when you insert some document.
Ex:- db.Stud.insert({{Name:”Jiya”})

db.collection.drop() Example:- • MongoDB's db.collection.drop() is used to drop a

db.Stud.drop() collection from the database.

© Kalasalingam academy of research and education

 CRUD OPERATIONS

Insert

Find

Update

Delete

© Kalasalingam academy of research and education

 CRUD OPERATIONS

Insert

Find

Update

Delete

© Kalasalingam academy of research and education

 CRUD OPERATION -INSERT

• The insertOne () Method:-Insert one value to the database

• The insertMany() Method: Insert more than one values at the same time.
 To insert one value
Syntax:- db.collection_name.insertOne()
Ex- db.mca.insertOne({name: “nosql”, year:1, class:”B”})
 To insert many values at the same time
Syntax:- db.collection_name.insertMany()
Ex- db.mca.insertOne({name: “nosql”, year:1, class:”B”},{ name: “maths”, year:1, class:”B”},{ name: “CO OS”,
year:1, class:”B”},{ name: “dsa”, year:1, class:”B”})

© Kalasalingam academy of research and education

 CRUD OPERATION –READ

•Read operations retrieve documents from a collection; i.e. query a collection for documents.
Syntax: db.collection.find()
Ex: db.mca.find({})
db.mca.find ( { name:nosql } ) .limit(5)
db.mca.find ( { name: { $in: [“nosql”,”maths”] } } )
Although you can express this query using the $or operator, use the $in operator rather than the $or operator when
performing equality checks on the same field.

© Kalasalingam academy of research and education

 Functions used in find()

db.stud.find( { rno: { $gt:5} } ) db.stud.find().sort( { Rno: 1 } )

Shows all documents whose rno>5 Sort on age field in Ascending order (1)

db.stud.find( { rno: { $gt: 0, $lt: 5} } ) db.stud.find().sort( { Rno: -1 } )

Shows all documents whose rno greater than 0 and Sort on age field in Ascending order(-1)
less than 5

db.stud.find({name: “Jiya”},{Rno:1}) db.stud.find().count()

To show the rollno of student whose name is equal Returns no of documents in the collection
to Jiya (by default _id is also shown)

db.stud.find({name: “jiya”},{_id:0,Rno:1}) db.stud.find({Rno:2}).count()

show the rollno of student whose name is equal to Returns no of documents in the collection which
Jiya (_id is not shown) satisfies the given condition Rno=2

© Kalasalingam academy of research and education

 db.stud.find().limit(2)
Returns only first 2 documents db.stud.find({ rno: { $gt:5} } ).limit(2)
Returns only first 2 documents whose rno is greater than 5
db.stud.find().skip(5)
Returns all documents except first 5 documents
db.stud.find({ rno: { $gt:5} } ).skip(5)
Returns all documents except first 5 documents whose rno
db.stud.findOne() - Find first document only is greater than 5

db.stud.find({name:{$in:[“riya”,”jiya”]}}) Find information

db.stud.find({“Address.city”: “Pune”})- whose name is riya or jiya
Finding in Multicolumned attribute

db.stud.find({name: “Riya”,age:20}) db.stud.find({Rno:{$nin:[20,25]}}) Find information

Find documents whose name is Riya and Rno is whose rollno is not 20 or 25
20

© Kalasalingam academy of research and education

 db.stud.distinct(“Address”)
Find from which different cities students are coming db.collection.stats()

db.stud.find({name:/^n/})
Find students whose name starts with n

db.collection.explain().find()
db.stud.find({name:/n/}) Find students
whose name contains n letter

db.stud.find({name:/n$/}) Find students db.collection.explain().find().help()

whose name ends with n

© Kalasalingam academy of research and education

 CRUD OPERATION – UPDATE

To update a document, MongoDB provides update operators, such as $set, to modify field values.
 db.collection.updateOne(<filter>, <update>, <options>)
 db.collection.updateMany(<filter>, <update>, <options>)
 db.collection.replaceOne(<filter>, <update>, <options>)

 Replacing values in document

db.inventory.replaceOne(
{ item: "paper" },
{ item: "paper", instock: [ { warehouse: "A", qty: 60 }, { warehouse: "B", qty: 40 } ]
}

© Kalasalingam academy of research and education

 UPDATING SINGLE DOCUMENT

Update a Single Document

The following example uses the db.collection.updateOne() method on the inventory collection to update the first
document where item equals "paper":
db.inventory.updateOne(
{ item: "paper" },
{
$set: { "size.uom": "cm", status: "P" },
$currentDate: { lastModified: true }
}
)
• uses the $set operator to update the value of the size.uom field to "cm" and the value of the status field to "P",
• uses the $currentDate operator to update the value of the lastModified field to the current date. If lastModified field does not
exist, $currentDate will create the field. See $currentDate for details.

© Kalasalingam academy of research and education

 UPDATING MANY DOCUMENTS

Updating many documents

db.inventory.updateMany(
{ "qty": { $lt: 50 } },
{
$set: { "size.uom": "in", status: "P" },
$currentDate: { lastModified: true }
}
)
• uses the $set operator to update the value of the size.uom field to "in" and the value of the status field to "P",
• uses the $currentDate operator to update the value of the lastModified field to the current date. If lastModified
field does not exist, $currentDate will create the field. See $currentDate for details.

© Kalasalingam academy of research and education

 CRUD - DELETE

Delete operations remove documents from a collection. MongoDB provides the following methods to delete
documents of a collection:
• db.collection.deleteOne()
To delete at most a single document that matches a specified filter (even though multiple documents may match the
specified filter) use the db.collection.deleteOne() method.
• db.collection.deleteMany()
To delete all documents from a collection, pass an empty filter document {} to the db.collection.deleteMany()
method. To delete all documents that match a deletion criteria, pass a filter parameter to the deleteMany() method.
 In MongoDB, delete operations target a single collection. All write operations in MongoDB are atomic on the level
of a single document.
 You can specify criteria, or filters, that identify the documents to remove. These filters use the same syntax as read
operations.
© Kalasalingam academy of research and education
 Thank you !

© Kalasalingam academy of research and education

 UNIT -4
Indexing Working with NoSQL
Databases

© Kalasalingam academy of research and education

 SCRUTINIZING THE SEARCH PROCESS

• MongoDB uses indexing in order to make the query processing more efficient.
• If there is no indexing, then the MongoDB must scan every document in the collection and retrieve only those
documents that match the query.
• Indexes are special data structures that stores some information related to the documents such that it becomes easy for
MongoDB to find the right data file.
• The indexes are order by the value of the field specified in the index.

Creating Index

MongoDB provides a method called createIndex() that allows user to create an index.

Syntax - db.COLLECTION_NAME.createIndex({KEY:1})

The key determines the field on the basis of which you want to create an index and 1 (or -1) determines the order in
which these indexes will be arranged(ascending or descending).

© Kalasalingam academy of research and education

 Creating Indexes

db.mycol.createIndex({“age”:1}) {
“createdCollectionAutomatically” : false,
“numIndexesBefore” : 1,
“numIndexesAfter” : 2,
“ok” : 1 }
The createIndex() method also has a number of optional parameters. They are:

• background (Boolean)
• unique (Boolean)
• name (string)
• sparse (Boolean)
• expireAfterSeconds (integer)
• hidden (Boolean)
• storageEngine (Document)

© Kalasalingam academy of research and education

 Drop Indexes

The basic syntax of DropIndex() method is as follows().

db.COLLECTION_NAME.dropIndex({KEY:1})
Here, "key" is the name of the file on which you want to remove an existing index. Instead of the index specification
document (above syntax), you can also specify the name of the index directly as:
dropIndex("name_of_the_index")

Example
db.mycol.dropIndex({"title":1})
{
"ok" : 0,
"errmsg" : "can't find index with key: { title: 1.0 }",
"code" : 27,
"codeName" : "IndexNotFound"
}

© Kalasalingam academy of research and education

 LISTING THE INDEXES

• getIndexes() method
To get indexes

• This method returns the description of all the indexes

Description
int the collection.

• db.COLLECTION_NAME.getIndexes()
Syntax

• db.mycol.createIndex({"title":1,"description":-1})
Example

• we have dropped 2 indexes in the named mycol

Description
collection

© Kalasalingam academy of research and education

 Thank you !

© Kalasalingam academy of research and education

 UNIT -4
CAPPED COLLECTION Working with NoSQL
Databases

© Kalasalingam academy of research and education

What is Capped Collection?

Capped collections are fixed-size circular collections that follow the insertion order to support high performance for
create, read, and delete operations.
By circular, it means that when the fixed size allocated to the collection is exhausted, it will start deleting the oldest
document in the collection without providing any explicit commands.

To create a capped collection, we

In addition to collection size, we
use the normal createCollection
can also limit the number of If you want to check whether a
command but with capped option If there is an existing collection
documents in the collection using collection is capped or not, use
as true and specifying the which you are planning to convert
the max parameter − the following isCapped command
maximum size of collection in to capped, you can do it with the
>db.createCollection("cappedLog −
bytes. following code −
Collection", db.cappedLogCollection.isCapped
db.createCollection("cappedLogC >db.runCommand({"convertToCa
{capped:true,size:10000,max:100 ()
ollection", pped":"posts",size:10000})
0})
{capped:true,size:10000})
By default, a find query on a capped collection will display results in insertion order. But if you want the
documents to be retrieved in reverse order, use the sort command as shown in the following code −
>db.cappedLogCollection.find().sort({$natural:-1})
There are few other important points regarding capped collections worth knowing
• We cannot delete documents from a capped collection.
• There are no default indexes present in a capped collection, not even on _id field.
• While inserting a new document, MongoDB does not have to actually look for a place to accommodate new
document on the disk. It can blindly insert the new document at the tail of the collection. This makes insert
operations in capped collections very fast.
• Similarly, while reading documents MongoDB returns the documents in the same order as present on disk.
This makes the read operation very fast.
 New Features in Version 6

Convert a Collection to Capped

You can convert a non-capped collection to a capped collection with the convertToCapped command:
db.runCommand({"convertToCapped": "mycoll", size: 100000});
>The size parameter specifies the size of the capped collection in bytes.
Change a Capped Collection's Size
You can resize a capped collection using the collMod command's cappedSize option to set the cappedSize in
bytes. cappedSize must be greater than 0 and less than 1e+15 (1 PB).
>db.runCommand( { collMod: "log", cappedSize: 100000 } )
Change the Maximum Number of Documents in a Capped Collection
To change the maximum number of documents in a capped collection, use
the collMod command's cappedMax option. If cappedMax is less than or equal to 0, there is no maximum
document limit.
>db.runCommand( { collMod: "log", cappedMax: 500 } )
© Kalasalingam academy of research and education
 Thank you !

© Kalasalingam academy of research and education

 UNIT -4
MAP REDUCE Working with NoSQL
Databases

© Kalasalingam academy of research and education

 Map Reduce

• Map-reduce is a data processing programming model that helps to perform operations on large data sets and produce
aggregated results.
• MongoDB provides the mapReduce() function to perform the map-reduce operations.
• This function has two main functions, i.e., map function and reduce function.
• The map function is used to group all the data based on the key-value and the reduce function is used to perform
operations on the mapped data.
• So, the data is independently mapped and reduced in different spaces and then combined together in the function
and the result will save to the specified new collection.
• This mapReduce() function generally operated on large data sets only.
• Using Map Reduce you can perform aggregation operations such as max, avg on the data using some key and it is
similar to groupBy in SQL.

© Kalasalingam academy of research and education

Example

DATASET
{"id":1, "sec":A, "marks":80} var
MAP map = function() REDUCE
{"id":2, "sec":A, "marks":90} {emit(this.sec, var reduce =
{"id":1, "sec":B, "marks":99} function(sec,marks)
{"id":1, "sec":B, "marks":95} this.marks)}; {return Array.max(marks);};
{"id":1, "sec":C, "marks":90}

A dataset of 5 values is added Inside the map function, we use Reduce function is where actual
into the database. emit(this.sec, this.marks) function, aggregation of data takes place.
and we will return the sec and marks
of each record(document) from the
emit function.
Contd…..

• We have reduced the records now we will output them into a new collection.{out :”collectionName”}
db.collectionName.mapReduce(map,reduce,{out :"collectionName"});
• In the above query we have already defined the map, reduce.
• Then for checking we need to look into the newly created collection we can use the query
db.collectionName.find() we get:

{"id":"A", value:90}
{"id":"B", value:99}
{"id":"C", value:90}