BIG DATA

No SQL Databases

By
Dr. Tene Ramakrishnudu
Associate Professor
Department of Computer Science &Engineering
National Institute of Technology(NIT), Warangal, TS, India
 Outline

❖NoSQL Database
❖Types of NoSQL databases
❖CAP Theorem
❖MongoDB

25-09-2024 RK-CSE-NITW 2
 NoSQL Databases

❖Not only SQL

❖First coined by Carlo Strozzi in 1998

❖Reintroduced by Eric Evans in 2009

❖Non-relational

❖Open source

❖Distributed Database
25-09-2024 RK-CSE-NITW 3
 NoSQL Databases

❖Types of NoSQL Databases

❖1. Key-Value or Big hash table: It maintains big hash

tables of keys and values.
▪ Ex: Raik, Redis, Dynamo
❖2. Document Oriented: It maintains data in collections
constituted from documents. constituted: form
▪ MongoDB, Apache CouchDB, MakLogic
❖3. Column Oriented: Each Storage block has data from
only one column
▪ Cassandra, Hbase, Hypertable
❖4. Graph Database: A graph stores data in nodes
▪ Neo4J, HyperGraphDB

25-09-2024 RK-CSE-NITW 4
 Advantages of NoSQL

❖Scale up and down: NoSQL database supports scaling rapidly and

elastically.
▪ Cluster scale (100+ nodes), performance scale (100,000+ read/write per second)
and data scale (1 billion+ documents)
❖Schema less: No pre-defined schema, flexible,

❖Easy to implement: ease to implement and deployment

❖Low-cost: less operational cost

❖Relaxes on Data consistency requirement: Most of the NoSQL

databases compromises on consistency (follows CAP theorem)

❖Sharding: NoSQL support auto-shading, automatically spread data

across an arbitrary number of servers,
❖Replication:
❖Multiple copies of data are stored across the cluster and data centers
(High availability and fault tolerance)
25-09-2024 RK-CSE-NITW 5
 NoSQL Brewer's theorem
❖CAP theorem also named Brewer's theorem

❖ Many nodes contain replicas of partitions of data

❖ Consistency :
▪ all replicas contain the same version of data
▪ a data item behaves as if there is one copy

❖ Availability:
▪ system remains operational on failing nodes
▪ Node failures do not prevent survivors from continuing to operate

❖ Partition tolarence:
▪ multiple entry points system remains operational on system split
▪ The system continues to operate despite network partitions

A P
25-09-2024 RK-CSE-NITW 6
 MongoDB

❖MongoDB is a powerful, flexible, and scalable general-

purpose database.

❖NoSQL
❖Non relational
❖Document-oriented database
❖Cross-platform
❖Open Source
❖Distributed

25-09-2024 RK-CSE-NITW 7
 MongoDB

❖MongoDB comes with a simple but powerful JavaScript

shell, which is useful for the administration of MongoDB
instances and data manipulation.

❖Documents in MongoDB can be thought of as “JSON-like”

in that they are conceptually similar to objects in
JavaScript.

❖JSON is a simple representation of data

❖BSON (binary JSON) beeson

25-09-2024 RK-CSE-NITW 8
 MongoDB: Replication

Client
Application

Write

Primary

Replica Replica Replica

Secondary Secondary Secondary

Process of Replication in MongoDB

25-09-2024 RK-CSE-NITW 11
 MongoDB: Sharding

Collection 1
1 TB Database

Shard 1 Shard 2 Shard 3 Shard 4

256GB 256GB 256GB 256GB

Logical Database(collection 1)

Process of Sharding in MongoDB

25-09-2024 RK-CSE-NITW 13
 MongoDB

❖Document:

❖The basic unit of data for MongoDB

❖Equivalent to a row in a relational database management system (but

much more expressive).

❖an ordered set of keys with associated values

❖With different fields/key-value pairs

❖Different order of fields

❖Every document has a special key, "_id", that is unique within a

collection.

25-09-2024 RK-CSE-NITW 14
 MongoDB

❖Example:

{
RollNo:123456
Name: abcdef
ContactNo: 0987654321
Email: [email protected]

25-09-2024 RK-CSE-NITW 15
 MongoDB

❖Collection:

❖a table in RDBMS.

❖a table with a dynamic schema.

❖Exists within a single database

❖Holds several MongoDB documents

❖A single instance of MongoDB host multiple independent
databases, each of which can have its own collections.
25-09-2024 RK-CSE-NITW 16
 MongoDB

❖ Example:

db.students.insert({
RollNo:123
Name: abc
ContactNo: 0987654321
Email: [email protected], [email protected]
}
{
RollNo:456
Name: def
ContactNo: 0987654321, 0987654321, 0987654321
Email: [email protected], [email protected], [email protected]
}
{
RollNo:789
Name: ghi
ContactNo: 0987654321, 0987654321
Email: [email protected]
})

25-09-2024 RK-CSE-NITW 17
 MongoDB

❖Database:

❖Collection of collections.

❖A database in RDBMS

❖Each database gets its own set of files on the file system

❖Single MongoDB server hosts multiple database

25-09-2024 RK-CSE-NITW 18
 MongoDB

❖The most common types are:

❖Null: Null can be used to represent both a null value and a
nonexistent field:
{"x" : null}
❖Boolean: which can be used for the values true and false:
{"x" : true}
❖Number: The shell defaults to using 64-bit floating point
numbers, these numbers look “normal” in the shell:
{"x" : 3.14} or: {"x" : 3}
For integers, use the NumberInt or NumberLong classes, which
represent 4-byte or 8-byte signed integers, respectively.
{"x" : NumberInt("3")} {"x" : NumberLong("3")}

25-09-2024 RK-CSE-NITW 19
 MongoDB

❖String: Any string of characters can be represented using the

string type:
{"x" : "foobar"}
❖Date: Dates are stored as milliseconds since the epoch. The
time zone is not stored:
{"x" : new Date()}
❖regular expression: Queries can use regular expressions
using JavaScript’s regular expression syntax:
{"x" : /foobar/i}
❖Array: Sets or lists of values can be represented as arrays:
{"x" : ["a", "b", "c"]}
❖embedded document: Documents can contain entire
documents embedded as values in a parent document:
{"x" : {"foo" : "bar"}}

25-09-2024 RK-CSE-NITW 20
 MongoDB

❖ObjectId: An object id is a 12-byte ID for documents.

{"x" : ObjectId()}

❖Binary data: Binary data is a string of arbitrary bytes. It

cannot be manipulated from the shell.

❖Code: Queries and documents can also contain arbitrary

JavaScript code: {"x" : function() { /* ... */ }}

25-09-2024 RK-CSE-NITW 21
 MongoDB: Installation

❖1. go to mongoDB website www.mongodb.com

❖2. select software ---> community server

❖3. choose version: current, platform: Windows, package: msi

❖4. double click on the downloaded file

❖5. accept terms & conditions, select default, leave the

remaining things as it is the two default paths.
▪ Continue up to the installation finish.
❖6. goto C --->Program Files ---> MongoDB ---> Server ---> 4.4 -
--> bin

25-09-2024 RK-CSE-NITW 22
 MongoDB: Installation

❖7. In bin folder verify the following files

▪ mongod.exe is the MongoDB Shard for windows daemon is a computer
program that runs as a background process
▪ mongo.exe (client) JavaScript shell interface to MongoDB.
▪ mongos.exe is the MongoDB Shard for windows

❖8. copy the file path C:\Program Files\ MongoDB\ Server\ 4.4\ bin

❖9. open command and paste (C:\Program Files\ MongoDB\

Server\4.4\bin) then press enter.

❖10. C:\Program Files\ MongoDB\ Server\4.4\bin>mongod

❖ 11. if any errors about the data folder go to C drive and create
data folder and inside data folder create db. or else go to step 12.

25-09-2024 RK-CSE-NITW 23
 MongoDB: Installation

❖12. open another command prompt and change the

directory to C:\Program Files\MongoDB\Server\4.4\bin
And execute
C:\Program Files\MongoDB\Server\4.4\bin>mongo
▪ It opens the JavaScript shell that is the mongoDB shell, then
execute your queries

❖13. >show dbs;

> use wsdc

❖14. Inside the wsdc wok with the CRUD operations.

25-09-2024 RK-CSE-NITW 24
 MongoDB: Installation (extra)

❖Special: installation setup

❖To run the mongod & mongo any where do the environment
setup

❖1. go to C:\Program Files\MongoDB\Server\4.4\bin and copy

the path

❖2. right click on the windows key then select the system

❖3. go to advanced system settings then select environment

variables.

❖4. select path create new and past (C:\Program Files\

MongoDB\ Server\ 4.4\bin) and click ok ok ok…
25-09-2024 RK-CSE-NITW 25
 MongoDB: Installation

❖5. now open the command prompt and execute the

mongod command and mongo command.

❖Note: once the system is restarted next time onwards no

need to the command mongod just run the command
mongod

25-09-2024 RK-CSE-NITW 26
 MongoDB

❖CRUD Operations

❖MongoDB use DATABASE_NAME is used to create

database.

❖Creates new database if its not exists or returns the

existing database.

❖Example:
> use wsdc
switched to db wsdc
25-09-2024 RK-CSE-NITW 27
 MongoDB

❖To check your currently selected database, use the

command db
>db
wsdc

❖To check list of databases created, use the command

show dbs
>show dbs
mydatabase
library
social_site
25-09-2024 RK-CSE-NITW 28
 MongoDB

❖To drop database use db.dropDatabase() .

>db.dropDatabase()

❖Select the database to delete

>show dbs
>use wsdc
switched to db wsdc
>db.dropDatabase()
>{“dropped”:”wsdc”, “ok”:1}
>show dbs

25-09-2024 RK-CSE-NITW 29
 MongoDB

❖To create collection use db.createCollection(name, options).

❖name is name of collection. Options is a document and is used to

specify configuration of collection.
>db.createCollection(name, options)
>db.createCollection(“students")
{ "ok" : 1

❖Options is optional
❖Fields in theoptions.
❖(i) Capped: Boolean (Optional)
❖If true, enables a capped collection.
❖Capped collection is a fixed size collection that automatically
overwrites its oldest entries when it reaches its maximum size.
❖If you specify true, you need to specify size parameter also.

25-09-2024 RK-CSE-NITW 30
 MongoDB

❖autoIndexID: Boolean (Optional)

❖If true, automatically create index on _id field.
❖Default value is false.

❖size: number (Optional)

❖Specifies a maximum size in bytes.
❖If capped is true, then you need to specify this field also.

❖max: number (Optional)

❖Specifies the maximum number of documents allowed in
the capped collection
25-09-2024 RK-CSE-NITW 31
 MongoDB

>db.createCollection(“students", { capped : true,

autoIndexID : true, size : 987654321, max : 10000 })
{ "ok" : 1 }

25-09-2024 RK-CSE-NITW 32
 MongoDB

❖MongoDB creates collection automatically, when you

insert some document.

>db.students.insert({
rollNo:123
name: “abc”
contactno: 0987654321
email: [email protected]
age: 25
})

25-09-2024 RK-CSE-NITW 33
 MongoDB

❖MongoDB provides the following methods for inserting

documents into a collection:

❖db.collection.insertOne() : inserts one document

❖db.collection.insertMany() : inserts multiple documents

❖db.collection.insert() :either a single document or an

array of documents.

25-09-2024 RK-CSE-NITW 34
 MongoDB

❖Read :
o db. students.find( { age: { $gt: 18 } }, { name: 1, address: 1 } ).limit(5)

❖Selects the documents in the students collection that match the

condition age is greater than 18.

❖To specify the greater than condition, query criteria uses the
greater than (i.e. $gt) query selection operator.

❖The query returns at most 5 matching documents

❖ The matching documents will return with only the _id, name
and address fields.

25-09-2024 RK-CSE-NITW 35
 MongoDB

❖Update: MongoDB provides the following methods for

updating documents in a collection:

❖db.collection.updateOne() :updates a single document

❖db.collection.updateMany(): updates multiple
documents
❖db.collection.replaceOne():replaces a single document.
❖db.collection.update() :modifies existing documents in a
collection

25-09-2024 RK-CSE-NITW 36
 MongoDB

>db. students.updateOne(
{ age: { $lt: 18 } },
{ $set: {contactno: 0987654300} } )

>db. students.updateMany(
{ age: { $gt: 18 } },
{ $set: { status: “accept" } } )

>db. students.replaceOne(
{ name: “abc" },
{ name: “baksdj", age : 25} )
25-09-2024 RK-CSE-NITW 37
 MongoDB

❖MongoDB provides the following methods for deleting

documents from a collection:

❖db.collection.deleteOne() :deletes a single document

❖db.collection.deleteMany() :deletes multiple documents
❖db.collection.remove():method deletes documents from
a collection

25-09-2024 RK-CSE-NITW 38
 MongoDB

>db. students.deleteOne( { status: "reject" } )

>db. students.deleteMany( { status: "reject" } )

>db.students.remove( { status: "D" } )

25-09-2024 RK-CSE-NITW 39
 MongoDB

❖Relational Operations:

❖$eq = equal to
db.students.find({gade:{$eq:’Good’}}).pretty()
❖$ne= not equal to
db.students.find({gade:{$ne:’Good’}}).pretty()
❖$gte= greater than or equal to
db.students.find({gade:{$gte:50}}).pretty()
❖$lte= less than or equal to
db.students.find({gade:{$lte:50}}).pretty()
❖$gt= greater than
db.students.find({gade:{$gt:50}}).pretty()
❖$lt= less than
db.students.find({gade:{$lt:50}}).pretty()
25-09-2024 RK-CSE-NITW 40
 MongoDB

❖$in= in the list

db.students.find({grade:{$in:[‘good’, ‘very good’]}}).pretty()

❖$nin= not in the list

db.students.find({grade:{$nin:[‘good’, ‘very good’]}}).pretty()

❖Grade is good and name= abc

db.students.find({grade:’good’, name:’abc’}).pretty()

❖Student name begins with ‘M’

db.students.find({name:/^M/}).pretty()
db.students.find({grade:{$regex:”^M”}}).pretty()
25-09-2024 RK-CSE-NITW 41
 MongoDB

❖Name ends with e

db.students.find({grade:/e$/}).pretty()
db.students.find({grade:{$regex:”e$”}}).pretty()

❖Name contains e
db.students.find({grade:/e/}).pretty()
db.students.find({grade:/.*e.*/}).pretty()
db.students.find({grade:{$regex:”e”}}).pretty()

❖Null values
db.students.find({age:{$eq:”null”}}).pretty()
25-09-2024 RK-CSE-NITW 42
 MongoDB

❖Count and sort:

db.students.count({grade:’good’}).pretty()
db.students.sort({grade:1}).pretty()
db.students.sort({grade:1,name:1}).pretty()
❖Grade Ascending order and name descending order
db.students.sort({grade:1,name:-1}).pretty()

❖Retrieve fist 3 documents from the students collections where the

grade is good.
db.students.find({grade:good}).limit(3).pretty()
❖ Skip the first 3 documents
db.students.find().skip(3).pretty()

❖Find fourth, fifth, sixth documents

db.students.find().skip(2).limit(3)pretty()

25-09-2024 RK-CSE-NITW 43
 MongoDB

❖arrays:
❖Create collection and each document should have a fruits array.
db.food.insert({_id:1,fruits:[“banana”,”apple”,”chery”]})
db.food.insert({_id:2,fruits:[“orange”,”mango”,”chery”})
db.food.insert({_id:3,fruits:[“gapes”,”apple”,”mango”]})
db.food.insert({_id:4,fruits:[“banana”,”apple”,”grape”]})
db.food.insert({_id:5,fruits:[“banana”,”mango”,”chery”]})

❖Verify the above insertions

db.food.find()
❖Find the document from food, which contains mango and banana
db.food.find({fruits:{$in:[‘banana’,’Mango’]}}).pretty()
db.food.find({fruits:{$in:[‘banana’]}}).pretty()

25-09-2024 RK-CSE-NITW 44
Design MongoDB for the social E-commerce website

25-09-2024 RK-CSE-NITW 45
 MongoDB GridFS

❖Store Binary Data:

❖MongoDB Provides GridFS to support the storage of

binary data

❖It store up to 4MB of data (sufficient for Image/ audio clip)

25-09-2024 RK-CSE-NITW 46
 Working with GridFS

❖GridFS is the MongoDB specification for storing and

retrieving large files such as
▪ images, audio files, video files, etc.

❖It is a kind of file system to store files but its data is stored
within the MongoDB collections.

❖GridFS has the capability to store files even greater than

its document size limit of 16MB.

❖GridFS divides a file into chunks and stores each chunk

of data in a separate document, each of maximum size
255k.
25-09-2024 RK-CSE-NITW 47
 Working with GridFS

❖GridFS by default uses two collections

▪ fs.files and fs.chunks to store the file's metadata and the chunks.

❖Each chunk is identified by its unique _id ObjectId field.

❖The fs.files serves as a parent document.

❖The files_id field in the fs.chunks document links the

chunk to its parent.

25-09-2024 RK-CSE-NITW 48
 Working with GridFS

❖GridFS by default uses two collections

▪ fs.files and fs.chunks to store the file's metadata and the chunks.

❖files._id : unique identifier for the document.

❖files.length: Size of the document in bytes.

❖files.chunkSize :The size of each chunk in bytes. The default

size is 255 kB.

❖files.uploadDate: It gives the date the document was first

stored by GridFS.

❖files.md5 : An MD5 hash of the complete file

25-09-2024 RK-CSE-NITW 49
 Working with GridFS

❖files.filename:
▪ A human-readable name for the GridFS file.
▪ It is optional.

❖files.contentType
▪ A valid MIME type for the GridFS file.
▪ It is optional.

❖files.aliases
▪ An array of alias strings.
▪ It is optional.

❖files.metadata
▪ the additional information
▪ metadata filed can be of any data.
▪ It is optional.

25-09-2024 RK-CSE-NITW 50
 Working with GridFS

❖Adding Files to GridFS

❖>mongofiles.exe -d gridfs put myfile.mp3

❖gridfs is the name of the database in which the file will be

stored.

❖If the database is not present, MongoDB will

automatically create a new document on the fly.
myfile.mp3 is the name of the file uploaded.

25-09-2024 RK-CSE-NITW 51
 Working with GridFS
❖Step1: Goto https://www.mongodb.com/try/download/database-
tools?tck=docs_databasetools

❖Step 2: Select the Version, Platform and Package then Download the
MongoDB Database tools.

❖Step 3: Install the mongoDB database tools

25-09-2024 RK-CSE-NITW 52
25-09-2024 RK-CSE-NITW 53
25-09-2024 RK-CSE-NITW 54
25-09-2024 RK-CSE-NITW 55
25-09-2024 RK-CSE-NITW 56
25-09-2024 RK-CSE-NITW 57
 Working with GridFS

❖Step 4: keep the files in the folder where the tools are running
>
❖Step 5: upload files on to the MongoDB using the following
command
goto database sever
C:\Program Files\MongoDB\Tools\100\bin
and execute the following command
>mongofiles.exe -d gridfs put VedaDance.mp3

❖Step 6: goto database sever

C:\Program Files\MongoDB\Server\5.0\bin
and execute the following command
>mongo

25-09-2024 RK-CSE-NITW 58
 Working with GridFS

❖Step 7: and execute the following commands

>use gridfs

>show collections

>db.fs.files.find()

>db.fs.files.find().pretty()

>db.fs.chunks.find({},{data:0})

>db.fs.chunks.find({},{data:0}) .pretty()
25-09-2024 RK-CSE-NITW 59
 Working with GridFS

❖To delete the uploaded file from the GridFS

mongofiles.exe -d gridfs delete VedaDance.mp4

❖Searching for the uploaded file

mongofiles.exe -d gridfs search VedaDance.mp4

25-09-2024 RK-CSE-NITW 61
 Working with GridFS

❖Listing all the files which are uploaded to the GridFS

mongofiles -d gridfs list

25-09-2024 RK-CSE-NITW 62
 ?

25-09-2024 RK-CSE-NITW 63
 Refeences

❖kristina chodorow, “MongoDB: The definitive guide”

❖Eric Brewer, “CAP Twelve years later: How the "Rules"

have Changed”.

25-09-2024 RK-CSE-NITW 64
 Thank You

25-09-2024 RK-CSE-NITW 65