Final Revision

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Query Processing and Optimization

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Draw a diagram to explain the steps
of query processor and optimizer.

Query
Scanning
Parsing
Validating
Intermediate form of Query
(query Tree)
Query
Optimizer
Catalog

Execution Plan
Query Code
Generator
Compile
d Query Executable Code
Code
Execution in
Runtime
processor

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Convert the following SQL statement to three
equivalent relational algebra statements:

SELECT *
FROM Staff s, Branch b
WHERE s.branchNo = b.branchNo AND s.position = ‘Manager’
AND b.city = ‘london’;

 Then choose the best according to the following data:

(1000 Tuples in Staff, ~ 50 Managers) - (50 Tuples in

Branch, ~ 5 London branches).

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Answer:

Results in these equivalent relational algebra statements

(1) s(position=‘Manager’)^(city=‘London’)^(Staff.branchNo=Branch.branchNo) (Staff X Branch)

(2) s(position=‘Manager’)^(city=‘London’) (Staff wvStaff.branchNo = Branch.branchNo Branch)

(3) [s(position=‘Manager’) (Staff)] wvStaff.branchNo = Branch.branchNo [s(city=‘London’) (Branch)]

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Answer:
s(position=‘Manager’)^(city=‘London’)^(Staff.branchNo=Branch.branchNo) (Staff X Branch)

 Requires (1000+50) disk accesses to read from Staff and Branch

relations
 Creates temporary relation of Cartesian Product (1000*50) tuples
 Requires (1000*50) disk access to read in temporary relation and test
predicate

Total Work = (1000+50) + 2*(1000*50) =

101,050 I/O operations

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Answer:
s(position=‘Manager’)^(city=‘London’) (Staff wvStaff.branchNo = Branch.branchNo
Branch)
 Again requires (1000+50) disk accesses to read from Staff and Branch
 Joins Staff and Branch on branchNo with 1000 tuples
(1 employee : 1 branch )

 Requires (1000) disk access to read in joined relation and check

predicate

Total Work = (1000+50) + 2*(1000) =

3050 I/O operations

3300% Improvement over Query 1

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Answer:
[ s(position=‘Manager’) (Staff) ] wv Staff.branchNo = Branch.branchNo [ s(city=‘London’) (Branch) ]


Read Staff relation to determine ‘Managers’ (1000 reads)

Create 50 tuple relation(50 writes)


Read Branch relation to determine ‘London’ branches (50 reads)

Create 5 tuple relation(5 writes)


Join reduced relations and check predicate (50 + 5 reads)

Total Work = 1000 + 2*(50) + 5 + (50 + 5) =

1160 I/O operations

8700% Improvement over Query 1

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Find the final query tree that is efficient to
execute the following Query using Heuristic
optimization.

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Find the final query tree that is efficient to
execute the following Query using Heuristic
optimization.

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Answer:

(a) Initial (canonical) query tree for SQL query Q.

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Answer:

(b) Moving SELECT operations down the query tree.

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Answer:

(c) Applying the more restrictive SELECT operation first.

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Answer:

(d) Replacing CARTESIAN PRODUCT and SELECT with

JOIN operations.

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Answer:

(e) Moving PROJECT operations down the query tree.

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

 Transaction Processing

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

What are the main Transaction problems?
 Lost Update Problem

 Temporary Update Problem

 Incorrect Summary Problem

 Unrepeatable Read Problem

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

 The Lost Update Problem

 if X = 80 at the start (originally there were 80 reservations on the flight), N = 5

 (T1 transfers 5 seat reservations from the flight corresponding to X to the flight corresponding
to Y), and M = 4 (T2 reserves 4 seats on X), the final result should be X = 79. However, in
the interleaving of operations shown in Figure 20.3(a), it is X = 84 because the update in T1
that removed the five seats from X was lost.

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 20-18

The Temporary Update Problem

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 20-19

The Incorrect Summary Problem

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 20-20

The Unrepeatable Read Problem
 Transaction T reads the same item twice
 Value is changed by another transaction T
between the two reads
 T receives different values for the two reads of
the same item

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 20- 21

Draw the state transaction diagram.

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

What are the properties of transaction?
 ACID properties
 Atomicity

Transaction performed in its entirety or not at all
 Consistency preservation

Takes database from one consistent state to another
 Isolation

Not interfered with by other transactions
 Durability or permanency

Changes must persist in the database

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

What are the conditions of Conflict
Operations?
 Two conflicting operations in a schedule
 Operations belong to different transactions
 Operations access the same item X
 At least one of the operations is a write_item(X)

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Consider the schedules below. Determine whether each schedule
is strict, cascadeless, recoverable, or nonrecoverable.

 S : r 2 (X); w 2 (X); r 1 (X); w 1 (X); r 1 (Y); w 1 (Y); C 1 ; C 2 ;

 Nonrecoverable

 S: r 1 (X); r 2 (X); w 1 (X); w 2 (X); C 2 ; r 1 (Y); w 1 (Y); C 1 ;

 Cascadeless

 S: r 1 (X); w 1 (X); r 2 (X); w 2 (X); r 1 (Y); w 1 (Y); C 1 ; C 2 ;

 Recoverable

 S: r 2 (X); r 1 (X); w 1 (X); r 1 (Y); w 1 (Y); C 1 ; w 2 (X); C 2 ;

 Strict
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 20- 25
Serializability Exercise

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 20-26

Serializability Exercise

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 20-27

Serializability Exercise

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 20-28

 Concurrency Control

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Compare between growing and shrinking
phases in Two-phase locking protocol.

Expanding (growing) phase
 New locks can be acquired but none can be released
 Lock conversion upgrades must be done during this phase


Shrinking phase
 Existing locks can be released but none can be acquired
 Downgrades must be done during this phase

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

What are the different types of Two-
Phase Locking?
 Basic 2PL
 Basic Growing and Shrinking phases.
 Conservative (static) 2PL
 Requires a transaction to lock all the items it
accesses before the transaction begins
 Strict 2PL
 Transaction does not release exclusive locks until
after it commits or aborts
 Rigorous 2PL
 Transaction does not release any locks until after it
commits or aborts
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 21- 31
What are the basic steps of Basic
Timestamp ordering protocol?
 Basic Timestamp Ordering
 1. Transaction T issues a write_item(X) operation:

If read_TS(X) > TS(T) or if write_TS(X) > TS(T), then an younger
transaction has already read the data item so abort and roll-back
T and reject the operation.

If the condition in part (a) does not exist, then execute
write_item(X) of T and set write_TS(X) to TS(T).

 2. Transaction T issues a read_item(X) operation:


If write_TS(X) > TS(T), then an younger transaction has already
written to the data item so abort and roll-back T and reject the
operation.

If write_TS(X)  TS(T), then execute read_item(X) of T and set
read_TS(X) to the larger of TS(T) and the current read_TS(X).

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

What are the basic steps of Strict
Timestamp ordering protocol?
 Strict Timestamp Ordering
 1. Transaction T issues a write_item(X) operation:

If TS(T) > read_TS(X), then delay T until the
transaction T’ that wrote or read X has terminated
(committed or aborted).
 2. Transaction T issues a read_item(X) operation:

If TS(T) > write_TS(X), then delay T until the
transaction T’ that wrote or read X has terminated
(committed or aborted).

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

What are the basic steps of Thomas’s
Write Rule?
 Thomas’s Write Rule
 If read_TS(X) > TS(T) then abort and roll-back T
and reject the operation.
 If write_TS(X) > TS(T), then just ignore the write
operation and continue execution. This is because
the most recent writes counts in case of two
consecutive writes.

 If the conditions given in 1 and 2 above do not

occur, then execute write_item(X) of T and set
write_TS(X) to TS(T).

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Explain the different phases on Validation
(Optimistic) concurrency techniques.
1. Read phase: A transaction can read values of committed data
items. However, updates are applied only to local copies
(versions) of the data items (in database cache).

2. Validation phase: Serializability is checked before

transactions write their updates to the database.

3. Write phase: On a successful validation transactions’

updates are applied to the database; otherwise,
transactions are restarted.

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Explain the different types of Intention
locks modes.
 Intention-shared (IS): indicates that a shared lock(s) will be
requested on some descendent nodes(s).

 Intention-exclusive (IX): indicates that an exclusive lock(s)

will be requested on some descendent node(s).

 Shared-intention-exclusive (SIX): indicates that the

current node is locked in shared mode but an exclusive
lock(s) will be requested on some descendent nodes(s).

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Illustrate the Lock Compatibility Matrix for
Multiple granularity Locking

IS IX S SIX X
IS yes yes yes yes no
IX yes yes no no no
S yes no yes no no
SIX yes no no no no
X no no no no no

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Exercise on data items hierarchy
 Given the following data items hierarchy; Write the required lock and
unlock operations in their correct order:


T1 wants to write the record r211
 LOCK: IX(db), IX(f2), IX(p21), X(r211)
Unlock: unlock(r211), unlock(p21), unlock(f2), unlock(db)


T2 wants to read the record r221
 LOCK: IS(db), IS(f2), IS(p21), S(r211)
 Unlock: unlock(r211), unlock(p21), unlock(f2), unlock(db)

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 21- 38

 Database Recovery

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Define the following terms:

 Immediate Update: As soon as a data item is modified in

cache, the disk copy is updated.

 Deferred Update: All modified data items in the cache is

written either after a transaction ends its execution or after a
fixed number of transactions have completed their
execution.

 Shadow update: The modified version of a data item does

not overwrite its disk copy but is written at a separate disk
location.

 In-place update: The disk version of the data item is

overwritten by the cache version.
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe
Define the following terms:

 Steal: Cache can be flushed before transaction commits.

 No-Steal: Cache cannot be flushed before transaction

commit.

 Force: Cache is immediately flushed (forced) to disk.

 No-Force: Cache is deferred until transaction commits

 Shadow Paging: The AFIM does not overwrite its BFIM but
recorded at another place on the disk. Thus, at any time a
data item has AFIM and BFIM (Shadow copy of the data
item) at two different places on the disk.

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Explain the different types of database failures.

 Transaction failure: Transactions may fail

because of incorrect input, deadlock, incorrect
synchronization.

 System failure: System may fail because of

addressing error, application error, operating
system fault, RAM failure, etc.

 Media failure: Disk head crash, power disruption,

etc.

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Explain the basic states of Write-Ahead Logging
(WAL) protocol.

 For Undo: Before a data item’s AFIM is

flushed to the database disk (overwriting the
BFIM) its BFIM must be written to the log and
the log must be saved on a stable store (log
disk).

 For Redo: Before a transaction executes its

commit operation, all its AFIMs must be written
to the log and the log must be saved on a
stable store.

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Explain the basic steps of Checkpointing
operation.

Checkpointing steps:

1. Suspend execution of transactions temporarily.

2. Force write modified buffer data to disk.
3. Write a [checkpoint] record to the log, save the log to disk.
4. Resume normal transaction execution.

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Exercise
 Consider the following transaction log from the start of the run of a
database system that is capable of running undo/redo logging with
checkpointing:
 Assume the log entries are in the format
<Tid, Variable, New value, Old value>

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Answer:

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Exercise
 Consider the following transaction log from the start of the run of a
database system that is capable of running undo/redo logging with
checkpointing:
 Assume the log entries are in the format
<Tid, Variable, Old value, New value>

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Exercise

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Answer

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Answer

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

 Database Security

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Explain the different types of Threats to database.

 Threats to databases
 Loss of integrity

Improper modification of information
 Loss of availability

Valid user cannot access data objects
 Loss of confidentiality

Unauthorized disclosure of confidential information

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Explain the different types of Control Measures.

 Control measures
 Access control

Handled by creating user accounts and passwords
 Inference control

Must ensure information about individuals cannot be
accessed
 Flow control

Prevents information from flowing to unauthorized
users
 Data encryption

Used to protect sensitive transmitted data
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe
Compare between DAC,MAC and RBAC

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 30- 54

Comparing MAC,DAC and RBAC

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 30- 55

What are the different security classes of
Mandatory Access Control

 Typical security classes are:

 Top Secret (TS)

 Secret (S)
 Confidential (C)
 Unclassified (U)
 TS is the highest level and U the lowest: TS ≥ S ≥ C ≥ U

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Given the original Employee table in (a): Write the
Appearance after filtering for C and U Users.

(b) Appearance of
EMPLOYEE after filtering for
classification C users

(c) Appearance of
EMPLOYEE after filtering
for classification U users

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

What are the Risks Associated with
SQL Injection?
 Unauthorized Access
 Data Disclosure
 Data Manipulation
 Code Execution
 Denial of Service (DoS)
 Application Defacement
 Elevation of Privilege

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 30- 58

What are the different SQL Injection
Methods? Give Examples
 SQL manipulation
 Changes an SQL command in the application
 Example: adding conditions to the WHERE clause

 Code injection
 Add additional SQL statements or commands that
are then processed
 SELECT * FROM `table`; DROP TABLE
`table`;

 Function call injection

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 30- 59
SQL Injection Examples
 Example 1: SQL Manipulation Injection

 -- SQL query in application code

SELECT * FROM users WHERE username = 'input_username' AND
password = 'input_password';

 -- SQL injection input

' OR '1'='1'; --

 -- Modified query executed by the database

SELECT * FROM users WHERE username = ‘ ' OR '1'='1'; -- ' AND
password = 'input_password';

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 30- 60

SQL Injection Examples
 Example 2: UNION-Based SQL Injection

 -- SQL query in application code

SELECT product_name, price FROM products WHERE category =
'input_category';

 -- SQL injection input

' UNION SELECT username, password FROM users; --

 -- Modified query executed by the database

SELECT product_name, price FROM products WHERE category = ‘ ’
UNION SELECT username, password FROM users; --';

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 30- 61

SQL Injection Examples
 Example 3: Error-Based SQL Injection

 -- SQL query in application code

SELECT * FROM products WHERE product_id = 'input_product_id';

 -- SQL injection input

' OR 1=CONVERT(int, (SELECT @@version)); --

 -- Modified query executed by the database

SELECT * FROM products WHERE product_id = ‘ ' OR
1=CONVERT(int, (SELECT @@version)); --';

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 30- 62

SQL Injection Examples
 Example 4: Time-Based Blind SQL Injection

 -- SQL query in application code

SELECT * FROM users WHERE username = 'input_username' AND
password = 'input_password';

 -- SQL injection input

' OR IF(1=1, SLEEP(5), 0); --

 -- Modified query executed by the database

SELECT * FROM users WHERE username = ‘ ' OR IF(1=1, SLEEP(5),
0); -- ' AND password = 'input_password';

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 30- 63

SQL Injection Examples
 Example 5: Out-of-Band SQL Injection

 -- SQL query in application code

SELECT * FROM feedback WHERE user_id = 'input_user_id';

 -- SQL injection input

'; EXEC xp_cmdshell('nslookup example.com'); --

 -- Modified query executed by the database

SELECT * FROM feedback WHERE user_id = ‘ '; EXEC
xp_cmdshell('nslookup example.com'); --';

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 30- 64

What are the SQL Injection
Protection Techniques?
 Use prepared statements or parameterized
queries: Prepared statements and parameterized
queries allow you to safely insert user input.

 Validate user input: Before using user input in a

SQL query, you should validate it to make sure that
it is not malicious.

 Stored Procedures: Use stored procedures to

encapsulate and execute SQL code on the
database server.
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 30- 65
What are the SQL Injection
Protection Techniques?
 Use a web application firewall (WAF): A WAF can
help to block malicious traffic, including SQL
injection attacks.

 Keep your software up to date: Software vendors

often release security patches that fix SQL injection
vulnerabilities. Make sure that you keep your
software up to date to protect against these
vulnerabilities.

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 30- 66

Security SQL Exercise

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

 Security SQL Exercise

Account D can retrieve any attribute of EMPLOYEE or

DEPENDENT and can modify DEPENDENT.

GRANT SELECT ON EMPLOYEE, DEPENDENT

TO D;

GRANT UPDATE ON DEPENDENT

TO D;

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

 Security SQL Exercise

- Account A can retrieve or modify any relation except

DEPENDENT and can grant any of these privileges to other
users.

GRANT SELECT, UPDATE

ON EMPLOYEE, DEPARTMENT, DEPT_LOCATIONS,

PROJECT, WORKS_ON

TO A

WITH GRANT OPTION;

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

 Security SQL Exercise

Account B can retrieve all the attributes of EMPLOYEE except

for SALARY.

CREATE VIEW EMPS AS

SELECT FNAME, MINIT, LNAME, SSN, BDATE,

ADDRESS, SEX,SUPERSSN, DNO

FROM EMPLOYEE;

GRANT SELECT ON EMPS

TO B;
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe
 NO-SQL

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Perform CRUD Operations
db.movies.insertOne(
{
title: "The Favourite",
genres: [ "Drama", "History" ],
runtime: 121,
rated: "R",
year: 2018,
directors: [ "Yorgos Lanthimos" ],
cast: [ "Olivia Colman", "Emma Stone", "Rachel Weisz" ],

type: "movie"
}
)

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Perform CRUD Operations
db.movies.insertMany([
{
title: "Jurassic World: Fallen Kingdom",
genres: [ "Action", "Sci-Fi" ],
runtime: 130,
rated: "PG-13",
year: 2018,
directors: [ "J. A. Bayona" ],
cast: [ "Chris Pratt", "Bryce Dallas Howard", "Rafe Spall" ],

type: "movie"
},
{
title: "Tag",
genres: [ "Comedy", "Action" ],
runtime: 105,
rated: "R",
year: 2018,
directors: [ "Jeff Tomsic" ],
cast: [ "Annabelle Wallis", "Jeremy Renner", "Jon Hamm" ],

type: "movie"
}
])

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Perform CRUD Operations

• Read All Documents in a Collection:

• To read all documents in the collection, pass an empty document as

the query filter parameter to the find method.

• The query filter parameter determines the select criteria.

• db.movies.find()

• This operation is equivalent to the following SQL statement:

SELECT * FROM movies

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Perform CRUD Operations

• Specify Equality Condition:

• To select documents that match an equality condition, specify the

condition as a <field>:<value> pair in the query filter document.

• To return all movies where the title equals Titanic from the movies
collection: db.movies.find( { "title": "Titanic" } )

• This operation corresponds to the following SQL statement:

SELECT * FROM movies WHERE title = "Titanic"

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Perform CRUD Operations

• Specify Conditions Using Query Operators:

• Use query operators in a query filter document to perform more

complex comparisons and evaluations.

• Query operators in a query filter document have the following form:

{ <field1>: { <operator1>: <value1> }, ... }

• To return all movies from the movies collection which are either rated
PG or PG-13:
db.movies.find( { rated: { $in: [ "PG", "PG-13" ] } } )
 SELECT * FROM movies WHERE rated in ("PG", "PG-13")

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Perform CRUD Operations

• Specify Logical Operators (AND / OR)

• A compound query can specify conditions for more than one field in
the collection's documents.

• Implicitly, a logical AND conjunction connects the clauses of a

compound query so that the query selects the documents in the
collection that match all the conditions.

• To return movies which were released in Mexico and have an IMDB

rating of at least 7:

db.movies.find( { countries: "Mexico", "imdb.rating": { $gte:

7}})

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Perform CRUD Operations

• Update Documents:

• The MongoDB shell provides the following methods to update

documents in a collection:

• To update a single document, use db.collection.updateOne().

• To update multiple documents, use db.collection.updateMany().

• To replace a document, use db.collection.replaceOne().

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Perform CRUD Operations

• Update Documents:

• To update a document, MongoDB provides update operators such as

$set, to modify field values.

• To use the update operators, pass to the update methods an update

document of the form:

{
<update operator>: { <field1>: <value1>, ... },
<update operator>: { <field2>: <value2>, ... },
...
}

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Perform CRUD Operations

• Update a Single Document:

• Use the db.collection.updateOne() method to update the first

document that matches a specified filter.

• To update the first document in the movies collection where title

equals "Twilight":

db.movies.updateOne( { title: "Twilight" },

{
$set: {
plot: "risks everything–including her life–."
},
$currentDate: { lastUpdated: true }
})
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe
Perform CRUD Operations

• Update Multiple Documents:

• Use the db.collection.updateMany() to update all documents that

match a specified filter.

• To update all documents in the sample_airbnb.listingsAndReviews

collection to update where security_deposit is less than 100:

use sample_airbnb
db.listingsAndReviews.updateMany(
{ security_deposit: { $lt: 100 } },
{
$set: { security_deposit: 100, minimum_nights: 1 }
}
)
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe
Perform CRUD Operations

• Replace a Document:

• To replace the entire content of a document except for the _id field,
pass an entirely new document as the second argument to
db.collection.replaceOne().

• When replacing a document, the replacement document must contain

only field/value pairs. Do not include update operators expressions.

• The replacement document can have different fields from the original
document.

• In the replacement document, you can omit the _id field since the _id
field is immutable; however, if you do include the _id field, it must
have the same value as the current value.
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe
Perform CRUD Operations

• Replace a Document:

• To replace the first document from the sample_analytics.accounts

collection where account_id: 371138:

db.accounts.replaceOne(
{ account_id: 371138 },
{ account_id: 893421, limit: 5000, products: [ "Investment",
"Brokerage" ] }
)

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Perform CRUD Operations

• Delete Documents:

• The MongoDB shell provides the following methods to delete

documents from a collection:

• To delete multiple documents, use db.collection.deleteMany().

• To delete a single document, use db.collection.deleteOne().

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Perform CRUD Operations

• Delete All Documents:

• To delete all documents from a collection, pass an empty filter

document {} to the db.collection.deleteMany() method.

• To delete all documents from the movies collection:

db.movies.deleteMany({})

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Perform CRUD Operations

• Delete All Documents that Match a Condition:

• You can specify criteria, or filters, that identify the documents to

delete.

• The filters use the same syntax as read operations.

• To specify equality conditions, use <field>:<value> expressions in

the query filter document.

• To delete all documents that match a deletion criteria, pass a filter

parameter to the deleteMany() method.

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Perform CRUD Operations

• Delete All Documents that Match a Condition:

• To delete all documents from the movies collection where the title
equals "Titanic":

db.movies.deleteMany( { title: "Titanic" } )

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Perform CRUD Operations

• Delete Only One Document that Matches a Condition:

• To delete at most a single document that matches a specified filter

(even though multiple documents may match the specified filter) use
the db.collection.deleteOne() method.

• To delete the first document from the movies collection where the
cast array contains "Brad Pitt":

db.movies.deleteOne( { cast: "Brad Pitt" } )

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Aggregation Operations

• Aggregation operations process multiple documents and return computed

results.

• You can use aggregation operations to:

• Group values from multiple documents together.

• Perform operations on the grouped data to return a single result.

• Analyze data changes over time.

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Aggregation Operations

• Insert some sample data into the "sales" collection:

db.sales.insertMany([
{ date: ISODate("2023-01-01"), product: "A", quantity: 10, amount: 100 },
{ date: ISODate("2023-01-01"), product: "B", quantity: 5, amount: 50 },
{ date: ISODate("2023-01-02"), product: "A", quantity: 8, amount: 80 },
{ date: ISODate("2023-01-02"), product: "B", quantity: 7, amount: 70 },
{ date: ISODate("2023-01-03"), product: "A", quantity: 12, amount: 120 },
{ date: ISODate("2023-01-03"), product: "B", quantity: 3, amount: 30 }
]);

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Aggregation Operations

• Calculate the Total Amount Sold for Each Product

db.sales.aggregate([
{
$group: {
_id: "$product",
totalAmount: { $sum: "$amount" }
}
}
]);

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Aggregation Operations

• Calculate the Total Amount Sold for Each Product

• Results:

{ "_id" : "B", "totalAmount" : 150 }

{ "_id" : "A", "totalAmount" : 300 }

• Equivalent SQL:

SELECT product, SUM(amount) AS totalAmount

FROM sales
GROUP BY product;

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Aggregation Operations

• Find the Average Quantity Sold for Each Product

db.sales.aggregate([
{
$group: {
_id: "$product",
avgQuantity: { $avg: "$quantity" }
}
}
]);

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Aggregation Operations

• Find the Average Quantity Sold for Each Product

• Results:

{ "_id" : "B", "avgQuantity" : 5 }

{ "_id" : "A", "avgQuantity" : 10 }

• Equivalent SQL:

SELECT product, AVG(quantity) AS avgQuantity

FROM sales
GROUP BY product;

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

 Aggregation Operations

• Identify the Date with the Highest Total Sales Amount

db.sales.aggregate([
{
$group: {
_id: "$date",
totalAmount: { $sum: "$amount" }
}
},
{
$sort: { totalAmount: -1 }
},
{
$limit: 1
}
]); Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe
Aggregation Operations

• Identify the Date with the Highest Total Sales Amount

• Results:

{ "_id" : ISODate("2023-01-01T00:00:00Z"), "totalAmount" : 150 }

• Equivalent SQL:

SELECT
date,
SUM(amount) AS totalAmount
FROM sales
GROUP BY date
ORDER BY totalAmount DESC
LIMIT 1;
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe
Aggregation Operations

• Calculate the Total Amount Sold on Each Date

db.sales.aggregate([
{
$group: {
_id: "$date",
totalAmount: { $sum: "$amount" }
}
},
{
$sort: { _id: 1 }
}
]);

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Aggregation Operations

• Calculate the Total Amount Sold on Each Date

• Results:
{ "_id" : ISODate("2023-01-01T00:00:00Z"), "totalAmount" : 150 }
{ "_id" : ISODate("2023-01-02T00:00:00Z"), "totalAmount" : 150 }
{ "_id" : ISODate("2023-01-03T00:00:00Z"), "totalAmount" : 150 }

• Equivalent SQL:

SELECT
date,
SUM(amount) AS totalAmount
FROM sales
GROUP BY date
ORDER BY date;
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe
 Aggregation Operations

• Find the Date with the Lowest Total Sales Amount

db.sales.aggregate([
{
$group: {
_id: "$date",
totalAmount: { $sum: "$amount" }
}
},
{
$sort: { totalAmount: 1 }
},
{
$limit: 1
}
]); Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe
Aggregation Operations

• Find the Date with the Lowest Total Sales Amount

• Results:

{ "_id" : ISODate("2023-01-01T00:00:00Z"), "totalAmount" : 150 }

• Equivalent SQL:

SELECT
date,
SUM(amount) AS totalAmount
FROM sales
GROUP BY date
ORDER BY totalAmount
LIMIT 1;
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe
Aggregation Operations

• Identify Products with Total Sales Amount Greater Than 80

db.sales.aggregate([
{
$group: {
_id: "$product",
totalAmount: { $sum: "$amount" }
}
},
{
$match: {
totalAmount: { $gt: 80 }
}
}
]);
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe
Aggregation Operations

• Identify Products with Total Sales Amount Greater Than 80

• Results:

{ "_id" : "B", "totalAmount" : 150 }

{ "_id" : "A", "totalAmount" : 300 }

• Equivalent SQL:

SELECT product, SUM(amount) AS totalAmount

FROM sales
GROUP BY product
HAVING totalAmount > 80;

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Aggregation Operations
• Calculate the Total Amount and Average Quantity Sold for Each
Product on January 1, 2023
db.sales.aggregate([
{
$match: {
date: ISODate("2023-01-01")
}
},
{
$group: {
_id: "$product",
totalAmount: { $sum: "$amount" },
avgQuantity: { $avg: "$quantity" }
}
}
]);
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe
Aggregation Operations

• Calculate the Total Amount and Average Quantity Sold for Each
Product on January 1, 2023
• Results:

{ "_id" : "B", "totalAmount" : 50, "avgQuantity" : 5 }

{ "_id" : "A", "totalAmount" : 100, "avgQuantity" : 10 }

• Equivalent SQL:

SELECT
product, SUM(amount) AS totalAmount, AVG(quantity) AS avgQuantity

FROM sales
WHERE date = '2023-01-01'
GROUP BY product;
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe
Aggregation Operations
• Filter Products with Total Sales Amount Greater Than 100

db.sales.aggregate([
{
$group: {
_id: "$product",
totalAmount: { $sum: "$amount" }
}
},
{
$match: {
totalAmount: { $gt: 100 }
}
}
]);

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Aggregation Operations
• Filter Products with Total Sales Amount Greater Than 100

• Results:

{ "_id" : "A", "totalAmount" : 120 }

{ "_id" : "B", "totalAmount" : 150 }

• Equivalent SQL:

SELECT product, SUM(amount) AS totalAmount

FROM sales
GROUP BY product
HAVING totalAmount > 100;

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

Aggregation Operations
• Projecting Fields for Total Quantity
db.sales.aggregate([
{
$group: {
_id: "$product",
totalQuantity: { $sum: "$quantity" }
}
},
{
$project: {
_id: 0,
product: "$_id",
totalQuantity: 1
}
}
]);
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe
Aggregation Operations
• Projecting Fields for Total Quantity

• Results:

{ "product" : "A", "totalQuantity" : 30 }

{ "product" : "B", "totalQuantity" : 15 }

• Equivalent SQL:

SELECT
product,
SUM(quantity) AS totalQuantity
FROM sales
GROUP BY product;

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe

 End of Revision

Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe