MongoDB lab

1. INTRODUCTION

• You should follow along with this tutorial by entering the commands into your
personal database, and completing the exercises where prompted.

• Ask the lab helpers if you get stuck.

NOTE: You should have received an email with the details for your Mongo
DB cloud account, if not then please contact [email protected] as
soon as possible to set this up. Your personal database will be the same as
your HWU username.

2. CONNECTING
You can use MongoDB Compass or Studio3D (free version), details for the setup for
each are shown below. Both are available for install on your own machine.

https://www.mongodb.com/products/tools/compass (free)

https://studio3t.com/download/ (commercial with free option)

MongoDB Compass is the preferred option

MongoDB Compass Setup
Click on the New Connection +

Open the Advanced Connection Options

section

Enter the details as follows:

[GENERAL Tab]

HOST: 140.238.98.146:27017

[AUTHENTICATION Tab]

Click on Username/Password

{change ab12 to your username}

Username: ab12
Password: check_your_email
Authentication Database: ab12

Click SAVE & CONNECT

Save the connection to favorites as:

Cloud-Mongo-1

2
Studio 3T Setup
Click on the CONNECT button (top left)

Choose Manually configure…

Click Next

Enter the details as follows:

Connection Name: Cloud-Mongo1

Server: 140.238.98.146

[AUTHENTICATION Tab]

Select Basic (SCRAM-SHA-256)

authentication mode

{change ab12 to your username}

Username: ab12
Password: the_password_emailed_to_you
Authentication Database: ab12

Click Save

Highlight the Cloud-Mongo-1 entry and

click Connect

3
3. CHANGE YOUR PASSWORD (OPTIONAL)
It is a good idea to change your password to something you can remember more
easily than the random password sent to you.

If you are using MongoDB Compass:

Once connected to the server open a Mongo shell.

This is where you can type in commands.

You’ll find it the bottom of the window, click on the words ‘>_MONGOSH’ to open it
up.

Once you’ve opened the MONGOSH (mongo shell) then switch to your database by typing:

{change ab12 to your username}

use ab12

From here you can issue the command to change your password.

Make sure you keep a note of it!

db.changeUserPassword("ab12","your new password here")

All going well you’ll see:

{ ok: 1 }

IMPORTANT:
You should now log out, change the connection settings to your new
password and log back in again. You’ll find the a Disconnect under the
Connect menu item.

Find your mongo-cloud-1 connection, click on it, go to the advanced connection

options, then the authentication tab, edit your password – save. Then connect.

4
If you are using Studio3T:
Issue the command to change your password from the query
window.

Make sure you keep a note of your new password!

{change ab12 to your username}

db.changeUserPassword("ab12","your new password here")

All going well you’ll see the result ok: 1.0

IMPORTANT:
You should now log out, change the connection settings
to your new password and log back in again.

Close the current connection (right click disconnect)

Return to the Connection Manager, edit the connection

updating your password (in the Authentication tab), Save
the connection details. Now connect again to the server
with this new password.

5
4. IMPORTING DATA
Open a terminal on your PC
- use terminal / shell for macOS / Linux
- on Windows you can use cmd to open the command prompt

Next download the data file for this lab from the terminal using curl:
curl -o labData.json www.macs.hw.ac.uk/~pb56/labData.json

Take a look at the file in a text editor (e.g. Notepad)

On Windows:

notepad labData.json

On mac / Linux:

cat labData.json

This is only a small file with a few records. You can see it is in JSON format and
contains some details about people such as their first_name, age, and title.

Next we need to create an empty collection and load in this data.

In MongoDB Compass:
Click on the + next to your username

Create Collection window should show

Enter hwuPeople as the collection name

Click Create Collection

The collection should open automatically

Now click on ADD DATA

Select Import JSON or CSV file

Select the labData.json file you downloaded

Click Import

5 documents should be imported

6
In Studio3T:

Right click – Add Collection…

Set the collection name as hwuPeople

Click Create

Right click on the collection (hwuPeople) and select Import Data…

If you are using the full version or still in the trial period you can now import the JSON file, select the
labData.json you downloaded earlier.

If you are using the free version without access to the JSON import
option then download this file using curl as you did before
www.macs.hw.ac.uk/~pb56/labData.agz then use the BSON –
mongodump archive load option instead.

Once you have loaded the dataset you should have 5 documents in your collection.

7
5. QUERYING

In MongoDB Compass:

Use the MongoShell to enter your commands, ensure you switch

to your database first. You should find the >MONGOSH at the
lower left of your screen – click on it to expand the space and
then drag it up to take up more of your screen.
{change ab12 to your username}

use ab12

>> it should not show ‘switch to db ab12’

In Studio3T:

Click on the IntelliShell

Use the shell on the right to enter your commands, results will be displayed
underneath.

You can change the output view from Tree View, to Table, or JSON

8
Filtering

First of all take a look at all the data loaded, there should be 5
documents:
db.hwuPeople.find()

Let us find all the people with role “ra”:

db.hwuPeople.find({role : "ra"})

Notice that Manni appears in the output: mongo automatically searches inside an
array (ie the role array).

To find all people named “Ken McLeod”:

db.hwuPeople.find({first_name: "ken",last_name: "mcleod"})

To find all people older than (greater than $gt ) 35:

db.hwuPeople.find({age : {$gt: 35}})

There are multiple ways to write the same query, e.g.:

db.hwuPeople.aggregate([{ $match : { age : {$gt: 35}}}])

Basic queries

 TASK: write a query to find all the RAs under 40 years old.
Hint: use $lt for less than.

9
Distinct

You can also find the unique values, such as distinct titles within the collection:
db.hwuPeople.distinct("title")

… which should return a list [ "dr", "mr" ] to get the number of items in the list
add .length to the previous command.

db.hwuPeople.distinct("title").length

Returned Fields

You can choose which fields in a document are returned. To display only names:
db.hwuPeople.find({},{first_name : 1, last_name : 1})

Searching for Text

It is possible to search for strings within text fields – try this (it shouldn’t work yet!):
db.hwuPeople.find({$text: {$search: "burger"}})

The above query did not work as we have not set up a text index yet.
To create an index on the first and last names we use the command:
db.hwuPeople.createIndex({"first_name": "text","last_name": "text"})

Now we can try running that search query again and it should work this time, giving
us the list of all the people who have the string "burger" in their name.

Another way to find ‘contains’ is using regex as follows (this should also work on
non-indexed fields):
db.hwuPeople.find({"last_name":/burger/i})

Note: the i makes it a case-insensitive search, remove the i to make it case-sensitive

You can use this query approach in conjunction with distinct – for example to find the
unique titles for people with ‘burger’ in their surname:

db.hwuPeople.distinct("title",{"last_name": /burger/i})

Note: this returns the distinct values for “title” based on the query “last_name” contains the value
‘burger’.

You can change the query to find people who have a surname starting ^ with burger or
ending $ in burger too as follows:
db.hwuPeople.distinct("first_name",{"last_name":/^burger/i})

db.hwuPeople.distinct("first_name",{"last_name":/burger$/i})

10
Multiple Conditions

You can specify multiple conditions using AND logic to add more conditions to the
query; alternatively you can specify OR to permit alternatives - for example find all
people that have a role “assistant prof” or “associate prof”:
db.hwuPeople.find({$or: [{"role": 'associate prof'}, "role": 'assistant prof'}]})

Exists

MongoDB does not enforce a schema and so each document can have different fields.
In this collection we may want to find everyone who does not have an “age”
specified:
db.hwuPeople.find({age : {$exists: false}})

Rather than listing the documents without an age, we can simply ask for the number
of documents using the count function:
db.hwuPeople.countDocuments({"age" : {$exists: false}})

You could also do this as follows:

db.hwuPeople.find({"age" : {$exists: false}}).count()

AND / OR conditions

Sorting

To sort the results of a “find” query based on a field append .sort({field: 1}) after the find()
statement. Use 1 for ascending order, and -1 for descending order.
 TASK: Write the query to sort the entire list of people in ascending order by age.

Aggregate / Group

If you need to get a summary for a group of values you can also sort after
aggregating, using the sortByCount function. This is like running a GROUP BY in
SQL. Here you should get the count for each title (e.g. 3 Dr,1 Dr, etc)

db.hwuPeople.aggregate([ {$sortByCount: "$title"},{$limit:3}])

11
 Note: The {$limit:3} is optional, here it demonstrates how you could retrieve just the top 3 groups
based on the count.

Try changing it to limit 2 to see what happens to the output.

Aggregation also allows you to run calculations per group such as finding the sum,
minimum, average etc:

db.hwuPeople.aggregate([{$group: {_id:"$title",total: {"$sum": '$age'}}} ])

6. INSERTING, UPDATING & REMOVING

To insert someone into the database:
db.hwuPeople.insertOne({"first_name" : "joe", "last_name" : "blogs", "age" : 21, "role": "msc", "id" :
"jb33"})
To prove this worked:
db.hwuPeople.find({"first_name" : "joe"})

Inserting data

 TASK: add yourself, specifying your _id manually.

HINT: treat “_id” as just another name/value pair.

Dr Burger has been promoted and so we need to change his “title” and his “role” to
“prof”:
db.hwuPeople.update({"last_name" : "burger"}, {$set: {"title" : "prof", "role" : "prof"}})

 QUESTION: What are the potential side effects of the previous statement?
HINT: “burger” is a rare last name in the UK and this is a small dataset

With a flexible schema you can add information to one document but not to others:
add an email address for Dr McLeod:
db.hwuPeople.update({"last_name": "mcleod"}, {$set: {"email": "[email protected]"}})
Look at the document for Manni:
db.hwuPeople.find({"first_name" : "manni"})

He is listed as being 37, but his birthday was last week; to increase his age:
db.hwuPeople.update({"first_name" : "manni"}, {$inc: {"age": 1}})

Manni has another role (lab assistant), to add this:

db.hwuPeople.update({"first_name": "manni"}, {$push: {"role": "lab assistant"}})

12
NOTE: $push only works for arrays.
 TASK: update your information to provide your email address and your title
(e.g., Mr, Ms etc.).

Updating existing entries

If you try to update a document that is not there, nothing happens:

db.hwuPeople.update({"first_name": "andy", "last_name": "proudlove", "role": "ra"}, {"age": 47})

db.hwuPeople.find({"first_name": "andy"})

Andy has not been added as no matching record was found. However, MongoDB
supports “upserts” (update or insert if there is no document found):

db.hwuPeople.update(

{ first_name: "andy", last_name: "proudlove", role: "ra" },

{ $set: { first_name: "andy", last_name: "proudlove", role: "ra", age: 47 } },

{ upsert: true }

);

db.hwuPeople.find({"first_name": "andy"})

Now remove everyone greater or equal to ($gte) 47 from the collection:

db.hwuPeople.remove ( {age : {$gte: 47} } )

Note: It should just be Andy that gets removed

13
 TASK: Use the remove command to delete Joe Bloggs from the database.
HINT: most of the operations that work with the insert command also work
with the remove command.

7. OPTIMISATION

For queries to be efficient they must use an index. To check if a query uses an index
use the .explain() method:

db.hwuPeople.find({"age" : {$gt: 35}}).explain()

COLLSCAN

Explain query plan - you will need to be in JSON view on Studio3T

We can see that the winning plan is COLLSCAN meaning that the collection is being
scanned, i.e. no index was used.

To see more details of the number of documents scanned add “executionStats” as

follows:

db.hwuPeople.find({"age" : {$gt: 35}}).explain("executionStats")

 e.g. totalDocsExamined: 6

To add an index to “age”:

db.hwuPeople.createIndex({"age": 1})

Now if we run the .explain() method again we see the winning plan is based on an
IXSCAN, i.e. it is using the index on the age field that we just created.

Also try the executionStats to see how many documents were examined this time.
db.hwuPeople.find({"age" : {$gt: 35}}).explain("executionStats")

 e.g. totalDocsExamined: 2

14
8. BACKUP
MongoDB Compass Studio3T

Click on a collection Right click on a collection

Click on Choose Export Collection

Choose Export the full collection Select a format (e.g. BSON – mongodump)

Choose JSON Click Configure

Hit the Export button Specify a Destination Folder

Specify a file path and filename Specify BSON – mongodump archive

Hit select button Hit the button

Check the file was written Check the file was written

9. DELETING A COLLECTION
Log into mongo and switch to your database (i.e. use ab12)
To delete the hwuPeople collection:
db.hwuPeople.drop()

To list all the collections in your database:

show collections

10. EXERCISE
Create a new collection (called “exercise”) that includes the following
information:
name: albert burger, role: supervisor
name: alasdair gray, role: supervisor
name: iain wiles, role: phd
name: steve smith, role: phd
name: hugh dollar, role: phd
Additionally, include the following relationships:
Alasdair supervises Iain and Steve.
Albert supervises Steve and Hugh.
HINT: use an array to hold the supervisor information against
each student.

 TASK: Now construct an optimized query to list all of Alasdair Gray’s

students.
 TASK: Write a query to find all the students with 2 supervisors.
HINT: use the condition $size: 2.
 TASK: Write a query to find those with more than 1 supervisor
HINT: use the condition $where and the javascript length feature.

15
11. USING JAVASCRIPT WITH MONGODB

Let’s use Javascript to create 50 robots in a collection from the MonogoDB shell. We
will use a FOR loop (iteration) and the assign the value to variable i

for (i=0;i<50;i++)

{
db.robots1.insertOne({"name":"robot"+i } )
}
Count the documents in the collection to make sure it all went as planned.
db.robots1.countDocuments()

…or you can take a look with:

db.robots1.find()

Note: you will need to type “it” to iterate through the results on MongoDB Compass; Studio3T has
page[<-][->]buttons for 50+ results

Now let’s add a unique index to the robot collection based on the name.
db.robots1.createIndex({"name":1},{unique:true})

If you try to run the Javascript again to insert the same 50 robots, you should get an
error about duplicates as a result of this unique constraint.

Comparing Collections
OK now let’s make another collection of robots numbered from 25 to 40 and this time
put them in a robot2 collection.
for (i=25;i<40;i++)

{
db.robots2.insertOne({"name":"robot"+i } )
}

We can check which robots don’t exist in both collection like this :
($nin means not in)

db.robots1.find({

name: { $nin: db.robots2.distinct("name") }

});

To remove the robots from a collection use a blank search filter like this:
db.robots1.deleteMany({})

To check they were deleted use countDocuments() or find ().

db.robots1.countDocuments()

--- END OF THE LAB ---

16