Notebooks/07_MongoDB.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction to Databases\n",
    "\n",
    "### Using [MongoDB](https://docs.mongodb.com/manual/administration/install-community/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [MongoDB Cheat Sheet](https://www.mongodb.com/developer/quickstart/cheat-sheet/) (Command line)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Connect to MongoDB Shell\n",
    "```\n",
    "mongo \n",
    "```\n",
    "Obs: connects to mongodb://127.0.0.1:27017 by default  \n",
    "```\n",
    "mongo --host <host> --port < port > -u < user > -p < pwd >  \n",
    "```\n",
    "Obs: omit the password if you want a prompt  \n",
    "```\n",
    "mongo \"mongodb://192.168.1.1:27017\"  \n",
    "```    \n",
    "#### Show Databases\n",
    "```  \n",
    "show dbs\n",
    "``` \n",
    "#### Switch Database  \n",
    "```    \n",
    "use < database_name > \n",
    "```\n",
    "#### Show Collections\n",
    "```    \n",
    "show collections\n",
    "```    \n",
    "#### Run JavaScript File\n",
    "```    \n",
    "load(\"myScript.js\")    \n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Create Databases (for a database 'db', and a collection 'coll')\n",
    "```\n",
    "db.coll.insertOne({name: \"Max\"})  \n",
    "db.coll.insert([{name: \"Max\"}, {name:\"Alex\"}]) // ordered bulk insert  \n",
    "db.coll.insert([{name: \"Max\"}, {name:\"Alex\"}], {ordered: false}) // unordered bulk insert  \n",
    "db.coll.insert({date: ISODate()})  \n",
    "db.coll.insert({name: \"Max\"}, {\"writeConcern\": {\"w\": \"majority\", \"wtimeout\": 5000}}) \n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Read Operations\n",
    "\n",
    "```\n",
    "db.coll.findOne() // returns a single document  \n",
    "db.coll.find()    // returns a cursor - show 20 results - \"it\" to display more  \n",
    "db.coll.find().pretty()  \n",
    "db.coll.find({name: \"Max\", age: 32}) // implicit logical \"AND\".  \n",
    "db.coll.find({date: ISODate(\"2020-09-25T13:57:17.180Z\")})  \n",
    "db.coll.find({name: \"Max\", age: 32}).explain(\"executionStats\") // or \"queryPlanner\" or \"allPlansExecution\"  \n",
    "db.coll.distinct(\"name\")  \n",
    "```\n",
    "#### Count  \n",
    "```\n",
    "db.coll.count({age: 32})          // estimation based on collection metadata  \n",
    "db.coll.estimatedDocumentCount()  // estimation based on collection metadata  \n",
    "db.coll.countDocuments({age: 32}) // alias for an aggregation pipeline - accurate count  \n",
    "```\n",
    "#### Comparison  \n",
    "```\n",
    "db.coll.find({\"year\": {$gt: 1970}})  \n",
    "db.coll.find({\"year\": {$gte: 1970}})  \n",
    "db.coll.find({\"year\": {$lt: 1970}})  \n",
    "db.coll.find({\"year\": {$lte: 1970}})  \n",
    "db.coll.find({\"year\": {$ne: 1970}})  \n",
    "db.coll.find({\"year\": {$in: [1958, 1959]}})  \n",
    "db.coll.find({\"year\": {$nin: [1958, 1959]}})  \n",
    "```  \n",
    "\n",
    "#### Logical   \n",
    "\n",
    "```\n",
    "db.coll.find({name:{$not: {$eq: \"Max\"}}})\n",
    "db.coll.find({$or: [{\"year\" : 1958}, {\"year\" : 1959}]})\n",
    "db.coll.find({$nor: [{price: 1.99}, {sale: true}]})\n",
    "db.coll.find({$and: [{$or: [{qty: {$lt :10}}, {qty :{$gt: 50}}]}, {$or: [{sale: true}, {price: {$lt: 5 }}]}]})  \n",
    "```\n",
    "\n",
    "#### Element   \n",
    "\n",
    "```\n",
    "db.coll.find({name: {$exists: true}})  \n",
    "db.coll.find({\"zipCode\": {$type: 2 }})  \n",
    "db.coll.find({\"zipCode\": {$type: \"string\"}})  \n",
    "```\n",
    "\n",
    "#### Aggregation Pipeline  \n",
    "\n",
    "```  \n",
    "db.coll.aggregate([{$match: {status: \"A\"}},{$group: {_id: \"$cust_id\", total: {$sum: \"$amount\"}}},{$sort: {total: -1}}])  \n",
    "```  \n",
    "\n",
    "#### Text search with a \"text\" index\n",
    "\n",
    "```\n",
    "db.coll.find({$text: {$search: \"cake\"}}, {score: {$meta: \"textScore\"}}).sort({score: {$meta: \"textScore\"}})  \n",
    "```  \n",
    "\n",
    "#### Regex  \n",
    "```\n",
    "db.coll.find({name: /^Max/})   // regex: starts by letter \"M\"  \n",
    "db.coll.find({name: /^Max$/i}) // regex case insensitive  \n",
    "\n",
    "```\n",
    "\n",
    "#### Array  \n",
    "\n",
    "```\n",
    "db.coll.find({tags: {$all: [\"Realm\", \"Charts\"]}})  \n",
    "db.coll.find({field: {$size: 2}}) // impossible to index - prefer storing the size of the array & update it  \n",
    "db.coll.find({results: {$elemMatch: {product: \"xyz\", score: {$gte: 8}}}})  \n",
    "```\n",
    "\n",
    "#### Projections  \n",
    "\n",
    "```\n",
    "db.coll.find({\"x\": 1}, {\"actors\": 1})               // actors + _id   \n",
    "db.coll.find({\"x\": 1}, {\"actors\": 1, \"_id\": 0})     // actors   \n",
    "db.coll.find({\"x\": 1}, {\"actors\": 0, \"summary\": 0}) // all but \"actors\" and \"summary\"   \n",
    "```\n",
    "\n",
    "#### Sort, skip, limit  \n",
    "\n",
    "```\n",
    "db.coll.find({}).sort({\"year\": 1, \"rating\": -1}).skip(10).limit(3)  \n",
    "```\n",
    "\n",
    "#### Read Concern  \n",
    "\n",
    "```\n",
    "db.coll.find().readConcern(\"majority\")   \n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Update Operations\n",
    "\n",
    "```\n",
    "db.coll.update({\"_id\": 1}, {\"year\": 2016}) // WARNING! Replaces the entire document\n",
    "db.coll.update({\"_id\": 1}, {$set: {\"year\": 2016, name: \"Max\"}})\n",
    "db.coll.update({\"_id\": 1}, {$unset: {\"year\": 1}})\n",
    "db.coll.update({\"_id\": 1}, {$rename: {\"year\": \"date\"} })\n",
    "db.coll.update({\"_id\": 1}, {$inc: {\"year\": 5}})\n",
    "db.coll.update({\"_id\": 1}, {$mul: {price: NumberDecimal(\"1.25\"), qty: 2}})\n",
    "db.coll.update({\"_id\": 1}, {$min: {\"imdb\": 5}})\n",
    "db.coll.update({\"_id\": 1}, {$max: {\"imdb\": 8}})\n",
    "db.coll.update({\"_id\": 1}, {$currentDate: {\"lastModified\": true}})\n",
    "db.coll.update({\"_id\": 1}, {$currentDate: {\"lastModified\": {$type: \"timestamp\"}}})\n",
    "```\n",
    "\n",
    "#### Array  \n",
    "\n",
    "```\n",
    "db.coll.update({\"_id\": 1}, {$push :{\"array\": 1}})\n",
    "db.coll.update({\"_id\": 1}, {$pull :{\"array\": 1}})\n",
    "db.coll.update({\"_id\": 1}, {$addToSet :{\"array\": 2}})\n",
    "db.coll.update({\"_id\": 1}, {$pop: {\"array\": 1}})  // last element\n",
    "db.coll.update({\"_id\": 1}, {$pop: {\"array\": -1}}) // first element\n",
    "db.coll.update({\"_id\": 1}, {$pullAll: {\"array\" :[3, 4, 5]}})\n",
    "db.coll.update({\"_id\": 1}, {$push: {scores: {$each: [90, 92, 85]}}})\n",
    "db.coll.updateOne({\"_id\": 1, \"grades\": 80}, {$set: {\"grades.$\": 82}})\n",
    "db.coll.updateMany({}, {$inc: {\"grades.$[]\": 10}})\n",
    "db.coll.update({}, {$set: {\"grades.$[element]\": 100}}, {multi: true, arrayFilters: [{\"element\": {$gte: 100}}]})\n",
    "```  \n",
    "\n",
    "#### Update many\n",
    "\n",
    "```\n",
    "db.coll.update({\"year\": 1999}, {$set: {\"decade\": \"90's\"}}, {\"multi\":true})\n",
    "db.coll.updateMany({\"year\": 1999}, {$set: {\"decade\": \"90's\"}})\n",
    "```\n",
    "\n",
    "#### FindOneAndUpdate\n",
    "\n",
    "```\n",
    "db.coll.findOneAndUpdate({\"name\": \"Max\"}, {$inc: {\"points\": 5}}, {returnNewDocument: true})\n",
    "```\n",
    "#### Upsert\n",
    "\n",
    "```\n",
    "db.coll.update({\"_id\": 1}, {$set: {item: \"apple\"}, $setOnInsert: {defaultQty: 100}}, {upsert: true})\n",
    "```\n",
    "\n",
    "#### Replace\n",
    "\n",
    "```\n",
    "db.coll.replaceOne({\"name\": \"Max\"}, {\"firstname\": \"Maxime\", \"surname\": \"Beugnet\"})\n",
    "```\n",
    "\n",
    "#### Save\n",
    "\n",
    "```\n",
    "db.coll.save({\"item\": \"book\", \"qty\": 40})\n",
    "```\n",
    "\n",
    "#### Write concern\n",
    "\n",
    "```\n",
    "db.coll.update({}, {$set: {\"x\": 1}}, {\"writeConcern\": {\"w\": \"majority\", \"wtimeout\": 5000}})\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Delete  \n",
    "\n",
    "```\n",
    "db.coll.remove({name: \"Max\"})\n",
    "db.coll.remove({name: \"Max\"}, {justOne: true})\n",
    "db.coll.remove({}) // WARNING! Deletes all the docs but not the collection itself and its index definitions\n",
    "db.coll.remove({name: \"Max\"}, {\"writeConcern\": {\"w\": \"majority\", \"wtimeout\": 5000}})\n",
    "db.coll.findOneAndDelete({\"name\": \"Max\"})\n",
    "```\n",
    "\n",
    "### Databases and Collections  \n",
    "#### Drop  \n",
    "\n",
    "```\n",
    "db.coll.drop()    // removes the collection and its index definitions\n",
    "db.dropDatabase() // double check that you are *NOT* on the PROD cluster... :-)\n",
    "```\n",
    "\n",
    "#### Create Collection  \n",
    "\n",
    "```\n",
    "// Create collection with a $jsonschema\n",
    "db.createCollection(\"contacts\", {\n",
    "   validator: {$jsonSchema: {\n",
    "      bsonType: \"object\",\n",
    "      required: [\"phone\"],\n",
    "      properties: {\n",
    "         phone: {\n",
    "            bsonType: \"string\",\n",
    "            description: \"must be a string and is required\"\n",
    "         },\n",
    "         email: {\n",
    "            bsonType: \"string\",\n",
    "            pattern: \"@mongodb\\.com$\",\n",
    "            description: \"must be a string and match the regular expression pattern\"\n",
    "         },\n",
    "         status: {\n",
    "            enum: [ \"Unknown\", \"Incomplete\" ],\n",
    "            description: \"can only be one of the enum values\"\n",
    "         }\n",
    "      }\n",
    "   }}\n",
    "})  \n",
    "```\n",
    "\n",
    "#### Other Collection Functions  \n",
    "\n",
    "```  \n",
    "db.coll.stats()\n",
    "db.coll.storageSize()\n",
    "db.coll.totalIndexSize()\n",
    "db.coll.totalSize()\n",
    "db.coll.validate({full: true})\n",
    "db.coll.renameCollection(\"new_coll\", true) // 2nd parameter to drop the target collection if exists\n",
    "```  \n",
    "\n",
    "#### Indexes  \n",
    "#### List Indexes  \n",
    "\n",
    "```  \n",
    "db.coll.getIndexes()\n",
    "db.coll.getIndexKeys()\n",
    "```  \n",
    "\n",
    "#### Create Indexes  \n",
    "\n",
    "```  \n",
    "// Index Types\n",
    "db.coll.createIndex({\"name\": 1})                // single field index\n",
    "db.coll.createIndex({\"name\": 1, \"date\": 1})     // compound index\n",
    "db.coll.createIndex({foo: \"text\", bar: \"text\"}) // text index\n",
    "db.coll.createIndex({\"$**\": \"text\"})            // wildcard text index\n",
    "db.coll.createIndex({\"userMetadata.$**\": 1})    // wildcard index\n",
    "db.coll.createIndex({\"loc\": \"2d\"})              // 2d index\n",
    "db.coll.createIndex({\"loc\": \"2dsphere\"})        // 2dsphere index\n",
    "db.coll.createIndex({\"_id\": \"hashed\"})          // hashed index\n",
    "\n",
    "// Index Options\n",
    "db.coll.createIndex({\"lastModifiedDate\": 1}, {expireAfterSeconds: 3600})      // TTL index\n",
    "db.coll.createIndex({\"name\": 1}, {unique: true})\n",
    "db.coll.createIndex({\"name\": 1}, {partialFilterExpression: {age: {$gt: 18}}}) // partial index\n",
    "db.coll.createIndex({\"name\": 1}, {collation: {locale: 'en', strength: 1}})    // case insensitive index with strength = 1 or 2\n",
    "db.coll.createIndex({\"name\": 1 }, {sparse: true})\n",
    "```\n",
    "\n",
    "#### Drop Indexes  \n",
    "\n",
    "```\n",
    "db.coll.dropIndex(\"name_1\")  \n",
    "```\n",
    "\n",
    "#### Hide/Unhide Indexes  \n",
    "\n",
    "```\n",
    "db.coll.hideIndex(\"name_1\")\n",
    "db.coll.unhideIndex(\"name_1\")\n",
    "```\n",
    "\n",
    "#### Handy commands  \n",
    "\n",
    "```\n",
    "use admin\n",
    "db.createUser({\"user\": \"root\", \"pwd\": passwordPrompt(), \"roles\": [\"root\"]})\n",
    "db.dropUser(\"root\")\n",
    "db.auth( \"user\", passwordPrompt() )\n",
    "\n",
    "use test\n",
    "db.getSiblingDB(\"dbname\")\n",
    "db.currentOp()\n",
    "db.killOp(123) // opid\n",
    "\n",
    "db.fsyncLock()\n",
    "db.fsyncUnlock()\n",
    "\n",
    "db.getCollectionNames()\n",
    "db.getCollectionInfos()\n",
    "db.printCollectionStats()\n",
    "db.stats()\n",
    "\n",
    "db.getReplicationInfo()\n",
    "db.printReplicationInfo()\n",
    "db.isMaster()\n",
    "db.hostInfo()\n",
    "db.printShardingStatus()\n",
    "db.shutdownServer()\n",
    "db.serverStatus()\n",
    "\n",
    "db.setSlaveOk()\n",
    "db.getSlaveOk()\n",
    "\n",
    "db.getProfilingLevel()\n",
    "db.getProfilingStatus()\n",
    "db.setProfilingLevel(1, 200) // 0 == OFF, 1 == ON with slowms, 2 == ON\n",
    "\n",
    "db.enableFreeMonitoring()\n",
    "db.disableFreeMonitoring()\n",
    "db.getFreeMonitoringStatus()\n",
    "\n",
    "db.createView(\"viewName\", \"sourceColl\", [{$project:{department: 1}}])\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using Mongo via Python API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#!pip install -U -q pymongo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import datetime\n",
    "import pprint\n",
    "\n",
    "import pymongo\n",
    "from pymongo import MongoClient"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Making a Connection with MongoClient\n",
    "\n",
    "The first step when working with PyMongo is to create a MongoClient to the running mongod instance. Doing so is easy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "client = MongoClient('localhost', 27017)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['admin', 'config', 'local']"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "client.list_database_names()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Getting/Creating a Database\n",
    "\n",
    "A single instance of MongoDB can support multiple independent databases. When working with PyMongo you access databases using attribute style access on MongoClient instances:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "#dir(client) #Checking the methds for \"client\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "db = client.test_database\n",
    "#db = client['test_database']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Check if Database Exists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['admin', 'config', 'local']\n"
     ]
    }
   ],
   "source": [
    "# The database only appears after we insert one collection/document\n",
    "\n",
    "print(client.list_database_names())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Getting a Collection\n",
    "\n",
    "A collection is a group of documents stored in MongoDB, and can be thought of as roughly the equivalent of a table in a relational database. Getting a collection in PyMongo works the same as getting a database:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "collection = db.test_collection\n",
    "#collection = db['test_collection']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[]\n"
     ]
    }
   ],
   "source": [
    "# The collection only appears after we insert one document\n",
    "\n",
    "print(db.list_collection_names())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Documents\n",
    "\n",
    "Data in MongoDB is represented (and stored) using JSON-style documents. In PyMongo we use dictionaries to represent documents. As an example, the following dictionary might be used to represent a blog post:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "post = {\"author\": \"Mike\",\n",
    "        \"text\": \"My first blog post!\",\n",
    "        \"tags\": [\"mongodb\", \"python\", \"pymongo\"],\n",
    "        \"date\": datetime.datetime.utcnow()}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(post)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating another Collection and inserting a Document\n",
    "\n",
    "To insert a document into a collection we can use the insert_one() method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ObjectId('61680dde6f22332ce261ada4')"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "posts = db.posts\n",
    "post_id = posts.insert_one(post).inserted_id\n",
    "post_id"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When a document is inserted a special key, \"_id\", is automatically added if the document doesn’t already contain an \"_id\" key. The value of \"_id\" must be unique across the collection. insert_one() returns an instance of InsertOneResult. For more information on \"_id\", see the documentation on _id.\n",
    "\n",
    "After inserting the first document, the posts collection has actually been created on the server. We can verify this by listing all of the collections in our database:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['admin', 'config', 'local', 'test_database']\n",
      "['posts']\n"
     ]
    }
   ],
   "source": [
    "print(client.list_database_names())\n",
    "print(db.list_collection_names())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Getting a Single Document With find_one()\n",
    "\n",
    "The most basic type of query that can be performed in MongoDB is find_one(). This method returns a single document matching a query (or None if there are no matches). It is useful when you know there is only one matching document, or are only interested in the first match. Here we use find_one() to get the first document from the posts collection:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'_id': ObjectId('61680dde6f22332ce261ada4'),\n",
       " 'author': 'Mike',\n",
       " 'text': 'My first blog post!',\n",
       " 'tags': ['mongodb', 'python', 'pymongo'],\n",
       " 'date': datetime.datetime(2021, 10, 14, 10, 59, 18, 843000)}"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "posts.find_one()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'dict'>\n"
     ]
    }
   ],
   "source": [
    "result1 = posts.find_one()\n",
    "print(type(result1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The result is a dictionary matching the one that we inserted previously.\n",
    "\n",
    "Note: The returned document contains an \"_id\", which was automatically added on insert.\n",
    "\n",
    "find_one() also supports querying on specific elements that the resulting document must match. To limit our results to a document with author “Mike” we do:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'_id': ObjectId('61680dde6f22332ce261ada4'),\n",
       " 'author': 'Mike',\n",
       " 'text': 'My first blog post!',\n",
       " 'tags': ['mongodb', 'python', 'pymongo'],\n",
       " 'date': datetime.datetime(2021, 10, 14, 10, 59, 18, 843000)}"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "posts.find_one({\"author\": \"Mike\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'_id': ObjectId('61680dde6f22332ce261ada4'),\n",
       " 'author': 'Mike',\n",
       " 'text': 'My first blog post!',\n",
       " 'tags': ['mongodb', 'python', 'pymongo'],\n",
       " 'date': datetime.datetime(2021, 10, 14, 10, 59, 18, 843000)}"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "posts.find_one({\"text\": \"My first blog post!\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'_id': ObjectId('61680dde6f22332ce261ada4'),\n",
       " 'author': 'Mike',\n",
       " 'text': 'My first blog post!',\n",
       " 'tags': ['mongodb', 'python', 'pymongo'],\n",
       " 'date': datetime.datetime(2021, 10, 14, 10, 59, 18, 843000)}"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "posts.find_one({\"tags\": \"python\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "posts.find_one({'date': datetime.datetime(2021, 2, 13, 12, 37, 2, 805000)})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'_id': ObjectId('61680dde6f22332ce261ada4'),\n",
       " 'author': 'Mike',\n",
       " 'text': 'My first blog post!',\n",
       " 'tags': ['mongodb', 'python', 'pymongo'],\n",
       " 'date': datetime.datetime(2021, 10, 14, 10, 59, 18, 843000)}"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "posts.find_one({'date': datetime.datetime(2021, 10, 14, 10, 59, 18, 843000)})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "posts.find_one({\"author\": \"Eliot\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "posts.find_one({\"tags\": \"R\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Querying By ObjectId\n",
    "\n",
    "We can also find a post by its _id, which in our example is an ObjectId:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ObjectId('61680dde6f22332ce261ada4')"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "post_id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'_id': ObjectId('61680dde6f22332ce261ada4'),\n",
      " 'author': 'Mike',\n",
      " 'date': datetime.datetime(2021, 10, 14, 10, 59, 18, 843000),\n",
      " 'tags': ['mongodb', 'python', 'pymongo'],\n",
      " 'text': 'My first blog post!'}\n"
     ]
    }
   ],
   "source": [
    "pprint.pprint(posts.find_one({\"_id\": post_id}))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that an ObjectId is not the same as its string representation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'61680dde6f22332ce261ada4'"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "str(post_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "posts.find_one({\"_id\": str(post_id)})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "bson.objectid.ObjectId"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(post_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A common task in web applications is to get an ObjectId from the request URL and find the matching document. It’s necessary in this case to convert the ObjectId from a string before passing it to find_one:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "from bson.objectid import ObjectId\n",
    "\n",
    "# The web framework gets post_id from the URL and passes it as a string\n",
    "def get(post_id):\n",
    "    # Convert from string to ObjectId:\n",
    "    document = client.db.collection.find_one({'_id': ObjectId(post_id)})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bulk Inserts\n",
    "\n",
    "In order to make [querying](https://docs.mongodb.com/manual/reference/operator/query/) a little more interesting, let’s insert a few more documents. In addition to inserting a single document, we can also perform bulk insert operations, by passing a list as the first argument to insert_many(). This will insert each document in the list, sending only a single command to the server:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "new_posts = [{\"author\": \"Mike\",\n",
    "              \"text\": \"Another post!\",\n",
    "              \"tags\": [\"bulk\", \"insert\"],\n",
    "              \"date\": datetime.datetime(2009, 11, 12, 11, 14)},\n",
    "             \n",
    "             {\"author\": \"Eliot\",\n",
    "              \"title\": \"MongoDB is fun\",\n",
    "              \"text\": \"and pretty easy too!\",\n",
    "              \"date\": datetime.datetime(2009, 11, 10, 10, 45)}]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "list"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(new_posts)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[ObjectId('6168105e6f22332ce261ada5'), ObjectId('6168105e6f22332ce261ada6')]"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result = posts.insert_many(new_posts)\n",
    "result.inserted_ids"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are a couple of interesting things to note about this example:\n",
    "\n",
    "+ The result from insert_many() now returns two ObjectId instances, one for each inserted document.\n",
    "+ new_posts[1] has a different “shape” than the other posts - there is no \"tags\" field and we’ve added a new field, \"title\". This is what we mean when we say that MongoDB is __schema-free__.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Querying for More Than One Document\n",
    "\n",
    "To get more than a single document as the result of a query we use the find() method. find() returns a Cursor instance, which allows us to iterate over all matching documents. For example, we can iterate over every document in the posts collection:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'_id': ObjectId('61680dde6f22332ce261ada4'),\n",
      " 'author': 'Mike',\n",
      " 'date': datetime.datetime(2021, 10, 14, 10, 59, 18, 843000),\n",
      " 'tags': ['mongodb', 'python', 'pymongo'],\n",
      " 'text': 'My first blog post!'}\n",
      "{'_id': ObjectId('6168105e6f22332ce261ada5'),\n",
      " 'author': 'Mike',\n",
      " 'date': datetime.datetime(2009, 11, 12, 11, 14),\n",
      " 'tags': ['bulk', 'insert'],\n",
      " 'text': 'Another post!'}\n",
      "{'_id': ObjectId('6168105e6f22332ce261ada6'),\n",
      " 'author': 'Eliot',\n",
      " 'date': datetime.datetime(2009, 11, 10, 10, 45),\n",
      " 'text': 'and pretty easy too!',\n",
      " 'title': 'MongoDB is fun'}\n"
     ]
    }
   ],
   "source": [
    "for post in posts.find():\n",
    "    pprint.pprint(post)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Just like we did with find_one(), we can pass a document to find() to limit the returned results. Here, we get only those documents whose author is “Mike”:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'_id': ObjectId('61680dde6f22332ce261ada4'),\n",
      " 'author': 'Mike',\n",
      " 'date': datetime.datetime(2021, 10, 14, 10, 59, 18, 843000),\n",
      " 'tags': ['mongodb', 'python', 'pymongo'],\n",
      " 'text': 'My first blog post!'}\n",
      "{'_id': ObjectId('6168105e6f22332ce261ada5'),\n",
      " 'author': 'Mike',\n",
      " 'date': datetime.datetime(2009, 11, 12, 11, 14),\n",
      " 'tags': ['bulk', 'insert'],\n",
      " 'text': 'Another post!'}\n"
     ]
    }
   ],
   "source": [
    "for post in posts.find({\"author\": \"Mike\"}):\n",
    "    pprint.pprint(post)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [Advanced queries](https://docs.mongodb.com/manual/reference/operator/query/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Counting\n",
    "\n",
    "If we just want to know how many documents match a query we can perform a count_documents() operation instead of a full query. We can get a count of all of the documents in a collection:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "posts.count_documents({})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "posts.count_documents({\"author\": \"Mike\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "posts.count_documents({\"author\": \"Mike\", 'tags':'bulk'})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Range Queries\n",
    "\n",
    "MongoDB supports many different types of advanced queries.  \n",
    "As an example, lets perform a query where we limit results to posts older than a certain date, but also sort the results by author:  \n",
    "Here we use the special \"$lt\" operator to do a range query, and also call sort() to sort the results by author."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "<table class=\"math-table\">\n",
       "<thead>\n",
       "<tr><th>Operation</th><th>Syntax</th><th>Description</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr><td>Equality</td><td>{“key” : “value”}</td><td>Matches values that are equal to a specified value.</td></tr>\n",
       "<tr><td>Less Than</td><td>{“key” :{$lt:”value”}}</td><td>Matches values that are less than a specified value.</td></tr>\n",
       "<tr><td>Greater Than</td><td>{“key” :{$gt:”value”}}</td><td>Matches values that are greater than a specified value.</td></tr>\n",
       "<tr><td>Less Than Equal to</td><td>{“key” :{$lte:”value”}}</td><td>Matches values that are less than or equal to a specified value.</td></tr>\n",
       "<tr><td>Greater Than Equal to</td><td>{“key” :{$gte:”value”}}</td><td>Matches values that are greater than or equal to a specified value.</td></tr>\n",
       "<tr><td>Not Equal to</td><td>{“key”:{$ne: “value”}}</td><td>Matches all values that are not equal to a specified value.</td></tr>\n",
       "<tr><td>Logical AND</td><td>{ “$and”:[{exp1}, {exp2}, …, {expN}] }</td><td>Joins query clauses with a logical AND returns all documents that match the conditions of both clauses.</td></tr>\n",
       "<tr><td>Logical OR</td><td>{ “$or”:[{exp1}, {&lt;exp2}, …, {expN}] }</td><td>Joins query clauses with a logical OR returns all documents that match the conditions of either clause.</td></tr>\n",
       "<tr><td>Logical NOT</td><td>{ “$not”:[{exp1}, {exp2}, …, {expN}] }</td><td>Inverts the effect of a query expression and returns documents that do not match the query expression.</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from IPython.core.display import display, HTML\n",
    "display(HTML('''\n",
    "<table class=\"math-table\">\n",
    "<thead>\n",
    "<tr><th>Operation</th><th>Syntax</th><th>Description</th></tr>\n",
    "</thead>\n",
    "<tbody>\n",
    "<tr><td>Equality</td><td>{“key” : “value”}</td><td>Matches values that are equal to a specified value.</td></tr>\n",
    "<tr><td>Less Than</td><td>{“key” :{$lt:”value”}}</td><td>Matches values that are less than a specified value.</td></tr>\n",
    "<tr><td>Greater Than</td><td>{“key” :{$gt:”value”}}</td><td>Matches values that are greater than a specified value.</td></tr>\n",
    "<tr><td>Less Than Equal to</td><td>{“key” :{$lte:”value”}}</td><td>Matches values that are less than or equal to a specified value.</td></tr>\n",
    "<tr><td>Greater Than Equal to</td><td>{“key” :{$gte:”value”}}</td><td>Matches values that are greater than or equal to a specified value.</td></tr>\n",
    "<tr><td>Not Equal to</td><td>{“key”:{$ne: “value”}}</td><td>Matches all values that are not equal to a specified value.</td></tr>\n",
    "<tr><td>Logical AND</td><td>{ “$and”:[{exp1}, {exp2}, …, {expN}] }</td><td>Joins query clauses with a logical AND returns all documents that match the conditions of both clauses.</td></tr>\n",
    "<tr><td>Logical OR</td><td>{ “$or”:[{exp1}, {&lt;exp2}, …, {expN}] }</td><td>Joins query clauses with a logical OR returns all documents that match the conditions of either clause.</td></tr>\n",
    "<tr><td>Logical NOT</td><td>{ “$not”:[{exp1}, {exp2}, …, {expN}] }</td><td>Inverts the effect of a query expression and returns documents that do not match the query expression.</td></tr>\n",
    "</tbody>\n",
    "</table>\n",
    "'''))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'_id': ObjectId('6168105e6f22332ce261ada6'),\n",
      " 'author': 'Eliot',\n",
      " 'date': datetime.datetime(2009, 11, 10, 10, 45),\n",
      " 'text': 'and pretty easy too!',\n",
      " 'title': 'MongoDB is fun'}\n",
      "{'_id': ObjectId('6168105e6f22332ce261ada5'),\n",
      " 'author': 'Mike',\n",
      " 'date': datetime.datetime(2009, 11, 12, 11, 14),\n",
      " 'tags': ['bulk', 'insert'],\n",
      " 'text': 'Another post!'}\n"
     ]
    }
   ],
   "source": [
    "d = datetime.datetime(2009, 11, 12, 12)\n",
    "for post in posts.find({\"date\": {\"$lt\": d}}).sort(\"author\"):\n",
    "    pprint.pprint(post)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Indexing\n",
    "\n",
    "Adding indexes can help accelerate certain queries and can also add additional functionality to querying and storing documents.  \n",
    "In this example, we’ll demonstrate how to create a unique index on a key that rejects documents whose value for that key already exists in the index.\n",
    "\n",
    "First, we’ll need to create the index:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['_id_', 'user_id_1']"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result = db.profiles.create_index([('user_id', pymongo.ASCENDING)], unique=True)\n",
    "sorted(list(db.profiles.index_information()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that we have two indexes now: one is the index on _id that MongoDB creates automatically, and the other is the index on user_id we just created.\n",
    "\n",
    "Now let’s set up some user profiles:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "user_profiles = [{'user_id': 211, 'name': 'Luke'},\n",
    "                 {'user_id': 212, 'name': 'Ziltoid'}]\n",
    "\n",
    "result = db.profiles.insert_many(user_profiles)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The index prevents us from inserting a document whose user_id is already in the collection:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "new_profile = {'user_id': 213, 'name': 'Drew'}\n",
    "duplicate_profile = {'user_id': 212, 'name': 'Tommy'}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "result = db.profiles.insert_one(new_profile)  # This is fine."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "ename": "DuplicateKeyError",
     "evalue": "E11000 duplicate key error collection: test_database.profiles index: user_id_1 dup key: { user_id: 212 }, full error: {'index': 0, 'code': 11000, 'keyPattern': {'user_id': 1}, 'keyValue': {'user_id': 212}, 'errmsg': 'E11000 duplicate key error collection: test_database.profiles index: user_id_1 dup key: { user_id: 212 }'}",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mDuplicateKeyError\u001b[0m                         Traceback (most recent call last)",
      "\u001b[0;32m/tmp/ipykernel_90727/2588562914.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdb\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprofiles\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minsert_one\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mduplicate_profile\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;31m## error\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m~/environments/default_env/lib/python3.8/site-packages/pymongo/collection.py\u001b[0m in \u001b[0;36minsert_one\u001b[0;34m(self, document, bypass_document_validation, session)\u001b[0m\n\u001b[1;32m    703\u001b[0m         \u001b[0mwrite_concern\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_write_concern_for\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msession\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    704\u001b[0m         return InsertOneResult(\n\u001b[0;32m--> 705\u001b[0;31m             self._insert(document,\n\u001b[0m\u001b[1;32m    706\u001b[0m                          \u001b[0mwrite_concern\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mwrite_concern\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    707\u001b[0m                          \u001b[0mbypass_doc_val\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mbypass_document_validation\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/environments/default_env/lib/python3.8/site-packages/pymongo/collection.py\u001b[0m in \u001b[0;36m_insert\u001b[0;34m(self, docs, ordered, check_keys, manipulate, write_concern, op_id, bypass_doc_val, session)\u001b[0m\n\u001b[1;32m    618\u001b[0m         \u001b[0;34m\"\"\"Internal insert helper.\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    619\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdocs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mabc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mMapping\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 620\u001b[0;31m             return self._insert_one(\n\u001b[0m\u001b[1;32m    621\u001b[0m                 \u001b[0mdocs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mordered\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcheck_keys\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmanipulate\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mwrite_concern\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mop_id\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    622\u001b[0m                 bypass_doc_val, session)\n",
      "\u001b[0;32m~/environments/default_env/lib/python3.8/site-packages/pymongo/collection.py\u001b[0m in \u001b[0;36m_insert_one\u001b[0;34m(self, doc, ordered, check_keys, manipulate, write_concern, op_id, bypass_doc_val, session)\u001b[0m\n\u001b[1;32m    607\u001b[0m             \u001b[0m_check_write_command_response\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mresult\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    608\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 609\u001b[0;31m         self.__database.client._retryable_write(\n\u001b[0m\u001b[1;32m    610\u001b[0m             acknowledged, _insert_command, session)\n\u001b[1;32m    611\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/environments/default_env/lib/python3.8/site-packages/pymongo/mongo_client.py\u001b[0m in \u001b[0;36m_retryable_write\u001b[0;34m(self, retryable, func, session)\u001b[0m\n\u001b[1;32m   1550\u001b[0m         \u001b[0;34m\"\"\"Internal retryable write helper.\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1551\u001b[0m         \u001b[0;32mwith\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_tmp_session\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msession\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0ms\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1552\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_retry_with_session\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mretryable\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfunc\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0ms\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1553\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1554\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_handle_getlasterror\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maddress\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0merror_msg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/environments/default_env/lib/python3.8/site-packages/pymongo/mongo_client.py\u001b[0m in \u001b[0;36m_retry_with_session\u001b[0;34m(self, retryable, func, session, bulk)\u001b[0m\n\u001b[1;32m   1436\u001b[0m         retryable = (retryable and self.retry_writes\n\u001b[1;32m   1437\u001b[0m                      and session and not session.in_transaction)\n\u001b[0;32m-> 1438\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_retry_internal\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mretryable\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfunc\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msession\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbulk\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1439\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1440\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_retry_internal\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mretryable\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfunc\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msession\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbulk\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/environments/default_env/lib/python3.8/site-packages/pymongo/mongo_client.py\u001b[0m in \u001b[0;36m_retry_internal\u001b[0;34m(self, retryable, func, session, bulk)\u001b[0m\n\u001b[1;32m   1468\u001b[0m                             \u001b[0;32mraise\u001b[0m \u001b[0mlast_error\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1469\u001b[0m                         \u001b[0mretryable\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1470\u001b[0;31m                     \u001b[0;32mreturn\u001b[0m \u001b[0mfunc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msession\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msock_info\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mretryable\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1471\u001b[0m             \u001b[0;32mexcept\u001b[0m \u001b[0mServerSelectionTimeoutError\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1472\u001b[0m                 \u001b[0;32mif\u001b[0m \u001b[0mis_retrying\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/environments/default_env/lib/python3.8/site-packages/pymongo/collection.py\u001b[0m in \u001b[0;36m_insert_command\u001b[0;34m(session, sock_info, retryable_write)\u001b[0m\n\u001b[1;32m    605\u001b[0m                 retryable_write=retryable_write)\n\u001b[1;32m    606\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 607\u001b[0;31m             \u001b[0m_check_write_command_response\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mresult\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    608\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    609\u001b[0m         self.__database.client._retryable_write(\n",
      "\u001b[0;32m~/environments/default_env/lib/python3.8/site-packages/pymongo/helpers.py\u001b[0m in \u001b[0;36m_check_write_command_response\u001b[0;34m(result)\u001b[0m\n\u001b[1;32m    227\u001b[0m     \u001b[0mwrite_errors\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mresult\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"writeErrors\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    228\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mwrite_errors\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 229\u001b[0;31m         \u001b[0m_raise_last_write_error\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mwrite_errors\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    230\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    231\u001b[0m     \u001b[0merror\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mresult\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"writeConcernError\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/environments/default_env/lib/python3.8/site-packages/pymongo/helpers.py\u001b[0m in \u001b[0;36m_raise_last_write_error\u001b[0;34m(write_errors)\u001b[0m\n\u001b[1;32m    208\u001b[0m     \u001b[0merror\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mwrite_errors\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    209\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0merror\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"code\"\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m11000\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 210\u001b[0;31m         \u001b[0;32mraise\u001b[0m \u001b[0mDuplicateKeyError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0merror\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"errmsg\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m11000\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0merror\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    211\u001b[0m     \u001b[0;32mraise\u001b[0m \u001b[0mWriteError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0merror\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"errmsg\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0merror\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"code\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0merror\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    212\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mDuplicateKeyError\u001b[0m: E11000 duplicate key error collection: test_database.profiles index: user_id_1 dup key: { user_id: 212 }, full error: {'index': 0, 'code': 11000, 'keyPattern': {'user_id': 1}, 'keyValue': {'user_id': 212}, 'errmsg': 'E11000 duplicate key error collection: test_database.profiles index: user_id_1 dup key: { user_id: 212 }'}"
     ]
    }
   ],
   "source": [
    "result = db.profiles.insert_one(duplicate_profile) ## error"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Delete Document / Collection / Database\n",
    "\n",
    "You can delete a document, a collection or a database in MongoDB, by using the methods:\n",
    "+ delete_one()\n",
    "+ delete_many()\n",
    "+ drop() \n",
    "+ drop_database()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Delete Document(s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]\n"
     ]
    }
   ],
   "source": [
    "# db = client.test_database\n",
    "mycol = db[\"customers\"]\n",
    "\n",
    "mylist = [{ \"_id\": 1, \"name\": \"John\", \"address\": \"Highway 37\"},\n",
    "          { \"_id\": 2, \"name\": \"Peter\", \"address\": \"Lowstreet 27\"},\n",
    "          { \"_id\": 3, \"name\": \"Amy\", \"address\": \"Apple st 652\"},\n",
    "          { \"_id\": 4, \"name\": \"Hannah\", \"address\": \"Mountain 21\"},\n",
    "          { \"_id\": 5, \"name\": \"Michael\", \"address\": \"Valley 345\"},\n",
    "          { \"_id\": 6, \"name\": \"Sandy\", \"address\": \"Ocean blvd 2\"},\n",
    "          { \"_id\": 7, \"name\": \"Betty\", \"address\": \"Green Grass 1\"},\n",
    "          { \"_id\": 8, \"name\": \"Richard\", \"address\": \"Sky st 331\"},\n",
    "          { \"_id\": 9, \"name\": \"Susan\", \"address\": \"One way 98\"},\n",
    "          { \"_id\": 10, \"name\": \"Vicky\", \"address\": \"Yellow Garden 2\"},\n",
    "          { \"_id\": 11, \"name\": \"Ben\", \"address\": \"Park Lane 38\"},\n",
    "          { \"_id\": 12, \"name\": \"William\", \"address\": \"Central st 954\"},\n",
    "          { \"_id\": 13, \"name\": \"Chuck\", \"address\": \"Main Road 989\"},\n",
    "          { \"_id\": 14, \"name\": \"Viola\", \"address\": \"Sideway 1633\"}\n",
    "         ]\n",
    "\n",
    "customers = mycol.insert_many(mylist)\n",
    "\n",
    "#print list of the _id values of the inserted documents:\n",
    "print(customers.inserted_ids)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Deleting one"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<pymongo.results.DeleteResult at 0x7f63bbbf9800>"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "myquery = {\"address\": \"Mountain 21\"}\n",
    "mycol.delete_one(myquery)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Deleting many"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2  documents deleted.\n"
     ]
    }
   ],
   "source": [
    "myquery = {\"address\": {\"$regex\": \"^S\"}}\n",
    "deleted = mycol.delete_many(myquery)\n",
    "\n",
    "print(deleted.deleted_count, \" documents deleted.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'n': 2, 'ok': 1.0}"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "deleted.raw_result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'_id': 1, 'name': 'John', 'address': 'Highway 37'}\n",
      "{'_id': 2, 'name': 'Peter', 'address': 'Lowstreet 27'}\n",
      "{'_id': 3, 'name': 'Amy', 'address': 'Apple st 652'}\n",
      "{'_id': 5, 'name': 'Michael', 'address': 'Valley 345'}\n",
      "{'_id': 6, 'name': 'Sandy', 'address': 'Ocean blvd 2'}\n",
      "{'_id': 7, 'name': 'Betty', 'address': 'Green Grass 1'}\n",
      "{'_id': 9, 'name': 'Susan', 'address': 'One way 98'}\n",
      "{'_id': 10, 'name': 'Vicky', 'address': 'Yellow Garden 2'}\n",
      "{'_id': 11, 'name': 'Ben', 'address': 'Park Lane 38'}\n",
      "{'_id': 12, 'name': 'William', 'address': 'Central st 954'}\n",
      "{'_id': 13, 'name': 'Chuck', 'address': 'Main Road 989'}\n"
     ]
    }
   ],
   "source": [
    "for entry in mycol.find():\n",
    "    print(entry)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Deleting all"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [],
   "source": [
    "deleted = mycol.delete_many({})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'n': 11, 'ok': 1.0}"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "deleted.raw_result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Delete Collection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['posts', 'profiles', 'customers']\n"
     ]
    }
   ],
   "source": [
    "print(db.list_collection_names())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [],
   "source": [
    "posts.drop()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['profiles', 'customers']\n"
     ]
    }
   ],
   "source": [
    "print(db.list_collection_names())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Delete Database"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['admin', 'config', 'local', 'test_database']\n"
     ]
    }
   ],
   "source": [
    "print(client.list_database_names())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "client.drop_database('test_database')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['admin', 'config', 'local']\n"
     ]
    }
   ],
   "source": [
    "print(client.list_database_names())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}