Indexes : Indexes, Types of indexes, Index properties, The various indexing strategies to be

considered. Replication and sharding, Multidocument translations, MongoDB security.

MongoDB - Indexing
Indexes support the efficient resolution of queries. Without indexes, MongoDB must scan
every document of a collection to select those documents that match the query statement.
This scan is highly inefficient and require MongoDB to process a large volume of data.
Indexes are special data structures, that store a small portion of the data set in an easy-to-
traverse form. The index stores the value of a specific field or set of fields, ordered by the
value of the field as specified in the index.

Indexes support the efficient execution of queries in MongoDB. Without indexes, MongoDB
must perform a collection scan, i.e. scan every document in a collection, to select those
documents that match the query statement. If an appropriate index exists for a query,
MongoDB can use the index to limit the number of documents it must inspect.

Indexes are special data structures [1] that store a small portion of the collection's data set in
an easy to traverse form. The index stores the value of a specific field or set of fields, ordered
by the value of the field. The ordering of the index entries supports efficient equality matches
and range-based query operations. In addition, MongoDB can return sorted results by using
the ordering in the index.

The concept of indexing is crucial for any database, and so for MongoDB also. Databases
having indexes make queries performance more efficient. When you have a collection with
thousands of documents and no indexing is done, your query will keep on finding certain
documents sequentially. This would require more time to find the documents. But if your
documents have indexes, MongoDB would restrict make it specific the number of documents
to be searched within your collection.

Creating an Index : 
MongoDB provides a method called createIndex() that allows user to create an index.  
Syntax – 
 
db.COLLECTION_NAME.createIndex({KEY:1})
The key determines the field on the basis of which you want to create an index and 1 (or -1)
determines the order in which these indexes will be arranged(ascending or descending).
Example – 
 
db.mycol.createIndex({“age”:1})
{
“createdCollectionAutomatically” : false,
“numIndexesBefore” : 1,
“numIndexesAfter” : 2,
“ok” : 1
}

1
The createIndex() method also has a number of optional parameters. 
These include: 
 
 background (Boolean) 
 
 unique (Boolean) 
 
 name (string) 
 
 sparse (Boolean) 
 
 Drop an Index 
 
In order to drop an index, MongoDB provides the dropIndex() method. 
Syntax – 
 
db.NAME_OF_COLLECTION.dropIndex({KEY:1})
The dropIndex() methods can only delete one index at a time. In order to delete (or
drop) multiple indexes from the collection, MongoDB provides the dropIndexes()
method that takes multiple indexes as its parameters. 
Syntax – 
 
db.NAME_OF_COLLECTION.dropIndexes({KEY1:1, KEY2, 1})
The dropIndex() methods can only delete one index at a time. In order to delete (or
drop) multiple indexes from the collection, MongoDB provides the dropIndexes()
method that takes multiple indexes as its parameters. 
Get description of all indexes : 
The getIndexes() method in MongoDB gives a description of all the indexes that exists in
the given collection. 
Syntax – 
 
db.NAME_OF_COLLECTION.getIndexes()
It will retrieve all the description of the indexes created within the collection.
 
In createIndex() method you can pass multiple fields, to create index on multiple
fields.
>db.mycol.createIndex({"title":1,"description":-1})
>

This method also accepts list of options (which are optional). Following is the list −

Parameter Type Description

Builds the index in the background so that building an index

background Boolean does not block other database activities. Specify true to build in
the background. The default value is false.

2
 Creates a unique index so that the collection will not accept
insertion of documents where the index key or keys match an
unique Boolean
existing value in the index. Specify true to create a unique index.
The default value is false.

The name of the index. If unspecified, MongoDB generates an

name string index name by concatenating the names of the indexed fields
and the sort order.

If true, the index only references documents with the specified

sparse Boolean field. These indexes use less space but behave differently in
some situations (particularly sorts). The default value is false.

Specifies a value, in seconds, as a TTL to control how long

expireAfterSeconds integer
MongoDB retains documents in this collection.

The weight is a number ranging from 1 to 99,999 and denotes

weights document the significance of the field relative to the other indexed fields in
terms of the score.

For a text index, the language that determines the list of stop
default_language string words and the rules for the stemmer and tokenizer. The default
value is English.

For a text index, specify the name of the field in the document
language_override string that contains, the language to override the default language. The
default value is language.

The dropIndex() method

You can drop a particular index using the dropIndex() method of MongoDB.
Syntax
The basic syntax of DropIndex() method is as follows().
>db.COLLECTION_NAME.dropIndex({KEY:1})
Here, "key" is the name of the file on which you want to remove an existing index.
Instead of the index specification document (above syntax), you can also specify
the name of the index directly as:
dropIndex("name_of_the_index")
Example
> db.mycol.dropIndex({"title":1})

3
{
"ok" : 0,
"errmsg" : "can't find index with key: { title: 1.0 }",
"code" : 27,
"codeName" : "IndexNotFound"
}

The dropIndexes() method

This method deletes multiple (specified) indexes on a collection.
Syntax
The basic syntax of DropIndexes() method is as follows() −
>db.COLLECTION_NAME.dropIndexes()
Example
Assume we have created 2 indexes in the named mycol collection as shown below
−
> db.mycol.createIndex({"title":1,"description":-1})

Following example removes the above created indexes of mycol −

>db.mycol.dropIndexes({"title":1,"description":-1})
{ "nIndexesWas" : 2, "ok" : 1 }
>

The getIndexes() method

This method returns the description of all the indexes int the collection.
Syntax
Following is the basic syntax od the getIndexes() method −
db.COLLECTION_NAME.getIndexes()
Example
Assume we have created 2 indexes in the named mycol collection as shown below
−
> db.mycol.createIndex({"title":1,"description":-1})

Following example retrieves all the indexes in the collection mycol −

> db.mycol.getIndexes()
[
{
"v" : 2,
"key" : {
"_id" : 1
},
"name" : "_id_",
"ns" : "test.mycol"
},
{

4
 "v" : 2,
"key" : {
"title" : 1,
"description" : -1
},
"name" : "title_1_description_-1",
"ns" : "test.mycol"
}
]
Types of Indexes in MongoDB
In MongoDB, the indexing is supported for a few types, and every type carries different
purposes. Following are the types of indexes supported:
1. Compound Index: This is a user-defined index supported in MongoDB where multiple
fields can be indexed. To create a compound index, the query is
db.collection.createIndex( { <fieldname1: 1/-1, fieldname2: 1/-1, … }). Here, the compound
index query is similar to a normal index query but with multiple fields.
2. Single Field Index: Simple user-defined index with a single field. This is the normal
indexing as we have seen earlier in the below example.
Query:
db.zips.createIndex({city:1})
Output:

3. MultiKey Indexing: Now this is one indexing that works on values stored in an array.
Here, for each element in an array, MongoDB creates an index.
Query: 
db.test.createIndex( {grade:1} )
Output:
One of the advantages of these multi-key indexing is the efficiency in queries against these
array fields. And this indexing can be implemented on arrays with nested documents and
arrays containing scalar variables.
4. Text Indexing: For the purpose of searching the string content or an array of string, ina
collection, indexing over the test is supported in MongoDB.
5. Hashed Index: Using the hashes of the indexed fields’ values, Hashed Indexing in
MongoDB is achieved. MongoDB also supports sharding based on the hash. The aim is to
reduce the size of the index. Range queries are not supported here.
Query: 
db.collectionname.createIndex( { name: "hashed" } )
Output:
Types of index 
MongoDB provides different types of indexes that are used according to the data type or
queries. The indexes supported by MongoDB is are as follows:
1. Single field Index: A single field index means index on a single field of a document.
This index is helpful for fetching data in ascending as well as descending order.  
Syntax:
db.students.createIndex({“<fieldName>” : <1 or -1>});
Here 1 represents the field is specified in ascending order and -1 for descending order.
Example:
db.students.createIndex({studentsId:1})
In this example we are creating a single index on studentsId field and the field is specified
in ascending order.

5
2. Compound Index: We can combine multiple fields for compound indexing and that will
help for searching or filtering documents in that way. Or in other words, the compound
index is an index where a single index structure holds multiple references.
Syntax:
db.<collection>.createIndex( { <field1>: <type>, <field2>: <type2>, … } )
Here, we can combine the required fields in this pattern. Also the value of these fields is
1(for ascending order) or -1(for descending order).
Note: Compound indexes may have a single hashed index field but a hashing function is
required by Hashed indexes to compute the hash of the value of the index field.
Example:
Here, we create a compound index on studentAge: 1, studentName:1
db.students.createIndex({studentAge: 1, studentName:1})

db.students.find().sort({"studentAge":1,"studentName":1}).pretty()
Here we are taking the sorting functionality based on “studentAge” followed by
“studentName” fields and hence in the below image, though there are 2 documents
matching for “studentAge = 25”, as studentName is an additional value given, as a second
document, studentName with value “Geek40” is displayed and after that only, as a third
document, studentName with value “GeeksForGeeksbest” is displayed. Hence, sometimes

6
there will be a need to create compound indexes when we want to have a closer level of
filtration.

3. Multikey Index: MongoDB uses the multikey indexes to index the values stored in
arrays. When we index a field that holds an array value then MongoDB automatically
creates a separate index of each and every value present in that array. Using these multikey
indexes we can easily find a document that contains an array by matching the items. In
MongoDB, you don’t need to explicitly specify the multikey index because MongoDB
automatically determines whether to create a multikey index if the indexed field contains an
array value. 
Syntax:
db.<collection>.createIndex( { <field>: <type>} )
Here, the value of the field is 1(for ascending order) or -1(for descending order).
Example:
In the students collection, we have three documents that contains array fields.

Now we create a multikey index:

db.students.createIndex({skillsets:1})

7
Now we view the document that holds skillsets:[“Java”, “Android”]
db.students.find({skillsets:["Java", "Android"]}).pretty()

4. Geospatial Indexes: It is an important feature in MongoDB. MongoDB provides two

geospatial indexes known as 2d indexes and 2d sphere indexes using these indexes we can
query geospatial data. Here, the 2d indexes support queries that are used to find data that is
stored in a two-dimensional plane. It only supports data that is stored in legacy coordinate
pairs. Whereas 2d sphere indexes support queries that are used to find the data that is stored
in spherical geometry. It supports data that is stored in legacy coordinate pairs as well as
GeoJSON objects. It also supports queries like queries for inclusion, intersection, and
proximity, etc.
Syntax of 2d sphere indexes:
db.<collection>.createIndex( { <Locationfield>: “2dsphere”} )
Example:
Let us assume the available data for “industries”

Now, let us create a 2d sphere index on the location field:

db.industries.createIndex({location:"2dsphere"})

8
Now, on the execution of the below query, we get
db.industries.find(
{
location:
{$near:
{
$geometry:{type: "Point", coordinates:[-73.9667, 40.78]},
$minDistance:1000,
$maxDistance: 5000
}
}
}
}.pretty()
Here, the “$near” operator returns documents that are in the specified range of at least
1000 meters from and at most 5000 meters from the specified GeoJSON point, and hence
we are getting only Tidal Park output. Similar to $near, it can support for $nearSphere,
$geoWithin, $geoIntersects,$geoNear etc.,

5. Text Index: MongoDB supports query operations that perform a text search of string
content. Text index allows us to find the string content in the specified collection. It can
include any field that contains string content or an array of string items. A collection can
contain at most one text index. You are allowed to use text index in the compound index.  
Syntax:
db.<collection>.createIndex( { <field>: “text”} )
We can give exact phrases also for searching by enclosing the search terms in double
quotes
db.<collectionname>.find( { $text: { $search: “\”<Exact search term>\”” } } )
As here enclosed in double quotes, the search results contain only exact searched data.
9
In case, if we want to exclude a few texts in our search term, then we can do as  
db.<collectionname>.find( { $text: { $search: “<search terms> -<not required search
terms>” } } )
Prepending a – character makes the search text to get ignored and the rest of the text is
considered.
In the text search, the results are available in unsorted order. To make it available in sorted
order of relevance score, $meta textScore field is needed and sort on it. Example:
db.singers.find(
{ $text: { $search: "Annisten" } },
{ score: { $meta: "textScore" } }
).sort( { score: { $meta: "textScore" } } )
Example:
In accessories collection we create text index:
db.accessories.createIndex({name: "text", description: "text"})

Now we display those documents that contain the string “Input”:

db.accessories.find({$text:{$search: "Input"}})

6. Hash Index: To maintain the entries with hashes of the values of the indexed
field(mostly _id field in all collections), we use Hash Index. This kind of index is mainly
required in the even distribution of data via sharding. Hashed keys are helpful to partition
the data across the sharded cluster.
Syntax:
db.<collection>.createIndex( { _id: “hashed” } )
From Version 4.4 onwards, the compound Hashed Index is applicable
7. Wildcard Index: MongoDB supports creating indexes either on a field or set of fields
and if the set of fields are mentioned, it is called as Wildcard Index. Generally, the wildcard
index does not include _id field but if you what to include _id field in the wildcard index
then you have to define it explicitly. MongoDB allows you to create multiple wildcard
indexes in the given collection. Wildcard indexes support queries for unknown or arbitrary
fields.
Syntax:
To create a wild card index on the specified field:
db.<collection>.createIndex( { “field.$**”:1 } )
To create a wild card index on all the field:
db.<collection>.createIndex( { “$**”:1 } )
To create a wild card index on multiple specified fields:
db.<collection>.createIndex(
10
  { “$**”:1 }, 
{“wildcardProjection”:
{“field1”: 1, “field2”:2}
})
Example:
In book collection we create the wildcard index:

Let us create an index for “authorTags” field

db.book.createIndex( { "authorTags.$**" : 1 } )
Since “index” is created on set of fields, we can easily query in the following way
db.book.find( { "authorTags.inclusions" : "RDBMS" } )
db.book.find( { "authorTags.usedin" : "Multipurpose" } )

11
 Index Properties

In addition to the numerous index types MongoDB supports, indexes can also have various
properties. The following documents detail the index properties that you can select when
building an index.
TTL Indexes
The TTL index is used for TTL collections, which expire data after a period of time.
Unique Indexes
A unique index causes MongoDB to reject all documents that contain a duplicate
value for the indexed field.
Partial Indexes
A partial index indexes only documents that meet specified filter criteria.
Case Insensitive Indexes
A case insensitive index disregards the case of the index key values.
Hidden Indexes
A hidden index is not visible to the query planner.
Sparse Indexes
A sparse index does not index documents that do not have the indexed field.

TTL Indexes
NOTE

TTL indexes are special single-field indexes that MongoDB can use to automatically remove
documents from a collection after a certain amount of time or at a specific clock time. Data
expiration is useful for certain types of information like machine generated event data, logs,
and session information that only need to persist in a database for a finite amount of time.
To create a TTL index, use the createIndex() method on a field whose value is either
a date or an array that contains date values, and specify the expireAfterSeconds option with
the desired TTL value in seconds.

For example, to create a TTL index on the lastModifiedDate field of the eventlog collection,

with a TTL value of 3600 seconds, use the following operation in mongosh:
db.eventlog.createIndex( { "lastModifiedDate": 1 }, { expireAfterSeconds: 3600 } )

Behavior

Expiration of Data
TTL indexes expire documents after the specified number of seconds has passed since the
indexed field value; i.e. the expiration threshold is the indexed field value plus the specified
number of seconds.

If the field is an array, and there are multiple date values in the index, MongoDB
uses lowest (i.e. earliest) date value in the array to calculate the expiration threshold.

12
If the indexed field in a document is not a date or an array that holds one or more date values,
the document will not expire.

If a document does not contain the indexed field, the document will not expire.

Delete Operations
A background thread in mongod reads the values in the index and removes
expired documents from the collection.

When the TTL thread is active, you will see delete operations in the output
of db.currentOp() or in the data collected by the database profiler.

Timing of the Delete Operation

MongoDB begins removing expired documents as soon as the index finishes building on
the primary. For more information on the index build process, see Index Builds on Populated
Collections.

The TTL index does not guarantee that expired data will be deleted immediately upon
expiration. There may be a delay between the time a document expires and the time that
MongoDB removes the document from the database.

The background task that removes expired documents runs every 60 seconds. As a result,
documents may remain in a collection during the period between the expiration of the
document and the running of the background task.

Because the duration of the removal operation depends on the workload of

your mongod instance, expired data may exist for some time beyond the 60 second period
between runs of the background task.

Replica Sets
On replica set members, the TTL background thread only deletes documents when a member
is in state primary. The TTL background thread is idle when a member is in
state secondary. Secondary members replicate deletion operations from the primary.

Support for Queries

A TTL index supports queries in the same way non-TTL indexes do.

Restrictions

 TTL indexes are a single-field indexes. Compound indexes do not support TTL and
ignore the expireAfterSeconds option.
 The _id field does not support TTL indexes.
 You cannot create a TTL index on a capped collection because MongoDB cannot
remove documents from a capped collection.
13
  You cannot create a TTL index on a time series collection. Similar functionality is
provided through automatic removal on time series collections instead.
 You cannot use createIndex() to change the value of expireAfterSeconds of an
existing index. Instead use the collMod database command in conjunction with
the index collection flag. Otherwise, to change the value of the option of an existing
index, you must drop the index first and recreate.
 If a non-TTL single-field index already exists for a field, you cannot create a TTL
index on the same field since you cannot create indexes that have the same key
specification and differ only by the options. To change a non-TTL single-field index
to a TTL index, you must drop the index first and recreate with
the expireAfterSeconds option.

Unique Indexes

A unique index ensures that the indexed fields do not store duplicate values; i.e. enforces
uniqueness for the indexed fields. By default, MongoDB creates a unique index on
the _id field during the creation of a collection.
NOTE
New Internal Format
Starting in MongoDB 4.2, for featureCompatibilityVersion (fCV) of 4.2 (or greater),
MongoDB uses a new internal format for unique indexes that is incompatible with earlier
MongoDB versions. The new format applies to both existing unique indexes as well as newly
created/rebuilt unique indexes.

Create a Unique Index

To create a unique index, use the db.collection.createIndex() method with the unique option

set to true.
db.collection.createIndex( <key and index type specification>, { unique: true } )

Unique Index on a Single Field

For example, to create a unique index on the user_id field of the members collection, use the
following operation in mongosh:
db.members.createIndex( { "user_id": 1 }, { unique: true } )

Unique Compound Index

You can also enforce a unique constraint on compound indexes. If you use the unique
constraint on a compound index, then MongoDB will enforce uniqueness on
the combination of the index key values.

For example, to create a unique index on groupNumber, lastname, and firstname fields of

the members collection, use the following operation in mongosh:
db.members.createIndex( { groupNumber: 1, lastname: 1, firstname: 1 }, { unique: true } )

14
The created index enforces uniqueness for the combination of groupNumber, lastname,
and firstname values.

For another example, consider a collection with the following document:

{ _id: 1, a: [ { loc: "A", qty: 5 }, { qty: 10 } ] }

Create a unique compound multikey index on a.loc and a.qty:

db.collection.createIndex( { "a.loc": 1, "a.qty": 1 }, { unique: true } )

The unique index permits the insertion of the following documents into the collection since
the index enforces uniqueness for the combination of a.loc and a.qty values:
db.collection.insertMany( [
{ _id: 2, a: [ { loc: "A" }, { qty: 5 } ] },
{ _id: 3, a: [ { loc: "A", qty: 10 } ] }
])
TIP
See also:

 Unique Constraint Across Separate Documents

 Unique Index and Missing Field

Behavior

Restrictions
MongoDB cannot create a unique index on the specified index field(s) if the collection
already contains data that would violate the unique constraint for the index.

You may not specify a unique constraint on a hashed index.

Building Unique Index on Replica Sets and Sharded Clusters

For replica sets and sharded clusters, using a rolling procedure to create a unique index
requires that you stop all writes to the collection during the procedure. If you cannot stop all
writes to the collection during the procedure, do not use the rolling procedure. Instead, build
your unique index on the collection by:

 issuing db.collection.createIndex() on the primary for a replica set, or

 issuing db.collection.createIndex() on the mongos for a sharded cluster.

Unique Constraint Across Separate Documents

The unique constraint applies to separate documents in the collection. That is, the unique
index prevents separate documents from having the same value for the indexed key.

Because the constraint applies to separate documents, for a unique multikey index, a

document may have array elements that result in repeating index key values as long as the

15
index key values for that document do not duplicate those of another document. In this case,
the repeated index entry is inserted into the index only once.

For example, consider a collection with the following documents:

{ _id: 1, a: [ { loc: "A", qty: 5 }, { qty: 10 } ] }
{ _id: 2, a: [ { loc: "A" }, { qty: 5 } ] }
{ _id: 3, a: [ { loc: "A", qty: 10 } ] }

Create a unique compound multikey index on a.loc and a.qty:

db.collection.createIndex( { "a.loc": 1, "a.qty": 1 }, { unique: true } )

The unique index permits the insertion of the following document into the collection if no
other document in the collection has an index key value of { "a.loc": "B", "a.qty": null }.
db.collection.insertOne( { _id: 4, a: [ { loc: "B" }, { loc: "B" } ] } )

Unique Index and Missing Field

If a document does not have a value for the indexed field in a unique index, the index will
store a null value for this document. Because of the unique constraint, MongoDB will only
permit one document that lacks the indexed field. If there is more than one document without
a value for the indexed field or is missing the indexed field, the index build will fail with a
duplicate key error.

For example, a collection has a unique index on x:

db.collection.createIndex( { "x": 1 }, { unique: true } )

The unique index allows the insertion of a document without the field x if the collection does
not already contain a document missing the field x:
db.collection.insertOne( { y: 1 } )

However, the unique index errors on the insertion of a document without the field x if the
collection already contains a document missing the field x:
db.collection.insertOne( { z: 1 } )

The operation fails to insert the document because of the violation of the unique constraint on
the value of the field x:
WriteResult({
"nInserted" : 0,
"writeError" : {
"code" : 11000,
"errmsg" : "E11000 duplicate key error index: test.collection.$a.b_1 dup key: { : null }"
}
})
TIP
See also:
Unique Partial Indexes

16
Unique Partial Indexes

New in version 3.2.

Partial indexes only index the documents in a collection that meet a specified filter
expression. If you specify both the partialFilterExpression and a unique constraint, the
unique constraint only applies to the documents that meet the filter expression.

A partial index with a unique constraint does not prevent the insertion of documents that do
not meet the unique constraint if the documents do not meet the filter criteria. For an
example, see Partial Index with Unique Constraint.

Sharded Clusters and Unique Indexes

You cannot specify a unique constraint on a hashed index.

For a ranged sharded collection, only the following indexes can be unique:

 the index on the shard key

 a compound index where the shard key is a prefix
 the default _id index; however, the _id index only enforces the uniqueness constraint
per shard if the _id field is not the shard key or the prefix of the shard key.

IMPORTANT

Uniqueness and the _id Index

If the _id field is not the shard key or the prefix of the shard key, _id index only
enforces the uniqueness constraint per shard and not across shards.
For example, consider a sharded collection (with shard key {x: 1}) that spans two
shards A and B. Because the _id key is not part of the shard key, the collection could
have a document with _id value 1 in shard A and another document
with _id value 1 in shard B.
If the _id field is not the shard key nor the prefix of the shard key, MongoDB expects
applications to enforce the uniqueness of the _id values across the shards.

The unique index constraints mean that:

 For a to-be-sharded collection, you cannot shard the collection if the collection has
other unique indexes.
 For an already-sharded collection, you cannot create unique indexes on other fields.

Sparse and Non-Sparse Unique Indexes

Starting in MongoDB 5.0, unique sparse and unique non-sparse indexes with the same key
pattern can exist on a single collection.

17
Unique and Sparse Index Creation
This example creates multiple indexes with the same key pattern and different sparse options:
db.scoreHistory.createIndex( { score : 1 }, { name: "unique_index", unique: true } )
db.scoreHistory.createIndex( { score : 1 }, { name: "unique_sparse_index", unique: true, sparse: true } )

Basic and Sparse Index Creation

You can also create basic indexes with the same key pattern with and without the sparse
option:
db.scoreHistory.createIndex( { score : 1 }, { name: "sparse_index", sparse: true } )
db.scoreHistory.createIndex( { score : 1 }, { name: "basic_index" } )

Basic and Unique Indexes With Duplicate Key Patterns

Starting in MongoDB 5.0, basic and unique indexes can exist with the same key pattern.

This duplication in key patterns allows for adding a unique index to already indexed fields.

In this example:

Create a basic index with the key pattern { score : 1 } and insert three documents.
db.scoreHistory.createIndex( { score : 1 }, { name: "basic_index" } )
db.scoreHistory.insert( { score : 1 } )
db.scoreHistory.insert( { score : 2 } )
db.scoreHistory.insert( { score : 3 } )

Create a unique index with the same key pattern { score : 1 }.

db.scoreHistory.createIndex( { score : 1 }, { name: "unique_index", unique: true } )

Try to insert a duplicate score document that fails because of the unique index.

db.scoreHistory.insert( { score : 3 } )

Partial Indexes

New in version 3.2.

Partial indexes only index the documents in a collection that meet a specified filter
expression. By indexing a subset of the documents in a collection, partial indexes have lower
storage requirements and reduced performance costs for index creation and maintenance.

Create a Partial Index

To create a partial index, use the db.collection.createIndex() method with

the partialFilterExpression option. The partialFilterExpression option accepts a document
that specifies the filter condition using:

18
  equality expressions (i.e. field: value or using the $eq operator),
 $exists: true expression,
 $gt, $gte, $lt, $lte expressions,
 $type expressions,
 $and operator at the top-level only

For example, the following operation creates a compound index that indexes only the
documents with a rating field greater than 5.
db.restaurants.createIndex(
{ cuisine: 1, name: 1 },
{ partialFilterExpression: { rating: { $gt: 5 } } }
)

You can specify a partialFilterExpression option for all MongoDB index types.

Behavior

Query Coverage
MongoDB will not use the partial index for a query or sort operation if using the index results
in an incomplete result set.

To use the partial index, a query must contain the filter expression (or a modified filter
expression that specifies a subset of the filter expression) as part of its query condition.

For example, given the following index:

db.restaurants.createIndex(
{ cuisine: 1 },
{ partialFilterExpression: { rating: { $gt: 5 } } }
)

The following query can use the index since the query predicate includes the
condition rating: { $gte: 8 } that matches a subset of documents matched by the index filter
expression rating: { $gt: 5 }:
db.restaurants.find( { cuisine: "Italian", rating: { $gte: 8 } } )

However, the following query cannot use the partial index on the cuisine field because using
the index results in an incomplete result set. Specifically, the query predicate includes the
condition rating: { $lt: 8 } while the index has the filter rating: { $gt: 5 }. That is, the query {
cuisine: "Italian", rating: { $lt: 8 } } matches more documents (e.g. an Italian restaurant with
a rating equal to 1) than are indexed.
db.restaurants.find( { cuisine: "Italian", rating: { $lt: 8 } } )

Similarly, the following query cannot use the partial index because the query predicate does
not include the filter expression and using the index would return an incomplete result set.
db.restaurants.find( { cuisine: "Italian" } )

19
Comparison with Sparse Indexes
Partial indexes are be preferred over sparse indexes. Partial indexes provide the following
benefits:

 Greater control over which documents are indexed.

 A superset of the functionality offered by sparse indexes.

Sparse indexes select documents to index solely based on the existence of the indexed field,
or for compound indexes, the existence of the indexed fields.

Partial indexes determine the index entries based on the specified filter. The filter can include
fields other than the index keys and can specify conditions other than just an existence check.
For example, a partial index can implement the same behavior as a sparse index:
db.contacts.createIndex(
{ name: 1 },
{ partialFilterExpression: { name: { $exists: true } } }
)

This partial index supports the same queries as a sparse index on the name field.

However, a partial index can also specify filter expressions on fields other than the index key.
For example, the following operation creates a partial index, where the index is on
the name field but the filter expression is on the email field:
db.contacts.createIndex(
{ name: 1 },
{ partialFilterExpression: { email: { $exists: true } } }
)

For the query optimizer to choose this partial index, the query predicate must include a
condition on the name field as well as a non-null match on the email field.

For example, the following query can use the index because it includes both a condition on
the name field and a non-null match on the email field:
db.contacts.find( { name: "xyz", email: { $regex: /\.org$/ } } )

However, the following query cannot use the index because it includes a null match on
the email field, which is not permitted by the filter expression { email: { $exists: true } }:
db.contacts.find( { name: "xyz", email: { $exists: false } } )

Restrictions

Starting in MongoDB 5.0, multiple partial indexes can be created using the same key

pattern as long as the partialFilterExpression fields do not express equivalent filters.

In earlier versions of MongoDB, creating multiple partial indexes is not allowed when using
the same key pattern with different partialFilterExpressions.

20
You cannot specify both the partialFilterExpression option and the sparse option.

_id indexes cannot be partial indexes.

Shard key indexes cannot be partial indexes.

Examples

Create a Partial Index On A Collection

Consider a collection restaurants containing documents that resemble the following
{
"_id" : ObjectId("5641f6a7522545bc535b5dc9"),
"address" : {
"building" : "1007",
"coord" : [
-73.856077,
40.848447
],
"street" : "Morris Park Ave",
"zipcode" : "10462"
},
"borough" : "Bronx",
"cuisine" : "Bakery",
"rating" : { "date" : ISODate("2014-03-03T00:00:00Z"),
"grade" : "A",
"score" : 2
},
"name" : "Morris Park Bake Shop",
"restaurant_id" : "30075445"
}

You could add a partial index on the borough and cuisine fields choosing only to index

documents where the rating.grade field is A:
db.restaurants.createIndex(
{ borough: 1, cuisine: 1 },
{ partialFilterExpression: { 'rating.grade': { $eq: "A" } } }
)

Then, the following query on the restaurants collection uses the partial index to return the
restaurants in the Bronx with rating.grade equal to A:
db.restaurants.find( { borough: "Bronx", 'rating.grade': "A" } )

However, the following query cannot use the partial index because the query expression does
not include the rating.grade field:
db.restaurants.find( { borough: "Bronx", cuisine: "Bakery" } )

21
Partial Index with Unique Constraint
Partial indexes only index the documents in a collection that meet a specified filter
expression. If you specify both the partialFilterExpression and a unique constraint, the
unique constraint only applies to the documents that meet the filter expression. A partial
index with a unique constraint does not prevent the insertion of documents that do not meet
the unique constraint if the documents do not meet the filter criteria.

For example, a collection users contains the following documents:

{ "_id" : ObjectId("56424f1efa0358a27fa1f99a"), "username" : "david", "age" : 29 }
{ "_id" : ObjectId("56424f37fa0358a27fa1f99b"), "username" : "amanda", "age" : 35 }
{ "_id" : ObjectId("56424fe2fa0358a27fa1f99c"), "username" : "rajiv", "age" : 57 }

The following operation creates an index that specifies a unique constraint on

the username field and a partial filter expression age: { $gte: 21 }.
db.users.createIndex(
{ username: 1 },
{ unique: true, partialFilterExpression: { age: { $gte: 21 } } }
)

The index prevents the insertion of the following documents since documents already exist
with the specified usernames and the age fields are greater than 21:
db.users.insertMany( [
{ username: "david", age: 27 },
{ username: "amanda", age: 25 },
{ username: "rajiv", age: 32 }
])

However, the following documents with duplicate usernames are allowed since the unique
constraint only applies to documents with age greater than or equal to 21.
db.users.insertMany( [
{ username: "david", age: 20 },
{ username: "amanda" },
{ username: "rajiv", age: null }
])

Case Insensitive Indexes

New in version 3.4.

Case insensitive indexes support queries that perform string comparisons without regard for
case.

You can create a case insensitive index with db.collection.createIndex() by specifying

the collation parameter as an option. For example:
db.collection.createIndex( { "key" : 1 },
{ collation: {
locale : <locale>,
strength : <strength>
22
 }
})

To specify a collation for a case sensitive index, include:

 locale: specifies language rules. See Collation Locales for a list of available locales.

 strength: determines comparison rules. A value of 1 or 2 indicates a case insensitive
collation.

For additional collation fields, see Collation.

Behavior

Using a case insensitive index does not affect the results of a query, but it can increase
performance; see Indexes for a detailed discussion of the costs and benefits of indexes.

To use an index that specifies a collation, query and sort operations must specify the same
collation as the index. If a collection has defined a collation, all queries and indexes inherit
that collation unless they explicitly specify a different collation.

Examples

Create a Case Insensitive Index

To use a case insensitive index on a collection with no default collation, create an index with
a collation and set the strength parameter to 1 or 2 (see Collation for a detailed description of
the strength parameter). You must specify the same collation at the query level in order to
use the index-level collation.

The following example creates a collection with no default collation, then adds an index on
the type field with a case insensitive collation.
db.createCollection("fruit")
db.fruit.createIndex( { type: 1},
{ collation: { locale: 'en', strength: 2 } } )

To use the index, queries must specify the same collation.

db.fruit.insertMany( [
{ type: "apple" },
{ type: "Apple" },
{ type: "APPLE" }
])
db.fruit.find( { type: "apple" } ) // does not use index, finds one result
db.fruit.find( { type: "apple" } ).collation( { locale: 'en', strength: 2 } )
// uses the index, finds three results
db.fruit.find( { type: "apple" } ).collation( { locale: 'en', strength: 1 } )
// does not use the index, finds three results

23
Case Insensitive Indexes on Collections with a Default Collation
When you create a collection with a default collation, all the indexes you create subsequently
inherit that collation unless you specify a different collation. All queries which do not specify
a different collation also inherit the default collation.

The following example creates a collection called names with a default collation, then creates
an index on the first_name field.
db.createCollection("names", { collation: { locale: 'en_US', strength: 2 } } )
db.names.createIndex( { first_name: 1 } ) // inherits the default collation

Insert a small collection of names:

db.names.insertMany( [
{ first_name: "Betsy" },
{ first_name: "BETSY"},
{ first_name: "betsy"}
])

Queries on this collection use the specified collation by default, and if possible use the index
as well.
db.names.find( { first_name: "betsy" } )
// inherits the default collation: { collation: { locale: 'en_US', strength: 2 } }
// finds three results

The above operation uses the collection's default collation and finds all three documents. It
uses the index on the first_name field for better performance.

It is still possible to perform case sensitive searches on this collection by specifying a

different collation in the query:
db.names.find( { first_name: "betsy" } ).collation( { locale: 'en_US' } )
// does not use the collection's default collation, finds one result

The above operation finds only one document, because it uses a collation with
no strength value specified. It does not use the collection's default collation or the index.

Hidden Indexes

New in version 4.4.

Hidden indexes are not visible to the query planner and cannot be used to support a query.

By hiding an index from the planner, users can evaluate the potential impact of dropping an
index without actually dropping the index. If the impact is negative, the user can unhide the
index instead of having to recreate a dropped index.

24
Behavior

Apart from being hidden from the planner, hidden indexes behave like unhidden indexes; i.e.

 If a hidden index is a unique index, the index still applies its unique constraint to the
documents.
 If a hidden index is a TTL index, the index still expires documents.
 Hidden indexes are included in listIndexes and db.collection.getIndexes() results.
 Hidden indexes are updated upon write operations to the collection and continue to
consume disk space and memory. As such, they are included in various statistics
operations, such as db.collection.stats() and $indexStats.
 Hiding an unhidden index or unhiding a hidden index resets its $indexStats. Hiding
an already hidden index or unhiding an already unhidden index does not reset
the $indexStats.

Restrictions

 To hide an index, you must have featureCompatibilityVersion set to 4.4 or greater.

However, once hidden, the index remains hidden even
with featureCompatibilityVersion set to 4.2 on MongoDB 4.4 binaries.
 You cannot hide the _id index.
 You cannot cursor.hint() a hidden index.

Examples

Create a Hidden Index

To create a hidden index, use the db.collection.createIndex() method with the hidden option
set to true.
NOTE
To use the hidden option with db.collection.createIndex(), you must
have featureCompatibilityVersion set to 4.4 or greater. However, once hidden, the index
remains hidden even with featureCompatibilityVersion set to 4.2 on MongoDB 4.4 binaries.

For example, the following operation creates a hidden ascending index on the borough field:
db.addresses.createIndex(
{ borough: 1 },
{ hidden: true }
);

To verify, run db.collection.getIndexes() on the addresses collection:

db.addresses.getIndexes()

The operation returns the following information:

[
{
25
 "v" : 2,
"key" : {
"_id" : 1
},
"name" : "_id_"
},
{
"v" : 2,
"key" : {
"borough" : 1
},
"name" : "borough_1",
"hidden" : true
}
]

The index option hidden is only returned if the value is true.

Hide an Existing Index

NOTE

 To hide an index, you must have featureCompatibilityVersion set to 4.4 or greater.

However, once hidden, the index remains hidden even
with featureCompatibilityVersion set to 4.2 on MongoDB 4.4 binaries.
 You cannot hide the _id index.

To hide an existing index, you can use the collMod command

or mongosh helper db.collection.hideIndex().

For example, create an index without hiding:

db.restaurants.createIndex( { borough: 1, ratings: 1 } );

To hide the index, you can specify either:

 the index key specification document to the db.collection.hideIndex() method:
db.restaurants.hideIndex( { borough: 1, ratings: 1 } ); // Specify the index key specification documen
 the index name to the db.collection.hideIndex() method:
db.restaurants.hideIndex( "borough_1_ratings_1" ); // Specify the index name

To verify, run db.collection.getIndexes() on the restaurants collection:

db.restaurants.getIndexes()

The operation returns the following information:

[
{
"v" : 2,
"key" : {
"_id" : 1
},
26
 "name" : "_id_"
},
{
"v" : 2,
"key" : {
"borough" : 1,
"ratings" : 1
},
"name" : "borough_1_ratings_1",
"hidden" : true
}
]

The index option hidden is only returned if the value is true.

Unhide an Existing Index

To unhide a hidden index, you can use the collMod command
or mongosh helper db.collection.unhideIndex(). You can specify either:
 the index key specification document to the db.collection.unhideIndex() method:
db.restaurants.unhideIndex( { borough: 1, city: 1 } ); // Specify the index key specification documen
 the index name to the db.collection.unhideIndex() method:
db.restaurants.unhideIndex( "borough_1_ratings_1" ); // Specify the index name

To verify, run db.collection.getIndexes() on the restaurants collection:

db.restaurants.getIndexes()

The operation returns the following information:

[
{
"v" : 2,
"key" : {
"_id" : 1
},
"name" : "_id_"
},
{
"v" : 2,
"key" : {
"borough" : 1,
"ratings" : 1
},
"name" : "borough_1_ratings_1"
}
]

The index option hidden no longer appears as part of the borough_1_ratings_1 index since

the field is only returned if the value is true.

27
Because indexes are fully maintained while hidden, the index is immediately available for use
once unhidden.

Sparse Indexes

Sparse indexes only contain entries for documents that have the indexed field, even if the
index field contains a null value. The index skips over any document that is missing the
indexed field. The index is "sparse" because it does not include all documents of a collection.
By contrast, non-sparse indexes contain all documents in a collection, storing null values for
those documents that do not contain the indexed field.
IMPORTANT

Changed in version 3.2: Starting in MongoDB 3.2, MongoDB provides the option to

create partial indexes. Partial indexes offer a superset of the functionality of sparse indexes. If
you are using MongoDB 3.2 or later, partial indexes should be preferred over sparse indexes.

Create a Sparse Index

To create a sparse index, use the db.collection.createIndex() method with the sparse option

set to true. For example, the following operation in mongosh creates a sparse index on
the xmpp_id field of the addresses collection:
db.addresses.createIndex( { "xmpp_id": 1 }, { sparse: true } )

The index does not index documents that do not include the xmpp_id field.
NOTE
Do not confuse sparse indexes in MongoDB with block-level indexes in other databases.
Think of them as dense indexes with a specific filter.

Behavior

sparse Index and Incomplete Results

If a sparse index would result in an incomplete result set for queries and sort operations,
MongoDB will not use that index unless a hint() explicitly specifies the index.

For example, the query { x: { $exists: false } } will not use a sparse index on the x field
unless explicitly hinted. See Sparse Index On A Collection Cannot Return Complete
Results for an example that details the behavior.

Changed in version 3.4.

If you include a hint() that specifies a sparse index when you perform a count() of all
documents in a collection (i.e. with an empty query predicate), the sparse index is used even
if the sparse index results in an incorrect count.
db.collection.insertOne( { _id: 1, y: 1 } );
db.collection.createIndex( { x: 1 }, { sparse: true } );
db.collection.find().hint( { x: 1 } ).count();
28
To obtain the correct count, do not hint() with a sparse index when performing a count of all
documents in a collection.
db.collection.find().count();
db.collection.createIndex( { y: 1 } );
db.collection.find().hint( { y: 1 } ).count();

Indexes that are sparse by Default

2dsphere (version 2), 2d, geoHaystack, and text indexes are always sparse.

sparse Compound Indexes
Sparse compound indexes that only contain ascending/descending index keys will index a
document as long as the document contains at least one of the keys.

For sparse compound indexes that contain a geospatial key (i.e. 2dsphere, 2d,

or geoHaystack index keys) along with ascending/descending index key(s), only the existence
of the geospatial field(s) in a document determine whether the index references the document.

For sparse compound indexes that contain text index keys along with ascending/descending
index keys, only the existence of the text index field(s) determine whether the index
references a document.

sparse and unique Properties
An index that is both sparse and unique prevents collection from having documents with
duplicate values for a field but allows multiple documents that omit the key.

Examples

Create a Sparse Index On A Collection

Consider a collection scores that contains the following documents:
{ "_id" : ObjectId("523b6e32fb408eea0eec2647"), "userid" : "newbie" }
{ "_id" : ObjectId("523b6e61fb408eea0eec2648"), "userid" : "abby", "score" : 82 }
{ "_id" : ObjectId("523b6e6ffb408eea0eec2649"), "userid" : "nina", "score" : 90 }

The collection has a sparse index on the field score:

db.scores.createIndex( { score: 1 } , { sparse: true } )

Then, the following query on the scores collection uses the sparse index to return the
documents that have the score field less than ($lt) 90:
db.scores.find( { score: { $lt: 90 } } )

Because the document for the userid "newbie" does not contain the score field and thus does
not meet the query criteria, the query can use the sparse index to return the results:
{ "_id" : ObjectId("523b6e61fb408eea0eec2648"), "userid" : "abby", "score" : 82 }

29
Sparse Index On A Collection Cannot Return Complete Results
Consider a collection scores that contains the following documents:
{ "_id" : ObjectId("523b6e32fb408eea0eec2647"), "userid" : "newbie" }
{ "_id" : ObjectId("523b6e61fb408eea0eec2648"), "userid" : "abby", "score" : 82 }
{ "_id" : ObjectId("523b6e6ffb408eea0eec2649"), "userid" : "nina", "score" : 90 }

The collection has a sparse index on the field score:

db.scores.createIndex( { score: 1 } , { sparse: true } )

Because the document for the userid "newbie" does not contain the score field, the sparse
index does not contain an entry for that document.

Consider the following query to return all documents in the scores collection, sorted by

the score field:
db.scores.find().sort( { score: -1 } )

Even though the sort is by the indexed field, MongoDB will not select the sparse index to
fulfill the query in order to return complete results:
{ "_id" : ObjectId("523b6e6ffb408eea0eec2649"), "userid" : "nina", "score" : 90 }
{ "_id" : ObjectId("523b6e61fb408eea0eec2648"), "userid" : "abby", "score" : 82 }
{ "_id" : ObjectId("523b6e32fb408eea0eec2647"), "userid" : "newbie" }

To use the sparse index, explicitly specify the index with hint():

db.scores.find().sort( { score: -1 } ).hint( { score: 1 } )

The use of the index results in the return of only those documents with the score field:
{ "_id" : ObjectId("523b6e6ffb408eea0eec2649"), "userid" : "nina", "score" : 90 }
{ "_id" : ObjectId("523b6e61fb408eea0eec2648"), "userid" : "abby", "score" : 82 }
TIP

See also: Sparse Index with Unique Constraint

Consider a collection scores that contains the following documents:
{ "_id" : ObjectId("523b6e32fb408eea0eec2647"), "userid" : "newbie" }
{ "_id" : ObjectId("523b6e61fb408eea0eec2648"), "userid" : "abby", "score" : 82 }
{ "_id" : ObjectId("523b6e6ffb408eea0eec2649"), "userid" : "nina", "score" : 90 }

You could create an index with a unique constraint and sparse filter on the score field using
the following operation:
db.scores.createIndex( { score: 1 } , { sparse: true, unique: true } )

This index would permit the insertion of documents that had unique values for
the score field or did not include a score field. As such, given the existing documents in
the scores collection, the index permits the following insert operations:
db.scores.insertMany( [
{ "userid": "AAAAAAA", "score": 43 },
{ "userid": "BBBBBBB", "score": 34 },
{ "userid": "CCCCCCC" },
{ "userid": "DDDDDDD" }
30
 ])

However, the index would not permit the addition of the following documents since
documents already exists with score value of 82 and 90:
db.scores.insertMany( [
{ "userid": "AAAAAAA", "score": 82 },
{ "userid": "BBBBBBB", "score": 90 }
])

Sparse and Non-Sparse Unique Indexes

Starting in MongoDB 5.0, unique sparse and unique non-sparse indexes with the same key
pattern can exist on a single collection.

Unique and Sparse Index Creation

This example creates multiple indexes with the same key pattern and different sparse options:
db.scoreHistory.createIndex( { score : 1 }, { name: "unique_index", unique: true } )
db.scoreHistory.createIndex( { score : 1 }, { name: "unique_sparse_index", unique: true, sparse: true } )

Basic and Sparse Index Creation

You can also create basic indexes with the same key pattern with and without the sparse
option:
db.scoreHistory.createIndex( { score : 1 }, { name: "sparse_index", sparse: true } )
db.scoreHistory.createIndex( { score : 1 }, { name: "basic_index" } )

31