MongoDB Indexing: A Simple Guide

1. What is MongoDB Indexing?

Definition:
MongoDB indexing is a method of organizing data to make searching faster and more
efficient, similar to how a book’s index helps you find specific information without reading
every page.

Key Questions Addressed:

• How can we speed up database queries?
• How do indexes improve application performance?
• What happens when we don’t use indexes?
• How do we choose which fields to index?

Importance:
Without proper indexing, databases can work for small amounts of data but will become
increasingly slow as data grows, eventually causing performance issues that affect the
entire application.

2. What are Indexes in MongoDB?

Definition:
Indexes in MongoDB are special data structures that store a small portion of the collection’s
data in an easy-to-traverse form, allowing MongoDB to quickly locate documents without
scanning every document in a collection.

What They Define:

• Search Optimization: Which fields MongoDB can quickly search through
• Query Processing: How MongoDB finds and retrieves data
• Sorting Efficiency: How quickly MongoDB can sort results
• Performance Impact: How fast queries are executed
• Space Usage: How much storage space is required

Timing:
Indexes should be created before an application experiences high traffic or stores large
amounts of data, as they form the foundation for efficient data retrieval.

3. Problems with Poor or Missing Indexes

Consequences:
1. Slow Queries – Searches take longer as MongoDB needs to scan every document
 2. High CPU Usage – The database server works harder to find data
3. Poor User Experience – Applications become unresponsive as data grows
4. Scalability Issues – The system struggles to handle more users or data
5. Inefficient Resource Usage – More computing power is needed for simple
operations

4. How to Create Effective Indexes

Process:
1. Identify Common Query Patterns
– Which fields are frequently searched?
– Which fields are used for sorting?
2. Understand Query Operations
– Are queries looking for exact matches or ranges?
– Are queries combining multiple fields?
3. Choose Fields to Index
– Select fields that appear in query conditions
– Prioritize fields with higher selectivity (more unique values)
4. Decide on Index Types
– Single-field indexes for simple queries
– Compound indexes for queries with multiple conditions
– Special indexes for text search or geospatial data
5. Monitor and Optimize
– Use analysis tools to check index usage
– Adjust indexes based on performance metrics

5. Types of MongoDB Indexes

1. Single Field Indexes
• What It Is: An index on one field of a document

• Advantages: Simple to set up, good for queries on a single field

• Disadvantages: Not efficient for complex queries with multiple conditions

• Example Code:

courseSchema.index({ country_id: 1 });

2. Compound Indexes
• What It Is: An index on multiple fields of a document

• Advantages: Efficient for queries that filter on multiple fields, works for sorting

• Disadvantages: Less efficient for queries that don’t use all indexed fields
 • Example Code:

courseSchema.index({ country_id: 1, education_level: 1 });

3. Unique Indexes
• What It Is: An index that ensures no two documents have the same value for a field

• Advantages: Enforces data integrity, offers O(1) lookups for unique values

• Disadvantages: Can’t be used when duplicate values are needed

• Example Code:

courseSchema.index({ courseTitle: 1 }, { unique: true });

4. Text Indexes
• What It Is: A specialized index for text search functionality

• Advantages: Enables full-text search capabilities

• Disadvantages: More resource-intensive, only one text index per collection

• Example Code:

courseSchema.index({ desc: 'text', courseTitle: 'text' });

5. Geospatial Indexes
• What It Is: Indexes for location-based queries

• Advantages: Enables efficient proximity searches

• Disadvantages: Specialized use case, more complex to use

• Example Code:

locationSchema.index({ coordinates: '2dsphere' });

6. Understanding Index Performance

Index Direction (1 vs -1)
• What It Means:

– 1 indicates ascending order (A→Z, small→large)

– -1 indicates descending order (Z→A, large→small)
• When to Use:

– Use 1 for most cases

– Use -1 when you frequently sort in descending order
• Example:
 courseSchema.index({ price: -1 }); // Optimized for sorting high
to low

Time Complexity
• Without Index (O(n)):
– MongoDB must scan every document
– Query time increases linearly with collection size
– Example: 1 million documents = ~17 minutes for a full scan
• With Index (O(log n)):
– MongoDB uses binary search techniques
– Query time increases logarithmically with collection size
– Example: 1 million documents = ~0.03 seconds
• Perfect Index (O(1)):
– Direct lookup with no search required
– Query time is constant regardless of collection size
– Example: Unique index lookup = instant (milliseconds)

7. Measuring Index Performance

Using explain() Method
• What It Does: Shows how MongoDB executes a query

• How to Use:

const stats = await Courses.find({ country_id:

someId }).explain("executionStats");
console.log(stats);

• Key Metrics:

– IXSCAN: Index is being used (good)

– COLLSCAN: Full collection scan (bad)
– executionTimeMillis: Query execution time
– nReturned: Number of documents returned
– totalDocsExamined: Number of documents scanned

Timing Queries in Express

• Using Date.now():

const start = Date.now();

const courses = await Courses.find({ country_id: someId });
const end = Date.now();
console.log(`Query took ${end - start} ms`);

• Using performance.now() (more precise):

 const { performance } = require('perf_hooks');
const start = performance.now();
const courses = await Courses.find({ country_id: someId });
const end = performance.now();
console.log(`Query took ${(end - start).toFixed(3)} ms`);

8. Indexing Best Practices

When to Create Indexes
• Always Index: Fields used in query conditions
• Consider Indexing: Fields used for sorting
• Avoid Indexing: Fields rarely used in queries
• Be Cautious With: Fields that change frequently

When Not to Use Indexes

• Very Small Collections: Under 100 documents may not benefit from indexes
• High Write, Low Read: Collections with many writes but few reads
• Low Selectivity Fields: Fields with few unique values
• Limited Resources: When storage space is a concern

Monitoring Index Usage

• Enable Mongoose Debug Mode:

mongoose.set('debug', true);

• Check Slow Queries:

db.system.profile.find().sort({ millis: -1 }).limit(5);

• Profile All Queries:

db.setProfilingLevel(2);

9. Real-World Performance Comparison

Impact of Indexing on Query Performance
Dataset Size Without Index With Index Improvement
1,000 docs ~1 second ~0.01 sec 100x faster
100,000 docs ~2 minutes ~0.02 sec 6,000x faster
1 million docs ~17 minutes ~0.03 sec 33,000x faster
1 billion docs ~11 days ~0.05 sec 20 million x
faster

Resource Usage
• Memory:
– Indexes are stored in memory for faster access
 – Each index typically uses 8KB of RAM per document
• Disk Space:
– Indexes typically add 5-10% to database size
– Text indexes can add up to 25% more space

10. Common Indexing Mistakes

Mistakes to Avoid
1. Over-indexing: Creating too many indexes increases write time
2. Wrong Order: Incorrect order in compound indexes
3. Redundant Indexes: Creating indexes that overlap with existing ones
4. Ignoring Write Impact: Not considering how indexes slow down writes
5. Not Monitoring: Never checking if indexes are being used

Solutions
1. Analyze Queries: Only create indexes for common queries
2. Use Compound Indexes Wisely: Put most-filtered fields first
3. Audit Existing Indexes: Remove unused or redundant indexes
4. Balance Read/Write: Consider write frequency when indexing
5. Regular Monitoring: Check explain() output regularly

11. Implementing Indexing in Mongoose

Basic Syntax
// Single field index
courseSchema.index({ field_name: 1 });

// Compound index
courseSchema.index({ field1: 1, field2: -1 });

// Unique index
courseSchema.index({ field_name: 1 }, { unique: true });

// Text index
courseSchema.index({ field_name: 'text' });

// Geospatial index
courseSchema.index({ location: '2dsphere' });

Example from Your Schema

// Add indexes for better query performance
courseSchema.index({ country_id: 1 });
courseSchema.index({ education_level: 1 });
courseSchema.index({ country_id: 1, education_level: 1 });
12. Conclusion
Key Points:
• Indexes are essential for database performance, especially as data grows
• The right indexes can make queries thousands of times faster
• Always monitor query performance and adjust indexes accordingly
• Balance between read performance and write overhead
• Choose indexes based on your specific query patterns

Final Thought:
Proper indexing is one of the most impactful optimizations you can make to improve
application performance. Taking time to understand and implement effective indexes will
save significant resources and provide a better user experience as your application scales.