Index Purpose and Benefits

What is an Index?

An index in a database is a data structure that enhances the speed of data retrieval operations on a
table. It is akin to the index at the back of a book, allowing you to quickly locate the desired content
without scanning every page.

Purpose of an Index

1. Faster Query Execution:

Indexes significantly improve the performance of SELECT queries by reducing the amount of
data scanned.
Useful for filtering, sorting, and joining large datasets.
2. Efficiency in Searching:
Provides efficient mechanisms for searching based on specific columns, especially in large
datasets.
3. Uniqueness Enforcement:
Enforce uniqueness constraints (e.g., primary keys) to avoid duplicate data.
4. Support for Sorting and Grouping:
Helps optimize ORDER BY and GROUP BY clauses by pre-sorting the indexed data.
5. Improves Joins:
Indexes on columns involved in JOIN operations speed up combining rows from multiple
tables.
6. Reduces I/O Operations:
Minimizes disk reads by allowing the database to fetch only relevant rows.

Caveats:

Indexes consume extra storage.

Excessive or poorly designed indexes can slow down INSERT, UPDATE, and DELETE operations due
to the overhead of maintaining the index.

Simple Index for Entry-Sequenced Files

An entry-sequenced file stores records in the order they are entered, without any logical relationship
between them. A simple index is used to quickly locate records in such files by creating an auxiliary
structure.

How a Simple Index Works:

1. Structure:

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 1/4
 A simple index is a table containing two fields:
Key field: The value used to identify a record (e.g., ID or Name).
Pointer field: A reference to the physical address (record location) in the file.
2. Indexing Process:
The index is created by scanning the file and storing key-pointer pairs for each record in the
index table.
The index is sorted by the key values for faster search.

Benefits:

1. Faster Search:
Instead of scanning the entire file, the index allows direct access to the desired record.
2. Efficient Updates:
For insertion or deletion, only the index needs adjustment.

Example:

Key (ID) Pointer (Location)

101 Address 1
102 Address 2

Limitations:

As file size grows, the index also grows, requiring more memory.
The index must be updated with every record addition or deletion.

Indexes Too Large to Hold in Memory

When an index becomes too large to fit in memory, efficient techniques are used to manage and access
it without compromising performance. Below are key aspects:

Challenges:

1. Memory Constraints:
Large datasets can lead to index structures that exceed the available memory.
Frequent disk I/O is required, which can slow down query execution.
2. Performance Overhead:
Scanning or updating large indexes stored on disk increases latency.

Solutions for Large Indexes:

1. Multi-Level Indexing:
Breaks the index into smaller parts.
Uses a hierarchy, such as a two-level or three-level index, where higher-level indexes fit in
memory, and lower levels reside on disk.
2. B-Tree and B+ Tree Indexes:
Disk-optimized structures that minimize disk reads.
Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 2/4
 B+ Trees store data in leaf nodes and maintain balanced height for efficient retrieval.
3. Partitioned Indexing:
Splits the index into smaller segments based on criteria like range or hash values.
Each partition can be loaded or queried independently.
4. Clustered Indexes:
Store the data physically sorted, reducing the need for large separate indexes.
5. Use of Cache:
Frequently accessed parts of the index are cached in memory.

Conclusion:

Disk-based index optimization techniques like B-Trees and multi-level indexing help manage large
indexes, balancing memory usage and query performance.

Indexing to Provide Access by Multiple Keys

Indexing for multiple keys allows efficient retrieval of records based on combinations of multiple
columns (keys). This is particularly useful in scenarios where queries involve multiple criteria.

Types of Multi-Key Indexing:

1. Composite Index:
A single index is created on multiple columns.
The index is sorted based on the combination of keys in the specified order.
Useful for queries like `WHERE key1 = X AND key2 = Y`.
2. Separate Indexes on Each Key:
Separate indexes are created for each key.
Database optimizers combine these indexes to filter results efficiently.
Useful for independent queries on each key.
3. Bitmap Index:
Uses bitmaps to represent key values and their presence in rows.
Efficient for combining multiple keys using logical operations like AND, OR.

Example of Composite Index:

Given a table with columns `City` and `Age`, a composite index on `(City, Age)` allows:

Query: `WHERE City = 'New York' AND Age = 30`

Direct lookup based on the combined key.

Benefits:

1. Speeds Up Multi-Condition Queries:

Reduces the need to scan the entire dataset.
2. Efficient Logical Operations:
Combines conditions for multiple keys seamlessly.

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 3/4
Considerations:

Composite indexes are order-sensitive; the sequence of keys matters for optimal use.

Retrieval Using Combinations of Secondary Keys

A secondary key is a non-primary attribute used to retrieve records from a database. Unlike primary
keys, secondary keys do not uniquely identify a record but allow multiple records to share the same
value. Retrieval using combinations of secondary keys involves accessing records based on multiple non-
primary attributes.

Process of Retrieval:

1. Separate Index for Each Key:

Individual indexes are created for each secondary key.
For example, a database table with columns `Department` and `Salary` might have separate
indexes for each.
2. Query Execution:
When a query involves multiple secondary keys (e.g., `WHERE Department = 'HR' AND Salary >
50000`), the database system retrieves matching records using each index.
3. Index Intersection or Union:
AND Condition:
The system performs an intersection of records matching each secondary key condition.
OR Condition:
A union operation retrieves records matching any condition.

Example:

For a table with indexes on `City` and `Age`, a query like:

`SELECT * FROM Employee WHERE City = 'New York' AND Age > 30`
will use the indexes for both keys to identify relevant records.

Benefits:

1. Efficient Filtering:
Combines indexes to narrow down results without scanning the entire table.
2. Scalability:
Useful for large datasets with complex queries.

Considerations:

May require sufficient memory to handle index operations.

Overhead in maintaining multiple secondary indexes during updates.

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 4/4