Introduction to Data Structures and Algorithms

Mathematical Foundations of Efficiency

Sumod Sethumadhavan

Hitachi Digital Service

January 8, 2025
Table of Contents

1 Algorithm Analysis Prerequisites

Background Required:
Mathematics:
2 Data Structures
Discrete Mathematics
Number Theory
3 Search Linear Algebra (Optional)
Computer Science:
4 Sorting
Basic Programming Concepts
Any Language Proficiency
5 Fibonacci Series Analysis (C/C++/Python)

6 Comparison and Practice

7 Key Takeaways
Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 2 / 36
What is Algorithm Analysis?

Definition: Measurement of algorithm efficiency.

Key Metrics:
Time Complexity: Execution time relative to input size.
Space Complexity: Memory usage during execution.
Objective: Optimize performance and scalability.

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 3 / 36
Mathematical Tools

Recurrence Relations: Solve recursive algorithms.

Asymptotic Notations:
Big-O: Upper bound (worst case).
Theta: Tight bound (average case).
Omega: Lower bound (best case).
Logarithms: Used in divide-and-conquer.
Summations: Analyze loops.

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 4 / 36
Algorithm Complexity Analysis

Time Complexity Guide:

Excellent: O(1) - Constant time

O(log n) - Logarithmic
Good: O(n) - Linear time
Fair: O(n log n) -
Linearithmic
Poor: O(n2 ) - Quadratic time
O(2n ) - Exponential
time

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 5 / 36
Data Structure Classifications

Figure: Visual representation of common data structures

Linear Data Structures Non-Linear Data Structures

Arrays Trees
Linked Lists Graphs
Stacks Hash Tables
Queues Heaps
Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 6 / 36
Linear Data Structures

Arrays
Contiguous memory allocation
Direct access: O(1)
Insertion/Deletion: O(n)

Linked Lists
Dynamic memory allocation
Sequential access
Efficient insertions/deletions

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 7 / 36
Non-Linear Data Structures

Trees
Hierarchical structure
Binary Search Trees: O(log n) operations
Applications: File systems, decision trees

Graphs
Network representation
Social networks, routing algorithms
BFS, DFS traversal methods

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 8 / 36
Linear Search

Definition
Linear search scans each element in a list sequentially until the target value is found or the end
of the list is reached.

Mathematics
Best Case: The target is the first element, O(1).
Worst Case: The target is not present, O(n).
Average case: O(n/2), simplifies to O(n).

Example
Given a list [2, 4, 6, 8, 10], search for 8:
Start with index 0, and compare each element sequentially.
Found at index 3 after 4 comparisons.
Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 9 / 36
Binary Search
Definition
Binary search works on sorted arrays by repeatedly dividing the search interval in half.

Mathematics
Best Case: Target is the middle element, O(1).
Worst Case: O(log n).
Average Case: O(log n).

Example
Given a sorted list [1, 3, 5, 7, 9, 11], search for 5:
Compare with middle element (7).
Move to left subarray [1, 3, 5].
Found at index 2 after 2 comparisons.
Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 10 / 36
Comparison of Linear and Binary Search

Search Method Time Complexity Use Case

Linear Search O(n) Unsorted or small datasets
Binary Search O(log n) Sorted datasets
Table: Comparison of Search Methods

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 11 / 36
Sorting Fundamentals

Definition
Sorting is a fundamental operation that arranges elements in a specific order:
Input: sequence (a1 , a2 , ..., an )
Output: permutation where a1′ ≤ a2′ ≤ ... ≤ an′

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 12 / 36
Merge Sort Overview

Key Characteristics
Divide-and-conquer strategy
Stable sorting algorithm
Consistent performance: Θ(n log n)

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 13 / 36
 Implementation

Python Code
1 def merge_sort ( arr ) :
2 if len ( arr ) <= 1:
3 return arr
4

5 mid = len ( arr ) // 2

6 left = merge_sort ( arr [: mid ])
7 right = merge_sort ( arr [ mid :])
8
9 return merge ( left , right )

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 14 / 36
 Implementation (continued)
Merge Function
1 def merge ( left , right ) :
2 result = []
3 i = j = 0
4
5 while i < len ( left ) and j < len ( right ) :
6 if left [ i ] <= right [ j ]:
7 result . append ( left [ i ])
8 i += 1
9 else :
10 result . append ( right [ j ])
11 j += 1
12
13 result . extend ( left [ i :])
14 result . extend ( right [ j :])
15 return result
Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 15 / 36
Recurrence Relation

Theorem
The time complexity follows the recurrence:
(
Θ(1) if n = 1
T (n) =
2T (n/2) + Θ(n) if n > 1

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 16 / 36
Recursion Tree Analysis

Analysis by Level
Level i contains:
2i nodes
Each subproblem size: n/2i
Work per node: Θ(n/2i )
Total work at level i: Θ(n)

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 17 / 36
Solution

Final Complexity
Tree height: log2 n levels
Work per level: Θ(n)
Total complexity: Θ(n log n)

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 18 / 36
Merge Sort: Divide & Conquer

Key Characteristics Merge Sort Process

Time Complexity: O(n log n)
Both average and worst case
Space Complexity: O(n)
Stable sort algorithm
Recurrence: T (n) = 2T (n/2) + n

Advantages
Predictable performance
Efficient for sequential access Figure: Merge Sort Visualization
39% fewer comparisons than quicksort’s
average case
Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 19 / 36
Space and Time Complexity

Time Complexity Space Complexity

Best: Θ(n log n) Array: Θ(n)
Average: Θ(n log n) Stack: Θ(log n)
Worst: Θ(n log n) Total: Θ(n)

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 20 / 36
Fibonacci Sequence Definition

Recursive Formula
F (n) = F (n − 1) + F (n − 2), F (0) = 0, F (1) = 1

Widely used in dynamic programming examples.

Benchmark for recursive vs. iterative approaches.

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 21 / 36
 Recursive Approach

1 def fibon ac c i _ re c u rs i v e ( n ) :
2 if n <= 1:
3 return n
4 return fi b o na c c i_ r e c ur s i ve (n -1) + fib o n ac c i _r e c ur s i ve (n -2)

Analysis
Time Complexity: T (n) = T (n − 1) + T (n − 2) + O(1) ⇒ O(2n ).
Space Complexity: O(n) due to stack usage.

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 22 / 36
 Dynamic Programming Approach

1 def fibonacci_dp ( n ) :
2 if n <= 1:
3 return n
4 fib = [0 , 1]
5 for i in range (2 , n +1) :
6 fib . append ( fib [i -1] + fib [i -2])
7 return fib [ n ]

Analysis
Time Complexity: O(n).
Space Complexity: O(n) for the array.

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 23 / 36
Matrix Exponentiation Approach

Mathematics Behind
   n  
F (n) 1 1 F (1)
=
F (n − 1) 1 0 F (0)

Analysis
Time Complexity: O(log n) using fast exponentiation.
Space Complexity: O(1).

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 24 / 36
Mathematical Proof and Analysis

Inductive Proof Eigenvalues √

λ1,2 = 1±2 5
 
1 1
For M = , prove:
1 0 ϕ = λ1
  Binet’s formula:
Fn+1 Fn
Mn = ϕn − (−ϕ)−n
Fn Fn−1 Fn = √
5
Base: n = 1 trivial
Step: n → n + 1:
 
Fn+1 + Fn Fn+1
M n+1 = M n M =
Fn + Fn−1 Fn

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 25 / 36
Fast Matrix Exponentiation Implementation

Square and Multiply Algorithm

For computing M n : 


I if n=0

M if n=1
Mn =


(M n/2 )2 if n is even
M · M n−1

if n is odd

Matrix Multiplication
For 2×2 matrices:      
a b e f ae + bg af + bh
· =
c d g h ce + dg cf + dh

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 26 / 36
Comparison of Methods

Method Time Complexity Space Complexity Notes

Recursive O(2n ) O(n) Exponential growth
Dynamic Programming O(n) O(n) Bottom-up approach
Matrix Exponentiation O(log n) O(1) Most efficient
Table: Fibonacci Algorithm Comparison

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 27 / 36
Think Before You Code

Mathematical Analysis First

Analyze time and space complexity requirements
Derive mathematical bounds for your solution
Verify if better theoretical bounds exist
Consider trade-offs between time and space

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 28 / 36
Efficient Implementation

Optimization Guidelines
Choose appropriate data structures
HashMaps for O(1) lookups
Heaps for priority operations
Balanced BSTs for ordered data
Avoid nested loops when possible
Use space-time trade-offs wisely

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 29 / 36
Test-Driven Development

Test Categories TDD Cycle

Base cases 1 Write test first
Edge cases 2 See it fail
Corner cases 3 Implement code
Boundary conditions 4 Pass the test
Large inputs 5 Refactor

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 30 / 36
Code Quality Standards

Best Practices
Follow language-specific conventions
Write self-documenting code
Add meaningful comments for complex logic
Use descriptive variable names
Break down complex functions

Remember
Clear, maintainable code is as important as efficient code

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 31 / 36
Practical Example: Binary Search

Mathematical Analysis Test Cases

Time: O(log n) Empty array
Space: O(1) Single element
Prerequisite: Sorted array Target not found
Duplicate elements

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 32 / 36
 Implementation Example

1 def binary_search ( arr , target ) :

2 """ Efficiently find target in sorted array """
3 if not arr : return -1
4 left , right = 0 , len ( arr ) - 1
5 while left <= right :
6 mid = left + ( right - left ) // 2
7 if arr [ mid ] == target : return mid
8 elif arr [ mid ] < target : left = mid + 1
9 else : right = mid - 1
10 return -1

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 33 / 36
Verification Checklist

Before Submission
Complexity matches requirements
All test cases pass
Code follows standards
Edge cases handled
Documentation complete
No memory leaks

Final Check
Always profile your code with real-world data sizes

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 34 / 36
Practice Problem

Task
Implement Fibonacci computation using:
Recursive method.
Dynamic programming.
Matrix exponentiation.

Discussion Points
Compare time and space complexities.
Identify the best use cases for each method.

Sumod Sethumadhavan (Hitachi Digital Service) Introduction to Data Structures and Algorithms January 8, 2025 35 / 36
Thank You! Questions?