Algorithm
Lecture: Introduction to Algorithms
What is an Algorithm?
• Definition: An algorithm is a step-by-step set of instructions to solve a problem or
perform a task.
• Example: A recipe to bake a cake, or instructions to solve a math problem.
Key Properties of Algorithms
1. Input: Zero or more inputs are externally supplied.
2. Output: At least one output is produced.
3. Definiteness: Each step is precisely defined.
4. Finiteness: The algorithm must end after a finite number of steps.
5. Effectiveness: Each step must be basic enough to be carried out.
Simple Example
Problem: Add two numbers.
Algorithm:
1. Start
2. Read number A
3. Read number B
4. Add A and B, store in SUM
5. Print SUM
6. End
Types of Algorithms
1. Brute Force – Try all possible solutions
2. Greedy – Make the best choice at each step
3. Divide and Conquer – Divide the problem, solve parts, combine
4. Dynamic Programming – Store results of subproblems to avoid recomputation
5. Backtracking – Try partial solutions and backtrack if needed
6. Recursive Algorithms – Call the function within itself
Complexity of Algorithms
• Time Complexity: How the running time grows with input size (Big O notation)
• Space Complexity: How much memory is used
Common Time Complexities:
Complexity Example Algorithm Description
O(1) Accessing an array element Constant time
O(log n) Binary Search Logarithmic time
O(n) Linear Search Grows linearly
O(n log n) Merge Sort Efficient sorting
O(n²) Bubble Sort Nested loops
O(2ⁿ) Recursive Fibonacci Exponential time
Common Algorithms to Learn
• Searching: Linear Search, Binary Search
• Sorting: Bubble Sort, Selection Sort, Merge Sort, Quick Sort
• Recursion: Factorial, Fibonacci
• Graph Algorithms: DFS, BFS
• Dynamic Programming: Knapsack, Fibonacci with memoization
Absolutely! Here's a detailed lecture on the complexity of algorithms, covering both time and
space complexity with intuitive explanations, examples, and tables — ideal for beginners or
early computer science students.
Lecture: Complexity of Algorithms
1. What is Algorithmic Complexity?
Algorithmic complexity refers to the resources an algorithm consumes, typically:
• Time: How long the algorithm takes to run
• Space: How much memory (RAM) it uses
This helps you:
• Compare different algorithms
• Choose the most efficient one for large inputs
• Understand how your code will scale
2. Time Complexity
What is Time Complexity?
It describes how the runtime of an algorithm increases with the size of the input, usually
represented by n.
We use Big O notation to describe the upper bound (worst-case) behavior.
Common Time Complexities
Complexity Name Example Algorithm Description
O(1) Constant Accessing array element Same time regardless of input size
O(log n) Logarithmic Binary Search Halves the problem at each step
O(n) Linear Linear Search Time grows directly with input size
O(n log n) Linearithmic Merge Sort, Quick Sort Efficient sorting
Complexity Name Example Algorithm Description
O(n²) Quadratic Bubble Sort Nested loops over the input
O(2ⁿ) Exponential Recursive Fibonacci Time doubles with each additional input
O(n!) Factorial Solving permutations Extremely slow growth
Examples
O(1) – Constant Time
def get_first_element(arr):
return arr[0]
→ Always one step, regardless of array size.
O(n) – Linear Time
def find_max(arr):
max_val = arr[0]
for val in arr:
if val > max_val:
max_val = val
return max_val
→ Must check each item once → grows with n.
O(n²) – Quadratic Time
def bubble_sort(arr):
for i in range(len(arr)):
for j in range(len(arr) - i - 1):
if arr[j] > arr[j + 1]:
arr[j], arr[j + 1] = arr[j + 1], arr[j]
→ Each item compared with each other → n × n.
How to Analyze Time Complexity
1. Count the number of basic operations (e.g., additions, comparisons).
2. Focus on the most dominant term (ignore constants and low-order terms).
o For 3n + 2, complexity is O(n)
o For 5n² + 4n + 7, complexity is O(n²)
3. Space Complexity
What is Space Complexity?
It measures how much additional memory an algorithm needs as the input size increases.
Example
def sum_array(arr):
total = 0
for num in arr:
total += num
return total
→ Uses constant space: one variable (total) → O(1)
When Space Matters
• Recursion: Each recursive call adds to the call stack
• Storing auxiliary structures: Hash tables, lists, matrices
• Dynamic Programming: Memoization uses extra space
Example – Fibonacci with memoization (O(n) space)
def fib(n):
memo = [0, 1]
for i in range(2, n+1):
memo.append(memo[i-1] + memo[i-2])
return memo[n]
4. Best, Worst, and Average Case
• Best Case: Minimum time (e.g., searching and finding on the first try)
• Worst Case: Maximum time (used for Big O)
• Average Case: Expected time for random inputs
Binary Search Example:
Case Time Complexity
Best O(1)
Worst O(log n)
Average O(log n)
5. Tradeoffs
Sometimes:
• Faster time needs more space (and vice versa)
• Choosing between algorithm efficiency and implementation simplicity
Algorithm Time Complexity Space Complexity
Merge Sort O(n log n) O(n)
Quick Sort O(n log n) O(log n)
Bubble Sort O(n²) O(1)
Dynamic Fib O(n) O(n)
Recursive Fib O(2ⁿ) O(n)
6. Practice Problems
Try analyzing the time/space complexity of these:
1. Count even numbers in a list
2. Multiply all elements in a 2D matrix
3. Check for duplicates in a list using a set
4. Compute factorial using recursion
5. Sort a list using insertion sort
7. Summary
Aspect Description
Time Complexity Measures how execution time scales
Space Complexity Measures how memory use scales
Big O Notation Describes upper bound (worst-case)
Focus on dominant terms Ignore constants and low-order terms
