Algorithms and
Asymptotic Analysis of
Algorithms
Course Instructor: Maheen Zulfiqar
What is an Algorithm?
• An algorithm is a finite set of well-defined instructions or steps to
solve a specific problem or perform a computation. It must be:
• Input: Takes zero or more inputs.
• Output: Produces at least one output.
• Definiteness: Each step is precisely defined.
• Finiteness: Must terminate after a finite number of steps.
• Effectiveness: Each step is basic enough to be carried out.
Examples of Algorithms
1.Sorting: Bubble Sort, Merge Sort, Quick Sort
2.Searching: Binary Search, Linear Search
3.Graph Algorithms: Dijkstra’s, BFS, DFS
4.Dynamic Programming: Knapsack, Fibonacci
Asymptotic Analysis
• Asymptotic analysis helps evaluate the performance of an
algorithm as the input size grows. It describes the growth
rate of runtime or space used, ignoring machine-specific
constants.
Why Asymptotic Analysis?
•Focuses on input size n → ∞.
•Compares efficiency independent of hardware/software.
•Useful for worst-case scenarios and scalability.
Types of Asymptotic Notations
3. Big Theta Notation (Θ):
1. Big O Notation (O): 2. Big Omega Notation (Ω):
• Definition:
• Definition: • Definition:
• Big Theta notation describes the
• Big O notation describes the upper • Big Omega notation describes the
tight bound of an algorithm's growth
bound of an algorithm's growth lower bound of an algorithm's rate. It provides an estimate of the
rate. It provides an estimate of the growth rate. It provides an average-case scenario.
worst-case scenario. estimate of the best-case
scenario. • Purpose:
• Purpose: • It indicates that the algorithm's time
• Purpose:
• It tells you the maximum amount or space complexity grows at a rate
• It tells you the minimum amount
of time or space an algorithm that is both bounded above and
of time or space an algorithm below by the same function.
could take to complete, given a
must take to complete, given a
certain input size. certain input size. • Example:
• Example: • Example: • If an algorithm has a time complexity
• If an algorithm has a time of Θ(n), it means that the algorithm's
• If an algorithm has a time execution time will grow linearly with
complexity of O(n), it means that complexity of Ω(1), it means that
in the worst case, the time it takes the input size, both in the best and
in the best case, the algorithm worst cases.
to run will grow linearly with the will always take a constant
input size 'n'. amount of time, regardless of the
input size.
Cont.
Notation Description Example
Big O (O) Worst-case upper bound O(n^2) for Bubble Sort
Omega (Ω) Best-case lower bound Ω(n) for Linear Search
Theta (Θ) Tight bound (both upper Θ(n log n) for Merge Sort
and lower)
Common Time Complexities
Complexity Description Example
O(1) Constant Accessing an array element
O(log n) Logarithmic Binary Search
O(n) Linear Linear Search
O(n log n) Linear logarithmic Merge Sort
O(n^2) Quadratic Bubble Sort
O(2^n) Exponential Recursive Fibonacci
O(n!) Factorial Brute-force TSP
Cont.
• O(1) constant: the operation doesn't depend on the size of its input, e.g. adding a
node to the tail of a linked list where we always maintain a pointer to the tail node.
• O(n) linear: the run time complexity is proportionate to the size of n. O(log n)
logarithmic: normally associated with algorithms that break the problem into
smaller chunks per each invocation, e.g. searching a binary search tree.
• O(n log n) just n log n: usually associated with an algorithm that breaks the
problem into smaller chunks per each invocation, and then takes the results of
these smaller chunks and stitches them back together, e.g. quick sort.
• O(n^2) quadratic: e.g. bubble sort.
• O(n^3) cubic: very rare.
• O(2^n) exponential: incredibly rare.
Analysing an Algorithm: Steps
1.Identify the basic operation (e.g., comparison in sorting).
2.Count the number of times the operation executes as n grows.
3.Express it using asymptotic notation.
4.Simplify using dominant term (ignore lower-order terms/constants).
Space Complexity
•Similar to time complexity but refers to memory usage.
•Includes:
•Input storage
•Auxiliary space
•Function call stack
Best, Worst, and Average Cases
•Best Case: Minimum time taken for any input of size n.
•Worst Case: Maximum time taken.
•Average Case: Expected time over all inputs.
Example: Linear Search
python
def linear_search(arr, key):
for i in range(len(arr)):
if arr[i] == key:
return i
return -1
•Best Case: Key at index 0 → O(1)
•Worst Case: Key not found or last element → O(n)
•Average Case: O(n)
