Introduction to Complexity in Algorithm

1.1. What is an Algorithm?

An algorithm is like a recipe for solving a problem. It has clear steps that tell the computer
exactly what to do, in the right order, to get the desired result.

Example of an Algorithm:

To find the largest number in a list:

1. Start with the first number in the list and call it the current largest.
2. Look at the next number in the list:
o If it is bigger than the current largest, replace the current largest with this new
number.
o If it’s not bigger, move to the next number.
3. Continue until you have checked all the numbers.
4. The final "current largest" number is the largest number in the list.

1.2. What is Complexity?

Algorithm Complexity measures how much time and memory an algorithm uses, depending on
the size of the input. It helps to know whether an algorithm will work efficiently as the problem
grows.

 Time Complexity: Measures how the running time increases as the input size increases.
 Space Complexity: Measures how much extra memory an algorithm uses as the input
size increases.

Example:

If you have 5 numbers and an algorithm takes 5 steps to process them, the time complexity is
O(5). If you have 1000 numbers, and it takes 1000 steps, the time complexity is O(1000).

1.3. Types of Algorithm Complexity

Understanding algorithm complexity helps to choose the best algorithm for a problem. Here are
the most common types of complexities:

1.3.1. Time Complexity:

This tells us how fast an algorithm runs. It describes the relationship between the input size and
the time it takes to run.

 O(1): Constant Time. The time it takes doesn’t depend on the input size.
o Example: Accessing an element from an array.
 O(n): Linear Time. The time it takes grows directly with the input size.
o Example: A linear search through a list to find a number.
 O(n2): Quadratic Time. The time grows as the square of the input size. Often happens in
algorithms that involve nested loops.
o Example: Bubble Sort, where you compare each pair of numbers.
 O(log n): Logarithmic Time. The time grows slowly as the input size increases. This
happens when the algorithm divides the problem in half with each step.
o Example: Binary Search, where you keep cutting the list in half to find a
number.
 O(n log n): Linearithmic Time. This is a combination of linear and logarithmic time,
commonly seen in efficient sorting algorithms.
o Example: Merge Sort and Quick Sort.
 O(2n): Exponential Time. The time grows very fast as the input size increases, making it
impractical for large inputs.
o Example: Calculating Fibonacci numbers using simple recursion.

1.3.2. Space Complexity:

This tells us how much extra memory is needed by an algorithm to complete its task.

 O(1): Constant Space. The memory used does not depend on the input size.
o Example: Sorting an array in-place.
 O(n): Linear Space. The memory grows directly with the input size.
o Example: Storing the input in an array.

1.4. Types of Algorithms

Different algorithms are designed to solve problems in different ways. Here are some types of
algorithms:

1.4.1. Divide and Conquer Algorithms

These algorithms divide a big problem into smaller problems, solve the smaller problems, and
then combine the solutions.

 Example: Merge Sort and Quick Sort. These are used to sort large lists efficiently.
o Merge Sort splits the list into smaller lists, sorts each, and merges them back
together.
1.4.2. Dynamic Programming

This approach solves problems by breaking them into overlapping subproblems and solving each
one just once. Results are stored to avoid repeating work.

 Example: Fibonacci Sequence. Instead of recalculating each Fibonacci number from

scratch, dynamic programming stores the results for future use.

1.4.3. Greedy Algorithms

Greedy algorithms make the best choice at each step to try to find the best solution.

 Example: Dijkstra’s Algorithm for finding the shortest path in a graph. It chooses the
closest node first, assuming that it leads to the shortest path.

1.4.4. Backtracking

Backtracking is used to find all possible solutions to a problem, and it keeps trying different
options until it finds the correct one.

 Example: Solving a Sudoku puzzle. The algorithm tries to fill the grid, and if it reaches
a dead end, it backtracks and tries a different approach.

1.4.5. Brute Force Algorithms

These algorithms try all possible solutions to find the best one. It’s simple but inefficient for
large problems.

 Example: Trying all possible password combinations to break into a locked account.

1.5. Uses of Algorithms

Algorithms are everywhere and are used in many fields, including:

 Data Sorting and Searching: Algorithms like QuickSort and Binary Search are used
to find or sort data quickly.
 Machine Learning: Algorithms like Gradient Descent help computers learn from data.
 Web and App Development: Algorithms handle everything from loading a webpage to
recommending products on an e-commerce site.
 Networking: Algorithms like Dijkstra's Algorithm help in routing data across
networks.
1.6. Advantages of Algorithms

 Efficiency: A well-designed algorithm can make a program run faster and use fewer
resources.
 Scalability: Good algorithms work well with both small and large datasets.
 Optimization: Algorithms that are designed efficiently can save time and memory,
especially for large inputs.
 Predictability: Knowing the complexity of an algorithm helps predict its performance on
different problem sizes.

1.7. Disadvantages of Algorithms

 Overhead: Writing complex algorithms can take time and might add unnecessary
complexity to a simple task.
 Not Always Accurate: The theoretical time complexity doesn’t always match real-world
performance due to factors like hardware limitations and programming language.
 Space Complexity: Some algorithms, even if they are fast, might use a lot of memory,
which can be problematic in resource-constrained environments.
Examples of Algorithms and Complexity

1. Searching for a Book in a Library

 Algorithm: Suppose you are looking for a specific book in a library with shelves
arranged alphabetically.
o Brute Force (Linear Search): Go through every book on every shelf until you
find the book you need.
 Time Complexity: O(n)O(n)O(n), where nnn is the total number of books.
o Efficient Search (Binary Search): If the books on each shelf are sorted, start in
the middle, check if the book is before or after the middle, and repeat until you
find the book.
 Time Complexity: O(log⁡n)O(\log n)O(logn).

2. Organizing a Grocery Shopping List

 Algorithm: You have a list of items to buy, and you want to organize it for quick
shopping.
o Brute Force: Go through your list randomly and search for the items one by one
in the store.
 Time Complexity: O(n2)O(n^2)O(n2), since you might go back and forth
many times.
o Optimized Sorting (Merge Sort): Arrange your shopping list based on the layout
of the store (e.g., all fruits together, then vegetables, etc.).
 Time Complexity: O(nlog⁡n)O(n \log n)O(nlogn).

3. Finding the Shortest Route (Google Maps)

 Algorithm: Suppose you need to travel from your home to a destination.

o Greedy Algorithm: Use a navigation app like Google Maps to choose the
shortest distance at each turn.
 Example: Dijkstra’s Algorithm is used here to calculate the shortest
path.
 Time Complexity: O(V+Elog⁡V)O(V + E \log V)O(V+ElogV), where
VVV is the number of locations and EEE is the number of roads.

4. Scheduling Appointments
  Algorithm: You have several appointments to attend in a day, and you want to minimize
travel time.
o Brute Force: Try every possible order of appointments to see which one gives the
shortest travel time.
 Time Complexity: O(n!)O(n!)O(n!), where nnn is the number of
appointments.
o Optimized Greedy Algorithm: Start with the closest appointment first, then go
to the next closest, and so on.
 Time Complexity: O(n2)O(n^2)O(n2).

5. Solving a Puzzle

 Algorithm: You are solving a Sudoku puzzle.

o Brute Force: Try every possible number in every empty cell until the puzzle is
solved.
 Time Complexity: O(9n)O(9^n)O(9n), where nnn is the number of empty
cells.
o Backtracking Algorithm: Start filling numbers in empty cells one by one. If a
number doesn’t work, go back and try another number.
 Time Complexity: O(9n)O(9^n)O(9n), but it’s much more efficient than
brute force in practice.

6. Sorting Exam Papers

 Algorithm: You need to sort a stack of exam papers in order of grades.

o Bubble Sort: Compare two papers at a time and swap them if they’re in the
wrong order. Repeat until the stack is sorted.
 Time Complexity: O(n2)O(n^2)O(n2).
o Quick Sort: Divide the stack into smaller stacks (e.g., higher grades in one stack,
lower grades in another), sort each smaller stack, and combine.
 Time Complexity: O(nlog⁡n)O(n \log n)O(nlogn).

7. Cooking a Meal

 Algorithm: You need to cook several dishes for a meal.

o Brute Force: Cook each dish one by one, starting from the first on the list.
 Time Complexity: O(n)O(n)O(n), where nnn is the number of dishes.
o Parallel Processing (Divide and Conquer): While one dish is boiling, start
chopping vegetables for another dish.
 Time Complexity: Much faster due to parallel execution.
8. Packing a Suitcase

 Algorithm: You want to fit as much as possible into a limited suitcase.

o Brute Force: Try all combinations of items to see which fits best.
 Time Complexity: O(2n)O(2^n)O(2n), where nnn is the number of items.
o Greedy Algorithm: Start with the most important or smallest items first until the
suitcase is full.
 Time Complexity: O(n)O(n)O(n).

9. Managing Finances

 Algorithm: You want to distribute your monthly budget among rent, food, utilities, and
savings.
o Dynamic Programming: Allocate amounts to each category based on the most
efficient use of your money (e.g., paying off high-interest loans first).
 Example: The Knapsack Problem, where you maximize value with
limited resources.
 Time Complexity: O(n⋅W)O(n \cdot W)O(n⋅W), where nnn is the number
of categories and WWW is the total budget.

10. Playing Chess

 Algorithm: You want to make the best move in a chess game.

o Brute Force: Analyze every possible move and every possible response up to the
end of the game.
 Time Complexity: Exponential (O(2n)O(2^n)O(2n)).
o Heuristic Algorithm: Use rules (e.g., "protect the king," "control the center") to
decide the next move.
 Time Complexity: Much faster but might not always find the best
solution.