Binary Search Algorithm

Md Habibul Basar Faruq

Introduction

● Binary Search is one of the fastest searching algorithms.

● It is used for finding the location of an element in a linear array.
● It works on the principle of divide and conquer technique.
● This algorithm only works for sorted array.
● Binary Search is also known as Half-interval search or Logarithm search.
Binary Search Algorithm Procedure
● There is a linear array ‘a’ of size ‘n’.
● Binary search algorithm is being used to search an element ‘item’ in this linear
array.
● If search ends in success, it sets loc to the index of the element otherwise it
sets loc to -1.
● Variables beg and end keeps track of the index of the first and last element of
the array or sub array in which the element is being searched at that instant.
● Variable mid keeps track of the index of the middle element of that array or
sub array in which the element is being searched at that instant.
Algorithm
Iterative Binary Search:
BinarySearch(arr, n, x):
low ← 0
high ← n - 1
while low ≤ high:
mid ← low + (high - low) / 2
if arr[mid] == x:
return mid
else if arr[mid] > x:
high ← mid - 1
else:
low ← mid + 1
return -1
Algorithm
Recursive Binary Search:
BinarySearch(arr, low,high, x):
if low > high:
return -1

mid ← low + (high - low) / 2

if arr[mid] == x:
return mid
else if arr[mid] > x:
return BinarySearch(arr, low, mid - 1, x)
else:
return BinarySearch(arr, mid + 1, high, x)
Example:
Consider an array arr[] = {5, 7, 8, 10, 16, 23, 25 56, 72, 100}, and the target = 10.

arr[] = 0 1 2 3 4 5 6 7
8 5 97 8 10 16 23 25 56 72 100

Output: 3

Code Link: Click Here

Time Complexity of Binary Search Algorithm
❏ Number of Steps in Binary Search:

Initially, the search space is n (size of the array).

After 1 iteration, the size becomes n/2.

After 2 iterations, the size becomes n/4.

After k iterations, the size becomes n/2k.

The process continues until the search space size reduces to 1.

N / 2k =1

Solving for k: N = 2k => k = log2N

Thus, the maximum number of steps (or iterations) required is proportional to log2N
Time Complexity of Binary Search Algorithm
❏ Complexity:

Each step involves a constant amount of work (calculating the middle index and performing a
comparison).

The total number of steps in the worst case is log2N.

Therefore, the time complexity is:

O(log2N)

Best-Case Time Complexity:

The best case occurs when the target is found at the middle index on the first iteration.

In this case, only 1 comparison is required:

O(1)
Time Complexity of Binary Search Algorithm
Worst-Case Time Complexity:

The worst case occurs when the target is not present, or it is located at the farthest possible
position (leftmost or rightmost in the last remaining subarray).

This requires log2N iterations:

O(log2N)

Average-Case Time Complexity:

On average, the target is found after half the possible iterations.

The average number of iterations is also logarithmic, so the average-case time complexity

is:

O(log2N)
Sorting Algorithms
Introduction
❏ A sorting algorithm is used to arrange elements of an array/list in a specific
order. For example:

An unsorted array:
200 111 35 77 6 30 55 29

Sorting algorithm
After sorting:
6 29 30 35 55 77 111 200
Different Sorting Algorithms
❏ There are several sorting algorithms, some of them are given below:
● Bubble Sort
● Selection Sort
● Insertion Sort
● Merge Sort
● Quick Sort
● Heap Sort
Bubble Sort Algorithm
❏ Bubble Sort is the simplest sorting algorithm that works by repeatedly
swapping the adjacent elements if they are in the wrong order.

This algorithm is not suitable for large data sets as its average and worst-case
time complexity are quite high.

It involves n phase where each phase performs n number of comparisons.

Bubble Sort Algorithm
The Bubble Sort algorithm involves following steps:

1. Start from the first element in the array.

2. Compare the current element with the next element.
● If the current element is greater than the next element, swap them.
1. Repeat Step 2 for all adjacent pairs in the array.
2. After each pass through the array:
● The largest unsorted element "bubbles up" to its correct position at the end.
● Ignore the last element in the next pass since it is already sorted.
1. Repeat the process for n−1 passes (where n is the size of the array) or until no swaps
are required.
Bubble Sort Algorithm
void bubbleSort ( int numbers[], int array_size )
{
int i, j, temp;
for( i = ( array_size - 1 ); i >= 0; i-- )
{
for( j = 1; j <= i; j++ )
{
if( numbers[j - 1] > numbers[j] )
{
temp = numbers[j - 1];
numbers[j - 1] = numbers[j];
numbers[j] = temp;
}
}
}
}
Bubble Sort Algorithm
Consider an array arr[] = {33,11,24,76,66,55,50}, and sort the array using Bubble sort
algorithm.

Input:
33 11 24 76 66 55 50

Output:
11 24 33 50 55 66 76

Code Link: Click here

Time complexity of Bubble sort
❏ Bubble Sort works by repeatedly swapping adjacent elements that are in the
wrong order. Here's a detailed analysis of its time complexity for different
cases:

Outer Loop: Executes n−1 passes in the worst case, where n is the number of
elements in the array.

Inner Loop: For each pass i, the inner loop executes n−i−1 comparisons
(fewer comparisons as the largest elements "bubble up" to their correct
positions).
Time complexity of Bubble sort
❏ Best Case (Already Sorted Array):

In the best case, the array is already sorted.

No swaps are required during any pass.
The algorithm can terminate early after the first pass due to the swapped flag
being false.
Number of Comparisons: n−1 (only one pass is needed to confirm the array is
sorted).
Time Complexity:
O(n)
Time complexity of Bubble sort
❏ Worst Case (Reverse Sorted Array):

In the worst case, the array is sorted in reverse order.

Every pair of adjacent elements needs to be swapped on each pass.
Number of Comparisons:
Outer loop: n-1 passes
Inner loop: (n−1)+(n−2)+ +1 Number of Comparisons
Total comparison: (n(n-1))/2
This is O(n2) comparisons.

Time complexity: O(n2)

Insertion Sort
❏ Insertion sort is a simple sorting algorithm that works by iteratively inserting each element of
an unsorted list into its correct position in a sorted portion of the list.

We start with second element of the array as first element in the array is assumed to be
sorted.

Compare second element with the first element and check if the second element is smaller
then swap them.

Move to the third element and compare it with the first two elements and put at its correct
position

Repeat until the entire array is sorted.

Insertion Sort Procedure
Insertion Sort Algorithm
for i = 1 to n-1:
key = arr[i]
j=i-1
while j >= 0 and arr[j] > key:
arr[j + 1] = arr[j]
j=j-1
arr[j + 1] = key
Insertion Sort
Example:

Input:
11 2 6 30 15 10

Output:
2 6 10 11 15 30

Code link: Click Here

Time Complexity of Insertion Sort Algorithm
Time Complexity of Insertion Sort

Best case: O(n) , If the list is already sorted, where n is the

number of

elements in the list.

Average case: O(n2 ) , If the list is randomly ordered

Worst case: O(n2) , If the list is in reverse order

Quick Sort Algorithm
Introduction
QuickSort is a comparison-based sorting algorithm that sorts an array or list of
elements using the divide-and-conquer strategy. It can be implemented using two
functions:

1. partition()
2. quickSort()
Partition()
It is the key process in the quicksort algorithm. It involves selecting a pivot element
and rearranging the array so that

- All elements smaller than the pivot are placed to its left, and
- The elements greater than the pivot are placed to its right.

The point where the pivot is placed is called the partitioning index and it is
returned to its caller quickSort().

50 70 60 12 23 55 33

After 1st partition

pivot
12 33 23 50 60 55 70
QuickSort()
It is the main recursive function that implements the divide-and-conquer strategy.

It divides the given array into two subarrays based on the partitioning index returned by partition() function.

Then it keeps calling itself for these two subarrays until the whole array is sorted.

If partition index is p

int p = partition(arr,s,e);

quicksort(arr,s,p-1);

quicksort(arr,p+1,e);
QuickSort Algorithm
Partition Function

➢ Input: An array arr, starting index s, and ending index e.

➢ Step 1: Select the first element arr[s] as the pivot.
➢ Step 2: Count the number of elements less than or equal to the pivot (cnt) in the subarray from s+1 to
e.
➢ Step 3: Calculate the pivot index (pivotIndex = s + cnt) and place the pivot in its correct position by
swapping arr[s] with arr[pivotIndex].
➢ Step 4: Initialize two pointers: i = s (start of the array) and j = e (end of the array).
➢ Step 5: Adjust elements around the pivot:
○ Increment i until arr[i] > pivot.
○ Decrement j until arr[j] ≤ pivot.
○ If i < pivotIndex and j > pivotIndex, swap arr[i] and arr[j].
➢ Step 6: Repeat until all elements around the pivot are correctly placed.
➢ Output: Return the final position of the pivot (pivotIndex).
QuickSort Algorithm
QuickSort Function

1. Input: An array arr, starting index s, and ending index e.

2. Step 1: If s >= e, return (base case).
3. Step 2: Partition the array using the partition function to determine the pivot index p.
4. Step 3: Recursively apply QuickSort to the two subarrays:
○ Subarray to the left of the pivot (s to p-1).
○ Subarray to the right of the pivot (p+1 to e).
Quicksort Algorithm
void quicksort(int *arr,int s, int e){

if(s>=e){

return;

int p = partition(arr,s,e);

quicksort(arr,s,p-1);

quicksort(arr,p+1,e);

}
Quicksort Example:
Input:
50 70 60 12 23 55 33

Output:

12 23 33 50 55 60 70

Code link: Click here

Time Complexity of Quicksort
Partition Function Analysis:

1. Counting Elements Smaller than Pivot:

● The loop from s+1 to e executes O(n) in the worst case, where n=e−s+1.
1. Swapping Pivot to Its Correct Position:
● This operation takes O(1).
1. Placing Elements on Correct Sides of the Pivot:
● The while loop adjusts the elements around the pivot. Each element is
compared at most once, so this step also takes O(n).

Overall Time Complexity of partition: O(n).

Time Complexity of Quicksort
QuickSort Function Analysis:

1. The partition function is called at each step, dividing the array into two subarrays.
2. Recursively, QuickSort is applied to the left and right subarrays.

Let T(n) represent the time complexity for sorting an array of size n:

T(n)=T(k)+T(n−k−1)+O(n)

k: Size of the left subarray.

n−k−1: Size of the right subarray.

O(n): Time for the partition step.

Time Complexity of Quicksort
Best case: The pivot divides the array into two equal halves.
Complexity: O(nlogn)
Worst case: The pivot always divides the array into one element and the rest of the
array (highly unbalanced split).
Complexity: O(n2)
Average case: On average, the pivot divides the array into two uneven parts, but
not as extreme as the worst case.
Complexity: O(nlogn)
Merge sort Algorithm
Introduction
Merge sort is a sorting technique based on the divide-and-conquer strategy.

It operates by recursively splitting the input array into smaller subarrays

Sorting these subarrays, and

Then combining them to form the final sorted array.

Merge sort
It follows the divide-and-conquer approach to sort a given array of elements.

Here’s a step-by-step explanation of how merge sort works:

Divide: Divide the list or array recursively into two halves until it can no more be
divided.

Conquer: Each subarray is sorted individually using the merge sort algorithm.

Merge: The sorted subarrays are merged back together in sorted order. The
process continues until all elements from both subarrays have been merged.
Merge sort
8 3 7 4 2 6 5 1

8 3 7 4 2 6 5 1
Divide

8 3 7 4 Divide 2 6 5 1

8 3 7 4 2 6 5 1
Divide

3 8 4 7 2 6 1 5
Merge
3 4 7 8 1 2 5 6
Merge
1 2 3 4 5 6 7 8
Merge sort Algorithm
MergeSort(Array, start, end):
if start> end
return
mid = (start+end)/2
mergeSort(Array, start, mid)
mergeSort(Array, mid+1, end)
merge(Array, start, mid, end)

Code Link: Click Here

Time Complexity of Merge sort
The recurrence relation of merge sort is:
Θ(1) if n=1

T(n) =

2T(n2)+Θ(n) if n>1

● T(n) Represents the total time time taken by the algorithm to sort an array of size n.
● 2T(n/2) represents time taken by the algorithm to recursively sort the two halves of the
array. Since each half has n/2 elements, we have two recursive calls with input size as
(n/2).
● O(n) represents the time taken to merge the two sorted halves
Time Complexity of Merge sort
Best Case: When the array is already sorted or nearly sorted.

Complexity: O(n log n)

Average Case: When the array is randomly ordered.

Complexity: O(n log n)

Worst Case: When the array is sorted in reverse order.

Complexity: O(n log n)

Linked List
Linked List
A linked list is a linear data structure that includes a series of connected nodes.
Here, each node stores the data and the address of the next node.

Head

Data Next Data Next Data Next Data Next Null

Linked list
Linked lists can be of multiple types:

● Singly Linked List

● Doubly Linked List,and
● circular linked list.
Linked List
Create a Node in singly linked list:

We can create a node using struct or class:

class Node{
public:
int data;
Node* next;

Node(int data, Node* next = nullptr){

this->data = data;
this->next = next;
}
};
Linked List
Create Nodes: Node* node1 = new Node(5);
Node* node2 = new Node(6);
Node* node3 = new Node(7);
Node* node4 = new Node(8);

Create Link:
Node* head = node1;

node1->next=node2;
node2->next=node3;
node3->next=node4;
Linked List Example
Here is a code to create a singly linked list and print the linked list:

Code Link: Click Here

Graphs
Definition of Graphs
A graph is a finite set of nodes with edges between nodes

Formally, a graph G is a structure (V,E) consisting of

A finite set V called the set of nodes, and

A set E that is a subset of VxV. That is, E is a set of pairs of the form (x,y) where x
and y are nodes in V
Examples of Graphs
● V={0,1,2,3,4}
● E={(0,1), (1,2), (0,3), (3,0), (2,2), (4,3)}

When (x,y) is an edge,

we say that x is adjacent to y, and y is
adjacent from x.

0 is adjacent to 1.
1 is not adjacent to 0.
2 is adjacent from 1.
Graph Representation
For graphs to be computationally useful, they have to be conveniently represented
in programs

There are two computer representations of graphs:

➔ Adjacency matrix representation

➔ Adjacency lists representation
Adjacency Matrix Representation
In this representation, each graph of n nodes is represented by an n x n matrix A,
that is, a two-dimensional array A

The nodes are (re)-labeled 1,2,…,n

A[i][j] = 1 if (i,j) is an edge

A[i][j] = 0 if (i,j) is not an edge

Algorithm to Implement Adjacency Matrix
● Define the number of vertices V in the graph
● Create a 2D array adjMatrix of size V x V.
● Initialize all elements of adjMatrix to 0.
● Define a function addEdge(u, v) to add an edge between two vertices:

Check if u and v are valid vertices (0 <= u, v < V).

If valid, set adjMatrix[u][v] = 1

Also set adjMatrix[v][u] = 1 as this is an undirected graph.

● Define a function printAdjMatrix() to print the adjacency matrix.

Iterate through the matrix using two for loops and print the value of the cells.
Example of Adjacency Matrix

Code: Click here

Pros and Cons of Adjacency Matrices
Pros:

● Simple to implement
● Easy and fast to tell if a pair (i,j) is an edge: simply check if A[i][j] is 1 or 0

Cons:

● No matter how few edges the graph has, the matrix takes O(n2) in memory
Adjacency Lists Representation
A graph of n nodes is represented by a one-dimensional array L of linked lists,
where

L[i] is the linked list containing all the nodes adjacent from node i.

The nodes in the list L[i] are in no particular order

Algorithm to Implement Adjacency List
Define the number of vertices V in the graph.

Create an array of vector adjList of size V.

Initialize each element of adjList to an empty list.

Define a function addEdge(u, v) to add an edge between two vertices:

Check if u and v are valid vertices (0 <= u, v < V).

If valid, add v to the vector of u.

Define a function printAdjList() to print the adjacency list:

Iterate through the array using a loop and print the vector for each vertex.
Example of Adjacency List Representation
L[0]: empty

L[1]: empty

L[2]: 0, 1, 4, 5

L[3]: 0, 1, 4, 5

L[4]: 0, 1

L[5]: 0, 1

Code Link: Click Here

Pros and Cons of Adjacency Lists
Pros:

● Saves on space (memory): the representation takes as many memory words

as there are nodes and edge.

Cons:

● It can take up to O(n) time to determine if a pair of nodes (i,j) is an edge: one
would have to search the linked list L[i], which takes time proportional to the
length of L[i].
Graph Traversal Techniques
The previous connectivity problem, as well as many other graph problems, can be
solved using graph traversal techniques

There are two standard graph traversal techniques:

- Depth-First Search (DFS)

- Breadth-First Search (BFS)
Graph Traversal (Contd.)
In both DFS and BFS, the nodes of the undirected graph are visited in a
systematic manner so that every node is visited exactly one.

Both BFS and DFS give rise to a tree:

When a node x is visited, it is labeled as visited, and it is added to the tree

If the traversal got to node x from node y, y is viewed as the parent of x, and x a
child of y
Depth-First Search

DFS follows the following rules:

1. Select an unvisited node x, visit it, and treat as the current node
2. Find an unvisited neighbor of the current node, visit it, and make it the new
current node;
3. If the current node has no unvisited neighbors, backtrack to the its parent, and
make that parent the new current node;
4. Repeat steps 3 and 4 until no more nodes can be visited.
5. If there are still unvisited nodes, repeat from step 1.
Illustration of DFS
Implementation of DFS
Observations:

● The last node visited is the first node from which to proceed.
● Also, the backtracking proceeds on the basis of "last visited, first to backtrack
too".
● This suggests that a stack is the proper data structure to remember the
current node and how to backtrack.

Code Link: Click here

Breadth-First Search
BFS follows the following rules:

1. Select an unvisited node x, visit it, have it be the root in a BFS tree being
formed. Its level is called the current level.
2. From each node z in the current level, in the order in which the level nodes
were visited, visit all the unvisited neighbors of z. The newly visited nodes
from this level form a new level that becomes the next current level.
3. Repeat step 2 until no more nodes can be visited.
4. If there are still unvisited nodes, repeat from Step 1.
Illustration of BFS
Implementation of BFS
Observations:

The first node visited in each level is the first node from which to proceed to visit
new nodes.

This suggests that a queue is the proper data structure to remember the order of
the steps.

Code Link: Click Here

Problem solving
Shortest path finding of an undirected graph: problem link

Topological sorting: problem link

Problem Solving
Component Count: A component in a graph is a set of vertices where each of them
are neighbors and none of them is neighbor of vertices outside of this set.

Count connected components in a graph.

Input: n = 5 and edges = [[0, 1], [1, 2], [3, 4]]

0 3

| |

1 --- 2 4

Output: 2

Problem link: Click Here

Solution: Click Here

Problem Solving
M-Coloring Problem: Given an undirected graph and a number m, the task is to
color the given graph with at most m colors such that no two adjacent vertices of
the graph are colored with the same color.
Problem Solving
M-Coloring Problem:

Input: graph = 0 1 1 1

1 0 1 0

1 1 0 1

1 0 1 0

Output: Solution Exists: Following are the assigned colors: 1 2 3 2

Explanation: By coloring the vertices with following colors,

adjacent vertices does not have same colors

Code Link: Click here

Dynamic Programming
Dynamic Programming
Dynamic Programming is an algorithm design technique for optimization problems: often
minimizing or maximizing.

Like divide and conquer, DP solves problems by combining solutions to subproblems.

It is mainly an optimization over plain recursion. Wherever we see a recursive solution that
has repeated calls for the same inputs, we can optimize it using Dynamic Programming.

The idea is to simply store the results of subproblems so that we do not have to re-compute
them when needed later. This simple optimization typically reduces time complexities from
exponential to polynomial.
When to Use Dynamic Programming (DP)?
Optimal Substructure:

The property Optimal substructure means that we use the optimal results of
subproblems to achieve the optimal result of the bigger problem.

Overlapping Subproblems:

The same subproblems are solved repeatedly in different parts of the problem
refer to Overlapping Subproblems Property in Dynamic Programming.
Approaches of Dynamic Programming (DP)
Dynamic programming can be achieved using two approaches:

1. Top-Down Approach (Memoization):

In the top-down approach, also known as memoization, we keep the solution

recursive and add a memoization table to avoid repeated calls of same
subproblems.

Before making any recursive call, we first check if the memoization table already
has solution for it.

After the recursive call is over, we store the solution in the memoization table.
Approaches of Dynamic Programming (DP)
2. Bottom-Up Approach (Tabulation):

In the bottom-up approach, also known as tabulation, we start with the smallest
subproblems and gradually build up to the final solution.

We write an iterative solution (avoid recursion overhead) and build the solution in
bottom-up manner.

We use a dp table where we first fill the solution for base cases and then fill the
remaining entries of the table using recursive formula.

We only use recursive formula on table entries and do not make recursive calls.
Fibonacci number using Recursion
Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34,55, …

One of the most basic, classic examples of this process is the fibonacci sequence.
It's recursive formulation is:

f(n) = f(n-1)+f(n-2) where n>=2, f(0)=0 and f(1) = 1

In C++ the code will be like this:

int f(int n) {
if (n == 0) return 0;
if (n == 1) return 1;
return f(n - 1) + f(n - 2);
}

Code link: click here

Fibonacci number using Recursion
The recursion tree of the above recursive solution.

The time complexity of the above approach is exponential and upper bounded by O(2n) as
we make two recursive calls in every function.
Fibonacci number using Dynamic Programming
We can clearly see that that recursive solution is doing a lot work again and again which is
causing the time complexity to be exponential.
To overcome the problem of recursive solution we use DP by following the mechanism given
below:
Identify Subproblems: Divide the main problem into smaller, independent subproblems,
i.e., F(n-1) and F(n-2)
Store Solutions: Solve each subproblem and store the solution in a table or array so that
we do not have to recompute the same again.
Build Up Solutions: Use the stored solutions to build up the solution to the main problem.
For F(n), look up F(n-1) and F(n-2) in the table and add them.
Avoid Recomputation: By storing solutions, DP ensures that each subproblem (for
example, F(2)) is solved only once, reducing computation time.
Fibonacci number using Dynamic Programming
Using Memoization Approach – O(n) Time and O(n) Space:
To accomplish this in our example, we use a memoization array initialized with -1. Before
making a recursive call, we check whether the corresponding position in the memo array
contains -1. If it does, it means the value hasn’t been computed yet, so we calculate it
recursively. Once computed, the result is stored in the memo array, allowing us to reuse it
directly if the same value is needed again later.

Code Link: Click Here

Memoization Table at Each Step
Fibonacci number using Dynamic Programming
Using Tabulation Approach – O(n) Time and O(n) Space:
In this method, we utilize an array of size n+1, typically referred to as dp[], to store Fibonacci
numbers. The array is initialized with base cases, such as dp[0] = 0 and dp[1] = 1. Using the
relation dp[i] = dp[i-1] + dp[i-2], we compute Fibonacci numbers iteratively from dp[2] to
dp[n]. This approach efficiently calculates Fibonacci numbers within a loop. Ultimately, the
value at dp[n] represents the Fibonacci number for the given input n, as each index in the
array corresponds to its respective Fibonacci number.

Code Link: Click Here

Iterative Table at Each Step