BRUTE FORCE APPROACHES

Module 2- Chapt 1

Mrs. Madhu Nagaraj

Assistant Professor
Dept of CSE-Data Science
ATMECE
Exhaustive Search
• Exhaustive search is simply a brute-force approach to combinatorial problems.
• It suggests generating each and every element of the problem domain, selecting those of
them that satisfy all the constraints, and then finding a desired element.

Method:
• generate a list of all potential solutions to the problem in a systematic manner.

• evaluate potential solutions one by one, disqualifying infeasible ones and, for an
optimization problem, keeping track of the best one found so far

• when search ends, announce the solution(s) found.

Example: Traveling Salesman Problem (TSP) and Knapsack Problem.

Traveling Salesman Problem
• Given n cities with known distances between each pair, find the shortest tour that passes through all the
cities exactly once before returning to the starting city
• More formally: Find shortest Hamiltonian circuit in a weighted connected graph.
• A Hamiltonian circuit is defined as a cycle that passes through all the vertices of the graph exactly
once.
Example:
2
a b
5 3
8 4

c d
7
 Tour Cost
a→b→c→d→a 2+3+7+5 = 17
a→b→d→c→a 2+4+7+8 = 21
a→c→b→d→a 8+3+4+5 = 20
a→c→d→b→a 8+7+4+2 = 21
a→d→b→c→a 5+4+3+8 = 20
a→d→c→b→a 5+7+3+2 = 17

Efficiency: # tours = O(# permutations of b,c,d) = O(n!)

Advantages:
• Always gives optimal solution
• Straight forward approach
• Simple technique

Disadvantages
• Inefficient method ((n-1)! possible routes)
• Time consuming to find optimal route.
Knapsack Problem
Given n items:
weights: w1 w2 … wn
values: v1 v2 … vn
a knapsack of capacity W
Find most valuable subset of the items that fit into the knapsack.
Total no of subsetsfor n elements is 2n. So the exhaustive search leads to Ω(2n)

Example: Knapsack capacity W=16

item weight value
1 2 $20
2 5 $30
3 10 $50
4 5 $10
Knapsack: Exhaustive Search
Efficiency: how
Subset Total weight Total value
{1} 2 $20 many subsets?
{2} 5 $30
{3} 10 $50
{4} 5 $10
{1,2} 7 $50
{1,3} 12 $70
{1,4} 7 $30
{2,3} 15 $80
{2,4} 10 $40
{3,4} 15 $60
{1,2,3} 17 not feasible
{1,2,4} 12 $60
{1,3,4} 17 not feasible
{2,3,4} 20 not feasible
{1,2,3,4} 22 not feasible
DECREASE-AND-CONQUER
Module 2- Chapt 2

Mrs. Madhu Nagaraj

Assistant Professor
Dept of CSE-Data Science
ATMECE
Decrease-and-Conquer
1. Reduce problem instance to smaller instance of the same problem
2. Solve smaller instance
3. Extend solution of smaller instance to obtain solution to original instance
4. Can be implemented either top-down or bottom-up.
5. Also referred to as inductive or incremental approach

3 Types of Decrease and Conquer

• Decrease by a constant (usually by 1)
• insertion sort
• topological sorting
Insertion Sort
To sort array A[0..n-1], sort A[0..n-2] recursively and
then insert A[n-1] in its proper place among the sorted
A[0..n-2]

 Usually implemented bottom up (nonrecursively)

Example: Sort 6, 4, 1, 8, 5

6|4 1 8 5
4 6|1 8 5
1 4 6|8 5
1 4 6 8|5
1 4 5 6 8
ALGORITHM InsertionSort(A[0..n − 1])
//Sorts a given array by insertion sort
//Input: An array A[0..n − 1] of n orderable elements
//Output: Array A[0..n − 1] sorted in nondecreasing order
for i ←1 to n − 1 do
v ←A[i]
j ←i − 1
while j ≥ 0 and A[j ]> v do
A[j + 1]←A[j ]
j ←j − 1
A[j + 1]←v
Analysis of Insertion Sort
Time efficiency
Cworst(n) = n(n-1)/2  Θ(n2)
Cavg(n) ≈ n2/4  Θ(n2)
Cbest(n) = n - 1  Θ(n) (also fast on almost sorted arrays)

Space efficiency: in-place

Stability: yes

Best elementary sorting algorithm overall

Binary insertion sort

Topological Sorting
A dag: a directed acyclic graph, i.e. a directed graph with no (directed) cycles

a b a b

a dag not a dag

c d c d
Arise in modeling many problems that involve prerequisite
constraints (construction projects, document version control)

Vertices of a dag can be linearly ordered so that for every edge its starting vertex is listed
before its ending vertex (topological sorting). Being a dag is also a necessary condition for
topological sorting be possible.
DFS-based algorithm for topological sorting
– Perform DFS traversal, noting the order vertices are popped off the traversal stack
– Reverse order solves topological sorting problem
– Back edges encountered?→ NOT a dag!

Example:
a b c d

e f g h
DFS: abefgh
Source removal algorithm
Repeatedly identify and remove a source (a vertex with no incoming edges) and all the
edges incident to it until either no vertex is left (problem is solved) or there is no source
among remaining vertices (not a dag)
Example:

a b c d

e f g h

Efficiency: same as efficiency of the DFS-based algorithm

DIVIDE-AND-CONQUER
Module 2- Chapt 3

Mrs. Madhu Nagaraj

Assistant Professor
Dept of CSE-Data Science
ATMECE
Divide-and-Conquer
The most-well known algorithm design strategy:
1. Divide instance of problem into two or more smaller instances
2. Solve smaller instances recursively
3. Obtain solution to original (larger) instance by combining these solutions

Divide-and-Conquer Examples
 Sorting: mergesort and quicksort
 Binary tree traversals
 Multiplication of large integers
 Matrix multiplication: Strassen’s algorithm
Divide-and-Conquer Technique
a problem of size n

subproblem 1 subproblem 2
of size n/2 of size n/2

a solution to a solution to
subproblem 1 subproblem 2

a solution to
the original problem
Mergesort
 Split array A[0..n-1] in two about equal halves and make copies of each half in arrays
B and C
 Sort arrays B and C recursively
 Merge sorted arrays B and C into array A as follows:
• Repeat the following until no elements remain in one of the arrays:
– compare the first elements in the remaining unprocessed portions of the arrays
– copy the smaller of the two into A, while incrementing the index indicating
the unprocessed portion of that array
• Once all elements in one of the arrays are processed, copy the remaining
unprocessed elements from the other array into A.
Pseudocode of Mergesort
ALGORITHM Mergesort(A[0..n − 1])
//Sorts array A[0..n − 1] by recursive mergesort
//Input: An array A[0..n − 1] of orderable elements
//Output: Array A[0..n − 1] sorted in nondecreasing order
if n > 1
copy A[0..[n/2] − 1] to B[0..[n/2] − 1]
copy A[[n/2]..n − 1] to C[0..[n/2] − 1]
Mergesort(B[0..[n/2] − 1])
Mergesort(C[0..[n/2] − 1])
Merge(B, C, A)
Pseudocode of Merge
ALGORITHM Merge(B[0..p − 1], C[0..q − 1], A[0..p + q − 1])
//Merges two sorted arrays into one sorted array
//Input: Arrays B[0..p − 1] and C[0..q − 1] both sorted
//Output: Sorted array A[0..p + q − 1] of the elements of B and C
i ←0; j ←0; k←0
while i <p and j <q do
if B[i]≤ C[j ]
A[k]←B[i]; i ←i + 1
else A[k]←C[j ]; j ←j + 1
k←k + 1
if i = p
copy C[j..q − 1] to A[k..p + q − 1]
else copy B[i..p − 1] to A[k..p + q − 1]
Analysis of Mergesort
 Number of key comparisons – recurrence:

C(n) = 2C(n/2) + C’(n) = 2C(n/2) + n-1

For merge sort – worst case:
C’(n) = n-1

All cases have same efficiency: Θ(n log n)

Number of comparisons in the worst case is close to theoretical minimum for comparison-
based sorting:
- Theoretical min: log2 n! ≈ n log2 n - 1.44n
General Divide-and-Conquer Recurrence

T(n) = aT(n/b) + f (n) where f(n)  (nd), d  0

Master Theorem: If a < bd, T(n)  (nd)

If a = bd, T(n)  (nd log n)
If a > bd, T(n)  (nlog b a )

Using log – Take log b of both sides:

Case 1: logb a < d, T(n)  (nd)
Case 2: logb a = d, T(n)  (nd logb n)
Case 3: logb a > d, T(n)  (nlog b a )
Mergesort Example
8 3 2 9 7 1 5 4

8 3 2 9 7 1 5 4

8 3 2 9 71 5 4

8 3 2 9 7 1 5 4

3 8 2 9 1 7 4 5

2 3 8 9 1 4 5 7

1 2 3 4 5 7 8 9
Quicksort
 Select a pivot (partitioning element) – here, the first element
 Rearrange the list so that all the elements in the first s positions are smaller than or
equal to the pivot and all the elements in the remaining n-s positions are larger than or
equal to the pivot (see next slide for an algorithm)

A[i]p A[i]p
 Exchange the pivot with the last element in the first (i.e., ) subarray — the pivot is
now in its final position
 Sort the two subarrays recursively
ALGORITHM Quicksort(A[l..r])
//Sorts a subarray by quicksort
//Input: Subarray of array A[0..n − 1], defined by its left and right
// indices l and r
//Output: Subarray A[l..r] sorted in nondecreasing order
if l < r
s ←Partition(A[l..r]) //s is a split position
Quicksort(A[l..s − 1])
Quicksort(A[s + 1..r])

Quicksort Example

5 3 1 9 8 2 4 7
Analysis of Quicksort
Recurrences:
• Best case: T(n) = T(n/2) + Θ (n)
• Worst case: T(n) = T(n-1) + Θ (n)

Performance:
• Best case: split in the middle — Θ(n log n)
• Worst case: sorted array! — Θ(n2)
• Average case: random arrays — Θ(n log n)
Analysis of Quicksort
• Problems:
– Duplicate elements

• Improvements:
– better pivot selection: median of three partitioning
– switch to insertion sort on small sub-arrays
– elimination of recursion – How?
These combine to 20-25% improvement

• Improvements: Dual Pivot

• Method of choice for internal sorting of large files (n ≥ 10000)
Binary Tree Traversals
Binary tree is a divide-and-conquer ready structure!

Ex. 1: Classic traversals (preorder, inorder, postorder)

Algorithm Inorder(T)
if T  
Inorder(Tleft)
print(root of T)
Inorder(Tright) 

Efficiency: Θ(n)
Binary Tree Algorithms (cont.)
Ex. 2: let us consider a recursive algorithm for computing the height of a binary tree.
Can be computed as the maximum of the heights of the root’s left and right subtrees plus 1.
ALGORITHM Height(T )
//Computes recursively the height of a binary tree
//Input: A binary tree T
//Output: The height of T
if T = ∅ return −1
TL TR
else return max{Height(Tlef t), Height(Tr ight)} + 1

Efficiency: Θ(n)
Write the binary tree traversals for the below tree
 Multiplication of Large Integers
• Some applications like modern cryptography require manipulation of integers that are
over 100 decimal digits long.
• such integers are too long to fit in a single word of a modern computer, they require
special treatment.
• In the conventional pen-and-pencil algorithm for multiplying two n-digit integers, each
of the n digits of the first number is multiplied by each of the n digits of the second
number for the total of n2 digit multiplications.
• The divide-and-conquer method does the above multiplication in less than n2 digit
multiplications.
 Multiplication of Large Integers
Consider the problem of multiplying two (large) n-digit integers represented by arrays of their
digits such as:

A = 12345678901357986429 B = 87654321284820912836

The grade-school algorithm:

a1 a2 … an
b1 b2 … bn
(d10) d11d12 … d1n
(d20) d21d22 … d2n
…………………
(dn0) dn1dn2 … dnn
First Divide-and-Conquer Algorithm
A small example: A  B where A = 2135 and B = 4014
A = (21·102 + 35), B = (40 ·102 + 14)
So, A  B = (21 ·102 + 35)  (40 ·102 + 14)
= 21  40 ·104 + (21  14 + 35  40) ·102 + 35  14

In general, if A = A1A2 and B = B1B2 (where A and B are n-digit, and A1, A2, B1, B2 are n / 2-
digit numbers),
A  B = A1  B1·10n + (A1  B2 + A2  B1) ·10n/2 + A2  B2

Recurrence for the number of one-digit multiplications M(n):

M(n) = 4M(n/2), M(1) = 1
Solution: M(n) = n2
Second Divide-and-Conquer Algorithm
A  B = A1  B1·10n + (A1  B2 + A2  B1) ·10n/2 + A2  B2

The idea is to decrease the number of multiplications from 4 to 3:

(A1 + A2 )  (B1 + B2 ) = A1  B1 + (A1  B2 + A2  B1) + A2  B2,

I.e., (A1  B2 + A2  B1) = (A1 + A2 )  (B1 + B2 ) - A1  B1 - A2  B2,

which requires only 3 multiplications at the expense of (4-1=3) extra additions and
subtractions (2 adds and 2 subs – 1 add)

Recurrence for the number of multiplications M(n):

M(n) = 3M(n/2), M(1) = 1
Solution: M(n) = 3log 2n = nlog 23 ≈ n1.585
Example of Large-Integer Multiplication

2135  4014
Strassen’s Matrix Multiplication
Strassen observed [1969] that the product of two matrices can be computed as
follows:

C00 C01 A00 A01 B00 B01

= *
C10 C11 A10 A11 B10 B11

M1 + M4 - M5 + M7 M3 + M5
=
M2 + M4 M1 + M3 - M2 + M6

 Example: M2 + M4 = (A10 + A11)  B00 + A11  (B10 - B00)

 =A B +A B
Formulas for Strassen’s Algorithm
M1 = (A00 + A11)  (B00 + B11)

M2 = (A10 + A11)  B00

M3 = A00  (B01 - B11)

M4 = A11  (B10 - B00)

M5 = (A00 + A01)  B11

M6 = (A10 - A00)  (B00 + B01)

M7 = (A01 - A11)  (B10 + B11)
Analysis of Strassen’s Algorithm
If n is not a power of 2, matrices can be padded with zeros.

Number of multiplications:
M(n) = 7M(n/2), M(1) = 1
Solution: M(n) = 7log 2n = nlog 27 ≈ n2.807 vs. n3 of brute-force alg.

Algorithms with better asymptotic efficiency are known but they

are even more complex.