Lec4, Algorithm Analysis & Design, Randamized Algorithms
Lec4, Algorithm Analysis & Design, Randamized Algorithms
Randomized Algorithms
DR/ Hossam Hawash
2 Randomized Algorithms
Probability and randomness often serve as tools for algorithm design and
analysis, by making part of the algorithm behave randomly.
We call an algorithm randomized if its behavior is determined not only by its
input but also by values produced by a random-number generator.
Random
Variable
3 Randomized Algorithms
Input
0 1 2 3 4 5
6 5 9 12 3 4
6 QuickSort Algorithm (Example)
0 1 2 3 4 5
partition(arr,0,5) 6 5 9 12 3 4
pivot= ?
Partition Initialization...
7 QuickSort Algorithm (Example)
0 1 2 3 4 5
partition(arr,0,5) 6 5 9 12 3 4
pivot=6
Partition Initialization...
8 QuickSort Algorithm (Example)
0 1 2 3 4 5
partition(arr,0,5) 6 5 9 12 3 4
pivot=6
left
Partition Initialization...
9 QuickSort Algorithm (Example)
0 1 2 3 4 5
partition(arr,0,5) 6 5 9 12 3 4
pivot=6
left right
Partition Initialization...
10 QuickSort Algorithm (Example)
0 1 2 3 4 5
partition(arr,0,5) 6 5 9 12 3 4
pivot=6
left right
0 1 2 3 4 5
partition(arr,0,5) 6 5 9 12 3 4
pivot=6
left right
12 QuickSort Algorithm (Example)
0 1 2 3 4 5
partition(arr,0,5) 6 5 9 12 3 4
pivot=6
left right
0 1 2 3 4 5
partition(arr,0,5) 6 5 9 12 3 4
pivot=6
left right
14 QuickSort Algorithm (Example)
0 1 2 3 4 5
partition(arr,0,5) 6 5 9 12 3 4
pivot=6
left right
15 QuickSort Algorithm (Example)
0 1 2 3 4 5
partition(arr,0,5) 6 5 9 12 3 4
pivot=6
left right
0 1 2 3 4 5
partition(arr,0,5) 6 5 4 12 3 9
pivot=6
left right
0 1 2 3 4 5
partition(arr,0,5) 6 5 4 12 3 9
pivot=6
left right
0 1 2 3 4 5
partition(arr,0,5) 6 5 4 12 3 9
pivot=6
left right
19 QuickSort Algorithm (Example)
0 1 2 3 4 5
partition(arr,0,5) 6 5 4 12 3 9
pivot=6
left right
0 1 2 3 4 5
partition(arr,0,5) 6 5 4 12 3 9
pivot=6
left right
21 QuickSort Algorithm (Example)
0 1 2 3 4 5
partition(arr,0,5) 6 5 4 12 3 9
pivot=6
left right
0 1 2 3 4 5
partition(arr,0,5) 6 5 4 3 12 9
pivot=6
left right
0 1 2 3 4 5
partition(arr,0,5) 6 5 4 3 12 9
pivot=6
left right
0 1 2 3 4 5
partition(arr,0,5) 6 5 4 3 12 9
pivot=6
left right
0 1 2 3 4 5
partition(arr,0,5) 6 5 4 3 12 9
pivot=6
left
right
26 QuickSort Algorithm (Example)
0 1 2 3 4 5
partition(arr,0,5) 6 5 4 3 12 9
pivot=6
left
right
0 1 2 3 4 5
partition(arr,0,5) 6 5 4 3 12 9
pivot=6
left
right
0 1 2 3 4 5
partition(arr,0,5) 3 5 4 6 12 9
pivot=6
left
right
0 1 2 3 4 5
partition(arr,0,5) 3 5 4 6 12 9
pivot=6
left
right
return 3
30 QuickSort Algorithm (Example)
0 1 2 3 4 5
QuickSort(arr,0,5)
3 5 4 6 12 9
pivot
position
0 1 2 3 4 5
QuickSort(arr,0,5)
3 5 4 6 12 9
QuickSort(arr,0,3) QuickSort(arr,4,5)
0 1 2 3 4 5
3 5 4 6 12 9
32 QuickSort Algorithm (Example)
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
partition(arr,0,3)
0 1 2 3 4 5
3 5 4 6 12 9
33 QuickSort Algorithm (Example)
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
partition(arr,0,3)
0 1 2 3 4 5
3 5 4 6 12 9
Partition Initialization...
34 QuickSort Algorithm (Example)
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
partition(arr,0,3)
0 1 2 3 4 5
3 5 4 6 12 9
left
Partition Initialization...
35 QuickSort Algoritm
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
partition(arr,0,3)
0 1 2 3 4 5
3 5 4 6 12 9
left right
Partition Initialization...
36 QuickSort Algorithm (Example)
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
partition(arr,0,3)
0 1 2 3 4 5
3 5 4 6 12 9
left right
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
partition(arr,0,3)
0 1 2 3 4 5
3 5 4 6 12 9
left right
38 QuickSort Algorithm (Example)
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
partition(arr,0,3)
0 1 2 3 4 5
3 5 4 6 12 9
left right
39 QuickSort Algorithm (Example)
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
partition(arr,0,3)
0 1 2 3 4 5
3 5 4 6 12 9
left
right
40 QuickSort Algorithm (Example)
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
partition(arr,0,3)
0 1 2 3 4 5
3 5 4 6 12 9
left
right
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
partition(arr,0,3)
0 1 2 3 4 5
3 5 4 6 12 9
left
right
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
partition(arr,0,3)
0 1 2 3 4 5
3 5 4 6 12 9
right & left CROSS!!!
left
1 - Swap pivot and arr[right]
right 2 - Return new location of pivot to caller
return 0
43 QuickSort Algorithm (Example)
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
0 1 2 3 4 5
3 5 4 6 12 9
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
QuickSort(arr,0,0) QuickSort(arr,1,3) 4 5
0 1 2 3
12 9
3 5 4 6
45 QuickSort Algorithm (Example)
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
QuickSort(arr,0,0) QuickSort(arr,1,3) 4 5
0 1 2 3
12 9
3 5 4 6
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
QuickSort(arr,0,0) QuickSort(arr,1,3) 4 5
0 1 2 3
12 9
3 5 4 6
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
QuickSort(arr,0,0) QuickSort(arr,1,3) 4 5
partition(arr,1,3)
0 1 2 3
12 9
3 5 4 6
Partition Initialization...
48 QuickSort Algorithm (Example)
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
QuickSort(arr,0,0) QuickSort(arr,1,3) 4 5
partition(arr,1,3)
0 1 2 3
12 9
3 5 4 6
Partition Initialization...
49 QuickSort Algorithm (Example)
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
QuickSort(arr,0,0) QuickSort(arr,1,3) 4 5
partition(arr,1,3)
0 1 2 3
12 9
3 5 4 6
left
Partition Initialization...
50 QuickSort Algorithm (Example)
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
QuickSort(arr,0,0) QuickSort(arr,1,3) 4 5
partition(arr,1,3)
0 1 2 3
12 9
3 5 4 6
left right
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
QuickSort(arr,0,0) QuickSort(arr,1,3) 4 5
partition(arr,1,3)
0 1 2 3
12 9
3 5 4 6
left right
52 QuickSort Algorithm (Example)
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
QuickSort(arr,0,0) QuickSort(arr,1,3) 4 5
partition(arr,1,3)
0 1 2 3
12 9
3 5 4 6
left right
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
QuickSort(arr,0,0) QuickSort(arr,1,3) 4 5
partition(arr,1,3)
0 1 2 3
12 9
3 5 4 6
right
left
54 QuickSort Algorithm (Example)
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
QuickSort(arr,0,0) QuickSort(arr,1,3) 4 5
partition(arr,1,3)
0 1 2 3
12 9
3 5 4 6
right
left
QuickSort(arr,0,3) QuickSort(arr,4,5)
QuickSort(arr,0,0) QuickSort(arr,1,3) 4 5
partition(arr,1,3)
0 1 2 3
12 9
3 5 4 6
right
left
QuickSort(arr,0,3) QuickSort(arr,4,5)
QuickSort(arr,0,0) QuickSort(arr,1,3) 4 5
partition(arr,1,3)
0 1 2 3
12 9
3 4 5 6
right
left
QuickSort(arr,0,3) QuickSort(arr,4,5)
QuickSort(arr,0,0) QuickSort(arr,1,3) 4 5
partition(arr,1,3)
0 1 2 3
12 9
3 4 5 6
right
left
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
QuickSort(arr,0,0) QuickSort(arr,1,3) 4 5
0
12 9
3 QuickSort(arr,1,2) QuickSort(arr,3,3)
1 2 3
4 5 6
59 QuickSort Algorithm (Example)
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
QuickSort(arr,0,0) QuickSort(arr,1,3) 4 5
0
12 9
3 QuickSort(arr,1,2) QuickSort(arr,3,3)
partition(arr,1,2)
1 2 3
4 5 6
60 QuickSort Algorithm (Example)
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
QuickSort(arr,0,0) QuickSort(arr,1,3) 4 5
0
12 9
3 QuickSort(arr,1,2) QuickSort(arr,3,3)
1 2 3
4 5 6
61 QuickSort Algorithm (Example)
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
QuickSort(arr,0,0) QuickSort(arr,1,3) 4 5
0
12 9
3 QuickSort(arr,1,2) QuickSort(arr,3,3)
QuickSort(arr,1,1) QuickSort(arr,2,2)
1 2
4 5
62 QuickSort Algorithm (Example)
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
partition(arr,4,5)
QuickSort(arr,0,0) QuickSort(arr,1,3) 4 5
0
12 9
3 QuickSort(arr,1,2) QuickSort(arr,3,3)
QuickSort(arr,1,1) QuickSort(arr,2,2)
1 2
4 5
63 QuickSort Algorithm (Example)
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
partition(arr,4,5)
QuickSort(arr,0,0) QuickSort(arr,1,3) 4 5
0
9 12
3 QuickSort(arr,1,2) QuickSort(arr,3,3)
QuickSort(arr,1,1) QuickSort(arr,2,2)
1 2
4 5
64 QuickSort Algorithm (Example)
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
QuickSort(arr,6,5)
0 4 5
3 QuickSort(arr,1,2) QuickSort(arr,3,3)
9 12
QuickSort(arr,1,1) QuickSort(arr,2,2)
1 2
4 5
65 QuickSort Algorithm (Example)
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
QuickSort(arr,6,5)
3 QuickSort(arr,1,2) QuickSort(arr,3,3)
9 12
QuickSort(arr,1,1) QuickSort(arr,2,2)
1 2
4 5
66 QuickSort Algorithm (Example)
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
QuickSort(arr,6,5)
0 4 5
3 QuickSort(arr,1,2) QuickSort(arr,3,3)
9 12
QuickSort(arr,1,1) QuickSort(arr,2,2)
1 2
4 5
67 QuickSort Algorithm (Example)
QuickSort(arr,0,5)
QuickSort(arr,0,3) QuickSort(arr,4,5)
QuickSort(arr,6,5)
3 QuickSort(arr,1,2) QuickSort(arr,3,3)
3 QuickSort(arr,4,4)
4
6
9
QuickSort(arr,1,1) QuickSort(arr,2,2)
QuickSort(arr,5,5)
1 2 5
4 5 12
68 QuickSort Algorithm (Output)
Sorted Array
0 1 2 3 4 5
3 4 5 6 9 12
69 QuickSort Algorithm (Worst-case time)
𝑇 𝑛
= 𝑐𝑛 + 𝑐 𝑛 − 1 + 𝑐 𝑛 − 2 + … . . + 2𝑐
= 𝑐 𝑛 + 𝑛 − 1 + 𝑛 − 2 + …..+ 2
𝑛
= 𝑐 𝑛+1 − 1 = 𝑂(𝑛2 )
2
70 QuickSort Algorithm (Best-case time)
𝑇 𝑛 = 𝑐𝑛 ∗ log 𝑛 = 𝑂(𝑛𝑙𝑜𝑔 𝑛)
71 QuickSort Algorithm (Average-case
time)
Suppose that your worst enemy has given you an array to sort with
quicksort, knowing that you always choose the rightmost element in each
subarray as the pivot, and has arranged the array so that you always get
the worst-case split. How can you foil your enemy?
you could randomly choose an element in the subarray, and use that
element as the pivot.
The partition function assumes that the pivot is in the rightmost position of
the subarray.
No problem—just swap the element that you chose as the pivot with the
rightmost element, and then partition as before.
74 Expected average-case performance
of Randomized Quicksort
Random Pivot Selection: The probability of choosing any particular element as the pivot is
1/𝑛 (assuming a uniformly random selection).
1
𝑇(𝑛) = 𝑂(𝑛) + × (𝑇(𝑚) + 𝑇(𝑛 − 𝑚 − 1))
2
𝑚
Now, we have a simplified expected time complexity recurrence relation that captures the
essence of Randomized Quicksort.
Solving this recurrence relation typically leads to an expected time complexity of 𝑂 𝑛log 𝑛
when considering average-case performance.
76 Randomized Binary Search Algorithm
1
𝑇(𝑛) = 𝑂(1) + × (𝑇(𝑚) + 𝑇(𝑛 − 𝑚))
2
𝑚
Step 1:
0 1 2 3 4 5
6 5 9 12 3 4
Rand(0,4)
87 Knuth Shuffle
Step 1:
0 1 2 3 4 5
6 5 9 12 3 4
0 1 2 3 4 5
6 4 9 12 3 5
88 Knuth Shuffle
Step 2:
0 1 2 3 4 5
6 4 9 12 3 5
Rand(0,3)
89 Knuth Shuffle
0 1 2 3 4 5
6 4 9 12 3 5
Rand(0,3)
90 Freivald’s Algorithm
Input : 𝐴 = , B = [], C = []
Output : Is that C = A x B? True or False
Algorithmic Steps:
Generate a random vector 𝑥 of appropriate dimensions
Vector 𝑥 is consisting of 0/1 only..
Compute 𝑌 = 𝐴 ⋅ (𝐵 ⋅ 𝑥) − 𝐶 ⋅ 𝑥.
Return true if 𝑌? = 0, 0, … , 0 𝑇
, otherwise, return false.
92 Freivald’s Algorithm: An Example
1 3 2 0 5 4
Input : 𝐴 = ,B = ,C =
5 1 1 2 11 8
Generate a random vector 𝑥 of appropriate size. Let's say 𝑥 = [1,0]𝑇 .
1 3 2 0 1 1 3 2 5
𝐴× 𝐵×𝑥 = × × = × =
5 1 1 2 0 5 1 1 11
5 4 1 5
𝐶×𝑥 = × =
11 8 0 11
5 5 0
Now, subtract the two results:Y = 𝐴 × (𝐵 × 𝑥) − 𝐶 × 𝑥 = − = .
11 11 0
93 Probabilistic Analysis of Freivalds' Algorithm
Cormen, T. H., Leiserson, C. E., Rivest, R. L., & Stein, C. (2022). Introduction to
algorithms. MIT press.
Motwani, Rajeev, and Prabhakar Raghavan. Randomized algorithms.
Cambridge university press, 1995.
Donald, E. K. (1999). The art of computer programming. Sorting and
searching, 3(426-458), 4.
Freivalds, R. (1979, September). Fast probabilistic algorithms. In International
Symposium on Mathematical Foundations of Computer Science (pp. 57-
69). Berlin, Heidelberg: Springer Berlin Heidelberg.