ÇUKUROVA UNIVERSITY
FACULTY OF ENGINEERING
COMPUTER ENGINEERING DEPARTMENT
Lecture 7: Probabilistic (or Randomized) Algorithms
Example 1: Dataset Comparison
𝑋 = 𝑥1 𝑥2 … 𝑥𝑛 , 𝑥𝑖 ∈ {0,1}
𝑌 = 𝑦1 𝑦2 … 𝑦𝑛 , 𝑦𝑖 ∈ {0,1}
Question: 𝑋 =? 𝑌
Classical Communication Protocol:
Comparing the two datasets bit by bit:
• Communication overhead: 𝑂(𝑛)
• Comparison overhead: 𝑂(𝑛) => 𝑛 = 1016 𝑏𝑖𝑡𝑠 (𝐵) ≈ 1136 𝑇𝐵
𝑣(𝑋) = 𝑣𝑎𝑙𝑢𝑒(𝑋) = ∑ 𝑥𝑖 . 2𝑛−𝑖
1
П(𝑘) = {𝑝 ∈ ℙ | 𝑝 ≤ 𝑘}, 𝜋(𝑘) = |П(𝑘)|
Randomized Equal algorithm:
𝐼𝑛𝑝𝑢𝑡: 𝑋 = 𝑥1 𝑥2 … 𝑥𝑛 , 𝑌 = 𝑦1 𝑦2 … 𝑦𝑛 , 𝑥𝑖 𝑦𝑖 ∈ {0,1}
𝐴 𝐵
1. 𝐴 randomly chooses a prime number 𝑝 ∈ П(𝑛2 )
2. 𝐴 calculates the “fingerprint of 𝑋 => s = v (X) mod p.
a. Note: X is considered as a binary number
3. 𝐴 sends s and p to 𝐵
4. 𝐵 calculates the “fingerprint of Y => t = v (Y) mod p
5. 𝐵 compares whether s = t? Are the two fingerprints the same?
a. Yes: equal -> A,
b. No: unequal -> A
Comparison overhead:
The Equal algorithm is a randomized communication protocol for the data comparison example.
It drastically reduces the comparison effort. Comparison overhead was previously 𝑂(𝑛), now
the overhead is reduced to 5 𝑠𝑡𝑒𝑝𝑠 → 𝑂(1)
CEN 345 Algorithms 1 Assoc. Prof. Dr. Fatih ABUT
ÇUKUROVA UNIVERSITY
FACULTY OF ENGINEERING
COMPUTER ENGINEERING DEPARTMENT
Communication overhead:
0 ≤ 𝑝, 𝑠 ≤ 𝑛²
𝑚 ∈ ℕ → ⌈log 𝑚⌉ 𝐵𝑖𝑡𝑠
𝑙(𝑠, 𝑝) ≤ 2 ∗ ⌈log 𝑛2 ⌉ ≤ 4 ∗ ⌈log 𝑛⌉
For 𝑛 = 1016 ;
≤ 4 ∗ 16 ∗⌈log 10⌉
≤ 4 ∗ 16 ∗ 4
= 256 Bits
Instead of 1016 , we would only transmit 256 bits.
𝑃𝑟𝑜𝑏𝑒𝑞𝑢𝑎𝑙 [𝐴 | 𝐵]
𝑋 = 𝑌 => 𝑡 = 𝑠
𝑋 ≠ 𝑌 <= 𝑡 ≠ 𝑠
𝑃𝑟𝑜𝑏𝑒𝑞𝑢𝑎𝑙 ["𝑢𝑛𝑒𝑞𝑢𝑎𝑙" | 𝑋 = 𝑌]) = 0
Example:
𝑛=5
𝑋 = 10011 𝑣(𝑋) = 19
𝑌 = 10001 𝑣(𝑌) = 17
𝑓𝑜𝑟 𝑝 = 11 => s = 8, t = 6
𝑋 = 10011 𝑣(𝑋) = 19
𝑌 = 10001 𝑣(𝑌) = 17
𝑓𝑜𝑟 𝑝 = 2 => s = 1, t = 1
𝑝 = 2 is a bad witness for the inequality of X and Y
Question: How many bad witnesses are in П(𝑛2 )?
𝑠 = 𝑡, i.e. 𝑣(𝑋) = 𝑣(𝑋) 𝑚𝑜𝑑 (𝑝), 𝑎𝑙𝑡ℎ𝑜𝑢𝑔ℎ 𝑋 ≠ 𝑌.
П+ (𝑛2 , 𝑋, 𝑌) = {𝑝 ∈ ℙ | 𝑣(𝑋) ≠ 𝑣(𝑌)(𝑝), 𝑋 ≠ 𝑌}
П− (𝑛2 , 𝑋, 𝑌) = {𝑝 ∈ ℙ | 𝑣(𝑋) = 𝑣(𝑌)(𝑝), 𝑋 ≠ 𝑌}
CEN 345 Algorithms 2 Assoc. Prof. Dr. Fatih ABUT
ÇUKUROVA UNIVERSITY
FACULTY OF ENGINEERING
COMPUTER ENGINEERING DEPARTMENT
Establishing the relationship between bad witnesses and total witness candidates:
|П− (𝑛2 , 𝑋, 𝑌)| 𝟐 𝐥𝐧 𝒏
𝑃𝑟𝑜𝑏𝑒𝑞𝑢𝑎𝑙 [𝑋 = 𝑌 | 𝑋 ≠ 𝑌]) = =
|П(𝑛2 )| 𝒏
2 ln 1016
𝑓𝑜𝑟 𝑛 = 1016 => ≈ 0.7 ∗ 10−14
1016
Multiple rounds of algorithm to further reduce the error:
𝑙 = 10 => 0.7 ∗ 10−144
𝑂(𝑙 ∗ log 𝑛)
𝐿 ⊆ Σ ∗ , 𝑤 ∈ Σ ∗ : 𝑤 ∈? 𝐿
𝐷 = {(𝑥, 𝑦) ∈ {0,1}𝑛 𝑥{0,1}𝑛 | 𝑛 ∈ Ν0, , 𝑥 ≠ 𝑦}
Given 𝑤 ∈ {0,1}𝑛 𝑥{0,1}𝑛
1. 𝑃𝑟𝑜𝑏𝑒𝑞𝑢𝑎𝑙 ["𝑤 ∈ 𝐷" | 𝑤 ∉ 𝐷] = 0
2 ln 𝑛
2. 𝑃𝑟𝑜𝑏𝑒𝑞𝑢𝑎𝑙 ["𝑤 ∈ 𝐷" | 𝑤 ∈ 𝐷] ≥ 1 − => 𝑅𝑃(𝜀(𝑛)), 𝜀(𝑛) 𝑖𝑠 𝑡ℎ𝑒 𝑒𝑟𝑟𝑜𝑟 𝑏𝑜𝑢𝑛𝑑
𝑛
3. Overhead: 𝑂(log 𝑛)
Example 2: Triangle Graph
∆𝐺𝑟𝑎𝑝ℎ = {〈𝐺〉 | 𝐺 𝑢𝑛𝑑𝑖𝑟𝑒𝑐𝑡𝑒𝑑 𝑔𝑟𝑎𝑝ℎ 𝑡ℎ𝑎𝑡 𝑐𝑜𝑛𝑡𝑎𝑖𝑛𝑠 𝑎𝑡 𝑙𝑒𝑎𝑠𝑡 𝑜𝑛𝑒 𝑡𝑟𝑖𝑎𝑛𝑔𝑙𝑒}
CEN 345 Algorithms 3 Assoc. Prof. Dr. Fatih ABUT
ÇUKUROVA UNIVERSITY
FACULTY OF ENGINEERING
COMPUTER ENGINEERING DEPARTMENT
Randomized Triangle Graph Algorithm:
1. 𝑇 (𝑡𝑒𝑠𝑡𝑒𝑟) randomly chooses an edge {𝑎, 𝑏}
2. 𝑇 randomly chooses a node 𝑐 ≠ {𝑎, 𝑏}
3. Test: Do 𝑐 and {𝑎, 𝑏} form a triangle? -> Yes or No
𝑃𝑟𝑜𝑏𝑇 ["𝑦𝑒𝑠" | 𝐺 ∉ ∆𝐺𝑟𝑎𝑝ℎ] = 0 ➔ no false positive statement
𝐺 = (𝑉, 𝐸)
|𝑉| = 𝑛, |𝐸| = 𝑚
3 1
𝑃𝑟𝑜𝑏𝑇 ["𝑦𝑒𝑠 𝑖𝑛 𝑜𝑛𝑒 𝑟𝑜𝑢𝑛𝑑" | 𝐺 ∈ ∆𝐺𝑟𝑎𝑝ℎ] ≥ ∗
𝑚 𝑛−2
3
𝑃𝑟𝑜𝑏𝑇 ["𝑛𝑜 𝑖𝑛 𝑜𝑛𝑒 𝑟𝑜𝑢𝑛𝑑" | 𝐺 ∈ ∆𝐺𝑟𝑎𝑝ℎ] ≤ 1 −
𝑚(𝑛 − 2)
𝑙
3
𝑃𝑟𝑜𝑏𝑇 ["𝑛𝑜 𝑖𝑛 𝑙 𝑟𝑜𝑢𝑛𝑑𝑠" | 𝐺 ∈ ∆𝐺𝑟𝑎𝑝ℎ] ≤ (1 − )
𝑚(𝑛 − 2)
𝑙
3
𝑃𝑟𝑜𝑏𝑇 ["𝑦𝑒𝑠 𝑖𝑛 𝑙 𝑟𝑜𝑢𝑛𝑑𝑠" | 𝐺 ∈ ∆𝐺𝑟𝑎𝑝ℎ] ≥ 1 − (1 − )
𝑚(𝑛 − 2)
1 𝑘 𝑘→∞ 1
(1 + ) → 𝑒 = 2.7182 …
𝑘
More general formulation with x instead of the special case with 1:
𝑥 𝑘 𝑘→∞
(1 + ) → 𝑒 𝑥
𝑘
Place a minus before x:
−𝑥 𝑘 𝑘→∞ −𝑥
(1 + ) → 𝑒
𝑘
We want to apply the Euler sequence for our probabilistic ∆𝐺𝑟𝑎𝑝ℎ algorithm:
𝑙→∞
𝑘 = 𝑙, 𝑥 = 𝑦𝑙 ➔ (1 − 𝑦)𝑙 → 𝑒 −𝑦𝑙
3 𝑚(𝑛−2)
𝑦 = 𝑚(𝑛−2) , 𝑙 = ➔𝑦∗𝑙 =1
3
𝑚(𝑛−2)
3 3 1 1 1
(1 − ) ≈ 𝑒 −1 ≈ ≈ <
𝑚(𝑛 − 2) 𝑒 2.7 2
CEN 345 Algorithms 4 Assoc. Prof. Dr. Fatih ABUT
ÇUKUROVA UNIVERSITY
FACULTY OF ENGINEERING
COMPUTER ENGINEERING DEPARTMENT
RP
1. 𝑃𝑟𝑜𝑏𝑇 ["𝑦𝑒𝑠" | 𝐺 ∉ ∆𝐺𝑟𝑎𝑝ℎ] = 0
1
2. 𝑃𝑟𝑜𝑏𝑇 ["𝑦𝑒𝑠" | 𝐺 ∈ ∆𝐺𝑟𝑎𝑝ℎ] ≥ 2
3. 𝑝𝑜𝑙𝑦𝑛𝑜𝑚𝑖𝑒𝑙
Example 3: Probabilistic Prime Number Tests
Little Fermat’s Theorem:
𝑝 ∈ ℙ, 𝑎 ∈ ℕ, (𝑎, 𝑝) = 1 => 𝑎𝑝−1 = 1(𝑝)
Question: 𝑛 ∈ ℕ, 𝑎 ∈ ℕ 𝑤𝑖𝑡ℎ 𝑎𝑛−1 = 1(𝑛) => 𝑛 ∈? ℙ
2𝑛−1 = 1(𝑛)
𝑛 = 3, 4, 5 …
n=341 – the number is not prime, but it pretends to be prime
Smallest pseudoprime number to base 2
Definition: Let 𝑚 be a composite number with (𝑎, 𝑚) = 1 and 𝑎𝑚−1 = 1(𝑚) or 𝑎𝑚 = 𝑎(𝑚),
then 𝑚 is called pseudoprime to base 𝑎
n=341 – the number is not prime, but it pretends to be prime
• n=341 is smallest pseudoprime number to base 2
• n=341 is not a pseudoprime to base 3
In other words: Pseudoprime numbers satisfy Fermat's little theorem even though they are not
prime
Existence of composite numbers that are pseudoprime to all (coprime) bases:
A composite number is called a Charmichael number iff 𝑎𝑚−1 = 1(𝑚)
or 𝑎𝑚 = 1(𝑚) applies to all bases with (𝑎, 𝑚) = 1
• m=561 is the smallest Carmichael number
• Carmichael numbers are free of squares
• Factoring Carmichael numbers contains at least 3 different prime factors
• There are infinitely many Carmichael numbers: 561, 1105, 1729, 2465, 2821, ...
CEN 345 Algorithms 5 Assoc. Prof. Dr. Fatih ABUT
ÇUKUROVA UNIVERSITY
FACULTY OF ENGINEERING
COMPUTER ENGINEERING DEPARTMENT
Theorem: For m ∈ ℕ, m ≥ 3, let
𝐹𝑚 = {𝑎 ∈ ℤ𝑚 | 𝑎𝑚−1 = 1(𝑚)}
be the set of bases for which m passes the Fermat test.
If 𝑚 is not a prime number, then 𝐹𝑚 contains the bases that “fool” the Fermat test.
Let m ∈ ℕ, m ≥ 3, be a composite and not a Carmichael number, then the following applies:
ℤ𝑚
|𝐹𝑚 | ≤
2
alg notPrime (𝒌 ∈ 𝑼+ , 𝒌 ≥ 𝟑)
Randomly pick an 𝒂 ∈ (𝟏, … , 𝒌 − 𝟏) with (𝒂, 𝒌) = 𝟏
If 𝒂𝒌−𝟏 ≠ 𝟏(𝒌)
then Output: 𝒌 𝒊𝒔 𝒏𝒐𝒕 𝒑𝒓𝒊𝒎𝒆
otherwise Output: 𝒌 𝒑𝒓𝒊𝒎𝒆?
endalg
ℙ = COMPOSITES ∈ RPP
̅|𝑘 ∉ℙ
(1) 𝑃𝑟𝑜𝑏[𝑘 ∈ ℙ ̅] = 0
̅] ≤ 1
̅|𝑘 ∈ℙ
(2) 𝑃𝑟𝑜𝑏[𝑘 ∉ ℙ 2
(3) notPrime is polynomial
1
The probability of error is therefore at most 2. If the algorithm is now carried out for l rounds
in which the base a is chosen anew at random and independently, then the probability of error
1
is at most 2𝑙; so, it can be made as small as you want.
1
➔ Executing the algorithm 𝑙 times leads to an error probability of ≤ 2𝑙
CEN 345 Algorithms 6 Assoc. Prof. Dr. Fatih ABUT
ÇUKUROVA UNIVERSITY
FACULTY OF ENGINEERING
COMPUTER ENGINEERING DEPARTMENT
Miller Rabin Algorithm and b-Sequences
ℤ𝑚 𝑖𝑠 𝑎 𝑓𝑖𝑒𝑙𝑑 iff 𝑚 ∈ ℙ
𝑥2 = 1
ℤ𝑚 : 𝑥 = 1, 𝑥 = −1 => 𝑡𝑟𝑖𝑣𝑖𝑎𝑙 𝑠𝑜𝑙𝑢𝑡𝑖𝑜𝑛s
𝑥 ∈ ℤ𝑚 is called a non-trivial square root of 1 modulo m if 𝑥 2 = 1 and 𝑥 ≠ 1 and 𝑥 ≠ −1.
e.g., 𝑚 = 15: 𝑥 = 4, 𝑥 = −4 => 𝑛𝑜𝑛 − 𝑡𝑟𝑖𝑣𝑖𝑎𝑙 𝑠𝑜𝑙𝑢𝑡𝑖𝑜𝑛s
In ℤ𝑚 , 𝑥 2 has the solution 𝑥 = ±1 iff 𝑚 ∈ ℙ
In other words: If there is a nontrivial square root modulo 𝑚, then m is a composite number.
𝑚∈ℕ
𝑠 = 𝑚𝑎𝑥{ 𝑟 ∈ ℕ | 2𝑟 𝑚 − 1}
𝑚−1
𝑑= 2𝑠
𝑏 ∈ ℤ𝑚 : 𝑏 − 𝑠𝑒𝑞𝑢𝑒𝑛𝑐𝑒
0 1 2 𝑠−1 𝑑 𝑠
〈𝑏 2 𝑑 , 𝑏 2 𝑑 , 𝑏 2 𝑑 , … , 𝑏 2 , 𝑏 2 𝑑 〉 𝑚𝑜𝑑 (𝑚)
If 𝑚 ∈ ℙ:
𝑠𝑑 𝑠 ∗𝑚−1
𝑏2 = 𝑏2 2𝑠 = 𝑏 𝑚−1 = 1(𝑚)
𝑝−1
Theorem: 𝐿𝑒𝑡 𝑏𝑒 𝑝 ∈ ℙ, 𝑠 = 𝑚𝑎𝑥{𝑟 | 2𝑟 𝑝 − 1}, 𝑑 = , 𝑏 ∈ ℕ 𝑤𝑖𝑡ℎ (𝑏, 𝑝) = 1
2𝑠
Then (1) 𝑏 𝑑 = 1 (𝑝) or
𝑟𝑑
(2) ∃𝑟 ∈ {0, 1, … , 𝑠 − 1}: 𝑏 2 = −1 (𝑝)
If 𝑝 ∈ ℙ, then the b-sequence has one of the following forms:
(1) 〈1, 1, … ,1〉
(2) 〈−1, 1, … , 1〉
(3) 〈?, ?, … , ? , −1, 1, … , 1〉
Last element of the b-sequence: 𝑏 𝑝−1 = 1 (𝑝)
CEN 345 Algorithms 7 Assoc. Prof. Dr. Fatih ABUT
ÇUKUROVA UNIVERSITY
FACULTY OF ENGINEERING
COMPUTER ENGINEERING DEPARTMENT
Use the reverse of the b-sequence:
𝑚 ∈ ℕ, 𝑏 ∈ ℤ𝑚 , 𝑤𝑖𝑡ℎ 𝑏 − 𝑠𝑒𝑞𝑢𝑒𝑛𝑐𝑒
(1) 〈?, ? , … ,1,1, … ,1〉
(2) 〈? , ? , … , ? , −1〉
(3) 〈?, ?, … , ? , ? 〉
➔𝑚∉ℙ
Definition: 𝑚 ∈ 𝑈+ , 𝑚 ≥ 3, 𝑚 − 1 = 2𝑠 𝑑 𝑤𝑖𝑡ℎ 𝑑 ∈ 𝑈+ , 𝑏 ∈ ℤ𝑚
𝑟𝑑
If 𝑏 𝑑 = 1(𝑚) or 𝑏 2 = −1(𝑚) holds for an 𝑟 ∈ {0, 1, … , 𝑠 − 1}, then 𝑚 is called strong
pseudoprime to base 𝑏.
Theorem: 𝑚 ∈ 𝑈+ , 𝑚 ≥ 3, 𝑐𝑜𝑚𝑝𝑜𝑠𝑖𝑡𝑒, then the number of bases for which 𝑚 is strongly
𝒎−𝟏
pseudoprime is at most 𝟒
➔ Miller Rabin Algorithm
algorithm MILLER-RABIN(n ∈ U+, n ≥ 3)
Compute d and s with n − 1 = d · 2s and d positive uneven
Randomly pick an a ∈ {2, 3, . . . , n − 2}
b := ad(n)
if b = 1 (n) or b = −1 (n): Output: n is prime?
for r := 1 to s − 1 do
b := b2 (n)
if b = −1 (n): Output: n is prime?
if b = 1 (n): Output: n is not prime
endfor
Output: n is prime?
endalgorithm MILLER-RABIN
CEN 345 Algorithms 8 Assoc. Prof. Dr. Fatih ABUT
ÇUKUROVA UNIVERSITY
FACULTY OF ENGINEERING
COMPUTER ENGINEERING DEPARTMENT
Miller Rabin Algorithm ∈ RPP
(1) 𝑃𝑟𝑜𝑏[𝑂𝑢𝑡𝑝𝑢𝑡: 𝑛 𝑖𝑠 𝑛𝑜𝑡 𝑝𝑟𝑖𝑚𝑒 | 𝑛 ∈ ℙ] = 0
1
(2) 𝑃𝑟𝑜𝑏[𝑂𝑢𝑡𝑝𝑢𝑡: 𝑛 𝑖𝑠 𝑝𝑟𝑖𝑚𝑒? | 𝑛 ∉ ℙ] ≤ 4
1
(3) 𝑃𝑟𝑜𝑏[𝑙 − 𝑡𝑖𝑚𝑒𝑠 𝑂𝑢𝑡𝑝𝑢𝑡: 𝑛 𝑖𝑠 𝑝𝑟𝑖𝑚𝑒? | 𝑛 ∉ ℙ] ≤
4𝑙
1
(4) 𝑃𝑟𝑜𝑏[𝑎𝑓𝑡𝑒𝑟 𝑙 𝑒𝑥𝑐𝑒𝑐𝑢𝑡𝑖𝑜𝑛𝑠 𝑂𝑢𝑡𝑝𝑢𝑡: 𝑛 𝑖𝑠 𝑛𝑜𝑡 𝑝𝑟𝑖𝑚𝑒 | 𝑛 ∉ ℙ] ≥ 1 −
4𝑙
(5) 𝑂(𝑙 ∗ (log 𝑛)) arithmetical operations or 𝑂(𝑙 ∗ (log 𝑛)³) bit operations
Outline:
2 ln 𝑛
𝐷𝑎𝑡𝑎𝑠𝑒𝑡 ∈ 𝑅𝑃 ( )
𝑛
1
∆𝐺𝑟𝑎𝑝ℎ ∈ 𝑅𝑃 ( )
2
1
𝑀𝑖𝑙𝑙𝑒𝑟 𝑅𝑎𝑏𝑖𝑛 ∈ 𝑅𝑃 ( )
4
Complexity Class RP:
Random/Probabilistic polynomial running algorithms with one-sided error
• No false positive statements.
• Conversely, RP algorithms make errors with a particular bound 𝜀.
• By repeating the execution of the algorithm, the total error can be reduced.
The algorithms that class RP defines are also called Monte Carlo algorithms. Such
algorithms allow a one-sided error.
CEN 345 Algorithms 9 Assoc. Prof. Dr. Fatih ABUT
