Week 5

Saksham Joshi

Presidential Graduate School, Kathmandu

MTH150: Discrete Mathematics

Professor Rayamajhi

April 7, 2024
 Probabilistic Algorithms

We know what an algorithm is. It is the step by step procedure to solve the given task or a

problem. We can also be clear that the probabilistic algorithms must also be some kind of algorithm. So

what is this probabilistic algorithm? Let’s understand probabilistic algorithm with an example. Let’s

say Sulav wants to cook Daal Bhat, and there is the simple recipe to cook Daal Bhat. But he is

different, he wants to cook Daal Bhat in a different way. There are certain ingredients that goes well

with the Daal, but he selects the ingredients based on his mood and want to see how it taste likes. This

is the randomness that took place into the decision-making process. He could have follow the recipe

that everyone follows but he made it in a different way. Although the result might not satisfied him, but

he can try again and again and until he meets the satisfaction. This is a simple example, this might not

be closely related to the topic but this somehow gives an idea to understand probabilistic algorithms.

With this we can understand that the probabilistic algorithms are the algorithms that make

random choices at one or more steps. They are the class of algorithms that makes use of randomness to

evaluate data, solve the issues and improve the performance as as per the need.

There are different types of probabilistic algorithms and they are:

1. Monte Carlo Algorithms

Monte-Carlo algorithms are computer algorithms that use repeated random sampling to

calculate their outcomes (GfG, 2023). Similarly, if the random algorithm occasionally returns an

incorrect result, it is a Monte-Carlo algorithm. Monte-Carlo algorithms and approaches are applied

when the current deterministic algorithm fails or when computing the solution to a given problem is not

feasible. Because Monte-Carlo methods are the most effective at repeatedly computing random

numbers, both mathematical and physical simulation systems can be solved using them. These Monte-

Carlo techniques are particularly helpful for materials, fluids, and biological systems that are

disordered. These methods are used in mathematics to compute definite integrals, which are given
complex boundary conditions for multidimensional integrals. Compared to previous strategies, this one

is the next one that takes risk analysis into account.

There isn't just one Monte Carlo method; rather, the phrase refers to a broad class of popular

techniques that follow a certain pattern.

• The domain's potential inputs are specified.

• Randomly create the inputs from the domain using a predetermined probability distribution.

• Execute a deterministic computation using these inputs.

• The results of each separate computation can be combined to obtain the ultimate result.

Here are the two examples of Monte Carlo Algorithms:

i. Karger’s Algorithm

An algorithm for determining a graph's minimum cut that is randomized is called Karger's

Minimum Cut (“Karger’s Minimum Cut Algorithm”, n.d.). It employs the Monte Carlo method,

therefore it should operate within a time limit and produce results with as little inaccuracy as possible.

It is possible to minimize the likelihood of an error by running the algorithm more than once.

Any two nodes in the network can be combined using the Karger's Algorithm to create a single

node known as a supernode. The edges that connect the other neighboring vertices can be joined to the

supernode, and the edge that separates the two nodes is contracted.

ii. The Metropolis-Hastings Algorithm

Markov Chain Monte Carlo’s (MCMC) fundamental concept is really straightforward(Navarro,

2023). In order to ensure that the stationary distribution of the Markov chain is, in fact, p(x), the idea is

to define a Markov chain over all conceivable x values. This means that, given n⟶∞, we can be

certain that xn~p(x). To do this, we will employ a Markov chain to create a series of x values,

designated as (x0, x1, x2,..., xn). The Metrolis-Hastings Algorithm is one of several alternative methods

for configuring a Markov chain with this feature.

The Las-Vegas algorithm is a randomized algorithm that consistently yields accurate results; the

only thing that varies from one to the next is the programs running time. Alternatively, the Las-Vegas

algorithm is randomized algorithm that consistently generates the right answer or alerts the user to a

failure (GfG, 2023). Conversely, a Las-Vegas algorithms assumes the risk associated with the

computational resources utilized, but not the result; that is, it produces the expected and accurate output

for the given issue.

Here are the two examples of Las-Vegas Algorithm:

iii. Quick-Sort Algorithm

The method of sorting Quick-sort uses a pivot element to split the array into two parts (GfG,

2023). The algorithm chooses the pivot element at random and sorts the two components recursively.

While Quick-sort's expected time complexity is O(n log n), its worst-case time complexity is O(n 2).

That being said, the worst-case scenario is improbable, and the algorithm works rather well.

iv. Cuckoo Hashing

By combining the concepts of relocation and multiple-choice, Cuckoo hashing ensures O(1) worst-case

lookup time (GfG, 2023a).

• Multiple-choice: We provide a key with two options for residing: h1(key) and h2(key).

• Relocation: It is possible for h1(key) and h2(key) to be distracted. The solution to this is to

mimic the Cuckoo bird, which, upon hatching, drives the other eggs and young birds out of the

nest. In a similar vein, updating a key in a cuckoo hashing table could cause an old key to shift.

Re-installing the older key now presents a challenge.

◦ There is no issue if the older key's alternate position is empty.

◦ If not, another key gets moved by the older key. This keeps on until the process discovers a

spot that is open or hits a loop. When there is a cycle, the entire data structure is "rehashed"

using fresh hash functions. It may take several retries before Cuckoo is successful.
 As long as the number of keys is maintained below half of the hash table's capacity, that is, as

long as the load factor is below 50%, insertion is predicted to occur O(1) (amortized) with high

probability, even when rehashing is taken into account.

Since deletion only necessitates looking at two places in the hash table, it is O(1) worst-case.

Conclusion

All in all, the probablistic algorithms are the algorithms that make random choices at one or

more steps. They are the class of algorithms that makes use of randomness to evaluate data, solve the

issues and improve the performance as as per the need. With the explanation of two-types of

probabilistic algorithms, with two different examples of each type such as Karger’s Algorithms and

Metropolis-Hastings algorithm comes under Monte-Carlo Algorithms and quick-sort and cuckoo

hashing algorithm comes under Las-Vegas algorithm.

GfG. (2023, January 11). Cuckoo Hashing Worst case O(1) Lookup. GeeksforGeeks.

https://www.geeksforgeeks.org/cuckoo-hashing/

GfG. (2023, June 1). Randomized Algorithms Set 2 (Classification and Applications). GeeksforGeeks.
https://www.geeksforgeeks.org/randomized-algorithms-set-2-classification-and-applications/
Karger’s minimum cut algorithm. (n.d.).
https://www.tutorialspoint.com/data_structures_algorithms/dsa_kargers_minimum_cut_algorithm.htm
Navarro, D. (2023, April 12). Notes from a data witch - The Metropolis-Hastings algorithm. Notes
From a Data Witch. https://blog.djnavarro.net/posts/2023-04-12_metropolis-hastings/
Rosen, K. (2019). Discrete mathematics and its applications (8th ed.). McGraw-Hill Higher Education.