spas Randomized germ. Wikipedia the ree encyclopedia Randomized algorithm From Wikipedia, the free encyclopedia A randomized algorithm is an algorithm which employs a degree of randomness as part of its logic. The algorithm typically uses uniformly random bits as an auxiliary input to guide its behavior, in the hope of achieving good performance in the “average case" over all possible choices of random bits. Formally, the algorithm's performance willbe a random variable determined by the random bits; thus either the running time, or the output (or both) are random variables, One has to distinguish between algorithms that use the random input to reduce the expected running time or ‘memory usage, but always terminate with a correct result (Las Vegas algorithms) in a bounded amount of time, and probabilistic algorithms, which, depending on the random input, have a chance of producing an incorrect result (Monte Carlo algorithms) or fail to produce a result either by signalling a failure or filing to terminate. Inthe second case, random performance and random output, the term "algorithm" for a procedure is somewhat questionable. In the case of random output, it is no longer formally effective.!"] However, in some cases, probabilistic algorithms are the only practical means of solving a problem?! In common practice, randomized algorithms are approximated using a pseudorandom number generator in place of a true sourve of random bits; such an implementation may deviate ftom the expected theoretical behavior. Contents = 1 Motivation = 2. Computational complexity = 3 History = 4 Applications = 4.1 Quicksort = 4.2 Randomized incremental constructions in geometry = 4.3 Verifying matrix multiplication = 4.4 Min cut = 5 Derandomization = 6 Where randommess helps = 7See also = 8 Notes = 9 References Motivation As a motivating example, consider the problem of finding an ‘a’ in an array of'n elements. Input: An array of n>2 clements, in which halfare ‘a’s and the other half are * orhipodiaorgiviliRardoized algo 1sons Randonized algerthm- Wkipoda the ee encyclopedia Output: Find an ‘a’ in the array. We give two versions of the algorithm, one Las Vegas algorithm and one Monte Carlo algorithm. Las Vegas algorithm: n) Randomly select one elerent out ef n elements. until ‘a! is found This algorithm succeeds with probability 1. The run time of.a single call varies and can be arbitrarily large, but the expected run time over many calls is (7). (See Big O notation) Monte Carlo algorithm: Randomly select one elenent out of n elenents. beet until i-k or ‘a! is found Ifan ‘a’ is found, the algorithm sueceeds, else the algorithm fails. Affer k iterations, the probability of finding an ‘a’ is: Prffind a] = 1 — (1/2)* This algorithm does not guarantee success, but the run time is fixed. The selection is executed exactly & times, therefore the run time is (1), Randomized algorithms are particularly useful when faced with a malicious "adversary" or attacker who deliberately tries to feed a bad input to the algorithm (see worst-case complexity and competitive analysis (online algorithm)) such as in the Prisoner's dilemma. It is for this reason that randomness is ubiquitous in cryptography. In cryptographic applications, pseudo-random numbers cannot be used, since the adversary can predict them, making the algorithm effectively deterministic. Therefore either a source of truly random numbers or a cryptographically secure pseudo-random number generator is required. Another area in which randommess is inherent is quantum computing, Inthe example above, the Las Vegas algorithm always outputs the correct answer, but its runing time is a random variable. The Monte Carlo algorithm (related to the Monte Carlo method for simulation) completes in a fixed amount of time (as a function of the input size), but allow a small probability of error. Observe that any Las ‘Vegas algorithm can be converted into a Monte Carlo algorithm (via Markov's inequality), by having it output an arbitrary, possibly incorrect answer ifit fails to complete within a specified time. Conversely, ifan efficient verification procedure exists to check whether an answer is correct, then a Monte Carlo algorithm can be converted into a Las Vegas algorithm by running the Monte Carlo algorithm repeatedly till a correct answer is obtained. orhipodiaorgiviliRardoized algo 29spas Randomized germ. Wikipedia the ree encyclopedia Computational complexity Computational complexity theory models randomized algorithms as probabilistic Turing machines. Both Las Vegas and Monte Carlo algorithms are considered, and several complexity classes are studied. The most basic randomized complexity class is RP, which is the class of decision problems for which there is an efficient (polynomial time) randomized algorithm (or probabilistic Turing machine) which recognizes NO-instances with absolute certainty and recognizes YES- instances with a probability of at least 1/2. The complement class for RP is co-RP. Problem classes having (possibly nonterminating) algorithms with polynomial time average case running time whose output is always correct are said to be in ZPP. The class of problems for which both YES and NO-instances are allowed to be identified with some error is called BPP. This class acts as the randomized equivalent of P, ic. BPP represents the class of efficient randomized algorithms. History Historically, the first randomized algorithm was a method developed by Michael ©. Rabin for the closest pair problem in computational geometry.1 The study of randomized algorithms was spurred by the 1977 discovery of a randomized primality test (ie., determining the primality ofa number) by Robert M. Solovay and Volker Strassen. Soon afterwards Michael O. Rabin demonstrated that the 1976 Miller's primality test can be turned into a randomized algorithm, At that time, no practical deterministic algorithm for primality was known, The Miller-Rabin primality test relies on a binary relation between two positive integers k and n that can be expressed by saying that k 'is a witness to the compositeness of" n. It can be shown that = Ifthere is a witness to the compositeness of n, then n is composite (i., » is not prime), and = If nis composite then at least three-fourths of the natural numbers less than m are witnesses to its compositeness, and = There is a fast algorithm that, given k and n, ascertains whether & is a witness to the compositeness of n. Observe that this implies that the primality problem is in Co-RP. Ifone randomly chooses 100 numbers less than a composite number n, then the probability of failing to find such a "witness" is (1/4)! so that for most practical purposes, this is a good primality test. Ifm is big, there may be no other test that is practical. The probability of error can be reduced to an arbitrary degree by performing enough independent tests, Therefore, in practice, there is no penalty associated with accepting a small probability of error, since with a little care the probability of error can be made astronomically small, Indeed, even though a deterministic polynomial-time primality test has since been found (see AKS primality test), it has not replaced the older probabilistic tests in cryptographic software nor is it expected to do so for the foreseeable future. Applications Quicksort orhipodiaorgiviliRardoized algo 38sata antorized doer Wipe te ron enelopeda Quicksort is a familiar, commonly used algorithm in which randomness can be usefull Any deterministic version of this algorithm requires O(n”) time to sort n numbers for some well-defined class of degenerate inputs (such as an already sorted array), with the specific class of inputs that generate this behavior defined by the protocol for pivot selection. However, ifthe algorithm selects pivot elements uniformly at random, it has a provably high probability of finishing in O(n log n) time regardless of the characteristics of the input. Randomized incremental constructions in geometry In computational geometry, a standard technique to build a structure like a convex hull or Delaunay triangulation is to randomly permute the input points and then insert them one by one into the existing structure. The randomization ensures that the expected number of changes to the structure caused by an insertion is small, and so the expected ‘running time of the algorithm can be upper bounded. This technique is known as randomized incremental construction (4) Verifying matrix multiplication Main article: Freivalds' algorithm Input: Matrix 4 © R"*?, BE RP*" and C © R"*" Output: True ifC =A - B; fae ifC#A-B ‘We give a Monte Carlo algorithm to solve the problem.{5) begin del repeat Choose compute c ifC rea endif until ink return TRUE end The running time of the algorithm is O(kn’). , . , ly Theorem: The algorithm is correct with probability at least | — (5) We will prove that if A B £ C'then Pr[A- B-r = C-r] < 1/2 fA. BF C, by definition we have D = A - B — C # 0. Without loss of generality, we assume that dy #0. On the other hand, Pr[A - B-r = C-r] = Pr{(A- B—C)-r =0]= Pr[D-r = 0} orhipodiaorgiviliRardoized algo 49sata Randomized algerthm- Wiipoda the oe eneelopedia If D) . - = Qo then the first entry of J) . pis 0, that is Yadyry = 0 jal Since dy, F 0), we cansove for M1: Ifwe fix al” except, the equality holds for at most one of the two choices for, € {0,1}. Therefore, Pr{ABr = Cr] < 1/2. We run the loop for k times. If’ = _4 . B, the algorithm is always correct; ifC’ # A - B, the probability of getting the correct answer is at kast | G jt Min cut Main article: Karger’s algorithm Input: A graph GV.) Memeoeoeeccecr BRARLEF FIN Output: A cut partitioning the vertices into L and R, with the minimum Figure 2: Successful run of Karger’s number of edges between L and R. algorithm on a 10-vertex graph. The minimum cut has size 3 and is Recall that the contraction of two nodes, w and v, in a (mult-)graph indicated by the vertex colours. yields a new node 1 ' with edges that are the union of the edges incident on either w or v, except fiom any edge(s) connecting u and v. Figure 1 gives an example of contraction of vertex A and B. Afier contraction, the resulting graph may have parallel edges, but contains no self loops. «(| 90 | Figure 1: Contraction of vertex A and B Karger's [4] basic algorithm: begin orhpodiaorgiviliRardorized algospas Randomized germ. Wikipedia the ree encyclopedia on repeat repeat Take @ random edge (u,v) B in G eplace u and y with the contraction ut watil only 2 nodes rena obtain the ding cut result C, wneil =n output the minimum eut among Cy/Cz¢--+/Cy end In each execution of the outer loop, the algorithm repeats the inner loop until only 2 nodes remain, the corresponding cutis obtained. The nun time of one execution is (7), and n denotes the number of vertices. After 1m times executions of the outer loop, we output the minimum cut among al the resuits. The figure 2 gives an example of one execution of the algorithm, Affer execution, we get a cut of size 3. Lemma 1: Let k be the min cut size, and let C= {¢1,¢2,-.4€g} be the min cut. If, during iteration i, no edge ¢ © C is selected for contraction, then C; = C. Proof: IfG is not connected, then G can be partitioned into L and R without any edge between them, So the min cut ina disconnected graph is 0. Now, assume G is connected, Let V=LU R be the partition of V induced by C : C={ {uv} €E:u € Lv ER) (well-defined since G is connected). Consider an edge {u,v} of C. Initially, u,v are distinct vertices. As long as we pick an edge f# ¢, u and v do not get merged. Thus, at the end of the algorithm, we have two compound nodes covering the entire graph, one consisting of the vertices of Z and the other consisting of the vertices of R. As in figure 2, the size of min cut is 1, and C= {(4,B)}. Ifwe don't select (4,B) for contraction, we can get the min cut Lemma 2: IfG is a multigraph with p vertices and whose min cut has size k, then G has at least pk/2 edges. Proof: Because the min cutis k, every vertex v must satisfy degree(v) > k. Therefore, the sum of the degree is at least pk. But itis well known that the sum of vertex degrees equals 2|Z]. The lemma follows. Analysis of algorithm The probability that the algorithm succeeds is 1 ~ the probability that all attempts fail. By independence, the probability that all attempts fail is Tec: #C)= qa —Pr(C; = C)). i=l By kmma 1, the probability that C; = C is the probability that no edge of C is selected during iteration i. Consider the inner loop and let G, denote the graph after j edge contractions, where j © {0,1,...98~3}. Gyhas n—j vertices. We use the chain rule of conditional possibilities. The probability that the edge chosen at iteration j is not in C, given that no edge of C has been chosen before, is 1 — jG) Note that G; still has min cut of size k, so i) by Lemma 2, it still has at least (= I)R edges 2 orhipodiaorgiviliRardoized algo aeen's Ranorizod Ager Wipe te ree enelopada >1 n-j-2 |IE(G)|~ n= G nj So by the chain rule, the ae offing tenet Ci Thus, 1 — n— (@ -3)n- 3,2. 1 PriG, = CO] 2 (=, ma synn EMR). Cancellation gives Pr[C; = C] > —? ss the probably tht the algorithm sueceeds at kas n(n—1) 7 n(n Form = 2 ~(1-—~_ 1) 1-(- Tara) In this is equivalent to — 4 the algorithm finds the n nin cut wit probabity 1 — 1 insime O(mn) = O(n* log n} : Derandomization Randomness can be viewed as a resource, like space and time, Derandomization is then the process of removing randomness (or using as litle of it as possibk). From the viewpoint of computational complexity, derandomizing an efficient randomized algorithm is the question, is P = BPP ? There are also specifie methods that can be employed to derandomize particular randomized algorithms: = the method of conditional probabilities, and its generalization, pessimistic estimators = discrepancy theory (which is used to derandomize geometric algorithms) = the exploitation of limited independence in the random variables used by the algorithm, such as the pairwise independence used in universal hashing = the use of expander graphs (or dispersers in general) to amplify a limited amount of initial randomness (this last approach is also referred to as generating pseudorandom bits from a random source, and leads to the related topic of pseudorandomness) ‘Where randomness helps ‘When the model of computation is restricted to Turing machines, itis currently an open question whether the ability to make random choices allows some problems to be solved in polynomial time that cannot be solved in polynomial time without this ability; this is the question of whether P = BPP. However, in other contexts, there are specific examples of problems where randomization yields strict improvements. * Based on the initial motivating example: given an exponentially long string of 2* characters, half.a's and half machine requires at least 24! lookups in the worst-case to find the index of an a; ifit is permitted to make random choices, it can solve this problem in an expected polynomial number of lookups. = Incommunication complexity, the equality of two strings can be verified using Jog: 7, bits of communication with a randomized protocol. Any deterministic protocol requires (12) bits. = The volume ofa convex body can be estimated by a randomized algorithm to arbitrary precision in polynomial time.{7! Barany and Fiiredi showed that no deterministic algorithm can do the same.!*! This is true unconditionally, ie. without relying on any complexity-theoretic assumptions. orkpodiaorgiviliRarderized algo 1 b’s, a random accesssata Randanizad algorithm Wikipedia. te ro enya = A more complexity-theoretic example of a place where randomness appears to help is the class IP. IP consists of all languages that can be accepted (with high probability) by a polynomially long interaction between an alt-powerfil prover and a verifier that implements a BPP algorithm, IP = PSPACE.”] However, ifit is required that the verifier be deterministic, then IP = NP. = Ina chemical reaction network (a finite set of reactions like A+B — 2C + D operating on a finite number of molecules), the abilty to ever reach a given target state from an initial state is decidable, while even approximating the probability of ever reaching a given target state (using the standard concentration-based. probability for which reaction will occur next) is undecidable, More specifically, a Turing machine can be simulated with arbitrarily high probability of running correctly for alltime, only ifa random chemical reaction network is used. With a simple nondeterministic chemical reaction network (any possible reaction can happen next), the computational power is limited to primitive recursive fimetions. = The inherent randomness of algorithms such as Hyper-encryption, Bayesian networks, Random neural networks and Probabilistic Cellular Automata was hamessed by Krishna Palem et al. to design highly efficient hardware systems using Probabilistic CMOS or PCMOS technology that were shown to achieve gains that are as high as a mukiplcative factor of 560 when compared to a competing energy-efficient CMOS based realizations.|101 See also «= Probabilistic analysis of algorithms Notes 1. * Probabilistic algorithms should not be mistaken with methods (which I refuse to call algorithms), which produce a result which has a high probability of being correct. It is essential that an algorithm produces correct results (discounting human or computer etrors), even if this happens after a very long time." Henri Cohen (2000). A Course in Computational Algebraic Number Theory. Springer-Verlag, p. 2. 2. 4 "In testing primality of very large numbers chosen at random, the chance of stumbling upon a value that fools the Fermat testis less than the chance that cosmic radiation will cause the computer to make an error in carrying out a ‘correct algorithm. Considering an algorithm to be inadequate for the first reason but not for the second, illustrates the difference between mathematics and engineering.” Hal Abelson and Gerald J. Sussman (1996). Structure and Interpretation of Computer Programs. MIT Press, section 1.2 (http://mitpress.mit.edw/sicp/full- text/book/book-Z-H-11 htmbifootnote_Temp_80) * Smid, Michiel. Closest point problems in computational geometry, Max-Planck-Institut fiir Informatiklyear=1995 * Seidel R. Backwards Analysis of Randomized Geometric Algorithms (hutp:/www.cs. berkeley. edu/~jrs/meshpapers/Seidel ps.g7) 5. * Michael Mitzenmacher, Eli Upfal. Probability and Computing:Randomized Algorithms and Probabilistic Analysis, April 2005. Cambridge University Press 6. * A.A Tsay, W. S. Lovejoy, David R. Karger, Random Sampling in Cut, Flow, and Network Design Problems, Mathematics of Operations Rescarch, 24(2)383—413, 1999, 7. * Dyer, M.; Frieze, A; Kannan, R, (1991), "A random polynomial-time algorithm for approximating the volume of convex bodies" (http:/portal.acm.orgicitation.cfim?id=102783), Journal of the ACM 38 (1): 1-17, doi:10.1145/102782. 102783 (http://dx. doi.org/10,1145%2F 102782. 102783) 8. * Piiredi, Z.; Bardny, I. (1986), "Computing the volume is difficult", Proc. 18th ACM Symposium on Theory of Computing (Berkeley, California, May 28-30, 1986), New York, NY: ACM, pp. 442447, doi:10.1145/12130.12176 (http:/dx.doi.org/10.1145%2F 12130. 12176), ISBN 0-89791-193-8 9. * Shamir, A. (1992), "IP = PSPACE", Journal of the ACM 39 (4): 869-877, doi:10.1145/146585.146609 (http:/dx.doi.org/10,1145%2F 146585.146609) Aw orhipodiaorgiviliRardoized algo aespas Randomized germ. Wikipedia the ree encyclopedia 10. * Lakshmi N. Chakrapani, Bilge E. S. Akgul, Suresh Cheemalavagu, Pinar Korkmaz, Krishna V. Palem and Balasubramanian Seshasayee. "Ultra Efficient Embedded SOC Architectures based on Probabilistic CMOS (PCMOS) Technology" (http://www. pubzone.org/dbip/confidate/ChakrapaniACKPS06). Design Automation and Test in Europe Conference (DATE), 2006. References = Thomas H. Cormen, Charles E, Leiserson, Ronald L, Rivest, and Clifford Stein, Introduction to Algorithms, Second Edition. MIT Press and McGraw-Hill, 1990. ISBN 0-262-03293-7. Chapter Probabilistic Analysis and Randomized Algorithms, pp. 91-122. © Jon Kleinberg and iva Tardos. Algorithm Design. Chapter 13: "Randomized algorithms". = Don Fallis, 2000, "The Reliability of Randomized Algorithms." (http//dx.doi.org/10.1093/bjps/S1.2.255) British Journal for the Philosophy of Science $1 255-11. + M, Mitzenmacher and E, Upfal. Probability and Computing : Randomized Algorithms and Probabilistic Analysis, Cambridge University Press, New York (NY), 2005. = Rajeev Motwani and P. Raghavan, Randomized Algorithms. Cambridge University Press, New York (NY), 1995. = Rajeev Motwani and P. Raghavan, Randomized Algorithms. (http//portal.acm org/citation.cfin? id=2343 13,234327) A survey on Randomized Algorithm. = Christos Papadimitriou (1993), Computational Complexity (Ist ed.), Addison Wesley, ISBN 0-201- 53082-1 Chapter 11: Randomized computation, pp. 241-278. = M.O. Rabin, (1980), "Probabilistic Algorithm for Testing Primality." Journal of Number Theory 12:128~ 38 = A.A Tsay, W. S. Lovejoy, David R. Karger, Random Sampling in Cut, Flow, and Network Design Problems, Mathematics of Operations Research, 24(2)383~413, 1999. Retrieved from "http//en. wikipedia. org/w/index.php?title~Randomized_algorithm&oldid~572067945" Categories: Randomized algorithms | Stochastic algorithms | Analysis of algorithms Probabilistic complexity theory = This page was last modified on 8 September 2013 at 15:40. = Text is available under the Creative Commons Attribution- ShareAlike License; additional terms may apply By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia® is a registered trademark ofthe Wikimedia Foundation, Inc., a non-profit organization. orhipodiaorgiviliRardoized algo 9