Scribd
Upload
38 views4 pages

Iid

Uploaded by

Bipin Attri
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
38 views4 pages

Iid

Uploaded by

Bipin Attri
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
You are on page 1/ 4
28/09/2022, 18:01 Independent and identically eistiovied random variables - Wikipedia WikiPeDiA Independent and identically distributed random variables In probability theory and statisties, a colletion of random variables is independent and identically distributed if each random variable has the same probability distribution asthe others and all are mutually independent. This property is usually abbreviated as i... ti, or IID. ITD was fist defined fn statistics and finds application indifferent fields sch as data mining and signal procesing Contents Introduction ‘03 Definition Definition fr two random variables Definition fr more than two random variables Detiition for ndependence Examples Example 4 Example 2 Example 3 Example 4 Generalizations Exchangeable random variables Levy process Inmachine learning ‘Why assume the data in machine learning re independent and identical cstibutod? See also References Furthor reading Introduction {In statistics, we commonly deal with random samples. A random sample canbe thought of as a set of objects that are chosen randomly. Or, more formally, its“a sequence of independent, identically distributed (IID) random variables" In other words, the terms random sample and JID are basically one and the same. In statistics, we usvally say “random sample,” but in probability it’s ore commen to say "IID." lontically Distributed means that the probabiltydstnbuton Independent maans thatthe sample items ar all independent events n other words, thay aren't connactad to each ater in any way. In other words, knowledge ofthe valie of ane variable gives na information about the valie of tre other and vce versa. reno overall rends-the ditbution doesn't fluctuate and all tems in the sample ae taken fram the same Application Independent and identically distributed random variables are often used as an assumption, which tends to simplify the under} practical applications of statistical modeling, however, the assumption may or may nat be realistic I mathematics. In The iid. assumption is also used in central limit theorem, which states thatthe probability distribution ofthe sum (or average) of L..d. variables with finie variance approaches a normal distribution Often the id. assumption arses inthe context of sequences of random variables. Then "independent and identically distributed” implies that an element inthe sequence is independent of the random variables that came before it. In this way, and. sequence i different from a Markov sequence, where the probability distbution for the nth random variable sa function ofthe previous random variable in the sequence (Tora first order Markov sequen). An id. sequence does not imply the probabilities forall elements ofthe sample space or event space must be the same Ll For example, repeated throws of loaded dice will produce a sequence that i ..d, despite the outcomes being biased Definition Definition for two random variables Suppose thatthe random variables X and ¥ are defined to assume values in J CR. Let F(a) = P(X < 2) and Fy(y) = P(Y < y) be the cumulative distribution functions of X and Y, respectively, and denote thr joint cumulative distribution funtion by Frzy(2,y) = P(X <2 AY 0, there is possibilty PQB]A). Generally, the occurrence of A has an effect on the probability of B, which called conditional probability ad only when the oocurrence of Ahas no effet an the ecumence of B, theres BIA) = PCB). Note: I P(A) > 0, PB) > 0 then A, B are mutually independent which cannot be established with mutually incompatible atthe same time, that is, independence must be compatible and mutual exclusion must be related Suppose A, B, Care three events. If PCAB) = PCA)P(B), P(BC) = P(B)P(C), PCAC) events A, B, Care independent ofeach other. PCADP(C), PARC) = PCA)P(R)P(C) are satisfied, then the A more general definition is there aren events, Ayy Azyonsy. Ifthe probabilities of the product events for any 2,3 ofthe probabilities ofeach event, Unem the events Ay Agy sy Ay are independent of each othe. 1m events are equal to the product Examples Example 1 [A soquence of outcomes of spins ofa fair or unfair roulette whcel is .d. One implication ofthis is that if the roulette ball lands on “red for example, 20 times in a roe, the next spin is no more or les likely tobe “black” than on anyother spin eee the Gamblers fallacy), A sequence of fir o loaded dice rolls isi [A sequence of fir or uni coin ips id Tm signal processing and image processing the notion of transformation to. implies two specifications, the ".@"part andthe "par: (Ga) the signa evel must be balanced on the ime axis; (the ig peru must be fastens ansfrmed by rng (su as ecovuto) to 2 white noise ig (ea ial where al equends are equally present Example 2 Toss a coin 0 times and record how many times does the con lands on head 1. Independent —each outcome of landing wil not atfectthe other outcome, which means the 10 results are independent frm each other 2 lntzay Osibued- the con 8a homogeneas nateal each time the probably or heads 8, which means he probably is ena fr Example 3 Rolla dice 10 ties and record how many time the results 1. Independent—each outcome ofthe sice wil ot afectthe next one, which means the 10 results are independent rom each other. 2 ntzaly Ostby the die a homogensvs materl each tne he probably forthe number Tis, whieh means he robb is Example 4 hitpsen wikipedia orgwikindependent_and_derticaly_cisrbuted_random variables 28 28/09/2022, 18:01 Independent and identically eistiovied random variables - Wikipedia Choose a card from a standard deck of cards containing 52 cards, then place th card back inthe deck. Repeat it for 52 times. Record appears amber of King 1. Independent — each outcome ofthe card will not affect he next one, which maans the 52 results are indepandent from each other 2 dentially Distributed after drawing one card fom i, each time the probably for Kings 4/82, which means the probably is identical foreach time, Generalizations Many results that were frst proven under the assumption that the random variables are iid. have been shown to be true even under a weaker distribational assumption. Exchangeable random variables The most general notion which shares the main properties of iid. variables are exchangeable random variables, introduced by Bruno de Finett ‘Exchangeabilty means that while variables may not be independent, fture ones behave like past ones ~ formally, any value ofa fnie sequence eas ikely as any permutation of thse values ~ the joint probability distribution is invariant under the syzumetric group, This provides a useful generalization ~for example, sampling without replacement is not independent, but i exchangeable, Lévy process In stochastic ealeulus, Li. variables are thought of asa discrete time Lévy process: exch variable gives how much one changes from one time to another. For example, a sequonce of Bernoulli trials is interpreted asthe Bernoall process. One may gencralize this to inlude continuous time Lévy processes, and sy Ley processes can be seen as limits of ..d.varables—for instance, the Wiener process is the limit of the Bernoulli process. In machine learning Why assume the data in machine learning are independent and identically distributed? “Machine learning uses currently aoquired massive quantities of data to deliver faster, more accurate results!l Therefore, we neod to use historieal data ‘with overall representativenest, If the data obtained isnot representative ofthe overall situation, then the rule wl be sunmmaried badly or wrongly Through ied. hypothesis, the number of individual cases in the training sample can be greatly reduced. This assumption makes maximization very easy to cileulate mathematically, Observing the assumption of independent and identical distribution in mathematics simplifies the calculation ofthe likelihood function in optimization problems. Because ofthe assumption of independence, the likelihood function can be written like this, 1) = Pleas 2a,23,--.2l6) = PCsl0)PCoal) les) In onder to mas P(=n|9) 1 the probability ofthe observed event, ake the log function and maximize the parameter 0. Thats to say to compute: ssemaxlog(t(0)) where log(t()) = log( P(e) + log P(22|6)) + log(Pl2s|6)) +... +log(P(=nI6)) ‘The computer is very efficient to calculate multiple addltions, butts not efficent to eleulate the multiplication. Tis simplification is the core reason for the increase in computational efficiency. And this Log transformation i eso inthe process of maximizing, turning many exponential functions into linear functions, or two reasons this hypothesis is easy to use the central limit theorem in practical applications. 1. Even ithe samg lit theorem to Ge rormal istration” 2. The second reason is thatthe accuracy ofthe model depends on the simplicty and representative power of the model unit, as well as the data quali. Because the spicy of the unit makes I easy to interpret and scale, and the representative power + scale out ofthe unit improves the madel accuracy Like in'a deep neural network, each neuron is very simple but has song representative power ayer by layer fo represent more complex features to improve model accuracy.) fed fom n approximately -omes from a mere complex non-Gaussian distribution, it can algo approximate wel. Bacau jsiancistibuion. For large number of observable samples "ine sum of many random var See also + De Finat's theorem + Paiwise independent variables + Geniza imi theorem References 4. Clauset, Aaron (2011). A brief primer on probably dsinbutions” (hp: /Muvalusantafe.edu/~aaronclcourses/7000/¢s:7000-001_2011_LO.pd (POF) Santa Fe stitute, hitpsen wikipedia orgwiki/ndependent_and_denticaly_cistrbuted_random variables sia 2809/2022, 18:01 Independent and identically eistiovted random variables - Wikipedia 2, Stephanie (2078-05-1") "ID Statistics: Independent and dentcally Distributed Defriton and Examples” (hips ivmw stats icshowto.comi-statstic si) Statistics How To. Reiriaved 2021-12-09, 8. Hampel, Frank (1998), "Is statistics too diffcul!”(Nos.Isomantiescholar or/paper02SacS74 0Sec47abSBeSecb28od8absbedbSBN), Conaclan sJoumal of Siatstes, 26 (3). 487-813, dol 10.280718318772 (hips /idoorg/10.2807%2F8315772), hd 20.500.11850/145503 (Nps. handle.ne¥20, '500, 1185052145603), JSTOR 3915772 (hip: jslor.orgslabe/3315772), S2CID 53717661 (Mipsi/ap, somaniicscholr.07/Corpusi0:53°° 765 1168) 4. lum, J. Rs Cheroff, H; Rosenblat, Teicher, H. (1958).

You might also like